Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/eval: another performance issue with recursive definitions #1044

Open
cueckoo opened this issue Jul 3, 2021 · 6 comments
Open

internal/eval: another performance issue with recursive definitions #1044

cueckoo opened this issue Jul 3, 2021 · 6 comments

Comments

@cueckoo
Copy link
Collaborator

cueckoo commented Jul 3, 2021

Originally opened by @philippseith in cuelang/cue#1044

With cue version 0.4.0 windows/amd64, the following combination of template, list, and default value causes cue vet/eval to hang causing high cpu load:

A: #TypedObject & {
	childs: {
    	B : #TypedObject & {
    		//type: "int"
    	}
    }
}

#TypedObject : {
	type:    #Type | *"string"
	childs: [_] : #TypedObject
}

#Type: #BasicType | #ComplexType | [#BasicType] | [#ComplexType]
#ComplexType: [_] : #Type

 #BasicType: "int" | "uint" | "string"

It successfully runs either when the A.childs.B.type is set explicitly or #Type is defined without List support, eg. as #Type: #BasicType | #ComplexType.

The same definition did work with cue 0.2.2

Strangely, relying on the default value for A.type does not cause cue to hang.

@felixhuettner
Copy link

felixhuettner commented Jul 28, 2021

I have a similar issue with the following snipped

#m1: {
	#f1: [...]
	f2: #f1
}

#m2: {
	#m1
	#f1: *[] | [...]
}

#s1: #m1
#s2: #m1
#s3: #m2
#s4: #m2
#s5: #m2


x: {
	#s1
	#s2
	#s3
	#s4
	#s5
}

Interstingly: at least 3 of s1 to s5 need to refer m2 for the issue to trigger

@verdverm
Copy link

verdverm commented Jul 28, 2021

This looks like a really nice and minimal reproducer for #758 !

Though could it also be related to one of the other roadmap/performance labelled issues? (😆e9)

@myitcv
Copy link
Member

myitcv commented Jul 28, 2021

@philippseith thanks for raising this, and for the repro (as well as @felixhuettner for your repro). All of these are almost certainly related to #803 which will be addressed with upcoming changes to the evaluation engine to support structure sharing (#804). I will leave this open however in case it turns out to be an edge case of any sort.

@myitcv myitcv changed the title cue vet hangs with template, list and default value internal/eval: another performance issue with recursive definitions Jul 28, 2021
cueckoo pushed a commit that referenced this issue Feb 4, 2022
The equality used in disjunction uses StructLits identity for
some elimination. However, as lists initially don't have those
associated these have to be created dynamically. This change
ensurse a unique StructLit is associated with each list so that
lists that are trivially equal can be identified as such.

This doesn't address the below issues completely (there are better
solutions), but in these particular cases it does improve
performance considerably.

Issue #758
Issue #1044

Change-Id: I5d3080aa6503584ed26563f24b68de70b446fe1b
Signed-off-by: Marcel van Lohuizen <[email protected]>
cueckoo pushed a commit that referenced this issue Feb 4, 2022
The equality used in disjunction uses StructLits identity for
some elimination. However, as lists initially don't have those
associated these have to be created dynamically. This change
ensurse a unique StructLit is associated with each list so that
lists that are trivially equal can be identified as such.

This doesn't address the below issues completely (there are better
solutions), but in these particular cases it does improve
performance considerably.

Issue #758
Issue #1044

Change-Id: I5d3080aa6503584ed26563f24b68de70b446fe1b
Signed-off-by: Marcel van Lohuizen <[email protected]>
cueckoo pushed a commit that referenced this issue Feb 4, 2022
The equality used in disjunction uses StructLits identity for
some elimination. However, as lists initially don't have those
associated these have to be created dynamically. This change
ensurse a unique StructLit is associated with each list so that
lists that are trivially equal can be identified as such.

This doesn't address the below issues completely (there are better
solutions), but in these particular cases it does improve
performance considerably.

Issue #758
Issue #1044

Change-Id: I5d3080aa6503584ed26563f24b68de70b446fe1b
Signed-off-by: Marcel van Lohuizen <[email protected]>
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/532656
Unity-Result: CUEcueckoo <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
Reviewed-by: Marcel van Lohuizen <[email protected]>
@njhartwell
Copy link

njhartwell commented Nov 3, 2022

I ran into something that I suspect is the same or related:

#Config & {
	output: {
	  processors: try: [...#Processor]
	}
}

#Config: {
	output?: #Output
}

#AllOutputs: { }

#Output: or([ for name, config in #AllOutputs {
	(name): config
}])

#Output: #Output & {
	processors?: [...#Processor]
}

#AllProcessors: {
	try: [...#Processor]
}

#Processor: or([ for name, config in #AllProcessors {
	(name): config
}])

#Processor: #Processor & {
	processors?: [...#Processor]
}

https://cuelang.org/play/?id=jhu3SJm2Q_k#cue@export@cue

Running cue eval against that (or in the playground) hangs indefinitely and pegs my CPU.

@felixhuettner
Copy link

My snippet from above no longer causes issues in v0.4.3

@njhartwell yours still hangs for me though

@myitcv myitcv added the zGarden label Jun 15, 2023
@mvdan mvdan removed the zGarden label Feb 8, 2024
@mvdan
Copy link
Member

mvdan commented Dec 10, 2024

See my reply in #803 (comment) relating to the new evaluator :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants