Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: Optimize for large multi-line string outputs #27746

Merged
merged 1 commit into from
Feb 11, 2021

Conversation

alisdair
Copy link
Contributor

When rendering a new, very large multi-line string output as part of terraform plan or terraform apply, we hit this code path:

if strings.Contains(val.AsString(), "\n") {
// It's a multi-line string, so we want to use the multi-line
// rendering so it'll be readable. Rather than re-implement
// that here, we'll just re-use the multi-line string diff
// printer with no changes, which ends up producing the
// result we want here.
// The path argument is nil because we don't track path
// information into strings and we know that a string can't
// have any indices or attributes that might need to be marked
// as (requires replacement), which is what that argument is for.
p.writeValueDiff(val, val, indent, nil)
break
}

The writeValueDiff multi-line string renderer calculates a longest common subsequence, which is an operation with quadratic time and space complexity. When the "two" strings are in fact the same value (as you can see from the call above), this results in a worst-case execution time. For very large strings, this can be several minutes:

$ time terraform-0.14.5 plan >/dev/null

real	4m3.381s
user	3m53.593s
sys	0m15.892s

This commit adds an early shortcut for this specific case, using simple string equality as the test, which directly prints the multi-line string without calculating an LCS. This dramatically improves the runtime in this specific edge case:

$ time terraform plan > /dev/null

real	0m0.104s
user	0m0.123s
sys	0m0.043s

Fixes #27743

@alisdair alisdair added cli 0.14-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged labels Feb 11, 2021
@alisdair alisdair requested a review from a team February 11, 2021 15:55
@alisdair alisdair self-assigned this Feb 11, 2021
@codecov
Copy link

codecov bot commented Feb 11, 2021

Codecov Report

Merging #27746 (943df48) into master (14936e6) will decrease coverage by 0.00%.
The diff coverage is 87.50%.

Impacted Files Coverage Δ
command/format/diff.go 90.63% <87.50%> (+0.05%) ⬆️
internal/providercache/dir.go 67.34% <0.00%> (-6.13%) ⬇️
dag/marshal.go 61.90% <0.00%> (-1.59%) ⬇️
terraform/evaluate.go 52.47% <0.00%> (-0.42%) ⬇️
states/statefile/version4.go 57.96% <0.00%> (-0.24%) ⬇️
terraform/node_resource_plan.go 98.05% <0.00%> (+1.94%) ⬆️

@alisdair alisdair merged commit 04e512d into master Feb 11, 2021
@alisdair alisdair deleted the alisdair/optimize-large-multi-line-string-outputs branch February 11, 2021 17:53
@ghost
Copy link

ghost commented Mar 14, 2021

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked as resolved and limited conversation to collaborators Mar 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
0.14-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged cli
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dramatically long time to show output in file function (or when apply)
2 participants