Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for_each arguments containing sensitive values if they aren't keys #27247

Merged
merged 2 commits into from
Dec 17, 2020

Conversation

pselle
Copy link
Contributor

@pselle pselle commented Dec 10, 2020

This adjusts the previous behavior that would disallow any value involving marks to for_each to be more nuanced, and allow maps and objects where the keys are unmarked strings to pass through this evaluation unscathed.

If the keys of an object are marked, a [cty] panic will occur when evaluating the expression, but there's an HCL PR to turn that into an error.

This is related to #27239, but does not fix it, a function call is used in the expression, and function calls always result in marked values when passed any marked arguments (docs PR that clarifies this: #27280) currently, since no functions are marks-aware at present.

Also related to #27180 but does not fix it, for the same reason (function calls).

@codecov
Copy link

codecov bot commented Dec 10, 2020

Codecov Report

Merging #27247 (a0c0273) into master (21d6fb5) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted Files Coverage Δ
terraform/eval_for_each.go 98.07% <100.00%> (+10.82%) ⬆️
communicator/communicator.go 77.19% <0.00%> (-3.51%) ⬇️
terraform/evaluate.go 52.89% <0.00%> (-0.42%) ⬇️
dag/marshal.go 63.49% <0.00%> (+1.58%) ⬆️
terraform/node_resource_plan.go 98.05% <0.00%> (+1.94%) ⬆️

@pselle pselle marked this pull request as draft December 10, 2020 21:11
@pselle pselle force-pushed the pselle/for_each-sensitive branch 2 times, most recently from c412cc5 to 4961109 Compare December 14, 2020 20:58
@pselle pselle marked this pull request as ready for review December 14, 2020 20:59
@pselle pselle force-pushed the pselle/for_each-sensitive branch from 4961109 to 2a292c0 Compare December 14, 2020 21:18
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid for_each argument",
Detail: "Sensitive variables, or values derived from sensitive variables, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource instance key.",
Copy link
Contributor Author

@pselle pselle Dec 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this "or derived from" be left to communicate the function-result situation? Unsure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that phrase, personally, and think it covers the function-result situation, but not in a "blocking merge" way!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use a term like "sensitive values" here? I think this diagnostic applies to all of sensitive variables, outputs, and attributes (with the experiment enabled).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alisdair excellent point, "values" is correct rather than variables! And thanks for the input @mildwonkey, I'll add it back

@pselle pselle force-pushed the pselle/for_each-sensitive branch 2 times, most recently from 613896c to cf64288 Compare December 16, 2020 20:35
@pselle pselle force-pushed the pselle/for_each-sensitive branch from cf64288 to 58717d2 Compare December 16, 2020 20:36
@pselle pselle requested a review from a team December 16, 2020 20:38
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid for_each argument",
Detail: "Sensitive variables, or values derived from sensitive variables, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource instance key.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that phrase, personally, and think it covers the function-result situation, but not in a "blocking merge" way!

diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid for_each argument",
Detail: "Sensitive variables, or values derived from sensitive variables, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource instance key.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use a term like "sensitive values" here? I think this diagnostic applies to all of sensitive variables, outputs, and attributes (with the experiment enabled).

func markSafeLengthInt(val cty.Value) int {
v, _ := val.UnmarkDeep()
return v.LengthInt()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR, but I'm kind of surprised that LengthInt can't be called on marked values. Do you think we could adjust it to do so in the future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. We could investigate it. There's at least a couple of Unmarking in the format code because of LengthInt as well, iirc, i honestly took it (LengthInt())
for granted as something that shouldn't change, but perhaps it should.

Rather than disallow values that have any marks
as for_each arguments, this makes the check more
nuanced to disallow cases where the whole value
is marked (a whole map, or any set). This allows
cases where a user may pass a map that has marked
values, but the keys are not sensitive
@pselle pselle force-pushed the pselle/for_each-sensitive branch from 58717d2 to a0c0273 Compare December 17, 2020 16:12
@pselle pselle merged commit 428d404 into master Dec 17, 2020
@pselle pselle mentioned this pull request Dec 17, 2020
@ghost
Copy link

ghost commented Jan 17, 2021

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked as resolved and limited conversation to collaborators Jan 17, 2021
@pselle pselle deleted the pselle/for_each-sensitive branch March 4, 2021 20:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants