Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Terraform v0.14.0 Panic with templatefile and sensitive variable #27253

Closed
DarkMukke opened this issue Dec 11, 2020 · 5 comments
Closed

Terraform v0.14.0 Panic with templatefile and sensitive variable #27253

DarkMukke opened this issue Dec 11, 2020 · 5 comments
Assignees
@alisdair
Labels
bug crash explained a Terraform Core team member has described the root cause of this issue in code new new issue not yet triaged

Comments

@DarkMukke
Copy link

DarkMukke commented Dec 11, 2020 •
edited
Loading

It worked fine before i marked a variable as sensitive, not sure if the panic is related.

Terraform Version

v0.14.0

Crash Output

Error: Error in function call

  on ../vault-config/helm.tf line 9, in resource "helm_release" "vault_config":
   9:     templatefile("${path.module}/values.yaml.tpl", {
  10:       node_selector = var.node_selector
  11:       cluster_name = var.cluster_name
  12:       extra_args = var.extra_args
  13:       image = format("%s:%s", var.terraform_image, var.terraform_tag)
  14:       consul_acl_token = var.consul_acl_token
  15:     })
    |----------------
    | path.module is "../vault-config"
    | var.cluster_name is "hb-mvp-ruben"
    | var.consul_acl_token is (sensitive value)
    | var.extra_args is "-var  dynamodb_role_arn='arn:aws:iam::123456789:role/vault_dynamodb20200715145628837100000001 -var dynamodb_role_name='vault_dynamodb20200715145628837100000001' -var         aws_region='eu-west-2'"
    | var.node_selector is "base"
    | var.terraform_image is "123456789.dkr.ecr.eu-west-2.amazonaws.com/terraform"
    | var.terraform_tag is "light"

Call to function "templatefile" failed: panic in function implementation:
value is marked, so must be unmarked first
goroutine 3113 [running]:
runtime/debug.Stack(0xc002473d68, 0x2f6d360, 0x37a3a70)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x9f
github.com/zclconf/go-cty/cty/function.errorForPanic(...)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/error.go:44
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues.func1(0xc002474780,
0xc002474790)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:217
+0x7f
panic(0x2f6d360, 0x37a3a70)
        /usr/local/go/src/runtime/panic.go:969 +0x175
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:123
github.com/zclconf/go-cty/cty.Value.AsString(0x384a680, 0xc000050400,
0x3142500, 0xc000f85bc0, 0x384a680, 0xc000050400)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1229 +0x4f
github.com/hashicorp/hcl/v2/hclsyntax.(*TemplateExpr).Value(0xc0006cfbc0,
0xc00226eb20, 0x3423b44, 0xa, 0xc00228f9d8, 0x1, 0x0, 0xf, 0x10)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression_template.go:83
+0xcc5
github.com/hashicorp/terraform/lang/funcs.MakeTemplateFileFunc.func2(0x3849380,
0xc0006cfbc0, 0x384a780, 0xc0022508c0, 0x304d3a0, 0xc002263140, 0x0, 0x0,
0x384a680, 0xc000050400, ...)
        /home/circleci/project/project/lang/funcs/filesystem.go:145 +0x66a
github.com/hashicorp/terraform/lang/funcs.MakeTemplateFileFunc.func3(0xc00222c440,
0x2, 0x2, 0x49312d0, 0xc00230c5d0, 0x0, 0x0)
        /home/circleci/project/project/lang/funcs/filesystem.go:169 +0x1d2
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues(0xc002231b90,
0xc00222c440, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:221
+0x454
github.com/zclconf/go-cty/cty/function.Function.Call(0xc002231b90,
0xc00222c440, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:228
+0xb5
github.com/hashicorp/hcl/v2/hclsyntax.(*FunctionCallExpr).Value(0xc00137e5a0,
0xc00226e220, 0x1, 0xc00226e880, 0xc002262ff0, 0x18, 0x18, 0x1013cdd,
0xc001597768)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:412
+0x10c5
github.com/hashicorp/hcl/v2/hclsyntax.(*TupleConsExpr).Value(0xc000d8b900,
0xc00226e220, 0x0, 0x6, 0xc0033ddc38, 0xc000ad4101, 0x3342ac0, 0xc00226e860,
0x2f6d360)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:712
+0xf0
github.com/hashicorp/hcl/v2/hcldec.(*AttrSpec).decode(0xc001de0960,
0xc000a90e40, 0x0, 0x0, 0x0, 0xc00226e220, 0x384a680, 0xc000050400, 0x0, 0x0,
...)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:205 +0x2ac
github.com/hashicorp/hcl/v2/hcldec.ObjectSpec.decode(0xc00131d3b0,
0xc000a90e40, 0x0, 0x0, 0x0, 0xc00226e220, 0xc00086aed0, 0x20, 0x20,
0xc00226e260, ...)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:79 +0x237
github.com/hashicorp/hcl/v2/hcldec.decode(0x3849600, 0xc00226e260, 0x0, 0x0,
0x0, 0xc00226e220, 0x384a440, 0xc00131d3b0, 0x0, 0x0, ...)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/decode.go:21 +0x11b
github.com/hashicorp/hcl/v2/hcldec.Decode(...)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/public.go:15
github.com/hashicorp/terraform/lang.(*Scope).EvalBlock(0xc003347c70,
0x3848e80, 0xc002231c50, 0xc000b012f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, ...)
        /home/circleci/project/project/lang/eval.go:67 +0x2ac
github.com/hashicorp/terraform/terraform.(*BuiltinEvalContext).EvaluateBlock(0xc002ad9520,
0x3848e80, 0xc002231c50, 0xc000b012f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/circleci/project/project/terraform/eval_context_builtin.go:280 +0x1ad
github.com/hashicorp/terraform/terraform.(*EvalDiff).Eval(0xc002477a98,
0x3885cc0, 0xc002ad9520, 0x0, 0x0, 0x0, 0x0)
        /home/circleci/project/project/terraform/eval_diff.go:156 +0x27c
github.com/hashicorp/terraform/terraform.(*NodePlannableResourceInstance).managedResourceExecute(0xc0014cf320,
0x3885cc0, 0xc002ad9520, 0x0, 0x40000)
        /home/circleci/project/project/terraform/node_resource_plan_instance.go:207
+0x58d
github.com/hashicorp/terraform/terraform.(*NodePlannableResourceInstance).Execute(0xc0014cf320,
0x3885cc0, 0xc002ad9520, 0xc00252a002, 0x309ac60, 0x33aa5e0)
        /home/circleci/project/project/terraform/node_resource_plan_instance.go:39
+0xd2
github.com/hashicorp/terraform/terraform.(*ContextGraphWalker).Execute(0xc0023ae5b0,
0x3885cc0, 0xc002ad9520, 0xf1fbc80, 0xc0014cf320, 0x0, 0x0, 0x0)
        /home/circleci/project/project/terraform/graph_walk_context.go:127 +0xbc
github.com/hashicorp/terraform/terraform.(*Graph).walk.func1(0x33aa5e0,
0xc0014cf320, 0x0, 0x0, 0x0)
        /home/circleci/project/project/terraform/graph.go:59 +0x962
github.com/hashicorp/terraform/dag.(*Walker).walkVertex(0xc000b790e0,
0x33aa5e0, 0xc0014cf320, 0xc0020bb000)
        /home/circleci/project/project/dag/walk.go:387 +0x375
created by github.com/hashicorp/terraform/dag.(*Walker).Update
        /home/circleci/project/project/dag/walk.go:309 +0x1246
.

But then running a separate test

test.tf

variable "my_name" {
  type = string
}

variable "my_password" {
  type = string
  sensitive = true
}

resource "local_file" "test" {
  filename = "name.txt"
  content = templatefile("test.tpl", {
    the_password = var.my_name
  })
}

resource "local_file" "test_senssitive" {
  filename = "text.txt"
  content = templatefile("test.tpl", {
    the_password = var.my_password
  })
}

test.tpl

${the_password}

Works fine, so i don't get it.

When I remove the sensitive = true from the variable definition, it works fine.
@DarkMukke DarkMukke added bug new new issue not yet triaged labels Dec 11, 2020
@alisdair alisdair self-assigned this Dec 11, 2020
@alisdair
Copy link
Contributor

Hi @DarkMukke, thanks for reporting this.

Like you, I was unable to reproduce the bug with the configuration you gave. From reading the stack trace, I think this is most likely a case of doubly-marked variables: a sensitive variable set as an input to a module with a sensitive variable. That bug was fixed in #27131, released as part of 0.14.1.

This allowed me to reproduce the same crash with 0.14.0 and the following configuration.

main.tf

variable "password" {
  type = string
  sensitive = true
  default = "password"
}

module "tmpl" {
  source = "./tmpl"
  password = var.password
}

tmpl/main.tf

variable "password" {
  type = string
  sensitive = true
}

resource "local_file" "test" {
  filename = "${path.module}/test.txt"
  content = templatefile("${path.module}/test.tpl", {
    password = var.password
  })
}

tmpl/test.tpl

${password}

Can you try upgrading to the 0.14.2 release and see if the issue is fixed for you?

@alisdair alisdair added the explained a Terraform Core team member has described the root cause of this issue in code label Dec 11, 2020
@danieldreier danieldreier added the waiting-response An issue/pull request is waiting for a response from the community label Dec 11, 2020
@DarkMukke
Copy link
Author

Spot on on the sensitive chain, i should have explained it better, it was a sensitive value in a wrapper module that was calling another module that also had the variable set as sensitive.

Testing with v0.14.2 it works as expected.

@ghost ghost removed the waiting-response An issue/pull request is waiting for a response from the community label Dec 14, 2020
@schnerring
Copy link

schnerring commented Jan 4, 2021 •
edited
Loading

I'm on v0.14.3 and reproduced the error. I think it has to do with sensitive variables of type list and set.

Given the file main.tf:

variable "passwords" {
  type      = list(string)
  default   = ["foo", "bar"]
  sensitive = true
}

output "test" {
  value     = templatefile("./template.yaml", { passwords = var.passwords })
  sensitive = true
}

and template.yaml:

%{ for password in passwords ~}
${password}
%{ endfor ~}

I get the error:

Error: Error in function call

  on main.tf line 8, in output "test":
   8:   value     = templatefile("./template.yaml", { passwords = var.passwords })
    |----------------
    | var.passwords is (sensitive value)

Call to function "templatefile" failed: panic in function implementation:
value is marked, so must be unmarked first
goroutine 45 [running]:
runtime/debug.Stack(0xc0006b1620, 0x237d540, 0x2bb5840)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x9f
github.com/zclconf/go-cty/cty/function.errorForPanic(...)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/error.go:44
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues.func1(0xc0006b21f8,
0xc0006b2208)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:217
+0x7f
panic(0x237d540, 0x2bb5840)
        /usr/local/go/src/runtime/panic.go:969 +0x175
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:123
github.com/zclconf/go-cty/cty.Value.ElementIterator(0x2c5c400, 0xc00059d640,
0x2551bc0, 0xc00059d680, 0x2551b01, 0xc00059d680)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1111 +0x4f
github.com/hashicorp/hcl/v2/hclsyntax.(*TemplateJoinExpr).Value(0xc00014b090,
0xc00059d260, 0xc0006b1a50, 0x41675b, 0x7f6c1311ab00, 0x100, 0x7f6c132eafff,
0x400, 0x7f6c1311ac00)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression_template.go:156
+0x190
github.com/hashicorp/hcl/v2/hclsyntax.(*TemplateExpr).Value(0xc0005fd3e0,
0xc00059d260, 0x2834347, 0xa, 0xc0005f7358, 0x1, 0x0, 0x2, 0x2)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression_template.go:33
+0x157
github.com/hashicorp/terraform/lang/funcs.MakeTemplateFileFunc.func2(0x2c5af80,
0xc0005fd3e0, 0x2c5c380, 0xc00014b000, 0x245cee0, 0xc0005e9020, 0x0, 0x0, 0x0,
0x0, ...)
        /home/circleci/project/project/lang/funcs/filesystem.go:145 +0x66a
github.com/hashicorp/terraform/lang/funcs.MakeTemplateFileFunc.func3(0xc0005c88c0,
0x2, 0x2, 0x3d43538, 0xc0005c2048, 0x0, 0x0)
        /home/circleci/project/project/lang/funcs/filesystem.go:169 +0x1d2
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues(0xc0005e8bd0,
0xc0005c88c0, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:221
+0x454
github.com/zclconf/go-cty/cty/function.Function.Call(0xc0005e8bd0,
0xc0005c88c0, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:228
+0xb5
github.com/hashicorp/hcl/v2/hclsyntax.(*FunctionCallExpr).Value(0xc00030a0f0,
0xc00059d0c0, 0x0, 0xc0005c3900, 0x1, 0x1, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:442
+0x10c5
github.com/hashicorp/terraform/lang.(*Scope).EvalExpr(0xc0005ae640, 0x2c5ae00,
0xc00030a0f0, 0x2c5c2c0, 0x3d43538, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/circleci/project/project/lang/eval.go:171 +0x1b7
github.com/hashicorp/terraform/terraform.(*BuiltinEvalContext).EvaluateExpr(0xc0005b1450,
0x2c5ae00, 0xc00030a0f0, 0x2c5c2c0, 0x3d43538, 0x0, 0x0, 0x62, 0x400, 0x62,
...)
        /home/circleci/project/project/terraform/eval_context_builtin.go:287 +0xbb
github.com/hashicorp/terraform/terraform.(*NodeApplyableOutput).Execute(0xc0005c8380,
0x2c97980, 0xc0005b1450, 0xc00003a002, 0x24aa880, 0x2683e80)
        /home/circleci/project/project/terraform/node_output.go:273 +0x60d
github.com/hashicorp/terraform/terraform.(*ContextGraphWalker).Execute(0xc0005b0b60,
0x2c97980, 0xc0005b1450, 0x7f6c12d6f100, 0xc0005c8380, 0x0, 0x0, 0x0)
        /home/circleci/project/project/terraform/graph_walk_context.go:127 +0xbc
github.com/hashicorp/terraform/terraform.(*Graph).walk.func1(0x2683e80,
0xc0005c8380, 0x0, 0x0, 0x0)
        /home/circleci/project/project/terraform/graph.go:59 +0x962
github.com/hashicorp/terraform/dag.(*Walker).walkVertex(0xc0005fc300,
0x2683e80, 0xc0005c8380, 0xc0005c8800)
        /home/circleci/project/project/dag/walk.go:387 +0x375
created by github.com/hashicorp/terraform/dag.(*Walker).Update
        /home/circleci/project/project/dag/walk.go:309 +0x1246
.

@alisdair
Copy link
Contributor

alisdair commented Jan 4, 2021

@schnerring Thanks for your additional report. I believe that is a duplicate of #27336, which has a pending fix in the upstream library.

@ghost
Copy link

ghost commented Jan 14, 2021

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked as resolved and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alisdair alisdair

Labels
bug crash explained a Terraform Core team member has described the root cause of this issue in code new new issue not yet triaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@alisdair @DarkMukke @schnerring @danieldreier