command/output: Raw output mode

[apparentlymart]

So far the output command has had a default output format intended for human consumption and a JSON output format intended for machine consumption.

However, until Terraform v0.14 the default output format for primitive types happened to be almost a raw string representation of the value, and so users started using that as a more convenient way to access primitive-typed output values from shell scripts, avoiding the need to also use a tool like "jq" to decode the JSON. Terraform v0.14 improved the default terraform output formatting to be consistent with how we print values in other commands such as terraform plan, but consequently it's no longer suitable for direct consumption by shell scripts.

Recognizing that primitive-typed output values are common and that processing them with shell scripts is common, this commit introduces a new -raw mode which is explicitly intended for that use-case, guaranteeing that the result will always be the direct result of a string conversion of the output value, or an error if no such conversion is possible.

Our policy elsewhere in Terraform is that we always use JSON for machine-readable output. We adopted that policy because our other machine-readable output has typically been complex data structures rather than single primitive values. A special mode seems justified for output values because it is common for root module output values to be just strings, and so it's pragmatic to offer access to the raw value directly rather than requiring a round-trip through JSON.

I'm not intending this to set any precedent for offering -raw options on any other commands. I would also consider any further complexity, such as selecting a deep primitive value out of a nested data structure using an expression/traversal, to be out of scope for this command. jq is a far better answer to those more complex cases, because it has a richer query language and various output post-processing options.

---

I've marked this for v0.14 backport in the hope of giving folks who were previously using terraform output for machine-readable data an easier upgrade path from v0.13 than installing and learning jq.

[codecov]

Codecov Report

Merging #27212 (8a88d3c) into master (30126ba) will increase coverage by 0.00%.
The diff coverage is 62.06%.

Impacted Files	Coverage Δ
command/output.go	68.94% <62.06%> (-3.12%)	⬇️
dag/marshal.go	61.90% <0.00%> (-1.59%)	⬇️

[mildwonkey]

I'm not thrilled about adding this flag to output when we've fought to avoid random flags, and have provided stable machine-readable outputs (on the other hand, -raw is not an uncommon flag on CLIs, so I'm not completely opposed either). Do you have any GH issues (or, I'll wager a guess, discuss posts) that might help me understand the impetus for adding this?

[apparentlymart]

I don't have them all handy right now but I've seen at least three variants of this already, two of which were about Terraform printing out a multi-line string with <<EOT/EOT around it and one about the quotes. The forum post was at least easy to find because there was a comment on it recently.

I share the concern about "random flags", but this one doesn't feel "random" to me... getting strings out of Terraform for use elsewhere is the main reason to use root module outputs, and shell scripts are the main way people string those together. While requiring jq (or similar) is justified for the other situations where we're returning complex data structures, it feels excessive to me here where root module output values are commonly just plain strings, like URLs or object ids, and when we already have a suitable building block (converting to string) to implement this without introducing lots of new complexity.

[mildwonkey]

Sounds good to me, thank you for the extra discussion!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.