Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to-do(dotenv): document key restrictions #5619

Closed
jeiea opened this issue Aug 2, 2024 · 12 comments · Fixed by #5745
Closed

to-do(dotenv): document key restrictions #5619

jeiea opened this issue Aug 2, 2024 · 12 comments · Fixed by #5745
Labels
docs dotenv good first issue Good for newcomers PR welcome A pull request for this issue would be welcome

Comments

@jeiea
Copy link

jeiea commented Aug 2, 2024

Is your feature request related to a problem? Please describe.

It seems that the current dotenv implementation restricts certain strings from being used as keys. For example, keys like '1a' and 'process.env.xxx' are not allowed.

Describe the solution you'd like

I'd like to see these restrictions removed so that these keys can be used.

Describe alternatives you've considered

I'd appreciate an explanation of the rationale behind these restrictions. Or is it possible that I just haven't found the right way to do this?

I found npm dotenv package allowing this.

@kt3k
Copy link
Member

kt3k commented Aug 2, 2024

I think this restriction comes from the restriction of env var names in shell script (because .env file is modeled after them). However maybe it's good idea to align it to other dotenv tools.

BTW Deno CLI also supports --env flag, which supports process.env.xxx, but it doesn't seem supporting 1a.

Can anybody check how other tools handles .env variable names?

@iuioiua
Copy link
Contributor

iuioiua commented Aug 9, 2024

Can anybody check how other tools handles .env variable names?

npm:dotenv supports this functionality, and that's the most popular package in the npm ecosystem. Frankly, I don't see any reasons not to support these types of keys.

@timreichen
Copy link
Contributor

https://dotenvx.com/docs/env-file#keys

@iuioiua
Copy link
Contributor

iuioiua commented Aug 9, 2024

Oh, sorry, I see. These restrictions are in place so that environment variables are portable between .env files and the CLI on various platforms. In that case, these restrictions are reasonable and should be documented. I'd happily look at a PR that does so.

@iuioiua iuioiua changed the title Loose dotenv key string constraint to-do(dotenv): document key restrictions Aug 9, 2024
@iuioiua iuioiua added good first issue Good for newcomers dotenv PR welcome A pull request for this issue would be welcome docs labels Aug 9, 2024
@jeiea
Copy link
Author

jeiea commented Aug 10, 2024

Thank you for your responses. I have a few more questions.

For the sake of portability (and sanity), environment variable names (keys) must consist solely of letters, digits, and the underscore (_) and must not begin with a digit. In regex-speak, the names must match the following pattern:

I'm not entirely sure what is meant by "portability" and "sanity" here. Are they suggesting that by restricting environment variable names for shell scripts, deno-std makes it easier to use across different environments? From my perspective, these restrictions in deno-std actually reduce portability because I’m unable to use the keys in permissive environment.

The mention of "sanity" is also confusing. I spent a significant amount of time debugging because I wasn’t aware that deno-std would ignore certain keys. My intention in raising this issue was to help improve the developer experience for other's sanity.

The point about following the same policy as Rust, as mentioned by kt3k, does seem more convincing. However, to my knowledge, the current implementations in deno-std and Rust aren't identical, so I don’t think that was the intended alignment.

If the restrictions on keys are to be maintained, I suggest providing a warning when environment variables are ignored. Also, I’m curious about what alternatives you would recommend for using .env files for esbuild’s define functionality under the current policy.

That said, I still support my original proposal as a way to avoid these issues altogether.

@kt3k
Copy link
Member

kt3k commented Aug 12, 2024

I suggest providing a warning when environment variables are ignored.

This sounds good to me. Let's do this

Also, I’m curious about what alternatives you would recommend for using .env files for esbuild’s define functionality under the current policy.

I don't see well what this mean. --define looks like something for preprocessing of JS expressions while bundling. How is that related to dotenv?

@jeiea
Copy link
Author

jeiea commented Aug 12, 2024

I don't see well what this mean. --define looks like something for preprocessing of JS expressions while bundling. How is that related to dotenv?

In my custom bundler, the process.env.xxx syntax originated from there. I wanted to pass the contents of a .env file as --define parameter, but this wasn’t permitted by this.

I could have used JSON or TOML for this, but I felt the .env format was the most natural choice.

@kt3k
Copy link
Member

kt3k commented Aug 16, 2024

As you can see from this discussion, .env is not meant for storing general purpose data, but for storing data usually mapped to environment variables, which usually have restrictions in key characters.

I think JSON/TOML/YAML are more appropriate for your use case.

@jeiea
Copy link
Author

jeiea commented Aug 16, 2024

As you can see from this discussion, .env is not meant for storing general purpose data, but for storing data usually mapped to environment variables, which usually have restrictions in key characters.

I think the only relavant ground on this discussion is timreichen's link, but I can't think of concrete problematic case for this. I described about this on my previous comment. So I'll appreciate if anyone give me concrete problem by loosing key constraint. I've searched dotenvx's issues but couldn't find such case.

It might also be helpful to clarify what is meant by "general purpose data". For example, if an external application requires the use of . in environment variable names, I wouldn't be able to use @std/dotenv. Is this intended?

@timreichen
Copy link
Contributor

I think the only relavant ground on this discussion is timreichen's link, but I can't think of concrete problematic case for this. I described about this on my previous comment. So I'll appreciate if anyone give me concrete problem by loosing key constraint. I've searched dotenvx's issues but couldn't find such case.

It might also be helpful to clarify what is meant by "general purpose data". For example, if an external application requires the use of . in environment variable names, I wouldn't be able to use @std/dotenv. Is this intended?

Why would an app need that? If a key value with special chars is relevant, I think one can use the variable to store the key CUSTOM_KEY=value.with.dots And work from there no?

@jeiea
Copy link
Author

jeiea commented Aug 16, 2024

Why would an app need that?

I admit I don't know such app, but I think such need exists from an issue.

If a key value with special chars is relevant, I think one can use the variable to store the key CUSTOM_KEY=value.with.dots And work from there no?

This approach seems more like a workaround than a proper solution. Given this restriction I see at least two issues:

  • It should document the restriction.
  • It should warn problematic key usage.

And from now on I suggest considering the following too.

  • It should guide recommended usage implementation (or forbid usage).

@timreichen
Copy link
Contributor

  • It should document the restriction.
  • It should warn problematic key usage.

I agree. Deno even throws if on invalid keys:

Dot in key:

FOO.BAR="bar"
Uncaught (in promise) ReferenceError: FOO is not defined

Number at beginning of the key:

1FOO="bar"
Warning Parsing failed within the specified environment file: .env at index: 0 of the value: 1FOO="bar"
error: The module's source code could not be parsed: Identifier cannot follow number at file:///path/to/.env:1:2

So I guess std dotenv should also throw on an invalid key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs dotenv good first issue Good for newcomers PR welcome A pull request for this issue would be welcome
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants