Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clearly specify .env file syntax (not like Docker, not like Bash; it's something else) #278

Open
yourcelf opened this issue Oct 12, 2020 · 3 comments
Labels
documentation Improvements or additions to documentation

Comments

@yourcelf
Copy link

The supported syntax of the .env file needs to be better specified (possibly as part of #88, but this may be a bigger issue than just that ticket).

Reading the regex-based parsing, it's clear that django-environ tries to support something bash-ish - it strips leading export declarations at the beginning of the line, and does some rudimentary quote processing (just stripping matching ' or " at the beginning and end).

However:

  • This differs from Docker's behavior for .env files. Docker does no quote parsing, so beginning/trailing quotes are considered as part of the value. An .env file that works in django-environ may not work in Docker. An example failure: a quoted JSON string like VAR='{"key": "value"}' will work in django-environ, but fail if Docker has parsed .env and populated the environment with it; the wrapping ' will be retained and not be valid json.
  • The rudimentary quote parsing is pathological in some cases (e.g. read_env doesn't parse dictionary properly from lines with quotes/apostrophes  #165), and certainly doesn't support all of the quoting complexity found in bash. A valid bash quoting syntax will not always work for django-environ.

In sum, the current behavior is:

  • Variables are expressed as VAR=VAL in single lines.
  • Beginning export keywords are stripped.
  • If VAL is wrapped in single or double quotes, those are stripped; but no further resolution of internal quoting structure is done.

Expecting a single .env file to successfully target more than one of Docker, django-environ, or bash is looking for trouble; care must be taken for what other contexts populate the environment from .env before django-environ runs. Docker-parity can be achieved by avoiding all wrapping quotes in the .env values, so long as no value requires wrapping quotes as part of its content. Bash parity is probably not possible without using a bash interpreter to parse the file.

@Dantemss
Copy link

Dantemss commented May 3, 2021

In particular, the output of shlex.quote() when it wraps quotes, which looks like '"'"', is not supported.
Workaround is to use a .env file with export on every line and force all scripts to source the .env file before launching django rather than relying on read_env.

@piscvau
Copy link

piscvau commented Aug 11, 2021

I am not an expert but sincerely understanding the syntax of dict form the 2 lines of the project description is difficult :
dict (BAR=key=val,foo=bar) #environ.Env(BAR=(dict, {}))

  • dict (BAR=key=val;foo=1.1;baz=True) #environ.Env(BAR=(dict(value=unicode, cast=dict(foo=float,baz=bool)), {}))

@sergeyklay sergeyklay added the documentation Improvements or additions to documentation label Sep 4, 2021
@pataquets
Copy link

As I'm now reviewing issues, I've stumbled up (again) with this one, and I'd like to share something.
Leaving aside what and how django-environ does now, I agree with @yourcelf that trying to implement different usages, patterns, syntaxes, whatever is clearly looking for trouble.
I'd like to share a comment I found insightful in a Terraform issue:
hashicorp/terraform#23906 (comment)
The most important point, IMO is that, although .env files are not a standard (at least not a complete one in the details), everyone's feels like their use case is the "expected standard" (be it Bash, Kubernetes, etc.).
This may create legitimate differences in expectations about what should djanjo-environ do or implement. The worst of it is that maybe implementing some tool's conventions breaks another's.

@joke2k I think stating clearly which features and expectations on the "standard" will django-eviron support (whatever you choose them to be) would ease settle those "Bash parses X vs. Kubernetes parses Y" discussions. Those discussions may come sometimes with statements like "the most common use case is/does... [insert your use case]" which are poisonous and may trigger heated debates, at some point.
I'd even go further and place it prominently visible in the introduction section in the docs. At least, some ".env are not a standard, just some loose convention". Realizing this helped me frame my views.

Hey, maybe even in a distant future, if the lib is made extensible, people can build their own "specific parsings" on top as some sort of add-on.
Also, for those looking for a deep and wide view on how people approach .env files, read the linked issue full thread (and for some amusement with the debate).
Just my 2c.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

5 participants