Optimizing Python Production: Mastering .env Files

In this post, I will share my experience in using .env files in Python properly so that you don't make the same mistakes.

I am assuming that you know why and how environment variables are used. The typical use case is to save API keys or any other credentials that you want to share (accidently or intentionally) while sharing/distributing your code. Apart from storing credentials, environment variables are also widely used for project configuration, etc. (We will look into this later in this blog).

Now it's tedious to set the environment variables especially when you just want to try a project. The process of setting up the environment varies as per your operating system.

.env File

This is where .env files comes into play, they let you mimic the actual environment variables with a text file rather than adding them to your operating system. A very popular library to handle this in Python is python-dotenv.

A .env file (. in the front of the name makes it hidden in Mac and Linux systems) is just a text file with each line representing an environment variable. This file is usually added to .gitignore so that it does not get pushed to the remote repo. (And all the effort of using this external file goes into vain 😔).

A quick example

.
├── .env
└── settings.py

import os
from dotenv import load_dotenv

#automatically gets the .env file from the root directory
load_dotenv()

SECRET_KEY = os.environ.get("SECRET_KEY")
DATABASE_PASSWORD = os.environ.get("DATABASE_PASSWORD")

The .env file contains the following

SECRET_KEY=abcdefghh
DATABASE_PASSWORD=123456

Each line in the .env file is the environment variable name along with its associated values. Here we have set up 2 environment variables.

To get the same result without using the .env file we would have to setup the environment variables either by using the environment variable manager in Windows or by using export SECRET_KEY=abcdefghh in Linux-based systems.

Using .env file for configuration

Another common use case is for Project configuration. For example - You want to choose between the production database and test database when developing or deploying your application, we can simply use a debug variable in the .env file instead of changing it in the code. This makes it easier for everyone working on the project to quickly test and push their changes after testing.

Let's look at the code below, I have mocked the actual database connection with a string for demonstration purposes.

.env file

DEBUG=True

import os

from dotenv import load_dotenv

load_dotenv()

DEBUG = os.environ.get("DEBUG")

if DEBUG:
    db="Test Database"
else:
    db="Production Database"

print(db)

Result

Test Database

Working, Yayy!! 😄
Let's do some testing by changing the DEBUG=False in the .env file and re-running the script.

Result

Test Database

Whatt!! 🤯
This is a very naive approach as the IF statement is only checking "if the variable is defined or not" and the "actual value of the variable".

Let's makes some changes in the code and set DEBUG=True in the .env file.

import os

from dotenv import load_dotenv

load_dotenv()

DEBUG = os.environ.get("DEBUG")

if DEBUG==True:
    db="Test Database"
else:
    db="Production Database"

print(db)

what do you think will be the output of the code? Let's run the script to find out.

D:\LEARNING\Python\dotenv>setting.py
Production Database

😲what just happened? We accidentally made changes to the production database, but we clearly used the debug variable to choose the debug database.

To understand what happened we see the return type of the os.environ.get() function. It returns an "str" and we are comparing it to a boolean True, so this "if" statement will never be true.

There is a quick fix to this => Compare the Debug variable with the string "True"

if DEBUG=="True":
    db="Test Database"
else:
    db="Production Database"

There is another way to fix this. we can convert the variable to a bool using strtobool from distutils library. I don't prefer this approach because it involves installing an external library.

I remember making this mistake in my early days. I got really frustrated because things were working fine locally as I had debug=True in my .env file but when I pushed it to production with debug=False, It was still using the test database.

Wrap-up

That was it for this blog. I hope you learned something new today. If you did, Please like/share/clap depending on which platform you are reading this so that it reaches others as well.
For any work related querries you can reach out to me at here.

✅Here are some more posts you might be interested in.