Taking Advantage of the docker-compose Environment Variables

If you need to define various configuration values, environment variables are your best friend. Like many tools, Docker and, more specifically, Docker Compose can both define and read environment variables to help keep your configurations clean and modular. This tutorial teaches you how to use docker-compose environment variables to define different containers, environments, and more.

Prerequisites

If you’d like to follow along step-by-step, ensure you have the following:

• A Linux host with admin privileges. This tutorial uses Ubuntu 18.04.5 LTS.
• Docker installed on the Linux host. This tutorial uses Docker v19.03.11.

Related:How to Install and Use Docker on Ubuntu (In the Real World)

Declaring Environment Variables in the Docker Compose File

Containers can demand a lot of configuration. And not every configuration in your containers will be unique. Some settings might be shared across some of your containers, like a MySQL database credential. Store the credentials manually in each container, and when the credentials change, you end up having to update the credentials multiple times.

Environment variables can reduce that hassle. Store those shared settings as environment variables! Reference the environment variables instead of repeating yourself in the containers. When the credentials change, you have to update just one setting: the environment variable.

Let’s start by declaring an environment variable and storing it in the Docker Compose file itself. The steps below will store settings for a hypothetical MySQL database in environment variables.

1. Open a terminal on your local machine.

2. Create a folder named ~/docker-compose-demo, then change (cd) the working directory to the folder you just created. The ~/docker-compose-demo folder will contain all of the files you’ll create in this tutorial.

mkdir ~/docker-compose-demo
cd ~/docker-compose-demo

3. Open your favorite text editor, copy/paste the code in the snippet below in the text editor. Save the file as docker-compose.yml inside the ~/docker-compose-demo directory. The docker-compose.yml stores the configurations for your application’s services.

In the snippet code below:

• mysql:5.7 is the base image that Docker Compose pulls and creates a new container.
• MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD, and MYSQL_RANDOM_ROOT_PASSWORDare three different environments variables within the Docker Compose file. Each environment variable is declared with its values. The value the environment variable gets comes after the: symbol.
• mysql service will create the container named mysql.

# Version of Docker compose file
version: "2.2"

services:
# Defining the service
  mysql:
# Defining the base image to be used 
    image: mysql:5.7
    hostname: mysql
    container_name: mysql
# Defining the environmental variable
    environment:
      # ENVIRONMET_VARIABLE_NAME: "environment variable value" 
      MYSQL_ROOT_PASSWORD: "root_password"
      MYSQL_ALLOW_EMPTY_PASSWORD: "password"
      MYSQL_RANDOM_ROOT_PASSWORD: "password"

Some containers rely on environment variables to work properly. The one in the example, MySQL, is one of these containers. If you don’t declare the environment variables the container is expecting, it will throw an error. You can see the error below.

4. Next, execute the docker-compose up command. The docker-compose up command reads the YAML file (docker-compose.yml) created in the previous step and creates the container. The docker-compose up command starts all the services configured in the Docker Compose file.

5. Now, verify if all three environment variables are present in the container by running the docker exec command and the env command.

Running the docker exec command will allow you to log in to the container. Running env will print the list of the current environment variables with their respective values.

# ee8a... is the container
docker exec -it ee8af8bfcd41 /bin/bash

Substituting the Environment Variables

In the previous section, you learned how to declare the environment variables directly within the Docker Compose file by hard coding them. Such an approach is not optimal if you need to secure your credentials. As an alternative, store the values for environment variables in a file named .env that only the admin can read.

Let’s practice using substitution for environment variables. In your terminal:

1. Create a file named .env in the same ~/docker-compose-demo directory and copy the code below in the .env file. The file will have all the environment variables with their respective values.

 # Defining the values of environmental variables
 MYSQL_ROOT_PASSWORD="root_password"
 MYSQL_ALLOW_EMPTY_PASSWORD="password"
 MYSQL_RANDOM_ROOT_PASSWORD="password"

2. Next, edit the permissions of the .env file you created in the previous step using the setfacl command. The command below ensures that the root user has read/write permissions to the .env file.

• m parameter allows you to set the permissions to the files/folders.
• u is the user (root) which will have permissions granted to the specified file ~/*docker-compose-demo/.env

setfacl -m "u:root:rw" ~/docker-compose-demo/.env

3. Next, edit open the docker-compose.yml file with your favorite editor and comment out the environment section in the docker-compose.yml from the previous section. Add the code below to set default values for the environment variables in the Compose file.

Docker sets the values via the command line or by reading files, such as the .env file in the example. In either case, Docker replaces the values accordingly.

# Defining the environmental variable using Hardcoded values in variables
    environment:
	    ###  Static way ###
      # MYSQL_ROOT_PASSWORD: "root_password" 
			# MYSQL_ALLOW_EMPTY_PASSWORD: "password"
      # MYSQL_RANDOM_ROOT_PASSWORD: "password"

			### Substitution ### 
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD} 
			MYSQL_ALLOW_EMPTY_PASSWORD: ${MYSQL_ALLOW_EMPTY_PASSWORD} 
			MYSQL_RANDOM_ROOT_PASSWORD:  ${MYSQL_RANDOM_ROOT_PASSWORD} 

The code defines the variables by applying the string interpolation method. Variables with string interpolation are declared as ${variable}. Docker sets the values of the environment variables at run time.

4. Again, execute the docker-compose up command. When you run the docker-compose up command, the docker-compose.yml file looks for the values of the environment variables in the .env file. The docker-compose command automatically looks for a .env file in the project directory or in the parent folder of your compose file.

As soon as docker-compose finds the value for the environment variables set in docker-compose.yml in the .env file, Compose substitutes the values accordingly and starts the service. Staring the service creates the container defined in the docker-compose.yml file.

Using Multiple Environment Variables Files for Multiple Environments

Up to now, you have learned two approaches to declare the environment variables. The first one is to declare environment variables within the docker-compose.yml file. The second approach is declaring the environment variables within a single .env file and applying substitution.

Both approaches are suitable when working with a single environment. If you have multiple environments, like Production and/or Testing, you need a different approach. Let’s check out how to create multiple .env files with different names to match your environments!

The example below will emulate the usual environments one might encounter in IT Ops: Dev, QA, and Prod. Back on your terminal:

1. Navigate to your ~/docker-compose-demo directory and create a .env.dev file. Copy/paste the content of the snippet code below in the file and save it. This file will hold the settings use for your MySQL development environment database.

# Enviornmental Variables file .env.dev
MYSQL_ROOT_PASSWORD="password_DEV"
MYSQL_ALLOW_EMPTY_PASSWORD="password1"
MYSQL_RANDOM_ROOT_PASSWORD="password1"

2. On the same directory, create a .env.qa file and save the contents of the snippet code below in the file. These are the settings for your MySQL quality assurance environment database.

# Enviornmental Variables  file .env.qa
MYSQL_ROOT_PASSWORD="password_QA"
MYSQL_ALLOW_EMPTY_PASSWORD="password2"
MYSQL_RANDOM_ROOT_PASSWORD="password2"

3. Now create a .env.prod file to store the settings for the hypothetical MySQL production environment database. The contents are below:

# Enviornmental Variables file .env.prod
MYSQL_ROOT_PASSWORD="password_PROD"
MYSQL_ALLOW_EMPTY_PASSWORD="password3"
MYSQL_RANDOM_ROOT_PASSWORD="password3"

4. Next, execute the docker-compose command with the --env-file option, specifying the .env.dev file. The docker-compose command looks for the .env.dev file in the current ~/docker-compose-demo directory.

docker-compose --env-file .env.dev up

Passing the file as an argument allows you to store the file anywhere and with an appropriate name.

Related:Use Docker Stop Containers Without Screwing Things Up!

As you can see below, you can easily pick the different environment files and deploy them by replacing the .env.dev file to either the .env.qa or the .env.prod file.

# Running the docker-compose command with the QA file
docker-compose --env-file .env.qa up
# Running the docker-compose command with the Prod file
docker-compose --env-file .env.prod up

5. Now, verify if the configuration file (.env.dev) has been successfully read by running the docker exec command. Running the docker exec command will allow you to log in to the container. Running env will print the list of the current environment variables.

Notice that all three environment variables (MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD, and MYSQL_RANDOM_ROOT_PASSWORD) are present in the container. Notice that their values come from the env.dev file.

Incorporating the env_file in the Docker Compose File

In the previous section, you saw how to declare the environment variables in .env files. When you store environment variables values in the .env file separately, you end up with lots of lines and references in the docker-compose.yml.

To reduce the references and the number of lines for environment variables in the docker-compose.yml file, consider incorporating the env_file in your docker-compose.yml file.

Let’s learn how to incorporate the env_file in the Docker compose file. Once again, in the terminal:

1. Create a file named var.env to hold the settings for your MySQL database. Copy/paste the following into the var.env file and save it to the same ~/docker-compose-demo directory.

MYSQL_ROOT_PASSWORD="password_NEW"
MYSQL_ALLOW_EMPTY_PASSWORD="password_NEW"
MYSQL_RANDOM_ROOT_PASSWORD="password_NEW"

2. Next, open the previously created docker-compose.yml file using your favorite editor. Replace the environment section from the docker-compose.yml that you created previously with env_file, and add the file’s path with - ./var.env.

Docker now looks for the ./var.env file in the same ~/docker-compose-demo directory.

version: "2.2"

services:
  mysql_svc:
    image: mysql:5.7
    hostname: mysql
    container_name: mysql
# Replacing the environment: with env_file: 
  # environment:
  #   MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
  #   MYSQL_ALLOW_EMPTY_PASSWORD: ${MYSQL_ALLOW_EMPTY_PASSWORD}
  #   MYSQL_RANDOM_ROOT_PASSWORD: ${MYSQL_RANDOM_ROOT_PASSWORD}
    env_file:
     - ./var.env

Notice that all of the three settings and references are now gone. You are left with a single reference. It may not seem like much in the example. But in the real world, the number of references can get out of hand real quick.

3. Next, execute the docker-compose command. The command looks for the values of the environment variable present in the var.env file and creates the container you defined in the docker-compose.yml file.

Docker creates the services, meaning that the Docker found the values for the environment variables in the var.env file. To confirm, run the docker exec and the env commands one last time. You will see all the environment variables (MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD, and MYSQL_RANDOM_ROOT_PASSWORD) and their values in the container.

Conclusion

In this tutorial, you learned different ways of declaring environment variables with Docker compose. The article showed you the great flexibility Docker allows in declaring the environment variables by either coding them directly or using them with separate files.

So which approach are you going to use next while running your Docker containers from Docker Compose?