Have you been searching for a platform to host your code, and build your DevOps workflow and processes all in one place? Why not use GitLab for CI/CD pipelines? GitLab lets you host your application code and use the GitLab CI CD feature to continuously test, build and deploy your application on every code change.
In this tutorial, you’ll learn how to configure a CI/CD pipeline on GitLab to test, build and deploy a Python flask application to an Ubuntu server on Digital Ocean.
Sounds interesting? Stay tuned and level up your workflow with GitLab CI!
Prerequisites
This tutorial will be a hands-on demonstration. If you’d like to follow along, be sure you have the following:
- A GitLab account, a repository already set up, and a personal access token.
- A Docker Hub account and a repository.
- An Ubuntu server droplet on DigitalOcean with Docker installed and running – You can sign up for the free $100 credit.
- UFW firewall enabled on the Ubuntu server droplet.
Initializing a GitLab Project
Before configuring a CI/CD pipeline on GitLab, you must first have a repository on GitLab that houses your application. This tutorial uses a Python flask application to illustrate GitLab CI CD in action.
1. Download all files in the ATA Scripts GitHub directory for gitlab-ci.
2. Once downloaded, open the directory containing your files from your root directory and open it up with your preferred code editor.
3. Next, run the command below to spin up (flask run) the flask application.
flask runThe output below indicates that the flask application is entirely functional and error-free.
4. Lastly, push the application up to your GitLab repository.
Configuring Jobs on GitLab
After initializing your GitLab project, you’ll need a way to execute tasks on your CI/CD pipeline. How? By configuring Jobs where you define tasks in a YAML file.
You’ll write the CI/CD pipeline as code in a YAML file, which GitLab will use to perform the CI/CD pipeline for your project.
There are two options involved when creating the YAML file:
- Create, and push the file in your project directory locally from your machine alongside the pipeline configurations to your remote GitLab repository.
- Create the file directly on your GitLab repository via the GitLab UI.
But in this tutorial, you’ll create the YAML file directly on your GitLab repository via the GitLab UI:
1. Click on the (+) icon shown below in your project’s root directory of your GitLab repository to initialize creating the (.gitlab-ci.yaml) file. Your browser redirects to a page where you can configure the new YAML file (step two).
2. Next, type in .gitlab-ci.yml in the input field with the filename placeholder, as shown below.
As you input the new YAML file’s name, you’ll notice the selected box automatically populates. This behavior indicates that GitLab has detected you’re about to configure a CI/CD pipeline for your project.
In GitLab CI CD, every stage or configuration is referred to as jobs, and each job name with more than one word is written in the snake case.
3. Add the code snippets below to the .gitlab-ci.yml file, which runs the test for the Python flask application.
run_tests:
# Use GitLab-managed runner with a python:3.9-slim-buster image to run the job
image: python:3.9-slim-buster
before_script:
# Update the system package index
- apt-get update
# Install Python3 and pip
- apt-get install -y python3-dev python3-pip
# Install Flask and pytest
- pip3 install Flask pytest
script:
# Run the pytest command to run the test for the Python flask application.
- pytestGitLab-managed runners use Docker containers to execute jobs configured for your CI/CD pipeline. So all the jobs for your CI/CD pipeline execute inside Docker containers depending on which image you’d like to use. By default, GitLab uses a Ruby image.
4. Now, scroll down to the page’s bottom, and click on the Commit changes button to commit the changes. Doing so makes GitLab detect the pipeline configuration code and run the job.
5. Lastly, navigate to the CI/CD section (left panel), and click on the Pipelines tab to view your pipeline.
You’ll see that the pipeline is currently in running status, as shown below.
Once executed successfully, your pipeline’s status changes to passed, which indicates your job ran successfully.
Securing Credentials by Creating Secret Variables
You’ve just successfully executed your first job. And now, you’re almost ready to build and push the Docker image for the Python flask project to both Docker Hub and GitLab container registries.
But first, you’ll need a way to store your login credentials for Docker Hub and GitLab securely. What’s that secure way? You’ll create secret variables to hold these credentials and keep them away from prying eyes.
1. Navigate to Settings (left panel), and click on the CI/CD tab, as shown below, to create a secret variable using GitLab project variables.
2. Next, scroll down on the new page, click on the Variables sub-section, and click Expand (top-right of the section) to expand the Variables section.
3. Once expanded, click Add variable (bottom-left) to add a secret variable.
4. Now, configure the new secret variable with the following:
Note that the names in the Key fields are arbitrary.
- Key – DOCKER_USERNAME.
- Value – Input your Docker Hub username.
5. Finally, repeat step four, and add more variables with the following details:
- Key – DOCKER_PASSWORD, Value – Provide your Docker Hub password.
- Key – GITLAB_ACCESS_TOKEN, Value – Input your GitLab access token.
You should now have four secret variables, as shown below.
Building and Pushing a Docker Image to Docker Hub
With your secrets kept a secret, it’s time to configure your second job. This job will build and deploy a Docker image for the Python flask application to Docker Hub.
1. Navigate to the CI/CD Pipeline Editor as you’ll need to add configurations to the pipeline.
2. Next, add the following code at the top of the run-test block. Be sure to replace dockerhub-user with your Docker Hub username.
variables:
IMAGE_NAME: dockerhub-user/flask-app
IMAGE_TAG: flask-app-1.03. Add the following code below the run_test block, which builds a Docker image for the Python flask application. Be sure to replace gitlab-user with your GitLab username in the script block.
build_image:
# Downloads a Docker client (docker:20.10.16).
image: docker:20.10.16
# Connects to a Docker daemon.
services:
- docker:20.10.16-dind
# Create a certificate directory (DOCKER_TLS_CERTDIR) to enable the
# Docker client and Docker daemon to share the same certificate directory.
# Enable Docker client and Docker daemon to authenticate each other.
variables:
DOCKER_TLS_CERTDIR: '/certs'
# Authenticate GitLab with login parameters described as secret variables.
before_script:
- docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
- docker login registry.gitlab.com -u $GITLAB_USERNAME -p $GITLAB_ACCESS_TOKEN
# Build and push the flask image to DockerHub and GitLab container registry.
script:
- docker build -t $IMAGE_NAME:$IMAGE_TAG .
- docker push $IMAGE_NAME:$IMAGE_TAG
- docker build -t registry.gitlab.com/gitlab-user/flask-app .
- docker push registry.gitlab.com/gitlab-user/flask-app4. Now, click on Commit changes again to save the changes, and trigger the updated pipeline.
You should now have the following output, which indicates your build is successful.
5. Navigate to the Packages & Registries (left panel) → Container Registry to confirm your Python flask image has been pushed to the GitLab container registry, as shown below.
6. Finally, switch to your Docker account and verify your Python flask application has been pushed to your Docker Hub registry.
Configuring Stages on the GitLab CI CD pipeline
Currently, each job in your pipeline is executed in isolation and running regardless of if any job is not successful, which is not good practice.
The best way to configure a pipeline is to make jobs execute accordingly, one after the other. With this behavior, if one job fails, other subsequent jobs will not execute.
1. Navigate to the Pipeline Editor and add the following code below the variables block in your pipeline.
The code below adds stages to the pipeline, referenced from jobs in the pipeline.
stages:
- test
- build2. Reference the stages in both the run_test and build_image blocks, as shown below.
3. Lastly, commit your changes to the pipeline, and navigate to Pipelines under the CI/CD section (left panel).
If all goes well, you’ll see each job executed one after the other in the Stages column, as shown below.
Once the run_tests job is done executing successfully, the build job begins immediately.
When the build job is executed successfully, you’ll see the following output showing two green checks under the Stages column.
Setting up NGINX as Reverse Proxy for the Ubuntu Server
Now that you have configured stages on your GitLab pipeline, you are ready to configure NGINX as a reverse proxy. Doing so lets you view your application over a web browser.
1. Run the command below to install NGINX on your server (your droplet on Digital Ocean).
sudo apt install nginx
2. Next, run the following rm command, which doesn’t provide output but deletes the NGINX default configuration file (/etc/nginx/sites-enabled/default). Doing so lets you write and use your configuration file.
sudo rm /etc/nginx/sites-enabled/default3. Create your NGINX configuration file (/etc/nginx/sites-enabled/flask-app), add the code below, and replace DROPLET_IP with your server’s IP address. Once added, save the changes, and close the editor.
The code below passes http://localhost:5000 to your remote server IP address.
server{
listen 80;
server_name DROPLET_IP;
location / {
proxy_pass http://localhost:5000;
include /etc/nginx/proxy_params;
}
}4. Now, run the ufw allow command below to allow http/tcp traffic on your firewall so that your browser can access your application.
sudo ufw allow http/tcp
5. Lastly, run the below systemctl restart command, which doesn’t show output but restarts NGINX to implement the settings you created.
sudo systemctl restart nginxDeploying Docker Image to an Ubuntu Server
After pushing the Docker image for your Python flask application to Docker Hub, you’re ready to configure a new pipeline. This pipeline deploys your application to an Ubuntu server on Digital Ocean.
1. Head back to GitLab and add a secret variable with the following:
- Key – Set any key as you like, but this tutorial’s choice is DIGITAL_OCEAN_SSH_KEY.
- Value – The SSH key you used to connect to your Ubuntu Server on Digital Ocean.
2. Next, add a new stage called deploy in your pipeline, as shown below.
3. Create a new job as you did in the “Configuring Jobs on GitLab” section, and name the job as deploy.
4. Add the following pipeline configurations in the Pipeline Editor, and commit your changes to create a third job called deploy.
deploy:
stage: deploy
before_script:
# Restrict the DIGITAL_OCEAN_SSH_KEY file
# to only be read and not to be modified by anyone else
- chmod 400 $DIGITAL_OCEAN_SSH_KEY
script:
# Connect to the Ubuntu server on digital ocean using SSH
- ssh -o StrictHostKeyChecking=no -i $DIGITAL_OCEAN_SSH_KEY [email protected]
# Login into your Docker account
docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD &&
# Pull the docker image from your docker registry
docker pull $IMAGE_NAME:$IMAGE_TAG &&
# Stop any existing containers
docker ps -aq | xargs docker stop | xargs docker rm || true &&
# Run the Python flask container in detached mode
docker run -d -p 5000:5000 $IMAGE_NAME:$IMAGE_TAG"After committing the changes, you’ll see the following output, which indicates all your jobs are executed successfully.
5. Run the following docker ps command to see the containers running in your server.
docker ps
6. Ultimately, open another tab on your browser and navigate to your server’s IP address, followed by port 5000 (http://SERVER-IP:5000).
If all goes well, you’ll see the message below on your browser.
Conclusion
In this tutorial, you’ve learned how to set up a GitLab CI CD pipeline with GitLab for the uninterrupted development of your application. You’ve created jobs to automate tasks, stages to define when to run jobs, and secret variables to keep sensitive information, well, as secrets.
You’ve seen that pushing a Docker image to GitLab’s container registry works like a charm. But what else will you configure for your pipeline? Perhaps deploy your Python flask application to AWS Elastic Beanstalk?
