Learn the Ins-and-Outs of Gitlab CI for DevOps

Published:29 September 2022 - 10 min. read

Block over 3 billion compromised passwords & strengthen your Active Directory password policy. Try Specops Password Policy for free!

Have you been searching for a platform to host your code, and build your DevOps workflow and processes all in one place? Why not use GitLab for CI/CD pipelines? GitLab lets you host your application code and use the GitLab CI CD feature to continuously test, build and deploy your application on every code change.

In this tutorial, you’ll learn how to configure a CI/CD pipeline on GitLab to test, build and deploy a Python flask application to an Ubuntu server on Digital Ocean.

Sounds interesting? Stay tuned and level up your workflow with GitLab CI!

Prerequisites

This tutorial will be a hands-on demonstration. If you’d like to follow along, be sure you have the following:

Initializing a GitLab Project

Before configuring a CI/CD pipeline on GitLab, you must first have a repository on GitLab that houses your application. This tutorial uses a Python flask application to illustrate GitLab CI CD in action.

1. Download all files in the ATA Scripts GitHub directory for gitlab-ci.

Download script files used in this tutorial.
Download script files used in this tutorial.

2. Once downloaded, open the directory containing your files from your root directory and open it up with your preferred code editor.

3. Next, run the command below to spin up (flask run) the flask application.

flask run

The output below indicates that the flask application is entirely functional and error-free.

Running the Python flask application
Running the Python flask application

4. Lastly, push the application up to your GitLab repository.

Configuring Jobs on GitLab

After initializing your GitLab project, you’ll need a way to execute tasks on your CI/CD pipeline. How? By configuring Jobs where you define tasks in a YAML file.

You’ll write the CI/CD pipeline as code in a YAML file, which GitLab will use to perform the CI/CD pipeline for your project.

There are two options involved when creating the YAML file:

  • Create, and push the file in your project directory locally from your machine alongside the pipeline configurations to your remote GitLab repository.
  • Create the file directly on your GitLab repository via the GitLab UI.

But in this tutorial, you’ll create the YAML file directly on your GitLab repository via the GitLab UI:

1. Click on the (+) icon shown below in your project’s root directory of your GitLab repository to initialize creating the (.gitlab-ci.yaml) file. Your browser redirects to a page where you can configure the new YAML file (step two).

Initializing creating (.gitlab-ci.yml) file on GitLab repository
Initializing creating (.gitlab-ci.yml) file on GitLab repository

2. Next, type in .gitlab-ci.yml in the input field with the filename placeholder, as shown below.

As you input the new YAML file’s name, you’ll notice the selected box automatically populates. This behavior indicates that GitLab has detected you’re about to configure a CI/CD pipeline for your project.

In GitLab CI CD, every stage or configuration is referred to as jobs, and each job name with more than one word is written in the snake case.

Creating the .gitlab-ci.yml file
Creating the .gitlab-ci.yml file

3. Add the code snippets below to the .gitlab-ci.yml file, which runs the test for the Python flask application.

run_tests:
    # Use GitLab-managed runner with a python:3.9-slim-buster image to run the job
    image: python:3.9-slim-buster
    before_script: 
        # Update the system package index
        - apt-get update
        # Install Python3 and pip
        - apt-get install -y python3-dev python3-pip
        # Install Flask and pytest
        - pip3 install Flask pytest
    script:
        # Run the pytest command to run the test for the Python flask application. 
        - pytest

GitLab-managed runners use Docker containers to execute jobs configured for your CI/CD pipeline. So all the jobs for your CI/CD pipeline execute inside Docker containers depending on which image you’d like to use. By default, GitLab uses a Ruby image.

4. Now, scroll down to the page’s bottom, and click on the Commit changes button to commit the changes. Doing so makes GitLab detect the pipeline configuration code and run the job.

Committing configuration in the (.gitlab-ci.yml) file to the GitLab repository
Committing configuration in the (.gitlab-ci.yml) file to the GitLab repository

5. Lastly, navigate to the CI/CD section (left panel), and click on the Pipelines tab to view your pipeline.

You’ll see that the pipeline is currently in running status, as shown below.

Viewing the pipeline in running state
Viewing the pipeline in running state

Once executed successfully, your pipeline’s status changes to passed, which indicates your job ran successfully.

Viewing a successful pipeline run
Viewing a successful pipeline run

Securing Credentials by Creating Secret Variables

You’ve just successfully executed your first job. And now, you’re almost ready to build and push the Docker image for the Python flask project to both Docker Hub and GitLab container registries.

But first, you’ll need a way to store your login credentials for Docker Hub and GitLab securely. What’s that secure way? You’ll create secret variables to hold these credentials and keep them away from prying eyes.

1. Navigate to Settings (left panel), and click on the CI/CD tab, as shown below, to create a secret variable using GitLab project variables.

Initializing creating secret variables
Initializing creating secret variables

2. Next, scroll down on the new page, click on the Variables sub-section, and click Expand (top-right of the section) to expand the Variables section.

Expanding the variables sub-section
Expanding the variables sub-section

3. Once expanded, click Add variable (bottom-left) to add a secret variable.

Initializing adding variables
Initializing adding variables

4. Now, configure the new secret variable with the following:

Note that the names in the Key fields are arbitrary.

  • Key – DOCKER_USERNAME.
  • Value – Input your Docker Hub username.
Adding the DOCKER_USERNAME variable
Adding the DOCKER_USERNAME variable

5. Finally, repeat step four, and add more variables with the following details:

  • Key – DOCKER_PASSWORD, Value – Provide your Docker Hub password.
  • Key – GITLAB_ACCESS_TOKEN, Value – Input your GitLab access token.

You should now have four secret variables, as shown below.

Confirming the number of secret variables
Confirming the number of secret variables

Building and Pushing a Docker Image to Docker Hub

With your secrets kept a secret, it’s time to configure your second job. This job will build and deploy a Docker image for the Python flask application to Docker Hub.

1. Navigate to the CI/CD Pipeline Editor as you’ll need to add configurations to the pipeline.

2. Next, add the following code at the top of the run-test block. Be sure to replace dockerhub-user with your Docker Hub username.

variables:
    IMAGE_NAME: dockerhub-user/flask-app
    IMAGE_TAG: flask-app-1.0

3. Add the following code below the run_test block, which builds a Docker image for the Python flask application. Be sure to replace gitlab-user with your GitLab username in the script block.

build_image:
    # Downloads a Docker client (docker:20.10.16).
    image: docker:20.10.16
    # Connects to a Docker daemon.
    services:
        - docker:20.10.16-dind
    # Create a certificate directory (DOCKER_TLS_CERTDIR) to enable the 
    # Docker client and Docker daemon to share the same certificate directory. 
    # Enable Docker client and Docker daemon to authenticate each other.
    variables:
        DOCKER_TLS_CERTDIR: '/certs'
    # Authenticate GitLab with login parameters described as secret variables.
    before_script:
        - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD 
        - docker login registry.gitlab.com -u $GITLAB_USERNAME -p $GITLAB_ACCESS_TOKEN
    # Build and push the flask image to DockerHub and GitLab container registry.
    script:
        - docker build -t $IMAGE_NAME:$IMAGE_TAG .
        - docker push $IMAGE_NAME:$IMAGE_TAG
        - docker build -t registry.gitlab.com/gitlab-user/flask-app .
        - docker push registry.gitlab.com/gitlab-user/flask-app

4. Now, click on Commit changes again to save the changes, and trigger the updated pipeline.

Committing changes to the pipeline
Committing changes to the pipeline

You should now have the following output, which indicates your build is successful.

Viewing pipelines
Viewing pipelines

5. Navigate to the Packages & Registries (left panel) → Container Registry to confirm your Python flask image has been pushed to the GitLab container registry, as shown below.

Confirming pushed image on GitLab container registry
Confirming pushed image on GitLab container registry

6. Finally, switch to your Docker account and verify your Python flask application has been pushed to your Docker Hub registry.

Confirming pushed image on Docker Hub
Confirming pushed image on Docker Hub

Configuring Stages on the GitLab CI CD pipeline

Currently, each job in your pipeline is executed in isolation and running regardless of if any job is not successful, which is not good practice.

The best way to configure a pipeline is to make jobs execute accordingly, one after the other. With this behavior, if one job fails, other subsequent jobs will not execute.

1. Navigate to the Pipeline Editor and add the following code below the variables block in your pipeline.

The code below adds stages to the pipeline, referenced from jobs in the pipeline.

stages:
    - test
    - build

2. Reference the stages in both the run_test and build_image blocks, as shown below.

Viewing and referencing the build stage
Viewing and referencing the build stage

3. Lastly, commit your changes to the pipeline, and navigate to Pipelines under the CI/CD section (left panel).

If all goes well, you’ll see each job executed one after the other in the Stages column, as shown below.

Viewing jobs executed one after another
Viewing jobs executed one after another

Once the run_tests job is done executing successfully, the build job begins immediately.

Viewing build job
Viewing build job

When the build job is executed successfully, you’ll see the following output showing two green checks under the Stages column.

Viewing jobs
Viewing jobs

Setting up NGINX as Reverse Proxy for the Ubuntu Server

Now that you have configured stages on your GitLab pipeline, you are ready to configure NGINX as a reverse proxy. Doing so lets you view your application over a web browser.

1. Run the command below to install NGINX on your server (your droplet on Digital Ocean).

sudo apt install nginx
Installing NGINX
Installing NGINX

2. Next, run the following rm command, which doesn’t provide output but deletes the NGINX default configuration file (/etc/nginx/sites-enabled/default). Doing so lets you write and use your configuration file.

sudo rm /etc/nginx/sites-enabled/default

3. Create your NGINX configuration file (/etc/nginx/sites-enabled/flask-app), add the code below, and replace DROPLET_IP with your server’s IP address. Once added, save the changes, and close the editor.

The code below passes http://localhost:5000 to your remote server IP address.

server{
    listen 80;
    server_name DROPLET_IP;

    location / {
            proxy_pass http://localhost:5000;
            include /etc/nginx/proxy_params;
        }
}

4. Now, run the ufw allow command below to allow http/tcp traffic on your firewall so that your browser can access your application.

sudo ufw allow http/tcp
Allowing http/tcp traffic with firewall rules
Allowing http/tcp traffic with firewall rules

5. Lastly, run the below systemctl restart command, which doesn’t show output but restarts NGINX to implement the settings you created.

sudo systemctl restart nginx

Deploying Docker Image to an Ubuntu Server

After pushing the Docker image for your Python flask application to Docker Hub, you’re ready to configure a new pipeline. This pipeline deploys your application to an Ubuntu server on Digital Ocean.

1. Head back to GitLab and add a secret variable with the following:

  • Key – Set any key as you like, but this tutorial’s choice is DIGITAL_OCEAN_SSH_KEY.
  • Value – The SSH key you used to connect to your Ubuntu Server on Digital Ocean.
Creating the DIGITAL_OCEAN_SSH_KEY secret variable
Creating the DIGITAL_OCEAN_SSH_KEY secret variable

2. Next, add a new stage called deploy in your pipeline, as shown below.

Adding deploy stage
Adding deploy stage

3. Create a new job as you did in the “Configuring Jobs on GitLab” section, and name the job as deploy.

4. Add the following pipeline configurations in the Pipeline Editor, and commit your changes to create a third job called deploy.

deploy:
    stage: deploy
    before_script:
        # Restrict the DIGITAL_OCEAN_SSH_KEY file 
        # to only be read and not to be modified by anyone else
        - chmod 400 $DIGITAL_OCEAN_SSH_KEY
    script:
        # Connect to the Ubuntu server on digital ocean using SSH
        - ssh -o StrictHostKeyChecking=no -i $DIGITAL_OCEAN_SSH_KEY [email protected]
            # Login into your Docker account
            docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD && 
            # Pull the docker image from your docker registry
            docker pull $IMAGE_NAME:$IMAGE_TAG &&
            # Stop any existing containers
            docker ps -aq | xargs docker stop | xargs docker rm || true &&
            # Run the Python flask container in detached mode
            docker run -d -p 5000:5000 $IMAGE_NAME:$IMAGE_TAG"

After committing the changes, you’ll see the following output, which indicates all your jobs are executed successfully.

Confirming the deploy job ran successfully
Confirming the deploy job ran successfully

5. Run the following docker ps command to see the containers running in your server.

docker ps
Verifying the application in the Ubuntu server
Verifying the application in the Ubuntu server

6. Ultimately, open another tab on your browser and navigate to your server’s IP address, followed by port 5000 (http://SERVER-IP:5000).

If all goes well, you’ll see the message below on your browser.

Verifying the application is accessible via a web browser
Verifying the application is accessible via a web browser

Conclusion

In this tutorial, you’ve learned how to set up a GitLab CI CD pipeline with GitLab for the uninterrupted development of your application. You’ve created jobs to automate tasks, stages to define when to run jobs, and secret variables to keep sensitive information, well, as secrets.

You’ve seen that pushing a Docker image to GitLab’s container registry works like a charm. But what else will you configure for your pipeline? Perhaps deploy your Python flask application to AWS Elastic Beanstalk?

Hate ads? Want to support the writer? Get many of our tutorials packaged as an ATA Guidebook.

Explore ATA Guidebooks

Looks like you're offline!