[Yandex Cloud documentation](../../index.md) > [Yandex API Gateway](../index.md) > [Tutorials](index.md) > Serverless-based workflows and automation > Building a CI/CD pipeline in GitLab using serverless products

# Building a CI/CD pipeline with serverless products

# Building a CI/CD pipeline in GitLab with serverless products


You can build a continuous integration/continuous delivery (CI/CD) pipeline using serverless products.

As an example, we will use a [Django](https://www.djangoproject.com/) web app simulating an online store shopping cart. The app uses a database to store product descriptions and the user session to store the shopping cart status. We will deploy the Django app in a [Serverless Containers container](../../serverless-containers/concepts/container.md) with secrets securely delivered to it through [Yandex Lockbox](../../lockbox/index.md). [Yandex API Gateway](../index.md) accepts user requests and redirects them to the app container.

The project uses two environments:
* `prod`: Production environment available to users.
* `testing`: Test environment for testing the app before the `prod` release.

For each environment, there is an individual Yandex Cloud [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) and separate static resources, e.g., a database and [service accounts](../../iam/concepts/users/service-accounts.md). This isolates the environments from each other at the [Yandex Identity and Access Management](../../iam/index.md) settings level.

There is also a shared folder named `infra` with a [registry](../../container-registry/concepts/registry.md) in [Yandex Container Registry](../../container-registry/index.md) containing all app's [Docker images](../../container-registry/concepts/docker-image.md). The system uses the `builder` service account to publish the images. The `prod` and `testing` environment service accounts have restricted permissions for the `infra` folder limited to [pulling Docker images](../../container-registry/operations/docker-image/docker-image-pull.md).

To build a CI/CD pipeline with serverless products:
1. [Create a GitLab instance](#create-gitlab).
1. [Configure GitLab](#configure-gitlab).
1. [Create a GitLab Runner](#runners).
1. [Upload files to the GitLab repository](#add-files).
1. [Create GitLab environment variables](#add-variables).
1. [Create a CI script configuration file](#add-ci).
1. [Check the result](#check-result).

If you no longer need the resources you created, [delete them](#clear-out).

## Required paid resources {#paid-resources}

The infrastructure support cost includes:

* Fee for [disks](../../compute/concepts/disk.md) and continuously running VMs (see [Yandex Compute Cloud pricing](../../compute/pricing.md)).
* Fee for using a [Yandex Managed Service for Kubernetes master](../../managed-kubernetes/concepts/index.md#master) (see [Managed Service for Kubernetes pricing](../../managed-kubernetes/pricing.md)).
* Fee for storing the created Docker images (see [Container Registry pricing](../../container-registry/pricing.md)).
* Fee for storing secrets (see [Yandex Lockbox pricing](../../lockbox/pricing.md)).
* Fee for the number of container invocations, computing resources allocated to the application, and outgoing traffic (see [Serverless Containers pricing](../../serverless-containers/pricing.md)).
* Fee for API gateway requests (see [API Gateway pricing](../pricing.md)).
* Fee for using [public IP addresses](../../vpc/concepts/address.md#public-addresses) (see [Yandex Virtual Private Cloud pricing](../../vpc/pricing.md#prices-public-ip)).

## Getting started {#before-begin}

### Download your project {#download-project}

Clone the [yc-serverless-gitlab-ci-cd repository](https://github.com/yandex-cloud-examples/yc-serverless-gitlab-ci-cd) using Git:

```bash
git clone https://github.com/yandex-cloud-examples/yc-serverless-gitlab-ci-cd.git
```

### Install additional dependencies {#prepare}

Install the following tools in the local environment:
* [Yandex Cloud CLI](../../cli/operations/install-cli.md).
* [`jq`](https://jqlang.org/download/), a JSON stream processor.
* [`yq`](https://github.com/mikefarah/yq#install), a YAML stream processor.
* [Python 3.8 or higher](https://www.python.org/downloads/).
* Python libraries listed in the `application/requirements.txt` project file:

  ```bash
  python -m pip install -r ./application/requirements.txt
  ```

### Set up your infrastructure {#deploy-infrastructure}

1. Navigate to the project directory; run the `bootstrap.sh` script specifying your [cloud](../../resource-manager/concepts/resources-hierarchy.md#cloud) ID:

   ```bash
   YC_CLOUD_ID=<cloud_ID> ./bootstrap.sh
   ```

   The script will deploy the basic infrastructure and create YAML resource description files in the `config` directory. You can edit the script to create additional resource folders. For example, this way you can add another test environment.

   {% note warning %}

   Once completed, the script will display the GitLab environment variables and their values. Save them, as you will need them later.

   {% endnote %}

1. [Create these service accounts](../../iam/operations/sa/create.md):
   * Service account for resources with the [editor](../../iam/roles-reference.md#editor) [role](../../iam/concepts/access-control/roles.md) for the [Yandex Managed Service for Kubernetes cluster](../../managed-kubernetes/concepts/index.md#kubernetes-cluster) folder. This service account will be used to create Managed Service for Kubernetes cluster resources.
   * Service account for [nodes](../../managed-kubernetes/concepts/index.md#node-group) with the [container-registry.images.puller](../../container-registry/security/index.md#choosing-roles) role for the Docker image registry folder. Nodes will use this account to pull the required Docker images from the registry.

   You can use the same service account for both operations.
1. [Create a Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-create.md#kubernetes-cluster-create) and a [node group](../../managed-kubernetes/operations/node-group/node-group-create.md). When creating the cluster, specify the service accounts you created for the resources and nodes.
1. [Configure security groups](../../managed-kubernetes/operations/connect/security-groups.md) for the Managed Service for Kubernetes cluster.
1. [Configure a security group](../../managed-gitlab/operations/configure-security-group.md) for the [Yandex Managed Service for GitLab instance](../../managed-gitlab/concepts/index.md#instance).

## Create a GitLab instance {#create-gitlab}

Create either a Managed Service for GitLab instance or a [VM](../../compute/concepts/vm.md) with a GitLab [image](../../compute/concepts/image.md) in the same [cloud network](../../vpc/concepts/network.md#network) as the Managed Service for Kubernetes cluster.

{% list tabs group=gl_installation %}

- Managed Service for GitLab instance {#instance-mgl}

  Create a Managed Service for GitLab instance by following [this guide](../../managed-gitlab/quickstart.md#instance-create).

- VM with a GitLab image {#gl-image-vm}

  Launch GitLab on a VM with a [public IP address](../../vpc/concepts/address.md#public-addresses).

  1. On the [folder dashboard](../../resource-manager/concepts/resources-hierarchy.md#folder) in the [management console](https://console.yandex.cloud), click **Create resource** and select `Virtual machine instance`.
  1. Under **Boot disk image**, in the **Product search** field, enter `Gitlab` and select a public [GitLab](https://yandex.cloud/en/marketplace/products/yc/gitlab) image.
  1. Under **Location**, select the [availability zone](../../overview/concepts/geo-scope.md) where your VM will reside. If you are not sure which one to choose, leave the default.
  1. Under **Computing resources**, navigate to the `Custom` tab and specify the [platform](../../compute/concepts/vm-platforms.md), number of vCPUs, and amount of RAM:
  
      * **Platform**: `Intel Ice Lake`.
      * **vCPU**: `4`.
      * **Guaranteed vCPU performance**: `100%`.
      * **RAM**: `8 GB`.
  
  1. Under **Network settings**:
      
      * In the **Subnet** field, select the network and subnet to connect your VM to. If the relevant [network](../../vpc/concepts/network.md#network) or [subnet](../../vpc/concepts/network.md#subnet) is missing, [create it](../../vpc/operations/subnet-create.md).
      * In the **Public IP address** field, keep `Auto` to assign the VM a random external IP address from the Yandex Cloud pool or select a static address from the list if you reserved one in advance.
  
  1. Under **Access**, select **SSH key** and specify the VM access credentials:
  
      * In the **Login** field, enter the username. Do not use `root` or other OS-reserved usernames. To perform operations requiring root privileges, use the `sudo` command.
      * In the **SSH key** field, select the SSH key saved in your [organization user](../../organization/concepts/membership.md) profile.
        
        If there are no SSH keys in your profile or you want to add a new key:
        
        1. Click **Add key**.
        1. Enter a name for the SSH key.
        1. Select one of the following:
        
            * `Enter manually`: Paste the contents of the public SSH key. You need to [create](../../compute/operations/vm-connect/ssh.md#creating-ssh-keys) an SSH key pair on your own.
            * `Load from file`: Upload the public part of the SSH key. You need to create an SSH key pair on your own.
            * `Generate key`: Automatically create an SSH key pair.
            
              When adding a new SSH key, an archive containing the key pair will be created and downloaded. In Linux or macOS-based operating systems, unpack the archive to the `/home/<user_name>/.ssh` directory. In Windows, unpack the archive to the `C:\Users\<user_name>/.ssh` directory. You do not need additionally enter the public key in the management console.
        
        1. Click **Add**.
        
        The system will add the SSH key to your organization user profile. If the organization has [disabled](../../organization/operations/os-login-access.md) the ability for users to add SSH keys to their profiles, the added public SSH key will only be saved in the user profile inside the newly created resource.
  
  1. Under **General information**, specify the VM name: `ci-tutorial-gitlab`.
  1. Click **Create VM**.
  
  It may take a few minutes to create your VM. When the VM status changes to `RUNNING` and GitLab starts, configure its settings.

{% endlist %}

## Configure GitLab {#configure-gitlab}

To configure GitLab and get ready for continuous integration (CI), create a new project and enter the CI authentication data:

{% list tabs group=gl_installation %}

- Managed Service for GitLab instance {#instance-mgl}

  1. Log in to the [Managed Service for GitLab instance](../../managed-gitlab/concepts/index.md#instance) web UI.
  1. Click **Create a project**.
  1. Click **Create blank project**.
  1. Fill out the fields as follows:
     * **Project name**: `gitlab-test`.
     * **Project URL**: Select the admin user in the field next to the Managed Service for GitLab instance FQDN.

     Leave the other fields unchanged.
  1. Click **Create project**.

- VM with a GitLab image {#gl-image-vm}

  1. On the Yandex Compute Cloud page, select the created [VM](../../compute/concepts/vm.md) and copy its [public IP](../../vpc/concepts/address.md#public-addresses).
  1. [Connect](../../compute/operations/vm-connect/ssh.md) to the VM over SSH.
  1. Get the GitLab administrator password with the following VM command:

     ```bash
     sudo cat /etc/gitlab/initial_root_password
     ```

  1. Copy the password without spaces from the `Password` line to the clipboard or a separate file.
  1. In your browser, open `http://<VM_public_IP_address>`. This will take you to the GitLab web UI.
  1. Log in as the administrator:
     * **Username or email**: `root`.
     * **Password**: Password you copied in the previous step.

     If you are unable to log in, [reset the administrator password](https://docs.gitlab.com/security/reset_user_password/#reset-your-root-password).
  1. [Change the administrator password](https://docs.gitlab.com/user/profile/user_passwords/#change-your-password).
  1. Log in as the administrator with the new password.
  1. Select **Create a project**.
  1. Specify the project name: `gitlab-test`.
  1. Click **Create project**.

{% endlist %}

## Create a GitLab Runner {#runners}

To run build tasks in your [Yandex Managed Service for Kubernetes cluster](../../managed-kubernetes/concepts/index.md#kubernetes-cluster), create a [GitLab Runner](https://docs.gitlab.com/runner/install/kubernetes/).To do this, install GitLab Runner by following [this guide](../../managed-kubernetes/operations/applications/gitlab-runner.md).

Once it is installed, you can run automated builds within your [Managed Service for Kubernetes cluster](../../managed-kubernetes/concepts/index.md#kubernetes-cluster).

For more information about installing and running GitLab Runner, see [this GitLab article](https://docs.gitlab.com/runner/install/).

## Upload files to the GitLab repository {#add-files}

1. [Add an SSH key to securely access GitLab](https://docs.gitlab.com/user/ssh/).
1. [Clone](https://docs.gitlab.com/topics/git/clone/) the `gitlab-test` repository over SSH.
1. Copy all files from the `yc-serverless-gitlab-ci-cd` repository to `gitlab-test`.
1. Navigate to the `gitlab-test` directory.
1. Index the new files:

   ```bash
   git add .
   ```

1. Commit the files:

   ```bash
   git commit -m "Add project files"
   ```

1. Push changes to the `gitlab-test` repository:

   ```bash
   git push
   ```

## Create GitLab environment variables {#add-variables}

1. In GitLab, navigate to **Settings** in the left-hand panel and select **CI/CD** from the pop-up list.
1. Click **Expand** next to **Variables**.
1. Add environment variables with the protection option disabled. The `bootstrap.sh` script will display all the required variables and their values:
   * `cloud_id`: Your cloud ID.
   * `CI_REGISTRY`: Container Registry ID in the `infra` folder, prefixed with `cr.yandex/`.
   * `CI_REGISTRY_KEY`: `builder` service account [key](../../iam/concepts/users/service-accounts.md#sa-key).
   * `cart_prod`: Yandex Cloud production folder name.
   * `DOCAPI_ENDPOINT_prod`: Document API endpoint of the [Yandex Managed Service for YDB](../../ydb/index.md) database in the `prod` folder.
   * `PROD_SA_ID`: `deployer` service account ID in the `prod` folder.
   * `SA_PROD_DEPLOYER_PRIVATE_KEY`: `deployer` service account key in the `prod` folder.
   * `prod_container_id`: Serverless Containers container ID in the `prod` folder.
   * `cart_testing`: Yandex Cloud test folder name.
   * `DOCAPI_ENDPOINT_testing`: Document API endpoint of the Managed Service for YDB database in the `testing` folder.
   * `TESTING_SA_ID`: `deployer` service account ID in the `testing` folder.
   * `SA_TESTING_DEPLOYER_PRIVATE_KEY`: `deployer` service account key in the `testing` folder.

   To add a variable:
   1. Click **Add variable**.
   1. In the window that opens, specify the variable name in the **Key** field and its value in the **Value** field.
   1. Disable **Protect variable**.
   1. Click **Add variable**.

## Create a CI script configuration file {#add-ci}

1. Open the `gitlab-test` project.
1. Click ![image](../../_assets/console-icons/plus.svg) in the repository navigation bar and select **New file** from the drop-down menu.
1. Name the file as `.gitlab-ci.yml` and add the following build stages to it:

   {% cut ".gitlab-ci.yml" %}

   ```yaml
   stages:
     - build
     - deploy-test-env
     - test
     - delete-test-env
     - release

   # Building a container image.
   build-job:
     stage: build
   # Using `kaniko` to create a container inside another container for enhanced security.
     image:
       name: gcr.io/kaniko-project/executor:debug
       entrypoint: [""]
     script:
       - mkdir -p /kaniko/.docker
   # Pushing the container image to the registry. The image is tagged with the commit hash.
       - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json
       - >-
         /kaniko/executor
         --context "${CI_PROJECT_DIR}/application"
         --dockerfile "${CI_PROJECT_DIR}/application/Dockerfile"
         --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"

   # Deploying the test environment using the built-in container image.
   deploy-test-env-job:
     stage: deploy-test-env
     image: alpine:3.15
     script:
   # Installing tools.
       - apk add -q --no-cache bash curl jq gettext
       - apk add yq --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
       - curl --fail -silent --location --remote-name https://storage.yandexcloud.net/yandexcloud-yc/install.sh
       - bash install.sh -i /usr/local/yandex-cloud -n
       - ln -s /usr/local/yandex-cloud/bin/yc /usr/local/bin/yc
    # Authenticating with the service account key.
       - echo "$SA_TESTING_DEPLOYER_PRIVATE_KEY" > key.json
       - yc config profile create sa-profile
       - yc config set service-account-key key.json
   # Setting up the API Gateway variables and creating a container.
       - export sa_id=$TESTING_SA_ID
       - export container_id=$(yc serverless container create --name testing --cloud-id ${cloud_id} --folder-name ${cart_testing} | yq .id)
   # Deploying a revision.
       - yc serverless container revision deploy --container-name testing --image "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" --cores 1 --memory 512mb --concurrency 1 --execution-timeout 10s --cloud-id ${cloud_id} --folder-name ${cart_testing} --service-account-id ${TESTING_SA_ID} --environment DOCAPI_ENDPOINT=${DOCAPI_ENDPOINT_testing},DB_TABLE_PREFIX='' --secret environment-variable=AWS_ACCESS_KEY_ID,name=cart-app,key=AWS_ACCESS_KEY_ID --secret environment-variable=AWS_SECRET_ACCESS_KEY,name=cart-app,key=AWS_SECRET_ACCESS_KEY --secret environment-variable=SECRET_KEY,name=cart-app,key=SECRET_KEY
   # Setting up a template and deploying API Gateway.
       - (cat ${CI_PROJECT_DIR}/apigw.yaml.j2 | envsubst) > apigw.yaml
       - cat apigw.yaml
       - yc serverless api-gateway create --name testing --spec=apigw.yaml --description "created from gitlab CI" --cloud-id ${cloud_id} --folder-name ${cart_testing}
       - mkdir output
       - export gwdomain=$(yc serverless api-gateway get testing --cloud-id ${cloud_id} --folder-name ${cart_testing} | yq .domain)
       - echo "https://"$gwdomain>output/gwurl
     artifacts:
       paths: [output/]

   e2e-test-job:
     stage: test
     image: alpine:3.15
     script:
       - apk add -q --no-cache bash curl
       - apk add yq --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
       - cat output/gwurl
       - export gwurlvar=$(cat output/gwurl)
       - curl $gwurlvar

   load-test-job:
     stage: test
     image: alpine:3.15
     script:
       - echo "Here goes load testing commands"
       - echo "Might even invoke bash with prepared bash script"
       - echo "Hello!"

   delete-test-env-job:
     stage: delete-test-env
     image: alpine:3.15
     script:
       - apk add -q --no-cache bash curl jq gettext yq
       - curl --fail --silent --location --remote-name https://storage.yandexcloud.net/yandexcloud-yc/install.sh
       - bash install.sh -i /usr/local/yandex-cloud -n
       - ln -s /usr/local/yandex-cloud/bin/yc /usr/local/bin/yc
       - echo "$SA_TESTING_DEPLOYER_PRIVATE_KEY" > key.json
       - yc config profile create sa-profile
       - yc config set service-account-key key.json
       - yc serverless api-gateway delete testing --cloud-id ${cloud_id} --folder-name ${cart_testing}
       - yc serverless container delete testing --cloud-id ${cloud_id} --folder-name ${cart_testing}

   release-job:
     stage: release
     image: alpine:3.15
     script:
       - apk add -q --no-cache bash curl jq gettext
       - curl --fail --silent --location --remote-name https://storage.yandexcloud.net/yandexcloud-yc/install.sh
       - bash install.sh -i /usr/local/yandex-cloud -n
       - ln -s /usr/local/yandex-cloud/bin/yc /usr/local/bin/yc
       - echo "$SA_PROD_DEPLOYER_PRIVATE_KEY" > key.json
       - yc config profile create sa-profile
       - yc config set service-account-key key.json
       - yc serverless container revision deploy --container-name prod --image "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" --cores 1 --memory 512mb --concurrency 1 --execution-timeout 10s --cloud-id ${cloud_id} --folder-name ${cart_prod} --service-account-id ${PROD_SA_ID} --environment DOCAPI_ENDPOINT=${DOCAPI_ENDPOINT_prod},DB_TABLE_PREFIX='' --secret environment-variable=AWS_ACCESS_KEY_ID,name=cart-app,key=AWS_ACCESS_KEY_ID --secret environment-variable=AWS_SECRET_ACCESS_KEY,name=cart-app,key=AWS_SECRET_ACCESS_KEY --secret environment-variable=SECRET_KEY,name=cart-app,key=SECRET_KEY
       - container_id=${prod_container_id}
   # Creating a production environment.
     environment:
       name: production/$CI_COMMIT_SHORT_SHA
   ```

   {% endcut %}

1. Add a comment in the **Commit message** field: `CI scripts`.
1. Click **Commit changes**.

The `.gitlab-ci.yml` file contains the following steps of the CI script:
* **build**: Building a Docker image with `Dockerfile` and pushing the image to Container Registry.
* **deploy-test-env**: App test deployment. Additionally, we described the [artifacts](https://docs.gitlab.com/ci/jobs/job_artifacts/) feature to transfer data from one step to another that we are not going to use here. You can configure it, if required.
* **test**: Testing the app. The tests involve E2E simulation and load testing. You can describe and set up custom tests as well.
* **delete-test-env**: Deleting the test app.
* **release**: Deploying the app in production. This step also uses [deployment environments](https://docs.gitlab.com/ci/environments/). The system creates and saves them each time the pipeline is run successfully. Use them to restore and deploy the previous app version.

After you save the `.gitlab-ci.yml` configuration file, the build script will start.

## Check the result {#check-result}

To check the build script results, select **Build** on the left-hand panel in the `gitlab-test` project, and then **Pipelines** from the drop-down menu. Make sure all five steps complete successfully.

You can access the app at the API Gateway service domain URL in the `prod` folder. You can find this URL in the [management console](https://console.yandex.cloud) or in the `domain` field of the `bootstrap.sh` script runtime log.

## Delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:
1. [Delete](../../resource-manager/operations/folder/delete.md) the `prod`, `testing`, and `infra` folders with their contents.
1. [Delete the Managed Service for GitLab instance](../../managed-gitlab/operations/instance/instance-delete.md) or the [GitLab VM](../../compute/operations/vm-control/vm-delete.md).
1. [Delete the Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-delete.md).
1. [Delete](../../vpc/operations/address-delete.md) the cluster public static IP address if you reserved one.
1. [Delete the service accounts](../../iam/operations/sa/delete.md).