[Yandex Cloud documentation](../../index.md) > [Tutorials](../index.md) > [Machine learning and artificial intelligence](index.md) > Development with Yandex Managed Service for GitLab > Managing the MLOps lifecycle with ML Registry

# Managing the MLOps lifecycle with ML Registry in Yandex Managed Service for GitLab

The machine learning model lifecycle comprises development, model training, quality evaluation, and model deployment. This guide explores GitLab's capabilities in the area of storing and versioning of ML artifacts you get during experiments with an ML model within the Yandex Cloud infrastructure. The experiments feature changing the model's hyperparameters, i.e., the settings configured prior to the start of model training that determine its architecture, training strategy, and overall behavior. Unlike the model's parameters (weights, coefficients), which are picked out in the course of data-based training, hyperparameters are selected by the researcher or ML engineer rather than changed automatically.

This diagram depicts the model development and testing lifecycle in a GitLab environment:

![mlops-pipeline](../../_assets/architecture/mlops.svg)

The development and testing environment is represented by a DSVM. Your GitLab instance with the [ML Registry](https://docs.gitlab.com/user/project/ml/model_registry/) package preinstalled is enabled by the scalable [Yandex Managed Service for GitLab](../../managed-gitlab/index.md). This makes the following MLOps tools available to you:

* Model Registry, which you can use to log metrics, analyze model application experiments, and perform quality assessment.
* The Model Experiments catalog used for model storage and versioning management.

The MLFlow [Python API library](https://mlflow.org/docs/latest/api_reference/python_api/index.html) serves for integrating the model with ML Registry.

In addition to Managed Service for GitLab, the user leverages [Yandex Compute Cloud](../../compute/index.md) to work with the model's source code, and [Yandex Virtual Private Cloud](../../vpc/index.md) for the network infrastructure.

## System architecture {#architecture}

### Network {#network}

As part of the solution, the user creates a Virtual Private Cloud [network](../../vpc/concepts/network.md#network) named `net-gitlab`.

#### Subnets {#subnets}

Within `net-gitlab`, the user creates a [subnet](../../vpc/concepts/network.md#subnet) in an [availability zone](../../overview/concepts/geo-scope.md) of their choice to host the Managed Service for GitLab instance and the VM.

#### Security groups {#security-groups}

Network access to resources within the infrastructure is managed through [security groups](../../vpc/concepts/security-groups.md). [Learn more on configuring security group rules for Managed Service for GitLab](../../managed-gitlab/operations/configure-security-group.md).

#### Resource addresses {#addresses}

The new infrastructure uses a [public IP address](../../vpc/concepts/address.md#public-addresses) for the new VM and the GitLab instance URL in the `gitlab.yandexcloud.net` domain.

### Managed Service for GitLab {#gitlab}

The [GitLab instance](../../managed-gitlab/concepts/index.md) is deployed on a VM managed by [Managed Service for GitLab](../../managed-gitlab/index.md). The instance can be accessed by its address via the standard GitLab web interface.

To execute GitLab CI/CD jobs, create and configure GitLab Runner for your Managed Service for GitLab instance.

### Compute Cloud {#compute}

Local testing and uploads of changes to the model's source code repository are done with the help of a [Data Science Virtual Machine](../../compute/operations/dsvm/index.md) virtual machine. A separate VM can be used to deploy GitLab Runner.

## Test model {#sample-ml}

The test ML model deployed in this guide simulates the development and versioning cycle of a credit pipeline model. The model is adapted for use in a cloud infrastructure.

To deploy the model's development environment in the Yandex Cloud environment:

1. [Get your cloud ready](#before-you-begin).
1. [Create the infrastructure](#deploy).
1. [Create a project and configure the environment](#project).
1. [Create an experiment and a model version](#experiment).

If you no longer need the resources you created, [delete them](#clear-out).

## Get your cloud ready {#before-you-begin}

Sign up for Yandex Cloud and create a [billing account](../../billing/concepts/billing-account.md):
1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud or create a new account.
1. On the **[Yandex Cloud Billing](https://center.yandex.cloud/billing/accounts)** page, make sure you have a billing account linked and it has the `ACTIVE` or `TRIAL_ACTIVE` [status](../../billing/concepts/billing-account-statuses.md). If you do not have a billing account, [create one](../../billing/quickstart/index.md) and [link](../../billing/operations/pin-cloud.md) a cloud to it.

If you have an active billing account, you can create or select a [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) for your infrastructure on the [cloud page](https://console.yandex.cloud/cloud).

[Learn more about clouds and folders here](../../resource-manager/concepts/resources-hierarchy.md).

### Required paid resources {#paid-resources}

* Managed Service for GitLab: use of computing resources pertaining to the instance (VM) and its data storage capacity (see [Managed Service for GitLab pricing](../../managed-gitlab/pricing.md)). Depending on where GitLab Runner is deployed, you may have to pay for the Compute Cloud VM used to install GitLab Runner.
* VMs: use of computing resources, storage, public IP address, and the OS (see [Compute Cloud pricing](../../compute/pricing.md)).
* Object Storage: storage of Managed Service for GitLab backups (see [Object Storage pricing](../../storage/pricing.md)).

## Create the infrastructure {#deploy}

{% note info %}

Before you start creating your infrastructure, [make sure](../../quota-manager/operations/list-quotas.md) your [cloud](../../resource-manager/concepts/resources-hierarchy.md#cloud) has enough unused [quotas](../../quota-manager/concepts/index.md) for resources.

{% endnote %}

{% list tabs group=instructions %}

- Manually {#manual}

  1. [Create a network](../../vpc/operations/network-create.md) named `net-gitlab`. with the **Create subnets** option disabled.
  1. In the `net-gitlab` network, [create](../../vpc/operations/subnet-create.md) a subnet with the following parameters in the `ru-central1-a` availability zone:
      * **Name**: `subnet-gitlab-a`
      * **Availability zone**: `ru-central1-a`
      * **CIDR**: `10.16.0.0/24`
  1. In the `net-gitlab` network, [create a security group](../../vpc/operations/security-group-create.md) named `gitlab-sg` for the [Managed Service for GitLab instance](../../managed-gitlab/concepts/index.md#instance) and the VM. Follow the [guide](../../managed-gitlab/operations/configure-security-group.md) to set up rules in this security group.
  1. [Create](../../iam/operations/sa/create.md) a service account named `gitlab-sa` and [assign](../../iam/operations/sa/assign-role-for-sa.md) it the `compute.admin`, `vpc.admin`, and `iam.serviceAccounts.user` [roles](../../iam/concepts/access-control/roles.md).
  1. [Create and activate a GitLab instance](../../managed-gitlab/operations/instance/instance-create.md) of any suitable configuration. When creating the instance, specify the subnet and security group you created earlier.
  1. [Create a VM from the DSVM image](../../compute/operations/dsvm/quickstart.md) named `vm-mlops` in the `ru-central1-a` availability zone and the subnet created earlier. When creating the VM, specify the security group you created earlier.

- Terraform {#tf}

    1. If you do not have Terraform yet, [install it](../infrastructure-management/terraform-quickstart.md#install-terraform).
    1. [Get the authentication credentials](../infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file.
    1. [Configure and initialize a provider](../infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf).
    1. Place the configuration file in a separate working directory and [specify the parameter values](../infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file.

    1. Download the [ml-ops-managed-gitlab.tf](https://github.com/yandex-cloud-examples/yc-ml-ops-managed-gitlab/blob/main/ml-ops-managed-gitlab.tf) configuration file to the same working directory.

        This file describes:

        * [Network](../../vpc/concepts/network.md#network).
        * [Subnet](../../vpc/concepts/network.md#subnet).
        * [Security group](../../vpc/concepts/security-groups.md) and rules the Managed Service for GitLab instance needs to operate.
        * Managed Service for GitLab instance.
        * VM with a [DSVM](https://yandex.cloud/en/marketplace/products/yc/dsvm) image.
        * Service account.

    1. In the `ml-ops-managed-gitlab.tf` file, specify the following settings:

        * `instance_name`: GitLab instance name.
        * `instance_login`: GitLab instance admin login.
        * `instance_email`: Instance admin email address.
        * `instance_domain`: Instance domain name in `<name>.gitlab.yandexcloud.net` format.
        * `vm_username` and `vm_public_key`: Username and absolute path to the [public key](../../compute/operations/vm-connect/ssh.md#creating-ssh-keys), both required to access the VM.
        * `sa_folder_id`: ID of the new service account's folder.

    1. Validate your Terraform configuration files using this command:

        ```bash
        terraform validate
        ```

        Terraform will display any configuration errors detected in your files.

    1. Create the required infrastructure:

        1. Run this command to view the planned changes:
        
           ```bash
           terraform plan
           ```
        
           If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
        
        1. If everything looks correct, apply the changes:
           1. Run this command:
        
              ```bash
              terraform apply
              ```
        
           1. Confirm updating the resources.
           1. Wait for the operation to complete.

        All the required resources will be created in the specified folder. You can check resource availability and their settings in the [management console](https://console.yandex.cloud).

{% endlist %}

## Create a project and configure the environment {#project}

1. [Create a GitLab project](https://docs.gitlab.com/ee/user/project/), select **Import project** on the home page, and specify the import settings:

   * **Import project from**: **Repository by URL**.
   * **Git Repository URL**: `https://github.com/yandex-cloud-examples/yc-ml-ops-managed-gitlab.git`.
   * **Project name**: `gitlab-mlflow`.

1. Deploy a [runner](../../managed-gitlab/concepts/index.md#runners) for the created GitLab project using the [instructions](../../managed-gitlab/tutorials/install-gitlab-runner.md). During deployment, specify the [previously created](#deploy) infrastructure components:

   * If installing the runner on your VM manually, select `subnet-gitlab-a` and the `gitlab-sg` security group when creating the VM.
   * If creating the runner using the management console, specify the `gitlab-sa` service account and the `gitlab-sg` security group.

### Configure the environment variables {#variables}

1. Open the GitLab `gitlab-mlflow` project.
1. Navigate to **Settings** in the left-hand panel and select **Access Tokens** from the drop-down list.
1. Create a new token with the following settings:
   * **Token name**: `mlflow`.
   * **Select a role**: `Maintainer`.
   * **Select scopes**: `api`, `manage_runner`, `read_repository`, `write_repository`.
1. Click **Create project access token**.
1. Copy the token value.
1. Select **Settings** on the left and, in the popup list, select **CI/CD**.
1. Under **Variables**, click **Expand**.
1. Add these environment variables:
      * `MLFLOW_TRACKING_TOKEN`: New token.
      * `MLFLOW_TRACKING_URI`: `https://<GitLab_instance_address>.gitlab.yandexcloud.net/api/v4/projects/4/ml/mlflow`.
      * `REPO_TOKEN`: The token you created earlier.

      To add a variable:
      * Click **Add variable**.
      * In the window that opens, specify a variable name in the **Key** field and its value in the **Value** field.
      * Click **Add variable**.

## Create an experiment and a model version {#experiment}

1. [Connect](../../compute/operations/vm-connect/ssh.md) to the `vm-mlops` VM over SSH.
1. [Add](https://docs.gitlab.com/user/ssh/) an SSH key for secure access to GitLab.
1. [Clone](https://docs.gitlab.com/topics/git/clone/) the `gitlab-mlflow` project repository using SSH.

1. Navigate to the repository directory and create a branch named `mlops-experiment-1`:

   ```bash
   git checkout -b mlops-experiment-1
   ```

1. Edit the model's parameters in the `loan_prediction.py` file, for example, edit `RANDOM_SEED`.
1. Upload the changes to GitLab:

   ```bash
   git add -A
   git commit -m "Change model parameter"
   git push
   ```

1. Open the GitLab project named `gitlab-mlflow` and [create a merge request](https://docs.gitlab.com/user/project/merge_requests/creating_merge_requests/) from the new branch. A training and testing scenario for the modified model will be created automatically.

1. Run the scenario:
    1. In the left-hand panel, select **Build**.
    1. Select **Pipelines** from the drop-down list.
    1. Click ![image](../../_assets/console-icons/play-fill.svg) in the **Actions** column and select **trigger_train**.

    Wait for the scenario to end.

1. Create a registry of model versions:
   1. In the left-hand panel, select **Deploy**.
   1. Select **Model Registry** from the drop-down list.
   1. Click **Create/import model** and select **Create new model**.
   1. Enter `loan-prediction-demo` as the model's name and click **Create**.

1. Check the results of the experiment:
   1. In the left-hand panel, select **Analyze**.
   1. Select **Model Experiments** from the drop-down list.
   1. Select the `Loan_prediction` experiment from the list in the right and navigate to the **Runs** tab. The list displays all the model training methods, their parameters, and training results.
   1. Click an item in the **Name** column. The **Artifacts** tab displays the training artifacts of the method you select.
   1. Click **Promote run** to register the model version. Select the `loan-prediction-demo` model from the drop-down list in the **Model** field, enter `0.0.1` in the **Version** field, and click **Promote**. A new version and all its training artifacts will be added to the version registry.

## Delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

{% list tabs group=instructions %}

- Manually {#manual}

    1. [Delete the GitLab instance](../../managed-gitlab/operations/instance/instance-delete.md).
    1. [Delete the VM](../../compute/operations/vm-control/vm-delete.md).

- Terraform {#tf}

    1. In the terminal window, go to the directory containing the infrastructure plan.
    
        {% note warning %}
    
        Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.
    
        {% endnote %}
    
    1. Delete resources:
    
        1. Run this command:
    
            ```bash
            terraform destroy
            ```
    
        1. Confirm deleting the resources and wait for the operation to complete.
    
        All the resources described in the Terraform manifests will be deleted.

{% endlist %}

## Useful links {#see-also}

* [Machine Learning Model Experiments](https://docs.gitlab.com/user/project/ml/experiment_tracking/)
* [Model Registry](https://docs.gitlab.com/user/project/ml/model_registry/)