[Yandex Cloud documentation](../../index.md) > [Yandex MetaData Hub](../index.md) > Data Catalog > Getting started > Getting started as a metadata steward

# Getting started as a metadata steward



{% note info %}

This feature is at the [Preview](../../overview/concepts/launch-stages.md) stage.

{% endnote %}



When using Yandex Data Catalog as a metadata steward, you can collect and organize metadata on Yandex Cloud objects and links between them.

Data Catalog can collect metadata from the following services:

* PostgreSQL
* MySQL®
* ClickHouse®
* Yandex StoreDoc/MongoDB
* OpenSearch
* Greenplum®
* Yandex Data Transfer
* WebSQL
* DataLens

To get started:

1. [Create a metadata catalog](#create-catalog).
1. [Create a metadata source](#create-source).
1. [Create data ingestions](#create-ingestions) from sources on a schedule.
1. [Test the ingested metadata](#test-data-ingestion).
1. [Create a classification and tags](#classification).
1. [Create a domain and subdomains](#domains).
1. [Create a glossary and terms](#glossary).
1. [Mark up the obtained data](#metadata-markup).

## Required paid resources {#paid-resources}

The cost of infrastructure support includes a fee for Yandex Managed Service for PostgreSQL cluster computing resources, storage and backup size (see [Managed Service for PostgreSQL pricing](../../managed-postgresql/pricing.md)).

## Getting started {#before-you-begin}

1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud, or sign up if you have not already.

1. If you do not have a resource folder, create one:

   1. In the [management console](https://console.yandex.cloud), in the top panel, click ![image](../../_assets/console-icons/layout-side-content-left.svg) or ![image](../../_assets/console-icons/chevron-down.svg) and select the [cloud](../../resource-manager/concepts/resources-hierarchy.md#cloud).
   1. To the right of the cloud name, click ![image](../../_assets/console-icons/ellipsis.svg).
   1. Select ![image](../../_assets/console-icons/plus.svg) **Create folder**.
   
      ![create-folder1](../../_assets/resource-manager/create-folder-1.png)
   
   1. Give your [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) a name. The naming requirements are as follows:
   
       * Length: between 3 and 63 characters.
       * It can only contain lowercase Latin letters, numbers, and hyphens.
       * It must start with a letter and cannot end with a hyphen.
   
   1. Optionally, specify the description for your folder.
   1. Select **Create a default network**. This will create a [network](../../vpc/concepts/network.md#network) with subnets in each availability zone. Within this network, you will also have a [default security group](../../vpc/concepts/security-groups.md#default-security-group), within which all network traffic will be allowed.
   1. Click **Create**.
   
      ![create-folder2](../../_assets/resource-manager/create-folder-2.png)

1. [Assign](../../iam/operations/roles/grant.md) your Yandex Cloud account the following roles for the resource folder:

    * `data-catalog.dataSteward`: To create and manage Data Catalog resources.
    * [vpc.user](../../vpc/security/index.md#vpc-user): To use the cluster [network](../../vpc/concepts/network.md#network).

    {% note info %}
    
    If you cannot manage roles, contact your cloud or organization administrator.
    
    {% endnote %}

1. [Create a Managed Service for PostgreSQL cluster](../../managed-postgresql/operations/cluster-create.md#create-cluster) to supply data from as a test load.

## Create a metadata catalog {#create-catalog}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), select the resource folder you [prepared earlier](#before-you-begin).
  1. [Navigate](../../console/operations/select-service.md#select-service) to **Yandex MetaData Hub**.
  1. In the left-hand panel, select ![image](../../_assets/console-icons/folder-magnifier.svg) **Data Catalog**.
  1. Click **Create catalog**.
  1. Set a **Name** for the metadata catalog.
  1. Set a **Description** for the metadata catalog.
  1. Click **Create**.

    {% note info %}
    
    When you create a metadata catalog, the **metadata AI markup** is on by default.
    
    With this option enabled, the AI assistant suggests descriptions, [domains](../concepts/data-catalog.md#domains-and-subdomains), [classifications and tags](../concepts/data-catalog.md#classifications-and-tags), [glossaries and terms](../concepts/data-catalog.md#glossaries-and-terms), and marks up your metadata using them. You can confirm, edit, or reject any suggestion your AI assistant makes by hovering over the **AI** icon next to the suggestion and selecting the action.
    
    After the catalog is created, you can manage the AI markup on the **Overview** page or when [updating](../operations/data-catalog/update-catalog.md) the catalog.
    
    {% endnote %}

{% endlist %}

## Create a metadata source and ingestion {#create-source}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/cloud-arrow-up-in.svg) **Data sources** tab and click **Create data source**.
  1. Set a **Name** for the source.
  1. Set a **Description** for the source.
  1. Select **Data source type**: **PostgreSQL**
  1. Under **PostgreSQL source**, set the following parameters:

     * **Folder ID**: Resource folder the Managed Service for PostgreSQL cluster was created in.
     * **Installation type**: **Cluster Managed Service for PostgreSQL**.
     * **Cluster for Managed DB**: Managed Service for PostgreSQL cluster you created earlier.
     * **Connection ID**: Connection to the previously created Managed Service for PostgreSQL cluster in Yandex Connection Manager.
     * **Database name** in the Managed Service for PostgreSQL cluster you created earlier.
     * **Upload from all databases**: Optionally, enable this setting if you need to upload data from all databases.
     * **Network ID**: Specify the network ID.


  1. Click **Create**. 
  1. This will open a page with a list of sources in the metadata catalog. You will see **No ingestion** in the line with the source you just created.
  1. Hover over this message and click **Create ingestion** in the window that opens. 
  1. Set a **Name** for the ingestion.
  1. Set a **Description** for the ingestion.  
  1. In the **Sched** field, select **Daily**.
  1. In the **Start time** and **End time** fields, specify the ingestion operation time.
  1. Click **Create**.

{% endlist %}

## Test the ingested metadata {#test-data-ingestion}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Navigate to the ![image](../../_assets/console-icons/database-magnifier.svg) **Metadata search** tab.

     In the window that opens, you will see the metadata you got from the Managed Service for PostgreSQL cluster created earlier.

{% endlist %}

{% note tip %}

The metadata ingested from the source will also appear on the **Uploaded data** tab in the same source.

{% endnote %}

## Create a classification and tags {#classification}

### Create a classification {#create-classification}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/tag.svg) **Tags and classifications** tab and click **Create classification**.
  1. Set a **Name** for the classification.
  1. Set a **Description** for the classification.
  1. Click **Create**.

{% endlist %}

### Create tags {#create-tags}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/tag.svg) **Tags and classifications** tab and open the [previously created](#create-classification) classification.
  1. Click **Create tag**.
  1. In the window that opens, set a **Name** for the tag.
  1. Set a **Description** for the tag.
  1. Click **Create**.

{% endlist %}

## Create a domain and subdomains {#domains}

### Create a domain {#create-domain}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/globe.svg) **Domains** tab and click **Create domain**.
  1. Set a **Name** for the domain. 
  1. Set a **Description** for the domain.
  1. Add one or more [previously created](#create-tags) tags.
  1. Click **Create**.

{% endlist %}

### Create a subdomain {#create-subdomain}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/globe.svg) **Domains** tab and select the [previously created](#create-domain) domain.
  1. Click **Add subdomain**.
  1. Set a **Name** for the subdomain. 
  1. Set a **Description** for the subdomain.
  1. Add one or more [previously created](#create-tags) tags.
  1. Click **Create**.

{% endlist %}

## Create a glossary and terms {#glossary}

### Create a glossary {#create-glossary}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/book.svg) **Terms and glossaries** tab and click **Create glossary**.
  1. Set a **Name** for the glossary.
  1. Set a **Description** for the glossary.
  1. Add one or more [previously created](#create-tags) tags.
  1. Click **Create**.

{% endlist %}

### Create terms {#create-terms}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Go to the ![image](../../_assets/console-icons/book.svg) **Terms and glossaries** tab and select the [previously created](#create-glossary) glossary.
  1. Click **Create term**.
  1. Set a **Name** for the term.
  1. Set a **Description** for the term.
  1. Specify synonyms for the term.
  1. Add one or more [previously created](#create-tags) tags.
  1. Add linked terms.
  1. Click **Create**.

{% endlist %}

## Mark up the obtained data {#metadata-markup}

{% list tabs group=instructions %}

- Management console

  1. In the [management console](https://console.yandex.cloud), navigate to the metadata catalog you [created earlier](#create-catalog).
  1. Navigate to the ![image](../../_assets/console-icons/database-magnifier.svg) **Metadata search** tab.
  1. Click ![image](../../_assets/console-icons/ellipsis.svg) next to the dataset you selected and select **Set domain**, **Edit tags**, or **Edit terms**.
  1. In the window that opens, select an object in the hierarchy of domains, tags, or terms. Use search, if required.
  1. Add the selected objects.

{% endlist %}

## What's next {#what-is-next}

* [Create a term](../operations/data-catalog/create-term.md) in a glossary.
* [Create a child term](../operations/data-catalog/create-term-child.md).
* [Update a glossary](../operations/data-catalog/update-glossary.md).
* [Edit a term in a glossary](../operations/data-catalog/update-term.md).
* [Create a tag](../operations/data-catalog/create-tag.md) in a classification.
* [Update a classification](../operations/data-catalog/update-classification.md).
* [Edit a tag](../operations/data-catalog/update-tag.md) in a classification.