# Yandex Tracker: Data export and visualization


Visualizing data from [Yandex Tracker](https://yandex.ru/support/tracker/ru) with Yandex DataLens allows you to build more advanced analytics than with the Tracker tools.

To visualize data from Tracker in DataLens:
* Set up regular export of data to an external storage.
* Visualize the required metrics and data using DataLens.

To visualize your data, follow these steps:

1. [Get your cloud ready](#before-you-begin).
1. [Create a database for storing your Tracker data](#database-create).
1. [Create an OAuth token to access Tracker](#oauth-token).
1. [Create a function in Cloud Functions for importing data](#function-import).
1. [Create a connection to DataLens](#connection-create).
1. [Create a dataset](#dataset-create).
1. [Create a chart](#chart-create).
1. [Create a dashboard in DataLens and add charts to it](#dashboard-create).

## Getting started {#before-you-begin}


{% note info %}

We recommend creating a separate Tracker account to use the service.

{% endnote %}


Sign up for Yandex Cloud and create a [billing account](../../billing/concepts/billing-account.md):
1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud or create a new account.
1. On the **[Yandex Cloud Billing](https://center.yandex.cloud/billing/accounts)** page, make sure you have a billing account linked and it has the `ACTIVE` or `TRIAL_ACTIVE` [status](../../billing/concepts/billing-account-statuses.md). If you do not have a billing account, [create one](../../billing/quickstart/index.md) and [link](../../billing/operations/pin-cloud.md) a cloud to it.

If you have an active billing account, you can create or select a [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) for your infrastructure on the [cloud page](https://console.yandex.cloud/cloud).

[Learn more about clouds and folders here](../../resource-manager/concepts/resources-hierarchy.md).


### Required paid resources {#paid-resources}

* Continuously running Managed Service for ClickHouse® cluster (see [Managed Service for ClickHouse® pricing](../../managed-clickhouse/pricing.md)).
* Using the function in Cloud Functions (see [Cloud Functions pricing](../../functions/pricing.md)).

If you no longer need the resources you created, [delete them](#clear-out).


## Create a database for storing your Tracker data {#database-create}

1. Navigate to the [management console](https://console.yandex.cloud).
1. Navigate to **Managed Service for ClickHouse**.
1. Click **Create ClickHouse cluster**.
1. Specify the cluster parameters:
    * Basic parameters:
        * **Environment**: `PRODUCTION`
        * **Version**: `22.8 LTS`
    * Resources:
        * **Platform**: `Intel Ice Lake`
        * **Type**: `standart`
        * **Host class**: `s3-c2-m8 (2 vCPU, 8 GB)`
    * Storage size: `30 GB`
    * Hosts:
        * **Public access**: `Enabled`
    * DBMS settings:
        * **User management via SQL**: `Disabled`
        * **Managing databases via SQL**: `Disabled`
        * **Username**: `tracker_data`
        * **DB name**: `db1`
    * Service settings:
        * **DataLens access**: `Enabled`
        * **Serverless access**: `Enabled`
    For the full list of settings, see the [Managed Service for ClickHouse® settings](../../managed-clickhouse/concepts/settings-list.md).
1. Click **Create cluster**. Wait for the new cluster status to change to `Alive`.
1. Copy and save the host name for further Cloud Functions setup.

## Create an OAuth token to access Tracker {#oauth-token}

1. Go to the [Create an app](https://oauth.yandex.ru/client/new) page.
1. In the **What app do you want to create?** window, leave `For user authorization` and click **Proceed to creation**. If you see a verification window for your Public Services Portal account, close it.
1. In the **Create app** window:

   * Enter your **Service name**.
   * Add an icon for your service.
   * Specify the email address for notifications about changes in external authorization.
   
   Click **Next**.

1. In the **App platforms** window, select `Web services`. In the **Redirect URI** field, specify the URL where to redirect the user after they have allowed or denied access to the app, or enter `https://oauth.yandex.ru/verification_code` for debugging.

   Click **Next**.

1. In the **User data access permissions** window, under **Additional**, specify:

   * `Read from tracker`
   * `Write to tracker`

   Click **Next**.

1. Check the data and click **Everything is correct, create app**.
1. Copy the app ID from the **ClientID** field.
1. In the window that opens, enter the following URL in the browser search bar:

    ```
    https://oauth.yandex.ru/authorize?response_type=token&client_id=<app_ID>
    ```

    Where `client_id` is the new app ID copied from the **ClientID** field.

1. Log in under the Tracker account you will use for data visualization.
1. Save the OAuth token you got.

## Create a function in Cloud Functions for importing data {#function-import}

1. Navigate to the [management console](https://console.yandex.cloud).
1. Navigate to **Cloud Functions**.
1. Click **Create function**.
1. Specify a name for the function and click **Create**.
1. In the **Editor** window that opens, select the `Python` runtime environment.
1. Click **Continue**.
1. In the **Code source** field, click **ZIP archive**.
1. Attach a [test archive](https://github.com/yandex-cloud-examples/yc-tracker-data-import/blob/main/build/tracker-data-import.zip).
1. In the **Entry point** field, specify `tracker_import.handler`.
1. Under **Parameters**, specify:
    * **Timeout**: `60`
    * **Memory**: `1024`
    * **Environment variables**:
        * `TRACKER_ORG_ID`: ID of the Yandex 360 for Business organization.
          
          {% note info "Note" %}

          If you are using a Yandex Identity Hub organization (you can check this on the [administration page](https://tracker.yandex.com/admin/orgs)), replace the `X-Org-ID` header with `X-Cloud-Org-Id` in the `tracker_import.py` function code.

          {% endnote %}

        * `TRACKER_OAUTH_TOKEN`: Tracker account [OAuth token](#oauth-token).
        * `CH_HOST`: [Host](#database-create) name.
        * `CH_DB`: [Database](#database-create) name.
        * `CH_USER`: [Username](#database-create).
        * `CH_PASSWORD`: [Password](#database-create).
        * `CH_ISSUES_TABLE`: `tracker_issues`.
        * `CH_CHANGELOG_TABLE`: `tracker_changelog`.
        * `TRACKER_INITIAL_HISTORY_DEPTH`: `1d`.
        * `CH_STATUSES_VIEW`: `v_tracker_statuses`.
1. Click **Save changes**.
1. In the **Testing** tab, click **Run test**.
1. The test result is a data import log:
    ```json
    {
        "statusCode": 200,
        "headers": {
        "Content-Type": "text/plain"
        },
        "isBase64Encoded": false,
        "body": "OK"
    }
    ```
1. Create a [trigger](../../functions/concepts/trigger/index.md) to regularly export new data to the database:
    1. Open the **Cloud Functions** section.
    1. Click ![trigger](../../_assets/console-icons/gear-play.svg) → **Create trigger**.
    1. Set the trigger type to **Timer**.
    1. In the **Cron expression** field, select `Every day`.
    1. Under **Function settings**, click **Create**.
    1. Enter the account name. By default, the account has the `functions.functionInvoker` role to work with the trigger.
    1. Click **Create**.
    1. Click **Create trigger**.

## Create a connection in DataLens {#connection-create}

1. Open the **Managed Service for ClickHouse®** [cluster](#database-create).
1. On the left side of the window, select ![datalens](../../_assets/console-icons/chart-column.svg) **DataLens**.
1. Click **Create connection**.
1. Set up your connection:
    * **Connection**: `Select in folder`.
    * **Cluster**: Cluster specified when [creating the database](#database-create).
    * **Hostname**: Host specified when [creating the database](#database-create).
    * **HTTP interface port**: `8443`.
    * **Username**: Username specified when [creating the database](#database-create).
    * **Password**: Password specified when [creating the database](#database-create).
    * **Cache TTL in seconds**: `Default`.
    * **Raw SQL level**: `Forbid`.
    * **HTTPS**: `Enabled`.

      ![Connection settings](../../_assets/datalens/connection-settings.png =680x665)

1. Click **Create connection**.

## Create a dataset {#dataset-create}

1. Go to the [DataLens](https://datalens.ru/promo) home page.
1. Click **Start in cloud**.
1. Go to the [connections page](https://datalens.ru/connections).
1. Select the [connection](#connection-create).
1. In the top-right corner, click **Create dataset**.
1. Drag one or multiple tables to the workspace:
    * `db1.v_tracker_issues`: Current (most recent) issue slice.
    * `db1.v_tracker_changelog`: Issue parameter change history.
    * `Db1.v_tracker_statuses`: Status transition time based on the issue change history.
1. Click **Save**.

## Create a chart {#chart-create}


1. Go to the [DataLens](https://datalens.ru/?skipPromo=true) home page.
1. In the left-hand panel, click ![image](../../_assets/console-icons/chart-column.svg) **Charts**.
1. Click **Create chart** → **Chart in Wizard**.
1. In the top-left corner, click ![image](../../_assets/console-icons/circles-intersection.svg) **Select dataset**.
1. In the **Datasets** drop-down list, select the [dataset](#dataset-create) you created in the previous step.


1. On the top panel, select the [visualization type](../visualization-ref/index.md). By default, the **Column chart** type is selected.

## Create a dashboard and add charts to it {#dashboard-create}

1. In the left-hand panel, click ![image](../../_assets/console-icons/layout-cells-large.svg) **Dashboards**.
1. Click **Create dashboard**.
1. At the top of the [dashboard](#dashboard-create) page, click **Add**→ **Chart**.
1. Fill in the widget parameters. Pay close attention to the following fields:
    * **Name**: It sets the widget name. It is displayed at the top of the widget.
    * **Chart**: It sets the widget you are adding.
    * **Description**: It sets the widget description. It is displayed at the bottom of the widget.
    * **Auto height**: It sets the automatic height for a widget of the **Table** or **Markdown** type. If it is disabled, you can set the widget height on the page using the mouse.
1. Click **Add**. The widget will appear on the dashboard.
1. Save the dashboard:

   1. In the top-right corner of the dashboard, click **Save**.
   1. Enter a name for the dashboard and click **Create**.

   For more information about setting up dashboards, see [this section](../concepts/dashboard.md).

{% cut "Example of a dashboard based on data from the `v_tracker_issues` table" %}

![Sample dashboard based on data from the v_tracker_issues table](../../_assets/datalens/dashboard-from-table-issues.png)

{% endcut %}

{% cut "Example of a dashboard based on data from the `db1.v_tracker_statuses` table" %}

![Sample dashboard based on data from the db1.v_tracker_statuses table](../../_assets/datalens/dashboard-from-table-statuses.png)

{% endcut %}

## How to delete the resources you created {#clear-out}

To stop paying for the resources you created:
* [Delete the ClickHouse® cluster](../../managed-clickhouse/operations/cluster-delete.md).
* [Delete the function in Cloud Functions](../../functions/operations/function/function-delete.md).

_ClickHouse® is a registered trademark of [ClickHouse, Inc](https://clickhouse.com)._