[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Apache Airflow™](../index.md) > [Concepts](index.md) > Zones of control

# Zones of control between the Yandex Managed Service for Apache Airflow™ users and Yandex Cloud

Yandex Managed Service for Apache Airflow™ users should be aware of how responsibilities are shared between [Yandex Cloud](#cloud-responsibilities) as a cloud provider and the [Yandex Cloud customer](#customer-responsibilities). Understanding these zones of control enables you to use Yandex Managed Service for Apache Airflow™ efficiently and streamline your interactions with support.

Yandex Cloud support only handles issues that fall within the Yandex Cloud scope of responsibility. Requests to troubleshoot problems within the customer's zone of control are outside the scope of support provided by Yandex Cloud.

## Service architecture {#architecture}

Each [Apache Airflow™ cluster](index.md#cluster) runs in a dedicated Yandex Managed Service for Kubernetes node group isolated from other clusters via virtual networks and native Kubernetes tools. Node groups are managed by a shared Kubernetes master controlled by Yandex Cloud.

A shared PostgreSQL cluster managed by Yandex Cloud stores Apache Airflow™ metadata. Access to this cluster is restricted.

## Yandex Cloud control zone {#cloud-responsibilities}

* **Physical and virtual infrastructure**. Maintaining the health of physical servers, virtual machines, disks, and network connectivity utilized by the service components.

* **Kubernetes infrastructure** Maintaining the health of the shared Kubernetes master and node group network infrastructure comprising a virtual network, security group, and service account. Isolation of node groups between clusters.

* **Managed PostgreSQL cluster for metadata**. Ensuring the health, availability, and backups of the shared PostgreSQL cluster used to store Apache Airflow™ metadata.

* **Apache Airflow™ components managed by Yandex Cloud**. Maintaining the health of the core [Apache Airflow™ components](index.md#components):

  * Web server
  * Scheduler
  * DAG processor
  * Workers
  * Triggerer

* **Backup and recovery**. Backup and recovery of the Apache Airflow™ cluster database with metadata. Recovery from a backup is not guaranteed if the problem is caused by an error on the component provider's side or data architecture issues. In such cases, Yandex Cloud support will identify the cause of the problem and provide recommendations to prevent the issue from recurring.

* **Cluster operations**. Cluster management through available cloud interfaces, including creating, modifying, scaling, updating, and deleting a cluster, as well as updating individual component configurations.

* **DAG file storage**. Ensuring platform-level integration between the Apache Airflow™ cluster and Yandex Object Storage.

* **Platform-level monitoring and logging**. Providing access to Apache Airflow™ component health metrics via Yandex Monitoring. Providing platform-specific Apache Airflow™ component logs via Yandex Cloud Logging.

* **Maintenance and updates**. Applying security updates to platform components and guest OS nodes. Applying optimal default settings when creating new clusters and updating Apache Airflow™ versions.

* **Integrations with Yandex Cloud services**. Providing platform integration with Yandex Cloud services:

  * Yandex Object Storage: Storing DAG files and task logs.
  * Yandex Identity and Access Management: Managing platform-level access to the service.
  * Yandex Virtual Private Cloud: Cluster network isolation.
  * Yandex Lockbox: Storing secrets, connections, and variables.
  * Yandex Cloud Logging and Yandex Monitoring: Collecting platform logs and metrics.

* **User documentation**. Developing and updating documentation for Yandex Managed Service for Apache Airflow™. Apache Airflow™ documentation is provided by the Apache Software Foundation.

## Yandex Cloud customer control zone {#customer-responsibilities}

* **Writing DAG files**. Developing, testing, debugging, and optimizing DAG files and Apache Airflow™ operators.

* **Task logic**. Ensuring proper business logic within operators, hooks, and sensors.

* **Python dependencies**. Managing custom Python libraries installed on the cluster. It is the customer’s responsibility to ensure these libraries are compatible with each other as well as with Python and Apache Airflow™ versions provided by Yandex Managed Service for Apache Airflow™.

  The Python version depends on the version of Apache Airflow™ provided by Yandex Cloud and may change after a cluster upgrade. Following an upgrade, the customer must verify that custom libraries remain compatible with the new Python and Apache Airflow™ versions. Yandex Cloud does not guarantee compatibility of custom libraries with any given version of Python or Apache Airflow™.

* **Task Instance diagnostics**. Analyzing task statuses, reviewing task logs, and diagnosing failure root causes at the code level.

* **Schedule planning**. Correctly configuring DAG schedule parameters, including `schedule_interval`, `start_date`, `catchup`, `max_active_runs`, etc.

* **DAG performance**. Optimizing DAGs in terms of the task count, parallel execution, and execution time. Preventing overload of the scheduler, DAG processor, and worker pool.

* **DAG file storage in Yandex Object Storage**. Uploading, versioning, and managing DAG files in a Yandex Object Storage bucket. Setting up and running CI/CD pipelines to deliver DAG files to the bucket. Ensuring the cluster service account has correct permissions to access the bucket.

* **Using GitLab as a DAG storage**. Ensuring stable operation of GitLab. Platform-level Yandex Managed Service for Apache Airflow™ errors related to GitLab fall within the Yandex Cloud scope of responsibility.

* **Cluster resources**. Ensuring that the cluster configuration matches the actual load. This includes selecting the appropriate resource types, number of component instances, minimum and maximum number of workers.

* **Apache Airflow™ settings**. Configuring Apache Airflow™ settings available to the customer. The customer assumes full risk if deviating from the recommended default values.

* **Maintenance window**. Scheduling a window for cluster maintenance.

* **Connections and variables**. Creating, configuring, and maintaining Apache Airflow™ connections and variables. Setting up integration with Yandex Lockbox or other secret storage backends. Managing secret lifecycle.

* **Access management**. Setting up roles and service accounts for access to Apache Airflow™.

* **Monitoring and responding to events**. Analyzing cluster performance metrics and responding promptly to alerts. Setting up additional alerts for custom task and DAG metrics. Analyzing task logs available via the Apache Airflow™ UI and Yandex Cloud Logging.

* **External systems**. Ensuring the availability and correct operation of external systems with which Apache Airflow™ tasks interface.

* **Network connectivity with external systems**. Configuring security groups, routing, and NAT to enable cluster access to target external resources.

* **Third-party software**. Configuring and operating CI/CD tools for DAG deployment and third-party monitoring systems.

* **Version upgrades**. Preparing for an Apache Airflow™ version update upon receiving an alert or reviewing the [Apache Airflow™ versioning policy](update-policy.md). This includes analyzing the release notes and and verifying that DAGs, operators, and dependencies are compatible with the target version.

  The customer must verify the compatibility of custom Python libraries with both the Python and Apache Airflow™ versions provided with the new cluster version before and after the upgrade. Updating or replacing incompatible libraries is also the customer's responsibility.

* **Pre-upgrade testing in production clusters**. Testing DAGs on the new Apache Airflow™ version before applying the upgrade to the production cluster.

* **Use of an outdated Apache Airflow™ version**. Resolving operational issues that arise from using an outdated version of Apache Airflow™ after the Yandex Cloud support windows have expired.

## Common scenarios {#scenarios}

| Scenario | Control zone |
| --- | --- |
| The Apache Airflow™ web server is down and the web UI is inaccessible. | Yandex Cloud |
| A task instance fails with an error in custom code. | Customer |
| The DAG processor fails to parse a DAG due to a platform error. | Yandex Cloud |
| The DAG processor fails to parse a DAG due to a syntax error in the DAG file. | Customer |
| The scheduler fails to run tasks due to an incorrect `schedule_interval` or `start_date`. | Customer |
| KEDA fails to scale workers due to a platform error. | Yandex Cloud |
| An error occurred during a cluster operation, such as creation or scaling via the API. | Yandex Cloud |
| Error connecting to an external database from an Apache Airflow™ task. | Customer |
| A DAG file was not uploaded to the target Yandex Object Storage bucket. | Customer |
| The native integration with Yandex Lockbox malfunctions. | Yandex Cloud |
| A custom Python library is incompatible with the current Apache Airflow™ or Python version. | Customer |
| A custom Python library breaks following a cluster Apache Airflow™ or Python version upgrade. | Customer |
| Metadata database backup fails at the platform level. | Yandex Cloud |
| GitLab is unavailable. | Customer |
| The GitLab connection fails due to a Yandex Managed Service for Apache Airflow™ platform error. | Yandex Cloud |

## Requesting support {#support}

Yandex Cloud support only addresses issues that fall within the Yandex Cloud scope of responsibility.

If your issue falls within the customer's zone of control, our support team cannot full investigation or resolution. Such issues include debugging DAGs, optimizing tasks, configuring connections, analyzing custom logs, ensuring compatibility of Python libraries, configuring GitLab CI/CD pipelines, and handling other operational tasks.

{% note info %}

If a customer lacks the expertise to independently perform actions within their control, they can contact their account manager. Your account manager will provide options for engaging certified partners or professional Yandex Cloud services.

{% endnote %}