[Yandex Cloud documentation](../../index.md) > [Yandex Audit Trails](../index.md) > [Tutorials](index.md) > Exporting audit logs to SIEM systems > Uploading audit logs to ArcSight SIEM

# Uploading audit logs to ArcSight SIEM

Create a trail to upload management event audit logs of resources in an individual folder to an Yandex Object Storage bucket with encryption enabled. Then configure continuous log delivery to ArcSight SIEM.

To complete the tutorial successfully, you must have an ArcSight instance installed.

The solution described in the tutorial follows the steps below:
1. A [trail](../concepts/trail.md) uploads logs to an Object Storage bucket.
1. A [bucket](../../storage/concepts/bucket.md) is mounted via a [FUSE](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) interface to a folder on an intermediate VM.
1. [SmartConnector](https://www.microfocus.com/documentation/arcsight/arcsight-smartconnectors/AS_SmartConn_getstart_HTML/) collects logs from the folder and delivers them to ArcSight for analysis.

For more information about the scripts for delivering audit logs to ArcSight, see [Yandex Cloud Security Solution Library](https://github.com/yandex-cloud-examples/yc-export-auditlogs-to-arcsight#two-log-shipping-scenarios).

{% note info %}

[Yandex Cloud Security Solution Library](https://github.com/yandex-cloud-examples/yc-security-solutions-library) is a public repo on GitHub with a set of examples and recommendations on how to build a secure infrastructure in Yandex Cloud.

{% endnote %}

To configure delivery of audit log files to ArcSight:

1. [Get your cloud ready](#before-begin).
1. [Set up your environment](#prepare-environment).
1. [Assign roles to the service accounts](#add-roles).
1. [Create a trail](#create-trail).
1. [Mount your bucket](#mount-bucket).
1. [Connect ArcSight SmartConnector](#configure-arcsight).

If you no longer need the resources you created, [delete them](#clear-out).

## Getting started {#before-begin}

If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

Sign up for Yandex Cloud and create a [billing account](../../billing/concepts/billing-account.md):
1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud or create a new account.
1. On the **[Yandex Cloud Billing](https://center.yandex.cloud/billing/accounts)** page, make sure you have a billing account linked and it has the `ACTIVE` or `TRIAL_ACTIVE` [status](../../billing/concepts/billing-account-statuses.md). If you do not have a billing account, [create one](../../billing/quickstart/index.md) and [link](../../billing/operations/pin-cloud.md) a cloud to it.

If you have an active billing account, you can create or select a [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) for your infrastructure on the [cloud page](https://console.yandex.cloud/cloud).

[Learn more about clouds and folders here](../../resource-manager/concepts/resources-hierarchy.md).

### Required paid resources {#paid-resources}

The infrastructure support cost includes:

* Fee for using VM instances (see [Compute Cloud pricing](../../compute/pricing.md)).
* Fee for storing data in a bucket (see [Object Storage pricing](../../storage/pricing.md#prices-storage)).
* Fee for data operations (see [Object Storage pricing](../../storage/pricing.md#prices-operations)).
* Fee for using KMS keys (see [Key Management Service pricing](../../kms/pricing.md#prices)).

## Set up your environment {#prepare-environment}

### Prepare an intermediate VM {#setup-vm}

You can use a VM that has access to an ArcSight instance or create a new one:

1. [Create a VM](../../compute/operations/vm-create/create-linux-vm.md) from a Linux image based on [Ubuntu 20.04](https://yandex.cloud/en/marketplace/products/yc/ubuntu-20-04-lts).
1. [Connect to the VM](../../compute/operations/vm-connect/ssh.md#vm-connect) over SSH.

### Create a bucket for audit logs {#create-backet}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), select the folder where you want to create a bucket, e.g., `example-folder`.
  1. Navigate to **Object Storage**.
  1. Click **Create bucket**.
  1. On the bucket creation page:
      1. Enter a name for the bucket by following the [naming conventions](../../storage/concepts/bucket.md#naming).
      1. Limit the maximum bucket size, if required.

         If the value is `0`, the size is not limited. It is similar to the enabled **No limit** option.

      1. Select the `With authorization` [access type](../../storage/concepts/bucket.md#bucket-access).
      1. Select the default [storage class](../../storage/concepts/storage-class.md).
      1. Click **Create bucket**.

{% endlist %}

### Create an encryption key in Key Management Service {#create-key}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), navigate to `example-folder`.
  1. Navigate to **Key Management Service**.
  1. Click **Create key** and specify:
     * **Name**: `arcsight-kms`.
     * **Encryption algorithm**: `AES-256`.
     * Keep the default values for all other parameters.
  1. Click **Create**.

{% endlist %}

### Enable bucket encryption {#backet-encoding}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), go to the bucket created earlier.
  1. In the left-hand panel, select **Security**.
  1. Open the **Encryption** tab.
  1. In the **KMS Key** field, select `arcsight-kms`.
  1. Click **Save**.

{% endlist %}

### Create service accounts {#create-sa}

You need to create two accounts: one for a trail and one for a bucket.

Create a service account named `sa-arcsight`:

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), navigate to `example-folder`.
  1. Navigate to **Identity and Access Management**.
  1. Click **Create service account**.
  1. Enter a name for the service account according to the naming requirements:
  
       * Length: between 3 and 63 characters.
       * It can only contain lowercase Latin letters, numbers, and hyphens.
       * It must start with a letter and cannot end with a hyphen.

       For example, `sa-arcsight`.
  
  1. Click **Create**.
  
{% endlist %}

Similarly, create a service account named `sa-arcsight-bucket`.

### Create a static key {#create-access-key}

You will need the key ID and secret key when mounting the bucket.

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), navigate to `example-folder`.
  1. Navigate to **Identity and Access Management**.
  1. In the left-hand panel, select ![FaceRobot](../../_assets/console-icons/face-robot.svg) **Service accounts**.
  1. In the list that opens, select `sa-arcsight-bucket`.
  1. Click **Create new key** in the top panel.
  1. Select **Create static access key**.
  1. Enter a description for the key and click **Create**.
  1. Save the ID and the secret key.

      {% note alert %}

      After you close this dialog, the key value will no longer be available.

      {% endnote %}

- CLI {#cli}

  1. Create an access key for the `sa-arcsight-bucket` service account:

      ```bash
      yc iam access-key create --service-account-name sa-arcsight-bucket
      ```

      Result:

      ```
      access_key:
        id: aje*******k2u
        service_account_id: aje*******usm
        created_at: "2022-09-22T14:37:51Z"
        key_id: 0n8*******0YQ
      secret: JyT*******zMP1
      ```

  1. Save the ID (`key_id`) and secret key (`secret`). This is the only time you can copy this key as it will not be shown again.

{% endlist %}

## Assign roles to the service accounts {#add-roles}

Assign the `audit-trails.viewer`, `storage.uploader`, and `kms.keys.encrypterDecrypter` roles to the `sa-arcsight` service account:

{% list tabs group=instructions %}

- CLI {#cli}

  1. `audit-trails.viewer` [role](../security/index.md#roles) for the folder:
     
      ```
      yc resource-manager folder add-access-binding \
      --role audit-trails.viewer \
      --id <folder_ID> \
      --service-account-id <service_account_ID>
      ```
      
      Where:
  
      * `--role`: Role being assigned.
      * `--id`: `example-folder` ID.
      * `--service-account-id`: `sa-arcsight` service account ID.

      For more information about the `yc resource-manager folder add-access-binding` command, see the [CLI reference](../../cli/cli-ref/resource-manager/cli-ref/folder/add-access-binding.md).
  
  1. Assign the `storage.uploader` [role](../../storage/security/index.md#storage-uploader) for the folder the bucket is in:
     
      ```
      yc resource-manager folder add-access-binding \
      --role storage.uploader \
      --id <folder_ID> \
      --service-account-id <service_account_ID>
      ```
  
      Where:
  
      * `--role`: Role being assigned.
      * `--id`: `example-folder` ID.
      * `--service-account-id`: `sa-arcsight` service account ID.

  1. `kms.keys.encrypterDecrypter` [role](../../kms/security/index.md#service) for the `arcsight-kms` encryption key:
  
      ```
      yc kms symmetric-key add-access-binding \
      --role kms.keys.encrypterDecrypter \
      --id <key_ID> \
      --service-account-id <service_account_ID>
      ```
  
      Where:
  
      * `--role`: Role being assigned.
      * `--id`: ID of the `arcsight-kms` KMS key.
      * `--service-account-id`: `sa-arcsight` service account ID.

{% endlist %}

Assign the `storage.viewer` and `kms.keys.encrypterDecrypter` roles to the `sa-arcsight-bucket` service account:

{% list tabs group=instructions %}

- CLI {#cli}

  1. `storage.viewer` role for the folder:
     
      ```
      yc resource-manager folder add-access-binding \
      --id <folder_ID> \
      --role storage.viewer \
      --service-account-id <service_account_ID>
      ```

      Where:
  
      * `--id`: `example-folder` ID.
      * `--role`: Role being assigned.
      * `--service-account-id`: `sa-arcsight-bucket` service account ID.
  
  1. `kms.keys.encrypterDecrypter` role for the `arcsight-kms` encryption key:
  
      ```
      yc kms symmetric-key add-access-binding \
      --role kms.keys.encrypterDecrypter \
      --id <key_ID> \
      --service-account-id <service_account_ID>
      ```
  
      Where:
  
      * `--role`: Role being assigned.
      * `--id`: ID of the `arcsight-kms` KMS key.
      * `--service-account-id`: `sa-arcsight-bucket` service account ID.
  
{% endlist %}

## Create a trail {#create-trail}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), navigate to `example-folder`.
  1. Navigate to **Audit Trails**.
  1. Click **Create trail** and specify:

     * **Name**: Name of the trail you want to create, e.g., `arcsight-trail`.
     * **Description**: Trail description (optional).

  1. Under **Destination**, configure the destination object:

     * **Destination**: `Object Storage`.
     * **Bucket**: Bucket name.
     * **Object prefix**: Optional parameter used in the [full name](../concepts/format.md#log-file-name) of the audit log file.
  
     {% note info %}
     
     Use a [prefix](../../storage/concepts/object.md#key) to store audit logs and third-party data in the same bucket. Do not use the same prefix for logs and other bucket objects because that may cause logs and third-party objects to overwrite each other.
     
     {% endnote %}

      * **Encryption key**: Specify the `arcsight-kms` encryption key used to [encrypt](../../storage/concepts/encryption.md) the bucket.
  
  1. Under **Service account**, select `sa-arcsight`.

  1. Under **Collecting management events**, configure the collection of management event audit logs:

     * **Collecting events**: Select `Enabled`.
     * **Resource**: Select `Folder`.
     * **Folder**: Automatically populated field containing the name of the current folder.

  1. Under **Collecting data events**, select `Disabled` in the **Collecting events** field.
  1. Click **Create**.

  {% note warning %}
  
  The solution will delete the logs from the bucket after they are exported to ArcSight. If you need to keep the logs in the bucket, create a separate bucket and trail.
  
  {% endnote %}

{% endlist %}

## Mount a bucket {#mount-bucket}

A bucket is mounted on an intermediate VM where ArcSight SmartConnector is installed.
To mount the bucket, create a file with the static access key of the `sa-arcsight-bucket` service account.

1. On the intermediate VM, create a file with the static access key:

   ```bash
   echo <access_key_ID>:<secret_access_key> > ${HOME}/.passwd-s3fs
   chmod 600 ${HOME}/.passwd-s3fs
   ```

1. Install [s3fs](https://github.com/s3fs-fuse/s3fs-fuse):

   ```bash
   sudo apt install s3fs
   ```

1. Create a directory to mount the bucket to, e.g., `mybucket` in your home directory:

   ```bash
   sudo mkdir ${HOME}/mybucket
   ```

1. Mount the bucket:

   ```bash
   s3fs <bucket_name> ${HOME}/mybucket -o passwd_file=${HOME}/.passwd-s3fs -o url=https://storage.yandexcloud.net -o use_path_request_style
   ```

1. Check that the bucket was mounted:

   ```bash
   ls ${HOME}/mybucket
   ```

## Install and configure ArcSight SmartConnector {#configure-arcsight}

{% note info %}

To complete this stage of the tutorial, you need an ArcSight SmartConnector distribution and access to an ArcSight instance.

{% endnote %}

1. On the intermediate VM, [install](https://www.microfocus.com/documentation/arcsight/arcsight-smartconnectors/AS_smartconn_install/) `ArcSight SmartConnector`:
    1. When installing it, select **ArcSight FlexConnector JSON Folder Follower** and specify the path to the `mybucket` folder.
    1. Specify **JSON configuration filename prefix**: `yc`.
1. [Download](https://github.com/yandex-cloud-examples/yc-export-auditlogs-to-arcsight/tree/main/arcsight_content) the `arcsight_content` files.
1. Copy the `yc.jsonparser.properties` file from the `flex` folder to the folder with this address: `<agent_installation_folder>/current/user/agent/flexagent`.
1. Copy the `map.0.properties` file from the `flex` folder to the folder with this address: `<agent_installation_folder>/current/user/agent/map`.
1. Edit the `<agent_installation_folder>/current/user/agent.properties` file:

    ```bash
    agents[0].mode=DeleteFile
    agents[0].proccessfoldersrecursively=true
    ```

1. Start the connector and make sure that events are received by ArcSight:

   ![image](../../_assets/audit-trails/tutorials/arcsight-events.png)

## How to delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

1. [Delete](../../storage/operations/buckets/delete.md) the Object Storage bucket.
1. [Delete](../../kms/operations/key.md#delete) the Key Management Service key.
1. [Delete](../../compute/operations/vm-control/vm-delete.md) the intermediate VM if you created it in Compute Cloud.