[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Kubernetes](../index.md) > [Step-by-step guides](index.md) > Working with external nodes > Configuring IPSec gateways to connect external nodes to a cluster # Configuring IPSec gateways to connect external nodes to a cluster With Yandex Managed Service for Kubernetes, you can connect servers from outside Yandex Cloud as [Kubernetes cluster](../concepts/index.md#kubernetes-cluster) nodes. To connect one, first, set up network connectivity between the remote network hosting the external server and the [cloud network](../../vpc/concepts/network.md#network) hosting your Managed Service for Kubernetes cluster. You can do this using a VPN. Below is an example of establishing network connectivity over the IPSec protocol. You can see the solution architecture in the diagram below: ![external-nodes-connect](../../_assets/tutorials/external-nodes-connect/external-nodes-connect.svg) As you see in the diagram, the external Kubernetes cluster node and VPN client reside in a remote network outside Yandex Cloud. To facilitate testing, the solution below uses a separate Yandex Virtual Private Cloud [cloud network](../../vpc/concepts/network.md#network) as the remote site, and a Yandex Compute Cloud [VM instance](../../compute/concepts/vm.md) located in that cloud network, as the VPN client. ## Getting started {#before-you-begin} 1. [Create](../../vpc/operations/network-create.md) your main cloud network with three [subnets](../../vpc/concepts/network.md#subnet) in different availability zones. {% note warning %} Do not change or delete Virtual Private Cloud resources used by the Managed Service for Kubernetes cluster. This may cause cluster errors or make it impossible to delete later. {% endnote %} 1. In the main network, [create](kubernetes-cluster/kubernetes-cluster-create.md) a Managed Service for Kubernetes cluster with a [highly available](../concepts/index.md#master) master. To create an external node group, the Managed Service for Kubernetes cluster must operate in [tunnel mode](../concepts/network-policy.md#cilium). You can only enable this mode when creating the cluster. 1. [Install kubect](https://kubernetes.io/docs/tasks/tools/install-kubectl) and [configure it to work with the new cluster](connect/index.md#kubectl-connect). 1. In the main network, [create](../../compute/operations/vm-create/create-linux-vm.md) a Compute Cloud VM with a [public IP address](../../vpc/concepts/address.md#public-addresses); name it `VM-1`. On this VM, you will set up the main IPSec gateway. 1. [Create](../../vpc/operations/network-create.md) an additional cloud network with one subnet. 1. In the additional network, [create](../../compute/operations/vm-create/create-linux-vm.md) a Compute Cloud VM with a public IP address; name it `VM-2`. You will use this VM to set up the additional IPSec gateway. ## Configuring security groups {#sg} 1. In the main network, [create](../../vpc/operations/security-group-create.md) a security group and associate it with `VM-1`. [Add](../../vpc/operations/security-group-add-rule.md) the following rules to the group: {% list tabs group=traffic %} - Outgoing traffic {#outgoing} Description | Port range | Protocol | Destination name | CIDR blocks --- | --- | --- | --- | --- `any` | `0-65535` | `Any` | `CIDR` | `0.0.0.0/0` - Incoming traffic {#incoming} Description | Port range | Protocol | Source | CIDR blocks --- | --- | --- | --- | --- `icmp` | `0-65535` | `ICMP` | `CIDR` | `0.0.0.0/0` `ssh` | `22` | `TCP` | `CIDR` | `0.0.0.0/0` `ipsec-udp-500` | `500` | `UDP` | `CIDR` | `/32` `ipsec-udp-4500` | `4500` | `UDP` | `CIDR` | `/32` `VM-2-subnet` | `0-65535` | `Any` | `CIDR` | `` {% endlist %} 1. In the additional network, [create](../../vpc/operations/security-group-create.md) a security group and associate it with `VM-2`. [Add](../../vpc/operations/security-group-add-rule.md) the following rules to the group: {% list tabs group=traffic %} - Outgoing traffic {#outgoing} Description | Port range | Protocol | Destination name | CIDR blocks --- | --- | --- | --- | --- `any` | `0-65535` | `Any` | `CIDR` | `0.0.0.0/0` - Incoming traffic {#incoming} Description | Port range | Protocol | Source | CIDR blocks --- | --- | --- | --- | --- `icmp` | `0-65535` | `ICMP` | `CIDR` | `0.0.0.0/0` `ssh` | `22` | `TCP` | `CIDR` | `0.0.0.0/0` `ipsec-udp-500` | `500` | `UDP` | `CIDR` | `/32` `ipsec-udp-4500` | `4500` | `UDP` | `CIDR` | `/32` `k8s-VM-1-subnets` | `0-65535` | `Any` | `CIDR` | ``, ``, `<` `cluster&services` | `0-65535` | `Any` | `CIDR` | ``, `` {% endlist %} 1. [Add](../../vpc/operations/security-group-add-rule.md) the following rule to the security group of the Managed Service for Kubernetes cluster and node groups: {% list tabs group=traffic %} - Incoming traffic {#incoming} Description | Port range | Protocol | Source | CIDR blocks --- | --- | --- | --- | --- `VM-2-subnet` | `0-65535` | `Any` | `CIDR` | `` {% endlist %} ## Configuring routing {#rt} 1. Configure routing for the main IPSec gateway: 1. In the main network, [create a route table and add a static route to it](../../vpc/operations/static-route-create.md): * **Destination prefix**: Specify the CIDR of the subnet hosting `VM-2`. * **IP address**: Specify the `VM-1` internal IP address. 1. Associate the route table with all subnets in your main network. 1. Configure routing for the additional IPSec gateway: 1. In the additional network, create a route table. 1. Add a static route for the route table: * **Destination prefix**: Specify the CIDR of the subnet hosting `VM-1`. * **IP address**: Specify the `VM-2` internal IP address. Repeat this step for each subnet in your main network. 1. Associate the route table with the `VM-2` subnet. ## Setting up IPSec gateways {#gw} 1. Set up the main IPsec gateway: 1. [Connect](../../compute/operations/vm-connect/ssh.md) to `VM-1` over SSH. 1. Install strongSwan: ```bash sudo apt update && sudo apt install strongswan ``` 1. Open the `ipsec.conf` configuration file: ```bash sudo nano /etc/ipsec.conf ``` 1. Replace the file contents with this text: ```text # basic configuration config setup charondebug="all" uniqueids=yes conn VM-1 type=tunnel auto=route keyexchange=ikev2 ike=aes256-sha2_256-modp2048! esp=aes256-sha2_256! authby=secret left= leftsubnet=,, leftsourceip= leftid= right= rightsubnet= aggressive=no keyingtries=%forever ikelifetime=86400s ``` For more information about parameters, see the [strongSwan guide](https://wiki.strongswan.org/projects/strongswan/wiki/connsection). 1. Open the `ipsec.secrets` file that is used for authentication: ```bash sudo nano /etc/ipsec.secrets ``` 1. Replace the file contents with this text: ```text : PSK "" ``` Learn more about the `ipsec.secrets` file format in the [strongSwan guide](https://wiki.strongswan.org/projects/strongswan/wiki/Ipsecsecrets). 1. Set up the additional IPSec gateway: 1. [Connect](../../compute/operations/vm-connect/ssh.md) to `VM-2` over SSH. 1. Install [strongSwan](https://docs.strongswan.org/docs/latest/howtos/introduction.html): ```bash sudo apt update && sudo apt install strongswan ``` 1. Open the `ipsec.conf` configuration file: ```bash sudo nano /etc/ipsec.conf ``` 1. Replace the file contents with this text: ```text # basic configuration config setup charondebug="all" conn VM-2 type=tunnel auto=route keyexchange=ikev2 ike=aes256-sha2_256-modp2048! esp=aes256-sha2_256! authby=secret left= leftid= leftsubnet= right= rightsubnet=,, rightsourceip= aggressive=no keyingtries=%forever ikelifetime=86400s lifetime=43200s lifebytes=576000000 dpddelay=30s ``` For more information about parameters, see the [strongSwan guide](https://wiki.strongswan.org/projects/strongswan/wiki/connsection). 1. Open the `ipsec.secrets` file that is used for authentication: ```bash sudo nano /etc/ipsec.secrets ``` 1. Replace the file contents with this text: ```text : PSK "" ``` Passwords must be the same for both VMs. Learn more about the `ipsec.secrets` file format in the [strongSwan guide](https://wiki.strongswan.org/projects/strongswan/wiki/Ipsecsecrets). 1. Restart strongSwan on both VMs: ```bash sudo ipsec restart ``` 1. Check the connection status on both VMs: ```bash sudo ipsec statusall ``` You should see `ESTABLISHED` in the command output, indicating a successfully established connection: ```text ... Security Associations (1 up, 0 connecting): VM-1[1]: ESTABLISHED 5 seconds ago, 10.128.*.**[46.21.***.***]...84.201.***.***[84.201.***.***] ... ``` If the connection was not established, try doing it manually. On `VM-1`, run this command: ```bash sudo ipsec up VM-1 ``` You may run this command on just one of the VMs. 1. Configure [MTU](../../vpc/concepts/mtu-mss.md) on both VMs: ```bash ETH_NIC=eth0 sudo iptables -t mangle -A FORWARD -i ${ETH_NIC} -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1361:1536 -j TCPMSS --set-mss 1360 sudo iptables -t mangle -A FORWARD -o ${ETH_NIC} -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1361:1536 -j TCPMSS --set-mss 1360 echo "net.ipv4.ip_no_pmtu_disc = 1" | sudo tee -a /etc/sysctl.conf sudo sysctl -p /etc/sysctl.conf ``` {% note warning %} If you keep the default MTU value, you may lose network traffic. {% endnote %} 1. [Connect](external-nodes-connect.md) `VM-2` to the Managed Service for Kubernetes cluster as its external node. ## Troubleshooting {#troubleshooting} ### Error when using the docker-ce and containerd packages on an external node {#docker-containerd} Follow these steps for troubleshooting: 1. View the list of services that are not running properly: ```bash sudo systemctl --failed ``` Result: ```text UNIT LOAD ACTIVE SUB DESCRIPTION docker.socket loaded failed failed Docker Socket for the API LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. ``` 1. Check the `docker.socket` status: ```bash sudo systemctl status docker.socket ``` Result: ```text docker.socket - Docker Socket for the API Loaded: loaded (/lib/systemd/system/docker.socket; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2024-02-10 09:53:37 UTC; 6s ago Triggers: ● docker.service Listen: /run/docker.sock (Stream) CPU: 1ms Feb 10 09:53:37 ext-node systemd[1]: Starting Docker Socket for the API... Feb 10 09:53:37 ext-node systemd[7052]: docker.socket: Failed to resolve group docker: No such process Feb 10 09:53:37 ext-node systemd[1]: docker.socket: Control process exited, code=exited, status=216/GROUP Feb 10 09:53:37 ext-node systemd[1]: docker.socket: Failed with result 'exit-code'. Feb 10 09:53:37 ext-node systemd[1]: Failed to listen on Docker Socket for the API. ``` 1. Look up errors in the system logs: ```bash sudo journalctl -xe ``` Result: ```bash ... Feb 10 09:56:40 ext-node maintainer[19298]: E: Sub-process /usr/bin/dpkg returned an error code (1) ... ``` 1. Reinstall the packages and fix the errors: ```bash sudo apt install -f ``` 1. When the installer prompts you for action with the `config.toml` file, enter `N` to keep the current version of the file.