Route Traffic from Kubernetes | Twingate

Route Traffic from Kubernetes

Introduction

The purpose of this guide is to demonstrate how the Twingate headless Client can be used as a router to connect to your remote resources from within a GKE cluster.

Production Environments

The information in this article should be used as a guide only. If you are deploying this method into a production environment, we recommend that you also follow all security and configuration best practices.

Images and code samples in this guide may contain references to specific versions of software or container images that may not be the latest versions available. Please refer to the official documentation for that software or container image for the most up-to-date information.

The general idea is to use the Twingate Headless Client in a dedicated Virtual Machine that acts as a Router from the perspective of the GKE Cluster:

We can then enter the required details:

You can leave the remaining default settings and click create.

Twingate Router Setup

We must first set up the virtual machine which will run the Twingate client and also act as a router for any internal services which need to communicate to resources within Twingate. This virtual machine will run in “headless” mode and therefore require a service account to be created.

Setting up a service account

To create a service account, open your Twingate admin page, select Team and then the services tab:

Click on Create Service Account and give it a sensible name:

You should then generate a dedicated service key:

Once you generate the key, take a copy of the Key Object as this is what the headless client uses to authenticate.

Handling Keys

As this key is used to authenticate, it is important you keep this safe.

Setting up the virtual machine

Again, use the GCP console to search for ” VM instances”, then select ” Create Instance“.

Enter a suitable name and place the virtual machine in the same region you set up the subnet in the VPC previously.

VM Sizing

For this example we have used the smallest machine type; this should be reviewed for production workloads depending on throughput.

Scrolling down to Boot Disk, select change:

Change the OS to Ubuntu:

And pick the version to the x86/64 image

Then click Select to select this as your machine image.

Next, expand the networking section.

Expand the Networking section and tick the IP Forwarding checkbox:

Scroll down to Network Interfaces and delete the pre-configured interface:

Click Add Network Interface and select the network and subnet you created previously:

( The remaining settings can be changed to suit your requirements but for the purpose of this guide we will just click the create button with no added configuration.)

Configuring the virtual machine

To enable access to the virtual machine, we will need to configure a firewall rule to allow SSH, navigate back to the VPC networking section and select the VPC network you created earlier.

Scrolling to the bottom of the page, select Firewalls and then click Add firewall rule.

Create a new firewall rule with a name of your choice. We will want to change the Targets to All instances in the network, the source IP to 0.0.0.0/0 and specifying tcp port 22.

This rule will expose port 22 to the internet; if you would like to restrict this further please see this official guide.

Now we can connect to the virtual machine by going back to the VM instance screen and clicking on the SSH button.

This will pop up a new window and you should now have access to the virtual machine shell.

Installing Twingate

We will be installing the Twingate Client in “headless” mode which means that it will authenticate using the service account we created earlier. It also means that it can be run in the background without any interaction.

The following steps are taken from the official guide here. If you are having issues with these steps, please refer to the official guide.

Installation

Enter the following command in the terminal / shell:

curl https://binaries.twingate.com/client/linux/install.sh | sudo bash

After a few moments the installation will complete.

Next, we need to create the file which will be used to configure the headless client, create a new file and copy the contents of the key you saved earlier:

nano /tmp/service_key.json

Save this file.

New to Nano?

More information on using the nano editor can be found at https://linuxize.com/post/how-to-use-nano-text-editor/

We can then use this file in the Twingate setup by running the following:

sudo twingate setup --headless /tmp/service_key.json

You will see that this has completed the setup:

Twingate Setup 1.0.58.46266 | 0.129.0
Copying service key
Setup is complete.

We can now start Twingate:

sudo twingate start

You should see that it has successfully started and connected:

Starting Twingate service in headless mode
Twingate has been started
Waiting for status...
online

Next we will need to set up the virtual machine so it can route the traffic from inside the network via the new Twingate connection.

First, we need to allow ip forwarding:

sudo nano /etc/sysctl.conf

Scroll down this file and uncomment net.ipv4.ip_forward=1:

Save the file.

Then we need to run the following command to apply the changes:

sudo sysctl -p

Next we need to make changes to iptables on the virtual machine by entering the following commands:

Working with Network Interfaces

ens4 in the commands below corresponds to the name of the Network Interface on the VM itself: yours may be different. If it is, replace ens4 with the name of your own interface.

sudo iptables --append FORWARD --in-interface ens4 --out-interface sdwan0 --jump ACCEPT

sudo iptables --append FORWARD --in-interface sdwan0 --out-interface ens4 --match state --state RELATED,ESTABLISHED --jump ACCEPT

sudo iptables -t nat --append POSTROUTING --out-interface sdwan0 --jump MASQUERADE

To ensure these rules remain when the VM is restarted we can install the following:

sudo apt install iptables-persistent -y

Enter YES on both screens when they pop up.

That is all the configuration required on the router.

The remote environment

We will need a remote network and resource that we allow our headless client to communicate with. Setting up the remote network in Twingate is beyond the scope of this guide, but you can find information on how to do this here.

For the purpose of this guide, we will use a remote network with a simple application deployed with an internal IP address of 10.43.40.159:

Click the Add Access button and give the GKE-Router service access to the resource you have created.

We can now test this access by going back to the VM in GCP and running a simple CURL command:

This can take a few minutes to update the permissions so you may need to wait a few minutes for this to work.

curl 10.43.40.159

Depending on what your resource is serving may mean you need to run a different command.

Which then returns the HTML from the Ghost web site:

...
<!DOCTYPE html>
<html lang="en">
<head>

<title>GHOST BLOG</title>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="HandheldFriendly" content="True" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

<link rel="stylesheet" type="text/css" href="/assets/built/screen.css?v=4c7ae18aa5" />
...

This shows the router has access to the Twingate resource, now we can create a GKE cluster and set up access from it.

Setting up the GKE cluster

Again using the search facility in the GCP console, search for GKE.

Click on create to create a new cluster, selecting a standard cluster.

Name the cluster and make sure it is in the same region and zone as the router.

This is minimal configuration for this guide so you may want to alter other settings in the GKE cluster to suit your requirements.

Select Networking and make sure the cluster is in the network and subnet you created earlier.

Click create to start creating the new GKE cluster; this process will take a few minutes.

Adding the static route

In order for the traffic to route properly from outside of the router VM (i.e., from the Kubernetes Cluster), we need to add a static route to the Twingate Resource IP address and ensure this is sent via the router.

Navigate back to the VPC network settings page and click on routes.

Click on create route.

Enter in the relevant details for your new route; below is an example based on the configuration in this guide.

You will see that the destination IP range is the IP address used in the example above: This should be your own resource IP address or range used in the remote network.

As shown below, make sure you specify the correct network and the next hop as the VM instance created earlier.

Click the create route button.

Updating firewall rules

By default, the GKE cluster and pods will not have access to “talk” to the router VM. Therefore, we need to add firewall rules to allow this.

On the VPC network page, click on Firewall and Create Firewall Rule.

Firewall Rules

This example adds two firewall rules with access to the whole subnet. You may want to consolidate into one rule and make these rules more restrictive based on your requirements.

Inbound from the cluster IPs:

You can find the IP addresses of the GKE nodes from the VM instances page

Inbound from the pod IP range:

You can find the pod and service IP range from the GKE cluster information page:

Then, we can add these to a firewall rule:

Then click on the create button.

Testing the route from GKE

We will deploy a simple pod to the GKE cluster to allow us to test the route. First we need to access the cluster, the following command will copy the connection information and authentication to your local KUBECONFIG.

Setting up the gcloud cli and kubectl is beyond the scope of this guide.

gcloud container clusters get-credentials twingate-demo --region europe-west1-b

Check you have access to the cluster:

kubectl get pods -A

We can then create a simple pod by creating a new file called ubuntu.yaml with the following contents:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
  labels:
    app: ubuntu
spec:
  containers:
  - name: ubuntu
    image: ubuntu:22.04
    command: ["/bin/sleep", "3650d"]
    imagePullPolicy: IfNotPresent
    resources:
      limits:
        memory: "128Mi"
        cpu: "500m"
    securityContext:
      capabilities:
        add:
          - NET_ADMIN
  restartPolicy: Always

Then we can apply this to the cluster:

kubectl apply -f ubuntu.yaml

You should see the pod being created and in a running state:

NAMESPACE     NAME                                                      READY   STATUS    RESTARTS   AGE
default       ubuntu                                                    1/1     Running   0          45m

We can then access the pod with the following command:

kubectl exec -it ubuntu /bin/bash

Curl is not required for this setup

We will install it to make sure connectivity between the Pods and the Twingate Resource works.

Once inside the pod, we need to install CURL by running the following:

apt update
apt install curl -y

Once CURL is installed, we can test the connection to our remote resource:

curl 10.43.40.159

All being well, you should see a response back from the remote resource:

As mentioned previously, depending on what the remote resource is, you may need to use a different command to test connectivity.

...
<!DOCTYPE html>
<html lang="en">
<head>

<link rel="stylesheet" type="text/css" href="/assets/built/screen.css?v=4c7ae18aa5" />
...

Pod Connectivity

Any pods within your cluster should now be able to communicate with this remote resource!

Thank you for taking the time to read this guide; if you have any questions or issues when using this guide, please contact us, we appreciate any feedback.

Last updated 5 months ago.