Connector Logging | Twingate

Connector Logging

Different types of logging available for Twingate Connectors.

Overview

This guide describes the different types of logging available for Twingate Connectors, how to enable them, and how to export, in real-time, this data to certain third-party tools.

To export historical data instead, see Exporting network traffic.

Service Logging

The Twingate Connector can output several levels of local service logging:

Level 3 (default) - Basic connectivity and ERROR messaging
Level 4 - WARN level messaging
Level 5 - INFO level messaging
Level 7 - DEBUG level messaging

Each level includes the messages logged by lower levels, so at level 7 you will see ERROR, WARN, INFO, and DEBUG messages. Level 7 is also what you should use if you want to try to diagnose a connectivity or configuration issue with a Connector.

The logging level is changed by updating /etc/twingate/connector.conf or changing the Docker configuration to include an environment variable, set to the following:

TWINGATE_LOG_LEVEL=3

Log level 3 is the default; change it to whichever level you want output in the logs.

Log level 7 is very verbose. If there are issues with log storage capacity on the host, we do not recommend using level 7 for a long duration.

Enabling real-time traffic logging

Real-time traffic logs are controlled by setting an environment variable before the Connector starts. If the environment variable TWINGATE_LOG_ANALYTICS is set to v3, the Connector will output network connection logs (described below in detail).

In all cases, local connection logs will be output to stdout in single-line JSON format. This allows log collection by SIEM platforms such as AWS CloudWatch, Datadog, Splunk, Promtail/Loki, etc.

As the logs will show both service and traffic logging mixed together, you will need to filter for lines starting with ANALYTICS, which will be followed by the JSON formatted traffic log detail.

Docker

If you are using Docker, you can add --env TWINGATE_LOG_ANALYTICS="v3" to the Docker run command to enable local network connection logs.

K8s Helm Chart

The official Twingate Helm Chart includes a configurable parameter env which can be used to set additional environment variables, including TWINGATE_LOG_ANALYTICS="v3" to enable real-time connection logs.

More information about this and other configurable parameters can be found in our Helm Chart’s README.

systemd

If you are using our systemd Connector, you can add the line TWINGATE_LOG_ANALYTICS=v3 to the file /etc/twingate/connector.conf.

Local connection logs can be read using journalctl. For example, the following command will output the most recent 100 logging lines and follow any new output:

journalctl -u twingate-connector -n 100 -f

Single-line JSON output schema (v3)

Connection logs are output as single-line JSON objects, making them straightforward to ingest and parse by SIEM and other log aggregation products. Each network connection line starts with ANALYTICS followed by a single JSON object.

An example log line is shown below:

ANALYTICS {"connection": {"cbct_freshness": -97, "client_ip": "192.0.2.0", "duration": 3034753, "id": "e755ba99988db558-24", "protocol": "tcp", "resource_ip": "1.2.3.4", "resource_port": 443, "rx": 234867, "tunnel_path": "direct", "tunnel_proto": "quic/udp", "tx": 23363}, "connector": {"id": "84014", "name": "nondescript-caterpillar"}, "device": {"id": "200903"}, "event_type": "closed_connection", "location": "{\"geoip\":{\"city\":\"Seattle\",\"country\":\"US\",\"lat\":47.6062,\"lon\":-122.3321,\"region\":\"WA\"}}}", "relays": [], "remote_network": {"id": "6938", "name": "AWS Network"}, "resource": {"address": "app.website.com", "applied_rule": "*.website.com", "id": "2255492"}, "timestamp": 1698356150045, "user": {"email": "user@twingate.com", "id": "113256"}}

For clarity, the JSON object in the log line above is formatted below for readability:

{
  "connection": {
    "cbct_freshness": -97,
    "client_ip": "192.0.2.0",
    "duration": 3034753,
    "id": "e755ba99988db558-24",
    "protocol": "tcp",
    "resource_ip": "1.2.3.4",
    "resource_port": 443,
    "rx": 234867,
    "tunnel_path": "direct",
    "tunnel_proto": "quic/udp",
    "tx": 23363
  },
  "connector": {
    "id": "84014",
    "name": "nondescript-caterpillar"
  },
  "device": {
    "id": "200903"
  },
  "event_type": "closed_connection",
  "location": "{\"geoip\":{\"city\":\"Seattle\",\"country\":\"US\",\"lat\":47.6062,\"lon\":-122.3321,\"region\":\"WA\"}}}",
  "relays": [],
  "remote_network": {
    "id": "6938",
    "name": "AWS Network"
  },
  "resource": {
    "address": "app.website.com",
    "applied_rule": "*.website.com",
    "id": "2255492"
  },
  "timestamp": 1698356150045,
  "user": {
    "email": "user@twingate.com",
    "id": "113256"
  }
}

Notes on the connection schema

connection.id is common to events for the same network connection. In success scenarios, this will correspond to a log with an event_type of established_connection and another with an event_type of closed_connection. Error states will not have a corresponding closed_connection event type.
connection.client_ip corresponds to the IP address of the client as visible from the internet. This usually means that this is an Internet-facing NAT address.
connection.resource_ip will always be the private IP address of the resource being accessed. For resources defined by a DNS address in Twingate, connection.resource_ip will be the local address as resolved by the Connector. See How DNS Works with Twingate for more information on how we handle DNS and local address translation.
connection.rx and connection.tx represent bytes that have been received and transmitted during the lifetime of the connection.
device.id is an internal ID that Twingate uses to identify the user’s device. This may not correspond to an ID presented by the device itself depending on the operating system. We will be standardizing this device ID in a future logging schema version.
resource.address is the address of the resource as defined in the Twingate Admin console. If this is a DNS address, it will appear as defined here, with the resolved IP address logged in the field connection.resource_ip.
location is a stringified JSON field that includes geolocation information about the user device IP address.

SIEM integration

Most SIEM solutions will support filtering and parsing via grok patterns or similar. The following example is a Vector configuration for filtering and parsing real-time connection logs.

[sources.twingate_connector]
type = "journald"
current_boot_only = true
include_units = ["twingate-connector"]

[transforms.tg_analytics_filter]
type = "filter"
inputs = ["twingate_connector"]
condition = """starts_with!(.message, "ANALYTICS")"""

[transforms.tg_analytics_transform]
type = "remap"
inputs = ["tg_analytics_filter"]
source = """.message = parse_json!(parse_grok!(.message, "ANALYTICS%{SPACE}%{GREEDYDATA:json_event}").json_event)"""
drop_on_abort = true