Tightening the wire: encrypted agent-to-agent comms with SSH tunnels

We run a fleet of autonomous AI agents. Each one is a Hermes agent with its own identity, tools, and memory. They communicate using a peer-to-peer protocol called A2A (Agent-to-Agent). Orders, reports, and bearer tokens travel between them over plain HTTP. On a LAN behind a firewall, that's arguably fine. But fine isn't how you build something you intend to sell to law firms.

We had three agents to wire up across two hosts. Each listens on a fixed local port for inbound messages from peers. By default, those listeners were bound to 0.0.0.0, accessible to anything that could reach the port. Not ideal.

The fix uses exactly one tool that was already in the stack: the fleet SSH key. Zero new software, zero new secrets to manage, nothing to configure at the application layer.

The hardware

Every agent carries a single fleet-wide SSH key. This key gives us passwordless root access to every host in the fleet. It's the skeleton key: one master secret, locked in a 600-perm file, never shared. If you lose it, you rotate every host. That's the trade you make when you choose a single key over per-host keys: simpler mesh, louder alarm if it leaks.

The architecture is a star: one host (the hub) initiates SSH connections to every other host in the fleet, holding loopback port forwards open. The agents themselves never connect outward. They bind to 127.0.0.1 and wait for traffic arriving through the tunnel.

Link-local forwards (-L) bring remote ports to the hub. Reverse forwards (-R) punch return paths back so the remote host can reach agents on the hub. Every binding is to 127.0.0.1: nothing listens on a routable address.

The systemd unit

The whole thing is a single systemd service with Restart=always. If the SSH session drops (network blip, host reboot), systemd reconnects it within seconds. The forwards are restored atomically because SSH refuses to start unless every forward succeeds.

ExecStart=/usr/bin/ssh -N -T \
  -o BatchMode=yes \
  -o ServerAliveInterval=15 \
  -o ServerAliveCountMax=3 \
  -o ExitOnForwardFailure=yes \
  -L 127.0.0.1:PORT_X:127.0.0.1:PORT_A \
  -R 127.0.0.1:PORT_Y:127.0.0.1:PORT_A \
  -R 127.0.0.1:PORT_Z:127.0.0.1:PORT_B \
  remote-host

Let's walk what each flag does:

• -N: no remote commands; the session exists solely for port forwarding. No shell, no stdin, no overhead.
• -T: disable pseudo-terminal allocation. A pipe, not a terminal session.
• BatchMode=yes: never prompt for passwords. If the key doesn't authenticate, fail immediately. No hanging, no interactive prompts.
• ServerAliveInterval=15 and CountMax=3: send a keepalive every 15 seconds. If three consecutive keepalives go unanswered (45 seconds of silence), SSH tears down the connection and systemd restarts it.
• ExitOnForwardFailure=yes: if the remote end refuses a forward (port already bound, wrong address), the entire SSH session fails. No partial connectivity. You either get the full mesh or you don't.
• -L: link-local forward. Traffic to 127.0.0.1:PORT_X on the hub is forwarded through the SSH tunnel to 127.0.0.1:PORT_A on the remote host.
• -R: remote forward. The reverse: traffic to 127.0.0.1:PORT_Y on the remote host arrives at 127.0.0.1:PORT_A on the hub.

Identity and trust at the application layer

What about the messages themselves? Every A2A message carries a bearer token that identifies the sender. The receiving agent maps that token to a trusted peer name from an environment variable:

# ~/.hermes/.env
A2A_PEER_TOKENS=<peer1>:<token>,<peer2>:<token>,<peer3>:<token>

When one agent sends a message to another, the receiver's A2A adapter reads the token, looks it up in A2A_PEER_TOKENS, and if it matches, the message is attributed to a named trusted crew member. If the token doesn't match any peer, the message is treated as untrusted input and the agent refuses to act on it.

This is a two-layer security model:

• Transport layer (SSH tunnel): encrypts traffic between hosts, authenticates both ends via the fleet SSH key.
• Application layer (bearer tokens): identifies individual agents within the trust boundary, prevents a compromised host from impersonating a peer.

Both layers are independent. Breaking one does not break the other. An attacker who steals the fleet key can reach the tunnel port, but still cannot send a message that one agent will treat as coming from another. They'd need the bearer token too.

The fleet certificate

The fleet SSH key is an Ed25519 keypair, generated once and distributed to every host in the fleet. It lives at ~/.ssh/fleet on every machine, permissions 600, owned by the local user that runs the agent gateway.

Adding a new host means appending the public half to ~/.ssh/authorized_keys on the target. One line. The private half never touches the new host's disk. It only exists on the hub (the initiating side). The remote host only needs the public half to authenticate inbound connections.

The agent read from that key is a step we explicitly own. No cloud secret store, no SSH CA, no third-party authentication provider. The key was generated by the operator, distributed by the operator, and if it needs to be rotated, the operator rotates it everywhere. That's the model we're building toward: the operator owns the infrastructure, the customer just uses the appliance.

ssh-keygen -t ed25519 -f ~/.ssh/fleet -C "hermes-fleet"

Verify the result with ssh-keygen -lf ~/.ssh/fleet.pub before distributing it, and keep the fingerprint out of anything public. A published fingerprint is a label for an attacker: it tells them which key to hunt for and confirms when they've found it. Ours stays in our own records.

What we didn't do

We didn't install WireGuard. We didn't set up Tailscale. We didn't configure mTLS or generate per-service certificates. We didn't add a sidecar proxy. All of those would work, and all of them would also be more software to update, more keys to manage, more surface area to audit.

The SSH tunnel approach is deliberately primitive. SSH is already on every Linux and macOS host. The fleet key already exists for remote administration. We reused the authentication layer we already had and tightened the bindings from 0.0.0.0 to 127.0.0.1. That's the whole change.

Lessons learned

• A single systemd unit for the whole mesh is fragile but teachable. One unit, one SSH session, one point of failure. If that session drops, every cross-host A2A link vanishes at once. The flip side: you can tell in one glance whether the mesh is healthy (systemctl is-active hermes-a2a-tunnel). For three agents on two hosts, simplicity wins over resilience engineering.
• Restartability matters. The first version of the tunnel unit used Restart=on-failure. A network blip triggered a reconnection that survived 24 hours fine, but on-failure didn't cover the case where systemd killed the process for something else. Restart=always is the correct answer for a tunnel whose job is to exist.
• Bind tight, not wide. The tunnel ports are bound to 127.0.0.1 explicitly: not 0.0.0.0, not localhost by convention. A 0.0.0.0-bound tunnel port on the hub means every container and every bridge interface on that host can reach the tunnel. 127.0.0.1 restricts it to processes running on the host itself.
• The A2A adapter re-reads peer tokens per message. No gateway restart needed to add a new peer. Set the token, verify the env loaded (pgrep -a hermes | head -1 to confirm the environment), and the next inbound message from that peer is recognized immediately.