Sign in Subscribe
Headscale

Hardening the Stack: Dynamic Machine Identity Injection Over Brittle Plaintext .env Files

Jamz Yaneza

• 3 min read

Lessons learned? There's more where that came from. The problem here is that whatever is git committed is also getting deployed ....

With the shared network provisioned and the database containers finally agreeing on port numbers, it was time to talk about security debt. The initial lab config was naked. Standard plaintext credentials sitting in stale text files on disk, and static OpenVPN keys that broke the second a client IP shifted. It felt wrong.

Time to move from simple, vulnerable configs to a zero-trust architecture without drowning the lab in an absolute nightmare of iptables firewall rules.

Pitfall 6: The Blind CI/CD Overwrite

The expectation was standard DevOps laziness: "CI/CD will make our lives easier, every commit automatically deploys". Wrote some basic GitHub Action runner playbooks to push changes automatically straight to the production root.

The problem? The runner was blind. It didn't care about git tracking boundaries. I'd fix a database password locally on an experimental test branch, commit a minor markdown change, and—poof—the runner would push the experimental branch directly to the production root, overwriting our working infrastructure config with a templated draft. We were fighting ourselves.

The discovery: Environment Isolation. The runner needed to be context-aware. Wrote a branch-aware staging workaround that splits development into a separate sandbox root (/opt/cti-dev) while production is strictly guarded behind a main branch trigger.

The Workaround: Branch-Aware Deployment

#!/bin/bash
# deploy-cti.sh: Context-aware pipeline routing logic
TARGET_BRANCH="${GITHUB_REF##*/}"

if [ "$TARGET_BRANCH" == "main" ]; then
    echo "Routing to production baseline conduit..."
    DEPLOY_ROOT="/opt/stacks/cti-prod"
else
    echo "Sandbox branch detected ($TARGET_BRANCH). Routing to staging sandbox..."
    DEPLOY_ROOT="/opt/cti-dev"
fi

mkdir -p $DEPLOY_ROOT
rsync -avz --exclude='.git' ./config/ $DEPLOY_ROOT/
cd $DEPLOY_ROOT && docker compose up -d --remove-orphans
echo "Deployment synchronized at $DEPLOY_ROOT."

Sandbox problem solved. Staging happens in the sandbox, production is guarded.

Pitfall 7: The Static VPN Tracking Debt

Remote access was the next brittle debt wall. Running standard OpenVPN with static profiles meant port-forwarding and constant dynamic IP tracking. One IP change on a client device broke the routing tables entirely, locking out the n8n automation ingress while I was away.

The drama: Teleport vs. Headscale. Teleport is incredibly shiny, but Headscale is lean, open-source, completely self-hosted, and fits the Proxmox DNA flawlessly.

The insight: The mesh network model. Re-routing our coordinator automation traffic through a dedicated Headscale node on LXC 137 allowed the stack to bridge from VLAN 107 (IoT) to VLAN 101 (CTI) via authenticated identity rules, not brittle network firewall parameters.

The Blueprint

graph LR subgraph Sovereign Mesh Client[Remote Client Device] -->|OIDC Identity Auth| HS[Headscale VPN LXC 137] HS -->|Secure Tunnel| CTI[CTI Stack Space VLAN 101] end

Zero-Trust isn't just an enterprise buzzword; it's the only way to scale orchestration access without losing your mind in a tangled mess of static routing tables.

Pitfall 8: Stale Text Files and plain-text .env Leaks

The final frontier for the week: the public repository trap. "The repository is public, my .env files are private—how do I bridge them without committing a credential to source control by mistake?". Worse, managing 10+ identical plaintext configuration files across different Proxmox nodes was a certified recipe for sync failures.

The breakthrough: Machine Identities. Wrote off text file management entirely, deployed a self-hosted Infisical instance, and moved directly to runtime environment injection.

The gold standard of server hardening is never writing a password to a physical host disk at all—securely authenticating the node instance itself via token-free local identity verification to pull what it needs at initialization.

# Verify the Infisical machine identity handle token offline
infisical login --method=universal-auth \
  --client-id=$INFISICAL_CLIENT_ID \
  --client-secret=$INFISICAL_CLIENT_SECRET

# Inject secrets straight into the container memory footprint at startup
infisical run --env=production -- docker compose up -d

And profit! Plaintext configurations completely expunged from the file system.

The stack is officially hardened, the automatic deployments are context-aware, and the plain text credentials have been wiped from the drive matrix. Act II is locked down. Next up in Post 4, we're building the enterprise SOAR bridge and migrating our n8n automation cluster database directly onto a PostgreSQL transactional storage tier to stop database lockouts under heavy thread loads.

See ya later. Happy tinkering!