The 20-Agent Shift: Clawing Back Resource Footprint with Bare-Metal Containers

Act III:Part 6 of the ThreatLabs CTI stack is officially in the wild, and today we’re talking about what happens when you take a good architectural idea and engineer it straight into a decentralized circus. After yanking the cords on the monolithic "God Bot" to save my execution queues, I went full-scale decentralized factory. Deployed a sprawling workforce of 23 hyper-focused micro-agents across my bare-metal Proxmox LXC containers.

The plan was beautiful on paper: complete functional decoupling, absolute role isolation, zero dropped webhooks.

Then the micro-agents started talking to each other, and the entire homelab slid into a high-latency middle-management nightmare.

The Workforce Blueprint: Decoupling the Brain

The initial design layout was divided into clean, isolated operational squads, each driven by a tight, single-purpose markdown playbook file hosted in the local Forgejo repository. No overlapping scopes, no shared variables, just pure telegraphic execution loops running across hand-me-down Intel Core hardware nodes.

We broke the 23-agent registry down into three distinct, hyper-focused divisions:

• The Intel Core Squad: Running deep algorithmic sorting, matching incoming indicators against historical datasets, and parsing TLP metrics.
• The Infra Operational Team: Micromanaging container system states, verifying network junctions, and triggering the fix-permissions.sh janitor loops.
• The Collection Sentinel Division: Continuously scraping OSINT leaks, parsing Telegram feeds, and plumbing the deep web crawling queues.

To keep things pristine, every single agent was deployed into its own isolated, resource-constrained container instance.

# Mass-provisioning the micro-agent workforce across the cti-net fabric
for AGENT_ID in {210..233}; do
  echo "Spinning up bare-metal micro-container instance ID: $AGENT_ID..."
  pct create $AGENT_ID local:vztmpl/debian-12-standard_12.2-1_amd64.tar.zst \
    -cores 1 -memory 1024 -swap 256 -features nesting=1 \
    -net0 name=eth0,bridge=vmbr0,tag=101,ip=dhcp -storage local-zfs -rootfs local-zfs:5
  pct start $AGENT_ID
done
echo "Sovereign workforce initialized. 23 containers on the grid."

Fired up the cluster, watched the containers initialize across the Proxmox VE dashboard, and it looked like a threat intelligence masterpiece.

For about forty-eight hours.

The Problem: Token Drift and Middle-Management Arguments

Then the data scaled, the context windows filled up, and the tools completely lost their collective minds.

When you create 23 individual micro-roles, your pipeline handoffs become intensely brittle. A single raw threat report dump would hit a Collection sentinel, which would extract a partial indicator string, pass it to an ingestion agent, which would then call a verification agent, which would loop back to an indexing subagent.

Because each subagent was running its own independent context window and parsing logic, small linguistic variances began compounding at every single layer of the pipeline. By the time a simple indicator packet traveled through six separate container handoffs, the original data structure had completely mutated.

Classic token drift.

Worse, the micro-agents started falling into context fragmentation traps. An Intel Core sorting agent would argue with a Collection sentinel over whether a specific string qualified as a true threat actor TTP or a standard script artifact. They were micromanaging each other's execution loops, generating massive token overhead loops, burning through system memory, and dropping actual intelligence alerts while they debated formatting parameters back into the Postgres backend logs.

The pipeline was completely bogged down by its own organizational sprawl. We didn't build a streamlined automation factory; we built a bureaucratic corporate committee inside a local Docker network.

The Realization: Breaking Under the Overhead

The breaking point was looking at the system metrics panel. The hyper-fragmented layout was chewing through processing cycles just to manage container-to-container webhook handoffs. My server processors were pinned at 90% utilization, the storage backplane was choking on internal REST API logs, and the fan noise out here was a physical symptom of pure architectural debt.

We over-engineered the engine into oblivion. Decentralization is an excellent design parameter, right up until the synchronization overhead starts eating your entire hardware compute budget.

The "Aha!" moment was recognizing that a granular 23-agent squad model is an unmitigated disaster for token stability. We needed to aggressively collapse the hierarchy, consolidate the windows, and stop the internal agent arguments before the system completely choked on its own context fragmentation.

The 23-agent sprawl trap is real, the token drift has been thoroughly mapped, and the container micro-roles are begging for a massive organizational purge. Act III is moving fast. Next week in Post 7, we are executing "The Great Collapse"—documenting the consolidation surgery where we ruthlessly collapse the sprawl of 23 micromanaging agents down to 4 elite Lead Squads running on a unified JSON envelope schema to kill off handoff overhead for good.

See ya later. Happy tinkering!