Zero-Spend Concurrency Auditing for Agent Sandboxes

High-concurrency sandbox systems can leak file descriptors, sockets, or child processes across repeated container lifecycles. Those failures are difficult to diagnose when every stress run also incurs live-model cost.

This article describes zero-spend concurrency auditing: replacing live-model calls with deterministic mocks so a harness can exercise container and proxy cleanup cheaply. The seven listed attempts found no net file-descriptor increase under their reported conditions. They do not prove that the system is leak-free under delayed cleanup, child-process failures, or live transport behavior.

1. Zero-Spend Diagnostic Protocol#

Our diagnostic protocol explicitly isolates the physical sandbox resource behavior from the actual LLM API intelligence. We realized we don’t need real intelligence to test infrastructure stability; we just need realistic compute loads.

Zero-Spend Concurrency Leak Diagnostic Protocol

Diagnostic Design#

Mock Harness Injection: We systematically replace live model API calls with deterministic, zero-latency mock handlers. These mock models return hardcoded, synthetic code edits and test execution commands instantly, perfectly simulating agent behavior and container I/O without hitting paid APIs like OpenAI or Anthropic.
High-Concurrency Stress Injection: The orchestrator spawns up to 500 parallel worker threads executing rapid docker exec bursts, file writes, and module imports. This intentionally high concurrency is designed to surface edge-case cleanup races.
Pre/Post File Descriptor Tracking: The testing framework drops down to the OS level and reads /proc/self/fd on the host control plane immediately before and after each stress run to measure exact open file descriptor counts. If the count goes up after all agent containers have finished and supposedly terminated, a leak definitively exists.

2. Empirical Diagnostic Sweep Results#

We executed 7 separate zero-spend diagnostic attempts across varying concurrency stress levels and trigger hypotheses. We specifically targeted architectural areas known to historically cause leaks: idle container executions, UDS socket proxies, and rapid container restart loops.

Attempt	Trigger Hypothesis	Concurrency Threads	Open FDs (Before)	Open FDs (After)	FD Leak Detected?	API Cost ($USD)
`Attempt 1`	`docker_exec_idle`	10 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 2`	`high_concurrency_socket`	50 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 3`	`module_import_loop`	50 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 4`	`uds_proxy_recycle`	100 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 5`	`container_restart_burst`	100 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 6`	`module_import_concurrent`	200 threads	24 FDs	24 FDs	`false`	$0.0
`Attempt 7`	`zero_spend_stress_exhaust`	500 threads	24 FDs	24 FDs	`false`	$0.0

Source dataset: ./data/fd_leak_audit_attempts.csv.

7 Zero-Spend Diagnostic Attempts Concurrency Stress vs Open FDs

What the diagnostic rows show#

No net FD increase in seven attempts: The listed runs begin and end at 24 FDs, including the 500-thread case. Record a high-water mark and a delayed post-cleanup sample before interpreting that as a no-leak result.
Pre-terminalization audit hypothesis: Cleanup checks before terminalization may make failures easier to attribute. The current rows do not isolate that change from other lifecycle behavior.

3. Practical Takeaways for Infrastructure Engineers#

Use Mock Engines for Resource Stress Testing: Never spend live LLM API credits to debug container concurrency, socket leaks, or OS file handle exhaustion. Mock out the LLM provider to isolate infrastructure bugs from model behavior and save thousands of dollars in development costs.
Track /proc/self/fd Programmatically: Do not rely on manual lsof terminal debugging when hunting leaks. Assert file descriptor count stability automatically before and after test execution in your CI/CD integration test suites. Let the tests catch the leaks.
Audit Security & Leaks Before Terminalization: Run your resource cleanup checks strictly prior to setting run terminalization states in the database. Once an agent run is marked “complete” and handed back to the user, all its associated ephemeral infrastructure must be mathematically proven to be destroyed.