Not faster, just more patient: pairing with an AI agent on infrastructure work
“Point an agent in a direction and it will check things thoroughly. It just can’t tell you which direction to look in.”
You may have seen Martin Lewis talking about a new Small Ideas Initiative. We built the website that helps source the ideas. In carrying out the prep work, we paired with an AI agent. Here’s how it helped, and what we found it was good and less good at.
What we did
We put Your Priorities — an open-source citizen participation platform — onto Dalmatian v2, our new hosting infrastructure at dxw. Working alongside an AI coding agent.
This was the first real application we’d deployed on v2. Everything before it had been a static site, or something we’d already proven on v1. So there were 2 things being tested at once: whether the app would run, and whether the platform was ready for something with moving parts.
The answer to both was, mostly, “yes”. We ended up with a working staging environment – app and worker running, WebSockets routed through CloudFront, media uploads landing in a private S3 bucket, and nightly AMI refreshes in place.
It cost about $140 in AI tokens, spread over a day and a half. For that we also got 2 bug fixes merged upstream, 6 problems identified in our own Terraform module, and a set of planning documents for the next environment.
Our rough estimate is that this saved several days of senior engineering time. But the reason it saved that time is not the reason you might expect.
Patience over speed
The saving didn’t come from the agent’s ability to work quickly, but from the fact that it was genuinely good at the slow, patient tasks that a tired human might be inclined to rush.
Reading a Terraform module line by line, without skimming. Tracing why a plan produced an unexpected change, instead of shrugging and re-running it. Bisecting a one-character bug by writing a test script to prove the cause, rather than guessing at the likely culprit and hoping.
These are not glamorous tasks. They’re the ones that get compressed when someone is 3 hours into a debugging session and wants to be finished. They’re also, very often, where the real problem is hiding.
The agent didn’t get bored. It didn’t develop a hunch and then go looking for evidence to support it. Asked to check something, it checked it — and when the check came back inconclusive, it said so. At one point it correctly pointed out that a “healthy” status didn’t mean the database was actually working. That’s the kind of small, unglamorous scepticism that saves a day later on.
What that patience turned up
6 issues in our own Dalmatian v2 Terraform:
- A missing Redis egress rule. The module opens ingress to ElastiCache but never opens egress from the ECS instances, so Redis is unreachable until you add a rule yourself.
- An encryption setting you can’t know in advance. transit_encryption_enabled shows as “known after apply”, so you don’t find out whether to use redis:// or rediss:// until it’s too late to plan around it.
- An instance-refresh Lambda that already existed, switched off. It does exactly what we needed. It also hardcodes x86_64, so it breaks on Graviton, and it silently can’t run if max_size equals min_size — which is the default.
- A WAF that can’t restrict access. It’s built as an allow-list on top of default-allow, so it can rate-limit but can’t make an environment private. v1 could do this with authenticated access at the edge. That didn’t carry over.
- A logic bug in the S3 log-delivery Terraform. An && and a || are combined in the wrong order, so the ACL can be created even when the bucket doesn’t exist.
- No wildcard subdomain support. Your Priorities needs a subdomain per community, but v2 hardcodes one hostname per environment — despite already generating the wildcard certificate that would make this straightforward.
And on the application side, a similar collection of small, undocumented obstacles: npm ci broken on Linux because of a macOS-only dependency (now fixed upstream), no Dockerfile that actually built the app (we wrote one), a seeding script that silently wipes and recreates every table, an app that won’t boot at all without S3 configured, and a database password containing a slash that broke the connection string with no useful error.
None of these are hard problems. Every one of them is the kind of thing that eats an afternoon.
Where the agent was weak
The AI agent was much less good at knowing what it didn’t know.
It made several confident claims that were simply wrong: that a file was missing when it wasn’t, that a change would trigger an instance refresh when it wouldn’t. It got document dates wrong, and a config value wrong. Most significantly, it designed a security approach that was worse than one we’d already built for v1 — not because the design was unreasonable, but because it had no idea the earlier work existed.
The pattern is clear enough. Point an agent in a direction and it will check things thoroughly. It just can’t tell you which direction to look in.
The single most valuable moment in the whole exercise was a colleague asking, “don’t we already do this in v1?” That question surfaced the better solution, when the agent’s own research never would have.
Keeping a human in the lead for every actual infrastructure command helped for the same reason. It forced the agent to explain what it expected to happen before it happened. Several of its mistakes surfaced in exactly that gap between prediction and reality.
What’s next
The work isn’t finished. We need to close off public access to staging, which the WAF fix alone won’t achieve. We need wildcard domain support before communities can self-serve. The Terraform issues above need fixing, and the health check logging needs quietening down. We’re also considering a switch from Redis to Valkey for a saving of around 20% — mostly straightforward, though untested with our queue library.
But the more useful takeaway is about how we’re approaching this work, not what we shipped.
If you’re weighing up where an AI agent fits into infrastructure work, we’d suggest the answer isn’t “the parts that take a long time”. It’s the parts that require sustained attention and offer no reward for it: careful reading, patient verification, and proving rather than assuming. That’s genuinely valuable, and it’s genuinely different from being fast.
The judgement about where to look is still ours.