Pedro Wiezel
back to blog

is that acceptable?

what changed when I had an AI agent write a spec and ask me things before it built anything

Partway through planning a small Mac app, the agent stopped and asked me: “‘Today’ can only count traffic while the app is running. Is that acceptable?”

The app is a floating window that shows what my connection is doing, second by second, with a running total for the day. Earlier that day I'd decided that closing the window quits the app, for reasons that had nothing to do with counting. A closed app counts nothing, so “Today” would have quietly skipped every hour the window was shut.

Nothing would have flagged it. The code would have done exactly what it was told, and the total would just have come up a little short every day, for a reason nobody had chosen.

the first time around

I built my first finished app, EXIF.ER, with Claude and Codex, describing what I wanted and checking what came back. It's on the App Store, people download it, and it once passed 569 tests while giving a video the wrong date.

Looking back, the tests and the code came out of the same conversations, so the tests checked what the code did rather than what I'd meant the app to do. A lot of small choices were made that way too. When I asked for a feature, the agent didn't stop at the parts I hadn't thought about. It picked something reasonable and kept going, and once it was built, the pick looked like it had always been the plan.

writing it down first

For the Mac app I tried something stricter, tlc-spec-driven, a skill for coding agents that Felipe Rodrigues wrote for Tech Lead's Club and that I later adapted for Apple apps. The rule is no code until there's a spec, and no spec until I've approved it. My approval sits in the file as a quote, “Spec approved, defaults included. Go on to design and tasks.”, and the task list needed a second one.

The requirements use EARS, a format Rolls-Royce engineers came up with in 2009 while working through the airworthiness rules for a jet engine's control system. Mine are for a network meter. “WHILE monitoring, the system SHALL read the counted interfaces' byte counters every 1.0 s.” That's a lot of capital letters for a window with a few numbers and a graph, but it's hard to write a fuzzy sentence in that shape.

The spec also lists what the app won't do, and why. Menu bar mode is out because it “doubles the UI to polish; the floating window is the product.” And it has hard limits, under 1% CPU and 60 MB of memory while monitoring. When the agent measured an early build, SwiftUI redrawing the text every second was eating most of 1.21% CPU. It suggested drawing the numbers with Core Animation instead, measured that at 0.30 to 0.50%, and asked whether to switch. I said yes. Without a number written down, I doubt either of us would have gone looking.

questions instead of surprises

Whenever the agent hit a product question, it asked me before building anything, and my answer went into a decision log next to the reason and the cost. There are twenty-three of them so far.

A couple were big. The speed test uses networkQuality, the command-line tool that comes with macOS, and neither of us knew whether a sandboxed Mac App Store app is even allowed to run it. So we picked the fallback first, “Ship live-only”, and then had a throwaway app try. It could.

Most were tiny. My design gave the window a green zoom button, but the window can't zoom, and macOS draws a control you can't use in gray, so the agent asked which I wanted. Same with the ⋯ menu, which I'd drawn lined up under its button while macOS opens it to the right. None of these would have broken anything. They'd just have been settled by whoever happened to be typing.

checking the tests

Passing tests had fooled me once already, so this time there was one more step. A verifier makes small, deliberate changes to the code and sees whether any test notices. On the first version it made three, and none of them were caught. One turned .test(enabled: online) into .test(enabled: true), so the speed test button stayed clickable with no connection, and every test still passed. The fixes added the checks that were missing and moved that button's rule to where the tests could reach it.

The skill wants the verifier to be a different agent from the one that wrote the code. I don't let agents start other agents, so it ran in a fresh session instead, and the record says “self-verified (degraded)”. That's weaker than the skill intends, but at least it's written down.

changing my mind

Four days after I approved that footnote, I changed my mind. “Today” now counts traffic from while the app was closed, by reading the system's counters the next time it opens, unless the Mac restarted in between. The spec had also ruled out a history of speed tests, and history went into the first version anyway, in a window of its own. Both changes are new entries in the log that point at the ones they replace, with my reasons in my own words.

So the spec didn't freeze anything. The app kept changing for the whole build, and every one of those changes has a date and an answer of mine next to it.

The app isn't out yet, and the footnote I approved is gone from it. The question that started all this is still in the log, as AD-004, marked superseded.