Delegation Studies · Study 01
Live StudyWhen should AI stop asking?
An AI agent pays a small design studio’s freelancer invoices. It checks each one against the agreed terms, and once it has a track record, it proposes paying without asking. Only the owner can grant that authority, and they can limit or take it back at any time. I designed and built the experience in code, then tested real AI models from three providers on it.
The models made the right decision. Their explanations were the problem.
When a trusted freelancer’s invoice arrived with new bank details, the models held it: the right call. But in some runs their notes called it a scam, or offered to change the bank account themselves, which only the owner can do. So in the design, the owner’s own rules decide, the model only flags what it noticed, and every message the owner reads is written by the product, never improvised by the model.
- 14 early test runs that changed the design
- 90 recorded runs that confirmed the pattern
- 3 AI providers tested
My role: framing, interaction and visual design, and model-behavior testing. Built with AI-assisted development.
Elsewhere in the study, recorded evidence appears where AI judgment matters most: “Noticed by 3 models in 15 of 15 runs.”
