From Prompt to Proof: Rebuilding Agent Studio Around Working Software

From Prompt to Proof: Rebuilding Agent Studio Around Working Software

Typing an agent idea into a box is easy. Producing a public, useful product that survives a real click is the hard part.

That distinction drove the latest Agent Studio work at a2a cloud. We did not want another builder that stopped when source code existed, a container started, or a URL returned HTTP 200. We wanted a system that could take a plain-language idea all the way to visible proof: a working frontend, a callable backend, a real result, a downloadable artifact, a persisted receipt, and a public distribution page.

This is the story of what broke, what we changed, and how we proved the new system in production.

The original failure mode: “deployed” was not the same as “working”

Our first production batch exposed several uncomfortable truths.

Two agents built and deployed, but automatic publishing failed because the Studio job token did not have the exact visibility permission required by the public-agent endpoint. A third agent exhausted its $10 build allocation before it reached deployment. The browser test reported success for the two deployed agents, yet one saved screenshot showed the button still saying “Building dashboard...” and the result panel still saying “Ready for the first run.”

Technically, the page was visible. Practically, the product had not demonstrated an outcome.

We found similar gaps elsewhere:

• A generated document existed in object storage, but its download response pointed users toward an internal storage URL.
• Public pages did not consistently make the live frontend obvious.
• The account policy correctly advertised three platform-funded calls followed by BYOK, but the MCP runtime had no platform-trial model configured, so a new account immediately received an LLM-key error.
• Retrying the idea factory could create duplicate jobs because the request did not carry a stable idempotency key.
• A low-budget build could spend too much time exploring and refining before it shipped anything.

These were not edge cases hidden in a test suite. They were failures found by using the landing page and the generated products like a customer.

The new contract: Studio must finish at proof

We changed the definition of “done.”

Agent Studio now builds a structured launch plan before it builds the agent. That plan describes the product recipe, expected outputs, required capabilities, browser journeys, account-access policy, and distribution package. The builder is no longer asked only to create an agent. It is asked to create a product with specific observable evidence.

The most important acceptance change is deliberately simple: the primary browser journey no longer passes when the result panel merely exists. It waits for:

[data-testid="agent-result"][data-state="success"]

Generated frontends must expose idle, loading, success, and error states. A loading placeholder cannot masquerade as a completed run.

The browser runner can now exercise the product instead of only looking at it. Its bounded actions include:

• filling form fields;
• attaching a real in-memory file;
• clicking the primary action;
• waiting for a completed result;
• asserting text and visibility;
• using Playwright's real download event and checking the suggested filename;
• collecting console and page errors; and
• saving a full-page screenshot.

This turns browser automation into product evidence.

Outputs became first-class

Agents are useful because of what they produce, so output expectations are now part of the build brief.

A recipe can require structured JSON, a table, a preview, an artifact, or a named download. Studio checks those expectations against the deployed agent. Generated frontends render the result appropriate to the output instead of dumping an opaque response.

We also fixed the Agent API download path. The control plane now reads the requested object and streams its bytes with the correct media type and Content-Disposition header. It no longer hands the caller an internal object-store location.

In the final production check, a document agent wrote a Markdown artifact to the caller's S3-backed workspace. The authenticated download returned HTTP 200, an attachment filename, text/markdown, and 1,881 bytes. The first line was the expected owner-update heading.

Low-budget builds now ship first

The $30 homepage budget is divided into three $10 build slots when the idea factory launches its top three agents. That constraint forced us to make the builder more disciplined.

For builds at or below $10, Agent Builder now follows a lean path:

1. scaffold once;
2. make one focused implementation pass;
3. run one sandbox test;
4. allow at most one repair pass; and
5. deploy.

It prefers deterministic product logic when an LLM is not required for the customer outcome. The goal is a useful first product, not an endlessly polished unpublished repository.

That change repaired the failed third slot: a People-team shared-inbox assistant. The new run built, sandbox-tested, deployed, published, and passed acceptance in one iteration under the same $10 allocation.

Account-funded trials now work end to end

Public agents can declare a new account-access primitive:

• an a2a cloud account is required;
• the platform funds a specified number of skill calls; and
• after the trial, the user supplies their own model key.

For this launch, each public agent received three platform-funded calls followed by BYOK.

The policy was already visible in the Agent Card, but the runtime path exposed a missing production setting: the control-plane processes that mint grants did not advertise an allowed platform model. We added gpt-5.5 to every production runtime that can resolve or forward an account trial.

Then we proved it with a completely new account that had no saved model key:

• call one generated a 2,079-byte document and persisted its receipt;
• call two did the same;
• call three did the same;
• call four was rejected with the BYOK setup URL; and
• the account-access endpoint reported 3 used, 0 remaining, and has_byok: false.

That is the intended commercial boundary expressed as a platform primitive rather than custom agent code.

The idea factory became a portfolio generator

Agent Studio can now turn one theme into exactly 100 ranked mini-startup ideas. Every idea includes:

• a target audience;
• a concrete goal;
• one of eight product recipes;
• usefulness, recurrence, monetization, platform-fit, and buildability scores; and
• a proposed MCP tool set.

The current recipes cover approval workflows, calculators, CSV tools, dashboards, document generators, email assistants, scheduled monitors, and custom products.

The production proof returned 100 unique IDs, 100 unique names, a complete rank sequence from 1 through 100, scores for all 100 ideas, and MCP tools for all 100. With building enabled, Studio selects the top three, allocates the $30 budget, creates public agents, and applies the three-call account trial.

Stable idempotency keys now keep a retried factory request tied to the same batch instead of silently creating duplicate work.

Public distribution is part of the build

A working agent should not be hidden behind an API hostname.

Studio's distribution report now checks the complete package:

• public agent page;
• prominent “Open web app” action;
• live frontend;
• Agent Card;
• MCP endpoint and tool count;
• CLI invocation;
• source repository; and
• verified status.

The three production agents are live:

• Agency Ops Dashboard
• People Inbox Assistant
• Sales Ops Dashboard

We reran strict browser checks against all three. Each moved from idle to success, rendered a populated result, produced its expected download, and emitted zero console or page errors.

We also tested the document-generator frontend with an actual uploaded text file. The browser forwarded the filename, MIME type, and base64 bytes. The generated document visibly incorporated the unique sentence from the uploaded file, persisted a receipt, and downloaded the expected Markdown filename.

What shipped

The production versions at the end of the work were:

• Agent Builder 0.1.17;
• Agent Studio 0.1.31;
• a2a-pack SDK 0.1.116; and
• the annotated SDK tag sdk-v0.1.116.

Agent Studio runs with three workers, while the shared control-plane workers use rolling updates so long-running builds are not discarded during deployment. The final Control Plane revision was Ready at 100 percent traffic, and every proof agent's latest Knative revision was Ready.

The larger lesson

An agent builder should not optimize for code generation. It should optimize for the shortest trustworthy path from an idea to a useful outcome.

That means treating the frontend, backend, authentication, model funding, artifacts, downloads, browser behavior, publishing, and distribution as one product contract. It also means letting production proof overrule a green-looking internal status.

Agent Studio is now much closer to that standard. A prompt can become a one-page startup, but the run does not earn “done” merely because a URL exists. It earns “done” when the product works, the user can see the result, and the evidence survives inspection.

That is the shift from prompt-to-agent to prompt-to-proof.