# Copy Agent The Copy Agent is a writing agent that learns from evidence, not a permanent style prompt. It prepares a person's real writing into a voice, chats with that voice, records their preferred edits, builds a new candidate from those edits, evaluates it on held-out cases, and changes the active agent only when the person promotes it. It's a command-line app built only on June's public packages: [`apps/copy-agent`](https://github.com/johno/june/tree/main/apps/copy-agent). ## Problem The hard part isn't generating prose. It's preserving authorship, turning corrections into structured evidence, and proving a new version is better without silently changing what answers. A model with a style prompt writes in its own voice, and it doesn't learn when a person fixes a draft. The Copy Agent works from what they've actually written: it retrieves samples like the request and drafts in their form. When they edit a reply, the edit is kept as evidence, and the next version is graded on cases it never learned from. ## Architecture In thirty seconds: ```text corpus → prepared voice → chat → edits → evidence split → candidate → held-out eval → promote ``` In full: ```text corpus ──prepare──▶ prepared voice (an agent) ──▶ chat ──▶ replies │ /good /bad /edit ◀──────┘ │ feedback in the session journal │ evidence: datasets and eval cases │ candidate: the active voice plus learnable edits │ evaluation: both agents on held-out cases and a benchmark │ promote (a person decides) ──▶ the active agent ``` > **A person decides.** Feedback, evidence, candidates and evaluations change nothing on their own. Promotion is the only way the active agent changes after the first one, and it refuses without a complete evaluation against the agent it would replace. - **Corpus**: a JSONL file or a folder of Markdown and text files, read into a June corpus. - **Agent**: a prepared voice, saved under a hash of its content. One agent is active at a time, named in `active.json`. - **Chat**: sessions are June chat journals. Each reply records which agent wrote it. - **Feedback**: ratings, notes with `#tags`, and preferred edits, stored as June feedback on the turn they judge. - **Evidence**: datasets and eval cases derived from every session, by rules the app owns. - **Candidate**: a new agent, the active voice plus the edits the evidence allows learning from, with a record of its parent. - **Evaluation**: both agents run the same workflow on the same cases, scored by deterministic and judged graders, and compared case by case. Everything is plain JSON under one directory, `.copy-agent/` by default. ## June modules used | Import | Why the Copy Agent needs it | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | `june` | A turn is a workflow (a `brief` step, then ``), so chat and evaluation run the same component, and every model call is on the trace | | `june/chat` | Conversations that survive restarts: `fileStore` journals, `agentResponder`, feedback on each turn, and the agent that wrote every reply | | `june/copy` | The voice: `prepareVoice` keeps the person's own writing intact, and `` retrieves exemplars, drafts candidates and keeps the best one | | `june/train` | Honest evidence and fair comparisons: datasets and eval cases from sessions, with provenance, then `evaluate` and `compareEvaluations` | It doesn't use `june/trainers`. Adding a person's edits to their voice needs no GPU or served model, and a handful of edits is too few to fine-tune on. The conversation dataset it writes is ready for `fineTune()` when there's enough. ## Execution flow ### One turn Each chat turn is one run of ``. The `brief` step reads the conversation into a copy request with structured output; its result is the argument to ``, so the write waits for the brief. The voice then retrieves exemplars like the request, drafts several candidates, scores them against the person's writing, and keeps the best. ```tsx {(fields) => ( {children} )} ``` From [`apps/copy-agent/src/agent.tsx`](https://github.com/johno/june/blob/main/apps/copy-agent/src/agent.tsx#L109-L121), lines 109–121. The writer never sees the chat, only the brief and the voice's exemplars. The brief carries facts and requests, never style, so the exemplars decide how the reply sounds. > **What this taught June.** The voice's model calls first ran outside the runtime, so a turn's trace and usage counted only the brief: 1,217 tokens for a five-call turn. `` made the write a step, and the same turns now report 5–7k tokens. > > **What a real model found in the app.** Every scripted test passed while the brief padded a tweet request with hashtags and an emoji, which the writer copied over the person's style. `/why` showed it, and the brief now carries facts, never style. A real corpus found another: free-form brief types like `homepage-hero` matched none of the corpus's labels, so every request retrieved the same few posts. The brief now picks from the voice's own kinds of writing. ### Chat The chat answers each turn by rendering `` through `agentResponder`. Its `version` is the agent's id, so every reply in the journal says which agent wrote it, even after another is promoted. A child step saves why the reply reads as it does (the request, the exemplars, every candidate's score), which `/why` prints. ```tsx return createChat({ store: fileStore(join(home, "sessions")), responder: agentResponder({ id: "copy-agent", version: agent.id, render: ({ sessionId, turnId, messages, input }) => ( {(write) => ( )} ), reply: ({ steps }) => replyOf(steps), }), }); ``` From [`apps/copy-agent/src/agent.tsx`](https://github.com/johno/june/blob/main/apps/copy-agent/src/agent.tsx#L147-L170), lines 147–170. ### Evidence `copy-agent evidence` reads every session. June's converters turn the journals into datasets and eval cases; the app decides which turns count. Edits are split by turn, deterministically, into what a candidate may learn from and what's held out, so nothing is graded on writing it was given. ```ts const edits = preferenceDatasetFromSessions(sessions).filter(kept); const split = edits.split({ ...SPLIT, key: (e) => turnKey(e.metadata) }); const keys = async (data: typeof edits) => (await data.examples()).map((example) => turnKey(example.metadata)); const learn = await keys(split.train); const heldOut = await keys(split.test); const learnable = new Set(learn); const preferences = split.train; const conversations = datasetFromSessions(sessions).filter(({ metadata }) => learnable.has(turnKey(metadata)), ); const cases = evalCasesFromSessions(sessions, { negative: true, positive: true, }).filter((c) => kept(c) && !learnable.has(turnKey(c.metadata))); ``` From [`apps/copy-agent/src/evidence.ts`](https://github.com/johno/june/blob/main/apps/copy-agent/src/evidence.ts#L109-L124), lines 109–124. Approved replies become cases that expect the reply, and rejected ones become cases to beat. A turn tagged `#exclude`, or one the Copy Agent didn't answer, is left out of everything. ### Candidate and evaluation `copy-agent candidate` adds the learnable edits to the active voice as new samples, each labelled with the brief of the turn it corrected, and saves a new agent with its parent, its evidence and the turns it learned from. It's never active when it's made. [Learning from edits](https://june.dev/docs/copy/learning.md) covers the loop this follows. `copy-agent eval` runs the current agent and the candidate on the same cases. The held-out edits are the point: the candidate is graded on corrections it never saw. The eval target is the same `` workflow a chat turn runs, so chat and evaluation can't drift apart: ```tsx const input = String(messages.at(-1)!.content); const { steps, usage } = await run( , { signal }, ); ``` From [`apps/copy-agent/src/evaluate.tsx`](https://github.com/johno/june/blob/main/apps/copy-agent/src/evaluate.tsx#L104-L114), lines 104–114. There are two suites, never mixed: the held-out cases from chat, and a fixed benchmark of 20 requests across nine writing modes. Both agents are scored by the same graders, with the active agent's voice as the reference, and `compareEvaluations()` lines the runs up case by case. A judge also picks which reply the person would rather have, asked twice with the replies swapped; if the answers disagree, it's a tie. > **What this taught June.** Graders can abstain. A grader that doesn't apply to a case returns `undefined` instead of failing it, so cases with and without an expected reply share one suite. ### Promotion `copy-agent promote` refuses unless there's an evaluation against the active agent, on the current evidence and benchmark, in which every case was written and scored. Then it shows that evaluation, asks, and records the decision: ```ts const evaluation = await evaluationFor(home, candidate, benchmark); const why = { previous: evaluation.baseline, evaluation: `evals/${evaluation.baseline}..${candidate}.json`, ...(reason && { reason }), }; await setActiveAgent(home, candidate, why); ``` From [`apps/copy-agent/src/decisions.ts`](https://github.com/johno/june/blob/main/apps/copy-agent/src/decisions.ts#L129-L135), lines 129–135. After a promotion the reasons are on disk: `active.json` names the evaluation, which names the candidate, whose `agent.json` names its parent, its evidence and the turns whose edits it learned. ## Using it Build June's packages, set a provider key, and run the commands in order from `apps/copy-agent`: ```sh bun run copy-agent prepare fixtures/wren.jsonl --name Wren bun run copy-agent chat bun run copy-agent evidence bun run copy-agent candidate bun run copy-agent eval bun run copy-agent promote bun run copy-agent status ``` - `prepare` learns a voice from a corpus. The first agent becomes active. - `chat` writes with the active agent. Inside it, `/why` shows the request, the samples it used and every candidate's score; `/good` and `/bad` rate the last reply, with an optional note and `#tags`; `/edit ` records what you'd have written instead, and `/edit` alone opens `$EDITOR`. - `evidence` turns every session's feedback into datasets and eval cases. - `candidate` makes a new agent from the active voice and the learnable edits. - `eval` compares the candidate with the active agent, and prints where the judge preferred the active agent and which cases got worse. - `promote ` makes it active if a person agrees; `reject --reason "…"` records that they didn't. - `status` shows the active agent and why, the evidence, and every candidate. `copy-agent help` lists every command, including `sessions`, `show` and `feedback`. The chat is also a default export, so June's own CLI runs it: `june ask src/chat.ts "…"`. ## Testing The tests run real June compositions with deterministic models at the edges, so they need no API key: - In-process tests use AI SDK mock models. - Tests that run the CLI as its own process use [msw](https://mswjs.io) to answer the Anthropic and OpenAI HTTP APIs, so the real provider code runs without a network. - `test/loop.test.ts` runs the whole loop through the real binary: prepare, chat with edits and ratings, evidence, candidate, eval, promote, then chat again with the promoted agent. It checks that every file on disk agrees with the others. - `test/failures.test.ts` checks that a refused or failed step leaves the home directory exactly as it was. Scripted models can't show whether the writing is any good. A real-model acceptance run (`openai:gpt-5-mini` writing, `openai:gpt-5` judging, six turns, three edits) evaluated a candidate on the evidence and the benchmark. The judge preferred the candidate 9 times, the active agent 8 times and tied 3. A candidate learned from three edits was no clear win, and that's what the evaluation reported. The benchmark is a JSON file of eval cases, each with a writing mode, a reason it exists and optional deterministic checks. Its graders are deterministic (`matches`, `voice`, `markers`, `constraints`) and judged (`brief`, `fidelity`, `authorship`, `invention`), and every judged score keeps the judge's reason. There's no single overall score: the evaluation shows each grader, the preference and the regressed cases for a person to weigh. ## What stayed in the app June provides sessions, feedback, datasets, eval cases, evaluation and comparison. These decisions belong to the Copy Agent: - **The agent manifest and the active pointer.** An agent is a prepared voice saved by content hash. `active.json` says which one answers chats. - **Candidate lineage.** A candidate records its parent, the evidence it was made from and the turns whose edits it learned. June has no concept of a candidate. - **The promotion decision.** What an evaluation must contain before a person can promote, and the record of every promotion and rejection in `decisions.jsonl`. - **Corpus ingestion.** Reading a folder of Markdown with front matter into June's corpus format. - **Curation rules.** Which turns are evidence, `#exclude` as the way to leave one out, and the learn/hold-out split. ## What building it taught June Building the Copy Agent was also a test of June's public API. [OBSERVATIONS.md](https://github.com/johno/june/blob/main/apps/copy-agent/OBSERVATIONS.md) logs each finding, and [AUDIT.md](https://github.com/johno/june/blob/main/apps/copy-agent/AUDIT.md) sums them up. Two changed June, and they're next to the parts they changed: the write became a step ([One turn](#one-turn)), and graders learned to abstain ([Candidate and evaluation](#candidate-and-evaluation)). The rest were fixes to the app itself, like keeping the brief to facts. June.dev's own copy is drafted with the Copy Agent, from a voice prepared from 134 samples of its author's writing. A person reviews every piece and records why each draft was approved or edited, and the voice's evaluation says it isn't ready to write unedited yet ([`apps/www/voice`](https://github.com/johno/june/tree/main/apps/www/voice)). ## Source and related docs - Source: [`apps/copy-agent`](https://github.com/johno/june/tree/main/apps/copy-agent), with its [README](https://github.com/johno/june/blob/main/apps/copy-agent/README.md) and [benchmark](https://github.com/johno/june/blob/main/apps/copy-agent/benchmark/README.md). - [React as the workflow](https://june.dev/docs/workflows.md) and [Agent and Step](https://june.dev/docs/workflows/agent-and-step.md): the runtime `` runs on. - [Chat](https://june.dev/docs/chat.md), [Responders](https://june.dev/docs/chat/responders.md) and [Feedback and preferred edits](https://june.dev/docs/chat/feedback.md): sessions, `agentResponder` and feedback. - [Copy and voice](https://june.dev/docs/copy.md) and [Writing with a voice](https://june.dev/docs/copy/voices.md): preparing a voice and ``. - [Learning from chat](https://june.dev/docs/chat/learning.md), [Evals](https://june.dev/docs/training/evals.md) and [Comparing candidates](https://june.dev/docs/training/comparing.md): evidence, evaluation and promotion. - [CLI](https://june.dev/docs/cli.md): `june run`, `june chat` and `june ask`.