# Voice preparation

`prepareVoice()` turns a [corpus](https://june.dev/docs/copy/corpora.md) into a voice's working material: the samples worth learning from, each with a reconstructed prompt and labels, as a dataset with deterministic splits. Most of it is deterministic bookkeeping, but one step calls a teacher model once per sample, and what that model writes is generated metadata you should review before relying on it.

```text
validate ─▶ exclude ─▶ dedupe ─▶ reconstruct each brief + infer labels ─▶ split
(on load)   (rules)    (text)    (one teacher call per kept sample)       (by id)
```

1. **Validation** happens when the [corpus loads](https://june.dev/docs/copy/corpora.md#validation), before preparation starts.
2. **Exclusion** drops samples `include` rejects. Deterministic.
3. **Deduplication** drops near-duplicates of an earlier sample. Deterministic.
4. **Teaching**: one teacher call per remaining sample reconstructs its brief and labels what its metadata leaves out. This is the only step that calls a model, and the only one whose output differs between runs.
5. **Splitting** pairs each prompt with the untouched text, and splits by sample id. Deterministic.

Preparation never changes a sample's text, and it's needed for two things: [retrieval](https://june.dev/docs/copy/retrieval.md) on a corpus that isn't fully labelled, and [fine-tuning](https://june.dev/docs/copy/training.md#fine-tuning-a-voice). A corpus whose samples already have good labels can write without it.

```ts
import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
});

prepared.samples; // kept samples, each with its prompt and inferred labels
prepared.removed; // [{ id, reason: "excluded" | "duplicate", of? }]
prepared.dataset; // every kept sample as a completion example
prepared.splits; // { train, validation, test }
prepared.corpus; // { source, size, fingerprint } of the corpus it came from
```

Its options:

| Option         |                                                                               |
| -------------- | ----------------------------------------------------------------------------- |
| `teacher`      | the model that reconstructs prompts and infers labels; required               |
| `include`      | which samples to keep; by default every sample not marked `authored: false`   |
| `dedupe`       | `{ threshold }` for near-duplicates (0.9 by default), or `false` to keep them |
| `instructions` | `"reverse"` (the default) or a function that writes each sample's prompt      |
| `infer`        | labels to fill in; `type`, `audience`, `intent` and `tone` by default         |
| `split`        | split shares and seed; 80/10/10 with seed 0 by default                        |
| `concurrency`  | how many teacher calls run at once (4 by default)                             |
| `signal`       | aborts the run                                                                |

## Exclusion

By default `prepareVoice()` keeps every sample except those marked `authored: false`. Pass `include` to decide yourself, for example to also leave out writing someone else edited, or writing from before a cut-off:

```ts
import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  include: (sample) =>
    sample.metadata.authored !== false &&
    sample.metadata.edited !== true &&
    (sample.metadata.date ?? "") >= "2022",
});
```

`include` replaces the default rule, so keep the `authored` check if you still want it. Each excluded sample is listed in `removed` with `reason: "excluded"`. See [authorship hygiene](https://june.dev/docs/copy/corpora.md#authorship-hygiene) for what to exclude.

## Deduplication

After exclusion, `prepareVoice()` drops near-duplicates, keeping the first of each group in corpus order. A sample is a duplicate of an earlier one when:

- their text matches once case, punctuation and spacing are ignored; or
- their lengths are within the threshold of each other, and their sets of word trigrams overlap by at least the threshold (shared trigrams over all distinct trigrams of both).

The threshold is 0.9 by default; `dedupe: { threshold: 0.8 }` catches looser copies, and `dedupe: false` keeps everything. It's deliberately simple and transparent: it catches cross-posted and lightly edited copies, not paraphrases. Each one is in `removed` with `reason: "duplicate"` and `of`, the id of the sample it duplicates.

## Reverse instructions

An archive of writing has the answers but not the questions: nobody saved the brief behind each homepage. Training and evaluation need `instruction → response` pairs, so by default the teacher reconstructs the brief: the instruction an editor or client most plausibly gave to get exactly this text, with its purpose, audience, format and length but none of its wording.

```text
Instruction (synthetic, from the teacher):
  Write a two-line homepage hero for founders that says Fonzie handles sourcing.

Response (authentic, never changed):
  Recruiting, minus the recruiting.
  Fonzie does the sourcing so you can do the building.
```

The distinction matters. **The completion is always the original text, byte for byte.** The teacher only writes prompts and labels; it's never asked to improve or rewrite anything, and nothing it says ends up in a completion. The teacher sees each sample's text and its string metadata, so it can use what's known (its type, its audience) when writing the brief.

To write prompts yourself instead, pass a function. It can be async, and must return a non-empty string:

```ts
import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  instructions: (sample) =>
    `Write a ${sample.metadata.type} for ${sample.metadata.audience}.`,
});
```

With your own prompts, the teacher is only called for samples that are missing a label in `infer`; pass `infer: []` to make no teacher calls at all.

## Metadata inference

Retrieval and evals work better when every sample says what it is. In the same call as the brief, the teacher fills in whichever of `infer` (by default `type`, `audience`, `intent` and `tone`; `topic` is also available) the sample doesn't already have. It's asked for `type` as a short kebab-case label, such as `homepage-hero` or `changelog`, and for the others in a few words.

Inferred values are guesses, not ground truth, so they're kept apart: in `prepared.samples[i].inferred`, and in each example's `metadata.inferred`. A sample's own metadata is never overwritten, and retrieval uses a sample's own label first, falling back to the inferred one.

Inferred `type` labels are free text, so a teacher may call similar samples `blog-post`, `essay` and `article`. Since retrieval matches labels by their words, a corpus with its own consistent `type` on every sample retrieves more predictably than one that relies on inference. Label `type` yourself when you can, and check what the teacher inferred when you can't.

## Failures and cost

Know this before you run it: preparation makes one teacher call per kept sample, four at a time by default, so a corpus of 130 samples is 130 calls to a capable model. If the teacher fails on any sample, `prepareVoice()` rejects with a `PrepareVoiceError` whose `failures` list each sample id and its error, in corpus order, so you can fix or drop those samples and run again:

```ts
import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice, PrepareVoiceError } from "june/copy";

try {
  await prepareVoice(copyCorpus("./fonzie.jsonl"), {
    teacher: openai("gpt-5"),
  });
} catch (error) {
  if (error instanceof PrepareVoiceError) {
    for (const { sample, error: cause } of error.failures) {
      console.error(sample, cause.message);
    }
  }
  throw error;
}
```

It's all or nothing: there's no partial result and no progress callback, so with a large corpus, check the teacher model and key on a few samples first. The teacher isn't deterministic, so preparing twice gives the same samples, removals and splits but not the same prompts and inferred labels.

## The result

```ts
import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";
import { fineTune } from "june/train";
import { trl } from "june/trainers";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  split: { train: 0.8, validation: 0.1, test: 0.1, seed: 42 },
});

await fineTune({
  model: "Qwen/Qwen3-8B",
  trainer: trl(),
  dataset: prepared.splits,
  method: "lora",
});
```

`prepared.dataset` holds every kept sample as a `completion` example, in corpus order: the prompt, the original text as the completion, and as metadata the sample's own metadata plus `sample` (its id) and `inferred`. So any example traces back to the writing it came from. `prepared.corpus` records the raw corpus's source, size (before anything was removed) and fingerprint, so a voice built from it can be [saved and checked](https://june.dev/docs/copy/training.md#manifests).

## Reviewing a preparation

Preparation's prompts and labels are a model's guesses, and everything downstream (retrieval, evaluation, fine-tuning) trusts them. Read what it made before you write with it:

- **What was removed, and why.** `removed` should hold no surprises.
- **The labels.** Count kept samples by `type` (their own, else inferred). These are the labels requests will have to [match](https://june.dev/docs/copy/retrieval.md#use-the-corpuss-own-labels), and a type with very few samples gives a request of that type little to go on.
- **A sample of prompts.** A reconstructed brief should describe the text's purpose, audience, form and length without quoting it.
- **The inferred labels on a few samples.** A `type` that disagrees with what the text plainly is will send requests to the wrong exemplars.

## Deterministic splits

Splits are [deterministic](https://june.dev/docs/training/datasets.md#deterministic-splits): 80/10/10 with seed 0 unless you pass `split`. They go by sample id, not by example content, because the teacher's prompts can differ between runs. Re-running preparation never moves a test sample into training, and the same corpus and seed always give the same splits on any machine.

Keep `test` for [evaluation](https://june.dev/docs/copy/scoring.md#evaluating-a-voice). Fine-tuning uses `train` and `validation`; retrieval uses every kept sample, so evaluate on held-out samples with a voice that can't retrieve them.

## Keeping a preparation

The teacher work is the expensive part, so keep what it made rather than preparing again casually. A `PreparedVoice` holds datasets, so it isn't plain JSON. Its `samples`, `removed` and `corpus` are, so write those out for review and the record. June can't load a saved preparation back yet: to write in another process with the inferred labels, either prepare again (which calls the teacher) or rebuild the `PreparedVoice` in your app from the samples you kept, as the Copy Agent does. A voice loaded from the raw corpus works too, but retrieves with the corpus's own labels only.