Corpora and authorship

A corpus is the person's writing, and everything a voice knows about them comes from it. What you leave out matters more than how much you put in: a voice imitates every sample it's shown, so one piece someone else wrote teaches it to sound like someone else. Optimize for writing that is surely theirs before you optimize for size.

Authorship hygiene

Before preparing, remove:

• other people's words: quoted text, testimonials, press quotes and excerpts from third parties;
• writing with unclear ownership: collaborative copy, ghostwritten pieces, and anything heavily edited by someone else;
• machine-generated text presented as theirs: drafts a model wrote that they published with light edits;
• thin boilerplate: legal footers, autogenerated emails, and lists, tables of contents or reference pages with little prose;
• duplicates, such as a post cross-posted to several places;
• pieces they've disowned, writing the person now considers bad or outdated.

You can remove samples before they reach the corpus, or mark them and let preparation decide. prepareVoice() drops samples with authored: false by default, and its include option takes your own rule, such as also dropping edited: true. Marking is better when you want a record of what was left out and why; dropping at collection is better for text that shouldn't be stored at all.

Get the person's consent first. A voice is a model of a real person's writing, and preparing and writing send their text to the models you choose; see corpora and consent.

Building a corpus from reviewed sources

This is the recommended way to build a real corpus. For anything bigger than a handful of samples, keep the decisions in a reviewed list rather than in code that scrapes. June.dev builds its author's corpus this way, and the shape works for any person or brand. Each entry says where a piece lives, which revision to read, what kind of writing it is, who wrote it, and whether it's in:

interface WritingSource {
  id: string;
  /** Where to read it: a URL, or a repository and path. */
  repo: string;
  path: string;
  /** A pinned revision, never a moving branch, so the corpus is reproducible. */
  ref: string;
  /** From a small, fixed vocabulary, since retrieval matches on it. */
  type: "essay" | "tutorial" | "short-post" | "readme" | "docs" | "page";
  date?: string;
  authorship: "theirs" | "mostly-theirs" | "uncertain" | "other";
  include: boolean;
  /** Why it's excluded, or anything a reviewer should know. */
  reason?: string;
}

Then a small script reads each included source at its revision, extracts the prose (dropping code blocks, markup, images and blockquotes), turns it into a sample whose metadata carries the entry's type, date and provenance, and writes the corpus file together with a summary of what was excluded and why. A few rules make this trustworthy:

• Pin revisions. Reading at a commit or snapshot means later edits, by anyone or anything, can't leak in, and the same list always builds the same corpus.
• When in doubt, leave it out. A smaller corpus of writing that is surely theirs beats a bigger one that isn't. Exclude uncertain and other until the person confirms.
• Fail whole. If a source can't be read, don't write a corpus without it: a corpus missing a source is silently a different corpus.
• Write the exclusions down. "84 words of prose once code is removed" or "a list of videos, not prose" is the review a person needs, and the reason a sample isn't there later.

The script's output is a JSONL file in the format below.

Input formats

copyCorpus() takes an array of samples, or the path or file: URL of a JSONL file:

import { copyCorpus } from "june/copy";

const corpus = copyCorpus([
  {
    text: "Recruiting, minus the recruiting.\nFonzie does the sourcing so you can do the building.",
    metadata: {
      date: "2025-03-14",
      source: "homepage",
      type: "homepage-hero",
      audience: "startup founders",
    },
  },
]);

A JSONL file has one sample per line, each a JSON object with text and optional id and metadata. Blank lines are skipped:

{"id":"launch-tweet","text":"Fonzie now books interviews for you. Pick a slot, we handle the rest.","metadata":{"date":"2026-08-20","source":"twitter","type":"tweet"}}
{"id":"changelog-ats","text":"Greenhouse sync is here. Candidates you like in Fonzie show up in Greenhouse, notes and all.","metadata":{"date":"2026-07-01","source":"changelog","type":"changelog"}}

A file is read on first use, relative to the current directory, and kept: later calls see the same samples even if the file changes. Call copyCorpus() again to reload.

Those are the only two formats. Anything else, such as a folder of Markdown posts, a CMS export or a mailbox, is ingestion your app does: read it, decide what's a sample, and hand copyCorpus() an array. The Copy Agent, for example, reads a folder of .md and .txt files, one sample per file, with key: value front matter as metadata.

Samples and ids

Text is kept byte for byte. June never trims, rewrites or normalizes it, and every completion it later trains on is this exact text. Strip what isn't the person's prose (code, quotes, navigation, signatures) before it reaches the corpus.

A sample's id defaults to a hash of its text, so it's stable across runs. Give your own ids when you have them, such as a post's slug: they make preparation reports, exemplars and training examples easy to trace. Two samples with the same explicit id are an error. Two with the same text and no id get sample-<hash> and sample-<hash>-2, and preparation drops the second as a duplicate.

Metadata

Every metadata field is optional:

Dates become ISO strings, and source, type, audience, intent, topic and tone are trimmed. Any other fields are kept as they are.

Metadata isn't bookkeeping; it's the leverage you have over what a voice does. Retrieval picks exemplars by comparing a request's labels with each sample's, type most of all, so a corpus with a small, consistent vocabulary of types (essay, tweet, changelog) retrieves far better than one where every sample has its own. The same labels let you count samples by kind, which is how you know what forms the voice can imitate. Label what you know; preparation can infer the rest.

Provenance

Provenance is part of a voice's quality, not just its compliance: when a draft reads wrong, the exemplars are usually why, and you can only fix a sample you can find. Keep enough in each sample's metadata to find the original again: a URL, or a repository, path and revision. June carries metadata through preparation into every training example, so any exemplar or training example traces back to the writing it came from, and a tuned voice's manifest records the datasets it was trained on:

{
  "id": "post-shipping-small",
  "text": "Ship the smallest thing that teaches you something…",
  "metadata": {
    "date": "2024-02-11",
    "source": "https://example.com/posts/shipping-small",
    "type": "essay",
    "authored": true,
    "provenance": {
      "repo": "example/site",
      "path": "posts/shipping-small.md",
      "ref": "4b1d0c3e…"
    }
  }
}

A corpus's fingerprint() is a SHA-256 over every sample's id, text and metadata, in order. A voice records it, so a saved voice refuses to load against a corpus that changed. source is the file path, or "memory" for an array, and size() counts samples.

Validation

Every sample is checked when the corpus loads, and an error names where it is:

june: ./fonzie.jsonl:12: "metadata.date" isn't a date

A sample needs a non-empty text string; metadata must be an object, id a string, and date something new Date() reads. Errors from an array name the sample's index (memory, sample 3).

Voice drift

People's writing changes. A blog post from 2014 and last month's launch email may not sound like the same person. Give samples a date, and decide how old writing should count:

• Leave it out. include: (sample) => sample.metadata.authored !== false && (sample.metadata.date ?? "") >= "2023" in prepareVoice() (include replaces the default rule, so keep the authored check), or filter before building the corpus.
• Weigh recent writing more. Retrieval already prefers newer samples, a little, and you can raise its recency weight.
• Mark eras. Extra metadata such as era: "early" is kept, so a custom retriever or filter can use it.

Dates are also how ties break in retrieval, so a dated corpus is more predictable than an undated one.

Drift is maintenance, not a one-time choice. Rebuild the corpus when the person's writing has moved on (a new role, a new product, a year of new posts), prepare it again, and evaluate the new voice against the old one on the same held-out writing before you switch.