Learning from chat

Real conversations are the best evidence of what a model or agent gets wrong. june/train turns chat sessions, and the feedback people left on them, into datasets and eval cases. Conversion only reads: it makes no model calls and never writes to a session's journal.

Used well, that evidence goes through one loop:

sessions ─▶ curated feedback ─▶ datasets and eval cases ─▶ candidate ─▶ held-out evaluation ─▶ a person promotes it
                (approved                                   (a fine-tune or
                 replies and edits)                          a changed agent)

A reply isn't training data just because it happened. Training on every reply teaches a model to repeat its own mistakes, so by default only replies a person approved (a positive rating) or corrected (a preferred edit) become training targets. An include function can change that rule, and the choice is yours.

Feedback is recorded with session.feedback(). june/train reads june/chat sessions; june/chat never depends on june/train. The Copy Agent is a complete application of this loop: edits in chat, evidence from june/train, a candidate evaluated against the active agent, and promotion by a person.

Curation

The converters' defaults decide which turns are evidence, and for what:

When a turn has several ratings or edits, the latest of each counts; the earlier ones stay in the journal. Ratings judge the original reply, so a negative rating, even one recorded after an edit, never discards the edit. A negative reply without a correction says what's wrong but not what's right, so it's eval material, never a training target.

Conversation datasets

import { createChat, fileStore, type Responder } from "june/chat";
import { datasetFromSessions } from "june/train";

declare const responder: Responder;

const chat = createChat({ responder, store: fileStore(".june/chat") });
const sessions = await Promise.all(
  ["4f9c0e1a", "b2d1aa07"].map((id) => chat.openSession(id)),
);

const sft = datasetFromSessions(sessions);
console.log(await sft.summary());

Each eligible turn becomes one conversation example: the completed turns before it, the user's input, and the target reply. Earlier turns appear as the full transcript, each with its steering and its original reply, as fullContext() would send them, so the example teaches the right answer to the conversation as it actually happened. If the chat used summaryContext() or a custom context, that's more than the responder saw.

Pass include to curate differently. It receives each completed turn with its context, latest rating and latest preferred reply, and the target is still the edit when there is one:

import type { Session } from "june/chat";
import { datasetFromSessions } from "june/train";

declare const sessions: Session[];

const toneFixes = datasetFromSessions(sessions, {
  include: ({ turn, preferred }) =>
    preferred !== undefined &&
    turn.feedback.some((feedback) => feedback.tags?.includes("tone")),
});

Preference datasets

import type { Session } from "june/chat";
import { preferenceDatasetFromSessions } from "june/train";

declare const sessions: Session[];

const prefs = preferenceDatasetFromSessions(sessions);

Every turn someone edited becomes a preference example: input is the conversation before the reply, chosen the latest edit, and rejected the original reply. An edit that doesn't change the reply is skipped.

Eval cases

import type { Session } from "june/chat";
import { evalCasesFromSessions, evalSuite, grader } from "june/train";

declare const sessions: Session[];

const suite = evalSuite({
  name: "chat-regressions",
  cases: evalCasesFromSessions(sessions, { negative: true }),
  graders: [
    grader<unknown, string, string>("short", ({ output }) =>
      output.length < 400 ? 1 : 0,
    ),
  ],
});

Each case's id is chat:<session id>:<turn id>, stable across reruns and edits, so runs line up in compareEvaluations(). A case's input is the conversation up to the reply, as AI SDK messages, so it runs against a model target directly. Edited turns expect the edit. Negatively rated turns without an edit only become cases with negative: true, and have no expected: June won't invent the right answer, so grade them on their own terms. positive: true adds approved replies as regression cases that expect the original.

Evaluate, then train only if it's justified

The outputs are ordinary datasets and cases, so they go wherever any other dataset does. Hold some sessions out first: a model graded on the turns it trained on shows whether it learned them, not whether it generalizes.

import { openai } from "june/sdk/openai";
import type { Session } from "june/chat";
import {
  datasetFromSessions,
  evalCasesFromSessions,
  evalSuite,
  evaluate,
  fineTune,
  grader,
  type Trainer,
} from "june/train";

declare const sessions: Session[];
declare const trainer: Trainer;

// Evaluate only on sessions training never sees.
const heldOut = sessions.slice(-5);
const training = sessions.slice(0, -5);

const suite = evalSuite({
  name: "chat",
  cases: evalCasesFromSessions(heldOut),
  graders: [
    grader<unknown, string, string>("matches-edit", ({ output, expected }) =>
      output.trim() === expected?.trim() ? 1 : 0,
    ),
  ],
});

// The model that answers these chats today.
const active = await evaluate(openai("gpt-5-mini"), suite);
console.log(active.scores);

// Only if it falls short in a way more examples can fix: tune the
// open-weight model you'd serve instead, then evaluate it on the same suite.
const { artifact, manifest } = await fineTune({
  model: "Qwen/Qwen3-8B",
  trainer,
  dataset: datasetFromSessions(training).split({
    train: 0.9,
    test: 0.1,
    seed: 7,
  }),
  method: "lora",
});
console.log(artifact.type, manifest.datasets.train.fingerprint);

• Evaluate first. The active model's scores on held-out cases say whether there's a problem worth training for. A better prompt, context or agent may fix it without a fine-tune; see Fine-tuning.
• Keep evaluation and training apart. Build eval cases only from sessions, or turns, that training never sees. Datasets covers deterministic splits.
• Compare on the same cases. Evaluate the candidate, a tuned model or a changed agent, on the same suite and compare the two runs with compareEvaluations(). Comparing candidates covers deciding what "better" means before you look.

The same stored sessions always produce the same examples, so a dataset's fingerprint identifies exactly which conversations a model was trained on.

Review before promotion

Nothing here changes what answers your chats. A fine-tune produces an artifact and an eval produces scores; deciding a candidate is better, and switching a chat's model or responder to it, is a deliberate step a person takes. June doesn't promote models, rewrite agents, or retrain on its own.

Before training, read what you're about to train on: feedback is written by people, edits can carry mistakes or private data, and a session's content went to whatever model answered it.

Provenance

Every example and case carries metadata saying where it came from:

{
  "source": "chat",
  "sessionId": "4f9c0e1a",
  "turnId": "7c3e91d2",
  "at": "2026-10-14T16:12:03.512Z",
  "responder": { "kind": "model" },
  "model": { "provider": "openai.responses", "modelId": "gpt-5-mini" },
  "tags": ["verbosity"],
  "target": "preferred"
}

target says whether the example teaches the human's edit or the original reply. Keep metadata when you write datasets out, so you can trace a tuned model's behavior back to the conversations it learned from.