# Local training

Use `trl()` when you want open-weight training on hardware you control. Use a [managed trainer](https://june.dev/docs/training/managed.md) when you'd rather the provider own compute and serving.

`trl()` fine-tunes open-weight models on your own machine with Hugging Face [TRL](https://huggingface.co/docs/trl) and [PEFT](https://huggingface.co/docs/peft). June orchestrates TRL; it doesn't embed an ML framework. The training runs in a separate Python process, and none of its dependencies are npm dependencies.

## Prerequisites

- [uv](https://docs.astral.sh/uv/getting-started/installation/). June runs its runner with `uv run --script`, which reads the runner's inline dependencies and installs them in an isolated, cached environment. You don't create a virtualenv or install anything by hand.
- A machine that can hold the model. See [Hardware](#hardware).
- For gated models, a Hugging Face token: `trl({ env: { HF_TOKEN } })`.

If uv isn't on `PATH`, `fineTune()` rejects before starting, with instructions.

To use a Python environment you manage instead, pass the command that runs a script in it. The environment needs the packages listed below:

```ts
import { trl } from "june/trainers";

// An absolute path, such as path.resolve(".venv/bin/python"): the runner starts in the run directory
const trainer = trl({ python: ["/home/me/project/.venv/bin/python"] });
```

## Hardware

Decide whether local training is realistic before you install anything. Real training wants a GPU:

| Machine                            | What's realistic                                                                |
| ---------------------------------- | ------------------------------------------------------------------------------- |
| CPU or Apple Silicon               | models under about 1B parameters, slowly: enough to try the pipeline end to end |
| a 24 GB NVIDIA GPU                 | LoRA on an 8B model in bf16, roughly                                            |
| a smaller NVIDIA GPU, around 10 GB | QLoRA on an 8B model, at some cost in speed; QLoRA needs CUDA                   |
| several large GPUs                 | full `sft`, which needs several times the model's size in GPU memory            |

These are rough guides, not promises: memory use depends on sequence length, batch size and the model. If none of these is your machine, use [managed training](https://june.dev/docs/training/managed.md) instead.

Normal tests and CI never train a model or need a GPU; see [Testing](#testing).

## A first run

```ts
import { dataset, fineTune } from "june/train";
import { trl } from "june/trainers";

const result = await fineTune({
  name: "fonzie",
  model: "Qwen/Qwen3-8B",
  trainer: trl(),
  dataset: dataset("./fonzie.jsonl"),
  method: { type: "lora", rank: 32, alpha: 64 },
  config: { epochs: 2, learningRate: 2e-4 },
});

result.artifact;
// { type: "adapter", format: "peft", uri: "/…/.june/trl/fonzie-trl-3f1a9c2e/output", baseModel: "Qwen/Qwen3-8B" }
```

## Methods and artifacts

| Method  | What it trains                                 | Artifact                              |
| ------- | ---------------------------------------------- | ------------------------------------- |
| `sft`   | every weight of the model                      | `checkpoint` in `transformers` format |
| `lora`  | a LoRA adapter on top of the frozen base model | `adapter` in `peft` format            |
| `qlora` | a LoRA adapter on a 4-bit quantized base model | `adapter` in `peft` format            |

LoRA options come from the method config: `rank` (16 by default), `alpha` (32), `dropout` (0.05) and `targetModules` (all linear layers). `config` sets `epochs`, `learningRate`, `batchSize`, `maxSequenceLength` and `seed`; TRL's defaults apply to anything unset.

Datasets must be all `completion` or all `conversation` examples, with string message content. TRL trains completion examples on the completion only. Conversations are trained on every turn, prompts included, because June's runner doesn't turn on TRL's `assistant_only_loss`. Preference examples aren't supported yet.

An adapter isn't a model you can call by itself. Load it on top of `baseModel` with PEFT, or serve it with a server that supports LoRA adapters (vLLM, TGI, llama.cpp after conversion), and point an AI SDK provider at that server to use it from June.

## The Python dependencies

The npm package stays small because the training never runs in JavaScript. June writes the data and a config to disk and starts its Python runner as a separate process; the runner declares its dependencies inline, so uv installs exactly these, in an isolated environment:

| Package        | Why                                               |
| -------------- | ------------------------------------------------- |
| `torch`        | runs the model                                    |
| `transformers` | loads the model and tokenizer                     |
| `trl`          | the supervised fine-tuning loop (`SFTTrainer`)    |
| `peft`         | LoRA adapters                                     |
| `datasets`     | `SFTTrainer` takes its data as a `datasets` table |
| `bitsandbytes` | 4-bit quantization; added only for `qlora`        |

`accelerate` comes with `trl`. bitsandbytes needs a CUDA GPU, which is why it's added only for QLoRA (with `uv run --with bitsandbytes`) instead of always.

## Runs and their files

Each run gets a directory under `.june/trl` (change it with `trl({ dir })`; add `.june` to `.gitignore`), named after the run's `name` and job id:

| File               | Written by | What                                                                          |
| ------------------ | ---------- | ----------------------------------------------------------------------------- |
| `config.json`      | June       | the model, method, hyperparameters and data files                             |
| `train.jsonl`      | June       | the training examples in TRL's format                                         |
| `validation.jsonl` | June       | the validation examples, if any                                               |
| `runner.py`        | June       | the exact runner script used                                                  |
| `pid`              | June       | the runner's process id and start time, so a reused pid isn't mistaken for it |
| `runner.log`       | the runner | its stdout and stderr                                                         |
| `status.json`      | the runner | `{ state, progress, error }`, updated as it trains                            |
| `result.json`      | the runner | `{ artifact: { type, format, path }, metrics }` when done                     |
| `output/`          | the runner | the adapter or checkpoint                                                     |

The runner keeps going if your process exits, and its state is all in these files. So a job can be checked on, waited for, or canceled from another process with `fineTuneJob(trl(), ref)`; see [Long-running fine-tunes](https://june.dev/docs/training/fine-tuning.md#long-running-fine-tunes). Canceling sends the runner's process group `SIGTERM`.

When a run fails, the job's error carries the runner's error and the last 20 lines of `runner.log` in `error.details.log`.

## Reproducibility

The run directory holds everything that went into the run: the config, the exact data, and the runner script. The manifest records the base model, method, hyperparameters and dataset fingerprints. Set `config.seed` for repeatable shuffling and initialization; GPU training still isn't bit-for-bit deterministic.

## Custom runners

`trl({ runner: "/home/me/project/my-runner.py" })` runs your script instead of June's. Like `python`, it must be an absolute path. It's run as `<python> <runner> config.json` in the run directory, and must follow the same contract: read `config.json`, write `status.json` (atomically: write a temp file and rename it) as it goes, and `result.json` before reporting `succeeded`.

## Testing

June's own tests run a fixture runner that follows this contract without training. A smoke test trains a tiny model with real TRL. It needs uv and a network connection, and is skipped unless you ask for it:

```sh
JUNE_TRL_SMOKE=1 make test path=packages/june/src/trainers
```