Watches
A watch runs a handler whenever a source has eligible work, one item at a time, and picks up from the last confirmed effect after a restart. A Linear board, open pull requests or a local queue can be a source; a June workflow, the Ship Agent or the Code Review Agent can be the handler.
import {
completed,
defineWatch,
deferred,
type Reconciliation,
type WatchSource,
} from "june/watch";
interface Ticket {
needsHardware: boolean;
}
declare const tickets: WatchSource<Ticket>;
declare function ship(ticket: Ticket): Promise<{ commit: string }>;
declare function findShipped(
key: string,
): Promise<Reconciliation<{ commit: string }>>;
export default defineWatch({
name: "tickets",
source: tickets,
idleFor: "1h",
async handle(item, { effect }) {
if (item.payload.needsHardware) return deferred("needs a signed device");
const receipt = await effect("ship", {
perform: () => ship(item.payload),
reconcile: () => findShipped(item.key),
});
return completed(receipt);
},
});
examples/watch/in-memory.ts is a complete watch over an in-memory queue.
Items and identity
A source returns pages of WatchItems: { scope, key, version?, payload, title?, url? }, and an opaque cursor for the next page.
scopeandkeyare the item's identity.SNO-12in two Linear workspaces, or PR 12 in two repositories, are different items because their scopes differ.itemId()derives the id the watch stores them under.versionis the source's revision of the item. The same version is the same work: an item completed at a version isn't handled again until the version changes. The source decides what goes into it, so a PR source can version by head SHA, and an issue source by a hash of the requirements that ignores comments and status churn.payloadis persisted with the item, so it must be JSON, and must hold no secret.
Sources
WatchSource<P>
namespace string
discover ({ cursor?, signal }) → SourcePage<P>: { items, next? }
claim? (item, context) → ClaimResult<P>
acknowledge? (item, outcome, context) → void
check? (signal) → CheckFinding[]
discoverreads one page. It must change nothing remotely: discovery runs on every poll, including the empty ones.claimtakes ownership just before the item is handled, re-reading what makes it eligible. It can return the item as re-read (claimed), say it left the queue (ineligible), or defer, retry or stop it for a person. Read-only sources leave it out.acknowledgereports the outcome back, such as moving an issue to review.checkmakes read-only checks forjune watch --check.
namespace is saved with the watch. Pointing a saved watch at a source with another namespace is an incompatible change.
Handlers and outcomes
A handler gets the item and a context: an AbortSignal, the durable attempt identity, effect() and link(). It returns one of:
| Outcome | Means |
|---|---|
completed(receipt?) | Done at this version. |
deferred(reason, { retryAt? }) | Blocked by something outside the watch. Other items go ahead; it is looked at again when its version changes or at retryAt. |
retryable(reason, { retryAt? }) | Worth trying again, after retryAt or the watch's backoff, up to retry.maxAttempts. |
terminal(reason) | Will never succeed at this version. |
needsAttention(reason) | Only a person can move it on. |
A handler can be called again for the same attempt after a restart, so anything it does outside the process goes through effect().
Effects
import type { EffectContext, Reconciliation } from "june/watch";
declare const context: EffectContext;
declare function postComment(signal: AbortSignal): Promise<{ id: string }>;
declare function findComment(
signal: AbortSignal,
): Promise<Reconciliation<{ id: string }>>;
// perform returns the receipt; reconcile answers { status: "applied", receipt },
// { status: "missing" } or { status: "unknown", reason }.
const receipt = await context.effect("comment", {
perform: (signal) => postComment(signal),
reconcile: (signal) => findComment(signal),
});
Effects are journaled by key for each item version. The intent is recorded before perform and the receipt after it, so once a key has a receipt it is returned without performing again.
When perform throws, the effect may still have happened: a timeout, a dropped connection or an abort can come after the service applied the request. So the key stays unknown, and the next call reconciles first: an applied effect is adopted, a missing one is performed, and an unknown one stops the item for a person. The same happens when the process dies part-way. Throw EffectNotAppliedError only when the effect certainly didn't happen, such as a request the service refused: that frees the key to be performed again without reconciling.
Keys are scoped by phase, so a source's claim, the handler and acknowledge never share one, whatever names they pick. Claim and handler keys last for the item version, across attempts: a PR a failed attempt opened is adopted by the next. Acknowledgement keys belong to one report: the attempt whose outcome is being reported. The host records the outcome before reporting it, and a report that a crash or a failed acknowledgement interrupted is finished, with the same outcome and the same keys, before the item is handled again.
memoryEffects() is an in-memory effect context with the same rules, for calling a handler or source directly in a test or a script.
link({ kind, id, url? }) records a reference to downstream work, such as a Ship task, a session or a pull request.
Running a watch
The june watch command runs a definition file. From TypeScript, createWatchHost does the same:
import { createWatchHost, type WatchDefinition } from "june/watch";
declare const watch: WatchDefinition;
const host = await createWatchHost(watch, { stateRoot: ".june/watch" });
await host.drain({ schedule: false }); // work until the queue is idle, then return
const controller = new AbortController();
await host.run({ signal: controller.signal }); // or keep going until aborted
await host.close();
A host works one item at a time:
- A pass pages through the source, enrolling each page before asking for the next.
- The next item is the first eligible one in the source's order. An attempt a stopped host left under way comes first, even if the source no longer lists it.
- An attempt claims the item, handles it and acknowledges the outcome.
- After every attempt the source is read again at once, so work that arrived while draining is found straight away.
- A pass with nothing actionable stores an absolute next wake: the end of the pass plus
idleFor, or sooner if a retry falls due first. An empty queue and a queue of blocked items both wait the sameidleFor. A host restarted before that wake waits for it; one restarted after it polls at once.
Discovery and inspection call no model, so an empty poll costs one pass over the source and nothing else.
| Outcome | The item is |
|---|---|
completed | done at this version |
deferred | skipped until its version changes, or until retryAt; other items go ahead |
retryable | retried after retryAt or an exponential backoff; after retry.maxAttempts, held as exhausted, and the source's acknowledge is told it needs a person |
terminal | held, and not retried at this version |
needs_attention | held until a person acts or its version changes |
A handler or source that throws counts as retryable, and an effect whose outcome can't be established as needs_attention. A claim that finds the item has left the queue defers it until the next idle wake without counting an attempt. A source that can't be reached throws SourceUnavailableError, optionally with the provider's retryAt, from discover, claim or acknowledge. That is a wait for the whole watch, not a failure of the item: the drain stops, no attempt is charged, an unfinished claim or report resumes later, and the host waits for the hint or its own backoff. Every retry time is at least retry.initialBackoff away, whatever the hint says.
Pass clock: manualClock() to run a watch on a clock that only moves when told to, so an hour of idle time takes no time in a test.
Operating a watch
watchStatus(stateRoot, name) and watchHistory(stateRoot, name) read a watch from any process, without taking it or calling anything remote. pauseWatch, resumeWatch and stopWatch write the watch's control.json, which the running host reads between attempts and while it sleeps (every controlPollMs, one second by default): a pause lets the attempt under way finish and starts no other, a resume polls at once, and a stop exits after the attempt under way. The CLI has the same commands.
Recovery
A watch can be killed at any moment: before a claim, after one, while a handler runs, after a service accepted an effect but before its receipt was written, or after everything was acknowledged. The next host to start picks up from what is on disk:
- The interrupted attempt comes first, before any other item is dispatched and whether or not the source still lists the item, since its own claim may have taken it out of the queue. It resumes as the same attempt, with the same identity, so its journaled effects are adopted.
- Each effect with no receipt is reconciled through the adapter's
reconcile, which looks at the service: an effect that happened is adopted, one that certainly didn't is performed, and one the service can't vouch for either way is recorded as unknown. A lookup that fails (a timeout, a 500) is no answer at all: the effect stays unrecorded and the item is retried to look again, or the whole watch waits if the provider threwSourceUnavailableError. - An unknown outcome stops only its item. The item is held as needing attention, with the effect named in its reason, and the other items go on. It isn't retried until a person acts or its version changes.
- A report that was under way is finished with the outcome recorded before it started, and the handler isn't run again.
history.jsonl records each step (attempt.started, attempt.resumed, effect.intended, effect.applied, effect.adopted, effect.unknown, effect.reconcile-failed, report.failed, attempt.ended), so the decisions a restarted host made can be read afterwards. Recovery never replays a June run's snapshot to repeat an effect: the journal and the service are the only evidence.
What this guarantees is at-least-once attempts with effects that happen once, as far as the service can be asked about them. That rests on the adapter:
reconcilehas to find an effect by something the effect carried, such as a marker in a comment, a branch name or a claim reference. Claim and handler effects last across attempts, so name their evidence after the item and its version (attempt.itemIdandattempt.version), notattempt.id, which changes with each attempt. Answerunknownonly when the service can't say; a lookup that fails should throw, and the item is retried to look again.- A host only coordinates with processes sharing its state directory. When another host, or a person, can write to the same service, the source's
claimhas to re-read ownership and stand down on evidence that someone else holds the item.
State and ownership
Each watch keeps its state under its own directory in a state root:
<state root>/<name>/
state.json the watch: its identity, discovery pass and cursor, next wake, and every item
state.json.bak the state before the last write
history.jsonl what happened, one event per line
owner.lock which process runs the watch
control.json what an operator asked: pause, resume, stop
- Writes are atomic. The state is replaced by renaming a complete file over it, so a process killed at any point leaves one write or the next. A discovery page and its cursor are written together, so the cursor never moves past items that aren't on disk.
- Nothing is overwritten that can't be read. A state that isn't valid JSON, was written by a newer June, or belongs to a definition whose name, source namespace or
confighas changed is refused with a message naming the file, and left as it is. - One process runs a watch.
owner.locknames the process holding it. A second process is refused while that one is alive, and takes the lease over only from a process on the same machine that has died. An owner on another machine is never presumed dead: the lease coordinates processes sharing one state directory on one host, not hosts. When several hosts could act on the same source, the source's own claim has to keep them apart. - No secrets and no transcripts. Items hold their source payload and effects hold their receipts, which are capped at 64 KB. Keep credentials in the environment and refer to downstream work with
link().
Two watches with different names keep separate state and leases, and run independently.
Definitions
defineWatch({ name, source, handle, idleFor, retry?, config? }) validates the definition and returns it.
nameis the watch's stable identity among the watches sharing a state directory: lowercase letters, digits and dashes.idleForis how long to wait after a pass that found nothing actionable: milliseconds or"90s","15m","1h","1d".retrybounds retries per item version:maxAttempts(3),initialBackoff(one minute) andmaxBackoff(one hour).configis JSON whose change makes saved state incompatible. Together with the name and the source's namespace it makes up the definition'sfingerprint. Keep secrets out of it.