How We’re Developing InterGen

This page is a running record of how InterGen — the assistant that ships inside InterGenOS — is being developed: what we built first, what it measured, where it was wrong, what three days of real use changed, and what we’re doing now. It is written in the order things happened, and the failures are recorded with the same weight as the successes, because a write-up that shows only the clean path teaches no one.

Every measured result here comes from a recorded run. Static package, catalog, and corpus facts come from the cited source tree; where a run is a sealed capture that is not published, the page says so rather than pretending the number is unsourced. Nothing here describes a thing as done that is not done.

The one-paragraph version

We started the way most people would: we imagined the questions a user would ask a system assistant, wrote several hundred of them, built a harness to run them against the assistant and grade the answers, and developed against that. It worked well enough to get a first release out the door, and it was wrong in ways we couldn’t see from the inside. In the three days of outside use that followed the first point release, one relative’s ordinary use — recipes, chickens, Medicare, the weather in her town — showed us a set of conversation shapes our corpus had never contained, and a set of failures our grading had never asked about. We had built the loop, but we had been using it as a gate on our own guesses, not as a loop on real use.

1. Where we started

1.1 What InterGen is, in one line

InterGen is an assistant built around a local language model, run by a daemon on the user’s own machine. Around the model sit a small set of tools it can call (package management, service control, web search) and two things the code answers itself: the clock and date, and retrieval from the system’s own wiki. The catalog holds three model sizes — a 2-billion-parameter model that every machine can run, a 9-billion model offered at roughly 7 GiB of discrete graphics memory, and a 35-billion model offered at roughly 22 GB — and setup lets the user pick among the sizes the hardware can hold rather than picking for them. Inference stays on the machine; assistant content leaves it only through an explicit network feature: a web search, or a request to a user-configured outside model that the user reviews and approves before it is sent. The mechanism is described in AI Assistant Internals.

1.2 Observability first: the assistant records its own turns

The earliest decision that mattered later was to make the assistant observable as a product feature, not a debug switch. By default, served D-Bus and web turns emit routing, assembled-prompt, model-output, timing, and final-delivery events to a per-user trace file (glass.jsonl) — sequenced and phase-tagged, size-capped and rotated. It can be turned off only loudly (an explicit environment setting), and a failure to write the trace never fails the request. The code is intergen/glass.py in the source tree.

Why this matters: the field-use findings in part 2 came from Glass. Corpus, package, and training measurements come from their respective source counts and run artifacts. When the first outside user’s laptop told us what had gone wrong, it told us through this file, with no instrumentation added after the fact.

1.3 The scenario harness: our picture of a user

We wrote a corpus of scenarios — a user’s message, what the correct routing is, what the answer must and must not contain — in thirteen families: conversational, device and peripheral questions, first-boot top-up, how-to teaching, personal memory, invariant classes, refusals, seeking information, software management, troubleshooting, web search, writing help, and a reconciliation set (intergen/tests/scenario/corpus/). Around it we built:

• a runner that drives the real assistant over its real D-Bus interface by default (an in-process transport and a mock exist for the harness’s own self-tests);
• a grader for the deterministic checks (did the right tool get called, is the forbidden string absent);
• a judge from a different model family than the one under test — the harness refuses a judge whose language backbone is the family being graded, so the assistant never grades itself — for the questions a pattern cannot answer (is this coherent, is it truthful, did it actually answer);
• a comparator for before/after runs; and
• a ratcheted coverage backlog that records known uncovered capability-inventory cells, so “what do we not test” is a list, not a feeling.

The corpus holds 610 scenarios as of 2026-08-25, counted by the harness’s own loader.

What we got right: running against the real interface, keeping the judge in a different model family, and treating coverage as an object. What we got wrong is the subject of part 2: the scenarios were predominantly single-turn, topic-driven, and written from a developer’s picture of a user. A developer asks “install htop”. A person says “yes, please” to an offer the assistant made a turn ago.

1.4 Baselines before training, and a grading rubric

Before any training touched a weight we ran the corpus against all three tiers on the hardware each tier is meant for, and sealed the results. The per-tier table from 2026-08-07 (140 conversations per tier; the run captures are sealed but not published):

Two things fell out of that table that we would never have guessed. Routing failures were almost tier-independent: the same five conversations failed on every model size — four of them terse or casual phrasings (“disk”, “GPU”, “get me htop”), the fifth a two-turn request to be taught how to update the system. A bigger model was never going to fix those; only the routing layer was. And the 9B tied or beat the 35B on conversations passed with roughly a quarter of the model-weight footprint (5.7 GB against 22 GB).

Reading that table produced the grading order we use since: was the question understood; was it answered; if a refusal was right, was it refused; was the answer coherent, correct, truthful; and only then, was the right tool called. Our first corpus had that upside down — tool routing was the only thing that could fail a conversation, and a non-linguistic garbage answer had passed every scripted check because the tool call was right.

1.5 Training: three rounds, one adapter shipped

We fine-tune with LoRA adapters on a from-source stack packaged in the distribution’s ai tier, not a separate environment. The training packages are available from the package repository but excluded from the installation image (as of 2026-08-25: unsloth 2026.7.4, peft 0.19.1, trl 0.24.0, transformers 5.5.0, bitsandbytes 0.49.2, torchao 0.16.0; serving through llama.cpp build b8796). Three rounds ran between 2026-08-11 and 2026-08-14. The short version:

• Rounds 1 and 2 were invalidated — not by the training, by the serving. The laptop we measured the 2B tier on was serving through an integrated-GPU Vulkan path that intermittently corrupted generations. Re-measuring the same stock model on the CPU moved the baseline from 95 to 108 passes with zero training. The “47% baseline” our first round had been scoped against was substantially a backend artifact. A serving-corruption canary now exists because of this.
• Round 3 trained all three tiers on a corrected pipeline. The earlier rounds’ training data was malformed in three measured ways: the correctly structured tool-call rows were skipped by the trainer and taught nothing; the rows that did carry a call taught a call form the product’s own template forbids; and the call arguments were stored in a form that template renders as a call with no arguments. The corrected pipeline’s raw emission — the complete product-form call with every argument — sits in the sealed exports beside the earlier form. Each trained model was then run three times against its own three stock baseline runs, and the difference was judged against the measured noise floor of the instrument — how much two identical stock runs disagree with each other on the same day (18 of 204 cells on the 2B, 24 on the 9B).
• Verdicts (2026-08-14): the 2B adapter was rejected — its movement was indistinguishable from the measured noise floor. The 35B adapter was rejected — no measurable effect, net zero cells. The 9B adapter shipped: discordance well above the floor, net +9 conversations, ten cells cured from 0-of-3 to 3-of-3, with two named regressions carried openly on the record (one deterministic dropped argument on package removal; one literal-mindedness about a service name). The first release ships the 2B and 35B models stock and the 9B model trained; the shipped 9B file is pinned by hash in intergen/data/models-manifest.json.

One out of three is the honest score, and the noise-floor method is the part worth copying: without three runs a side and a measured floor, ordinary run-to-run movement in both rejected adapters would have been easy to overinterpret.

2. What real use did to that

2.1 The first outside user

The first point release went to family and friends on 2026-08-22. One relative, on a laptop with a small NVIDIA card, used the assistant the way a person uses a thing: eight conversations, about sixty questions over three days — recipes, gardening, pets, Medicare, weather and events in her town, a car noise, a sewing machine. She asked for one system task all week: a package update on the first day. With her consent we read the assistant’s traces on 2026-08-25. Those traces are not published; the shapes they showed are.

2.2 What the traces showed

Five shapes of failure, none of which our six hundred scenarios contained:

1. The consent turn lost the question. The assistant staged five live-data offers — “I can look that up on the web — want me to?” — and she accepted four of them (“yes”, “please do”, a yes that restated the request); each time the next turn answered “I don’t know your location” (she had named her town in the accepting sentence itself) or answered from memory. No accepted offer ever ran a search: in three days exactly one search ran at all, the one time she used the literal phrase “search the web for”. Our corpus had no scenario in which the user’s whole message is a one-word reply to something the assistant said.
2. A live-data question routed to the clock. “What time will the sun set today” was answered with the current time. The intent classifier saw “what time” and stopped.
3. “wha?” replayed the previous answer verbatim. No clarification handling at all.
4. The embedding server failed on any input over 512 tokens (HTTP 500, four times), timed out twenty times across five daemon starts, and once raised a health alarm reading “sustained semantic-corruption flags (0 in the recent five-generation window)” — an alarm contradicting itself. With embeddings down, documentation retrieval fell back to keyword matching, silently.
5. The graphics card was unused by design of that release, and the status said the opposite: the tier logic assigned zero layers to the card while the vision model used it, and the status report read fully_offloaded: true with offloaded_layers: 0.

And a sixth, which is not a bug we can fix in code: the 2B model states wrong facts with confidence (“a chicken lays 2–3 eggs a day”, a Social Security milestone at “age 64”), and answers direct questions with a “To determine X, follow these steps: 1. Understand the Basics” template. Our user documentation had said the assistant “tells you rather than guessing”. That sentence was false and has been removed.

2.3 What that says about the corpus

None of these is exotic. They are the shapes of ordinary conversation: an offer followed by a one-word acceptance; a question whose answer depends on the previous turn; two intents that look alike on the surface; an elliptical reply; a question about the world right now. We had written six hundred scenarios about topics and almost none about shapes. The harness could run what we asked of it; the picture of the user it was fed was ours.

2.4 Where the instrument was wrong

The failures above are the model’s or the router’s. These are ours, and they cost more:

• The serving backend corrupted two rounds of measurement (part 1.5). We scoped a training round against numbers that were largely artifact. The fix was to re-measure stock on honest serving before believing any delta — and to build a canary.
• Post-training batteries launched against the wrong model. On 2026-08-12 the batteries meant to grade a trained model were started without the established preparation. A live-daemon single-instance guard stopped the in-process runner; an unpinned override then fell silently back to stock; and even a pinned text-only override would have disabled vision relative to the baseline. Caught at collection before any result was sealed; the dead-end logs are sealed instead.
• A false “evidence destroyed” claim. The same day, a report stated that two rounds’ logs had been deleted. They were in the designated archive the whole time; the author had searched two other places first. Retracted the same hour; the rule that came out of it — an absence claim requires searching the designated archive first — is now written down.
• The 35B took seven launches to train at all: four failed attempts (two out of memory, two structural) on a two-card workstation, one launch against the wrong tree, and one failed smoke test before a researched placement worked. It then trained cleanly, exported, served, and produced six measured runs — three stock, three trained — and the adapter added nothing. The null result is this round’s recipe on that model, not the machine.
• Smaller ones, kept because the pattern is the lesson: a run aborted by a timeout in a boundary reset; an identical prompt reused across runs so a real fix read as “not confirmed”; a listening-socket tool that returned an empty table on a machine with a live session; a case-sensitive glob that “proved” a package absent when its directory existed under a capital letter.

The through-line: in this arc we found the instrument broken at least as often as the model.

3. Where we are (2026-08-25)

• Shipped: the first release (R001, 2026-08-16) and a point release (R001.1, 2026-08-20): three tiers in the catalog, the 9B trained, observability on by default, a signed boot chain, and the harness documented in the tree.
• Fixed on the development branch inside a day of reading the traces: the consent turn now carries the pending question and the named place into the web search; the sunset question can no longer route to the clock; both proven red-first with her exact sentences as fixtures on the real router path. The embedding server now starts with room for its whole context, shortens an oversized input on token boundaries and says so, sends bounded requests, and is asked nothing until it reports itself ready; its health alarm states the count that fired it. Automatic requested-layer planning now uses detected graphics memory and model/projector fit when those inputs are readable, rather than the tier table; achieved offload on her laptop is where it will be measured. The model-quality floor of the 2B tier is what it is; the documentation now says so.
• The harness is being turned around. Rather than paraphrasing more topics, the next step is branch coverage per conversation shape — offer-then-consent, prior-turn context, look-alike intents, elliptical replies, live-data questions — ten to twenty variants each, generated in code. Her three days of traces are the holdout set: nothing from them becomes training data or a scenario; they are what the shapes are tested against. A fix must show red on that holdout before and green after, or the shape is not covered. This work is planned, not landed, as of this page’s date.
• The gate on a release now includes a signed record of the installed-system checks, verified against the release key before anything else is read. That is not an InterGen change; it is the same lesson applied to the release process — trust a measurement only when its provenance is checked.

4. Where we’re going

• Field data as the loop, not the gate. Every future scenario family starts from a recorded conversation shape, not from a guess. The people using the release keep producing the shapes; the harness keeps growing branch coverage over them; the traces from each release are the next release’s holdout.
• The next training round runs against the reshaped corpus, with the same rule: three runs a side, a measured noise floor, and a verdict only above it. We expect the 2B tier to stay stock under the current recipe, and to be replaced rather than trained when a better small model fits the same hardware.
• A different model for the workstation tier. The 35B model is a mixture-of-experts with about 3 billion parameters active per token, and it measured in the 2B band on system tasks. The planned candidate to replace it is a dense 27-billion-parameter model — fewer parameters in total, but every one of them works on every token. It replaces nothing until it passes its gates (native vision, license, serving support, measured speed on the same hardware, and trainability); the workstation that trained the 9B is the only machine here that can train it.
• Everything measured on real hardware, reported with what it does not prove. Each change in this project states its proven surface and its unproven residue in the same breath. This page will do the same as it grows: each section dated, each number pointing at a recorded run.

Closing, from the project’s primary maintainer

We started with our best educated guesses regarding how we thought people would interact with InterGen — how they would phrase requests, what types of things they would ask, how he would answer them. A week’s worth of real-world use and interactions later has shown us we missed half of what we were aiming for. So, this document now lives to keep the records — where we started, where we are now and how we got here, and more importantly — with measured data and known failures in hand — where we’re going. This is going to be quite the ride :)