Package & Build Failures

This page covers two distinct failure surfaces in InterGenOS:

1. Package-manager failures — problems while installing, removing, upgrading, or verifying packages on a live system with pkm.
2. From-source build failures — problems while constructing packages and the ISO with the build orchestrator, most commonly the validate-phase gates that run before any compilation.

Read the symptom carefully to decide which surface you are on. A pkm failure touches the live root filesystem and its package database; a build failure happens in an isolated build environment and never modifies your installed system.

InterGenOS is built from source and ships a machine you understand, can modify, and can trust. The package and build tooling are designed so that every state change is auditable. When something fails, the goal is to find the root cause, not to paper over it.

Part 1 — Package-manager (pkm) failures

pkm manages the installed operating system: installation, removal, querying, and integrity verification of pre-compiled binary archives (.igos.tar.gz). It operates on the live filesystem and keeps two coordinated records of state:

• A SQLite database at /var/lib/igos/pkm.db — the primary source of truth for fast queries and transactions.
• Text manifests under /var/lib/igos/packages/ — human-readable files generated alongside the database records for inspection.

This duality gives you performance without sacrificing auditability. If the two ever appear to disagree, treat the database as authoritative and report the divergence.

Integrity verification fails (pkm verify)

pkm verify recalculates the SHA-256 hash of every installed file on disk and compares it against the expected hash stored in the database’s files table. A mismatch means the file on disk differs from what was deployed — either accidental corruption or unauthorized modification.

What to do when verify reports a mismatch:

• Identify the file and its owning package. The database indexes files by path, so a single mismatched file points you directly at the package that owns it.
• Decide whether the change was intentional. Files under /etc/ are tracked as configuration and are expected to drift as you edit them. A mismatch on a binary or library is more serious.
• Do not dismiss a mismatch. A changed binary on a security-only-aligned system is a finding to investigate, not to acknowledge and move on.

Archive trust / verification rejected at install time

Before installation, the CLI supports an --archive-trust flag, which accepts three values: strict (the default), loose, and repo-only. In strict or repo-only modes, the incoming .igos.tar.gz archive’s SHA-256 hash is computed and checked against the trusted available index that was synced from the repository with pkm sync.

If installation is rejected on trust grounds:

• The archive hash does not match the index entry. Re-run pkm sync to refresh the available cache, then retry — your index may be stale relative to the published archive.
• If the hash still does not match after a fresh sync, stop. A persistent mismatch between a downloaded archive and a freshly-synced trusted index is exactly what archive verification exists to catch.

A directory-collision or invariant check aborts installation

Before deploying files, pkm runs invariant checks. The two it is documented to enforce:

• Directory collisions — it refuses to let a package replace a critical path (for example, turning the /lib symlink into a directory).
• Supersede ordering — if a package declares SUPERSEDES, the predecessor it supersedes must exist and be correctly ordered in the install queue.

If install aborts here, the package’s manifest is asking for a filesystem change that would damage the system layout, or an upgrade ordering that cannot be satisfied with the packages on hand. Install the predecessor first, or fix the queue order, rather than forcing past the check.

The filesystem deploy step is the point of no return: once files are written to the root filesystem, the package is committed to disk before the database transaction that records it. The invariant checks run before that write precisely so a bad manifest never reaches it.

Understanding upgrades: the “supersede” model

pkm does not do in-place version replacement; it uses a supersede model. When package B supersedes package A:

1. Files present in both A and B are overwritten on disk by B, and their database ownership transfers from A to B with updated SHA-256 hashes.
2. Files that existed in A but are not in B are left in place, still owned by A’s historical record.
3. A’s record is timestamped superseded_by, pointing at B.

This is how a package can split cleanly (for example, util-linux into util-linux-core and util-linux-extra) without orphaning files or breaking dependencies. If after an upgrade you find files you expected to be gone, this model — not a bug — is usually why: those files belonged only to the superseded record.

Removal leaves files behind, or warns about a modified file

Removal is deliberately cautious:

• Configuration files under /etc/ are skipped to preserve your data, unless you pass --force.
• If a file’s on-disk hash differs from the database hash (it was modified after install), pkm emits a warning but still removes it.
• Directories are removed only when they become completely empty.

So “leftover” files after a removal are usually preserved configuration or a non-empty shared directory, both by design. The modified-file warning is informational; it is telling you the file you removed was not the one originally deployed.

Config files: .new after an upgrade

pkm tracks the original hash of each /etc/ file in a config_files table. When an upgrade ships a new version of a config file you have modified, pkm will attempt a safe merge or leave the new version as a .new file beside your edited one. Review and reconcile .new files; they are how upgrades avoid silently overwriting your changes.

Part 2 — From-source build failures

The build orchestrator constructs InterGenOS from source in an isolated environment, then assembles a signed, verity-protected ISO. As of this writing the build runs 20 phases:

validate → verify-sources → setup → toolchain → chroot-prep → chroot-tools → core → config → core-extra → base → kernel → desktop → ai → extra → bootloader → image → manifest → squashfs → ukis-verity → iso, with an optional publish step.

Packages are organized into six tiers — toolchain, core, base, desktop, ai, and extra — totalling roughly 850 packages across the tree as of 2026-06-15. These counts drift as the tree evolves; derive the live count from the package tree rather than trusting a fixed number.

The overwhelming majority of “my build failed before it really started” reports are validate-phase gate failures, covered next. A genuine compilation failure happens later, inside the build environment, and looks different: a compiler error, a failing test, or a missing build dependency surfaced during a tier build.

Telling a validate-phase gate from a real build bug

The validate phase is the first phase and runs in roughly one second. It runs entirely on the host side: it reads package.yml files, the audit database, and source tarballs. It does not create or touch the isolated build root, which is set up in a later phase.

This has a practical consequence that saves a lot of time:

• A validate failure halts the orchestrator with exit 1 before any toolchain work.
• The build root and build scratch space are left untouched (on a clean baseline they do not exist yet).
• You can fix the finding and relaunch without reverting your build environment — there is no contamination to undo.

A fail in roughly 0.1–1 second is almost always a validate-phase gate, not a build bug.

The validate-phase gates, in order

A separate aggregate reconciliation report is also produced. It is advisory rather than a build gate, but its mismatch rows are real signals that should be triaged, not ignored.

The two gates that fire most often when you add or change a package are audit-coverage (Gate 2) and tier validation (Gate 3).

Gate 2 — Audit-coverage: “N packages need audit work”

A typical failure:

[audit-preflight] missing audit:     3
[audit-preflight] stale (version):   1
[audit-preflight] FAIL: 4 packages need audit work.

Every in-scope package must carry a current audit record. The gate distinguishes three failure classes:

• missing — no audit record at all. Cause: a newly-added package.
• stale — the audit record’s version does not match the package.yml version. Cause: a version bump.
• drift — the audit’s recorded build dependencies do not match the package.yml build dependencies. Cause: dependencies edited without re-auditing.

Understand the data flow or you will re-aggregate to no effect. The per-package auditor reads the package.yml and the source tarball and writes a per-package audit record into the private audit store. An aggregation step then ingests all those records into the build database that the gate actually reads. The gate reads the database, so you must re-aggregate after auditing; auditing alone does not update what the gate sees.

To resolve, for each flagged package: refresh its audit record, re-aggregate into the database, then re-run the audit-coverage check until it reports PASS. Commit the refreshed audit records to the private audit store.

Gate 3 — Tier validation: “MOVE→<tier>” or “UNCLEAR”

This gate computes each package’s canonical “natural tier” and compares it to the tier the package declares. Verdicts:

• MOVE→X — the natural tier differs from the declared tier.
• UNCLEAR — no natural tier could be determined.
• CROSS-TIER-DEP — a build/host dependency resolves to a later tier. This is a genuine ordering violation; fix the dependency, never demote the consumer.

The natural tier is derived in layers: explicit curated category lists and naming patterns first, then consumer inference from the build/host reverse-dependency graph (a package takes the earliest tier that consumes it). A subtle trap: the inference graph is built from build and host dependencies only. A package consumed only as a runtime dependency has no graph consumer, so inference cannot place it and it comes back UNCLEAR.

Do not blindly obey the verdict. Evaluate it against the tier source-of-truth document first. There are two correct outcomes:

1. The package really is in the wrong tier — move it: relocate its directory to the correct tier, update the tier: field in package.yml, fix builder wiring as needed, then re-audit and re-aggregate (the audit record carries the tier).
2. The package is in the correct tier but the heuristic misfires — encode the source-of-truth: add an explicit entry for the package to the validator’s appropriate category list. This is legitimate only when the tier document already classifies the package that way; you are encoding the documented truth, not gaming the gate. If the document is genuinely ambiguous, that is an operator judgment call, not a self-serve edit.

Reconciliation mismatches (advisory): a required dependency not in our build deps

The reconciliation report can flag that an upstream-required dependency is missing from a package’s declared build dependencies. This is advisory, but every mismatch is a real signal to triage:

• Real gap — the dependency is genuinely needed and was forgotten. Add it, re-audit, re-aggregate.
• Intentional divergence — InterGenOS builds the package differently from upstream, so the upstream dependency does not apply. Record the divergence at the comparison site with an inline justification. A permanent deviation from upstream requires operator authorization: propose it with evidence rather than self-authorizing.

There is also a per-package override mechanism that can skip a package’s audit checks temporarily, carrying a reason, an approver, and an expiry date. Reserve it for a genuinely temporary, whole-package gap. For a permanent single-dependency divergence, prefer the precise, self-documenting divergence record over a broad whole-package override.

After a fix: relaunch without reverting

Because the validate phase exits before the build root is created, the recovery loop for a validate gate is cheap:

1. Read the failing gate’s output and identify the packages and the failure class.
2. Apply the correct fix (refresh audits, move a tier, encode the source-of-truth, or triage a mismatch).
3. Re-run the checks locally until every gate is green.
4. Commit the changes (public tree for tier/validator edits, private store for audit records), then relaunch the build.

No revert of the build environment is needed for a validate-phase fix.

A note on packages changed after a successful build

When a from-scratch build has already succeeded and been captured as a reference, any package added or changed after that point is where unrecorded-but-intentional decisions hide. Before resolving a gate on such a package, review its history since the last good build so you preserve, rather than silently undo, a deliberate choice. A correct-but-unrecorded divergence can sit un-triaged for days precisely because the build was passing.

Where state lives

The history table is an append-only log of every install, removal, and supersede. When you need to reconstruct what changed and when, it is the audit trail.

Related pages

• Troubleshooting FAQ
• Forge install guide