Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

PyTorch / TensorFlow / JAX

This page covers the blessed path for running the major Python machine-learning frameworks (PyTorch, TensorFlow, JAX) on InterGenOS, and how that path relates to the GPU acceleration stack the system already provides.

InterGenOS is a built-from-source distribution. The guiding principle holds here as everywhere else: you should end up with a machine you understand, can modify, and can trust. ML tooling is no exception, and nothing on this page bypasses the system’s signed Secure Boot chain, dm-verity integrity, or the safety posture of the local assistant.

What ships today

InterGenOS 1.0-dev (build id v1.0-dev1) is assembled across six package tiers (toolchain, core, base, desktop, ai, extra) in a 20-phase build. The ai tier is small and focused: it carries the components of the InterGen local assistant rather than a general-purpose data-science stack.

The shipping AI layer is the local assistant stack:

  • InterGen is a tiered, hardware-detected, offline-first local assistant built on Qwen models with zero telemetry. It probes RAM and GPU on the host and selects an appropriately sized model. Inference is served locally over an HTTP API by a llama-server subprocess from llama.cpp.
  • InterGen Sentinel is a pluggable security scanner. Its default configuration runs entirely on-device: a fast local-rules pass plus an optional deep pass backed by a small local Qwen classifier. Six cloud providers (Claude (Anthropic), Gemini (Google), Copilot (Microsoft), ChatGPT (OpenAI), Grok (xAI), DeepSeek) are opt-in only; none is configured by default.

This matters for ML work because the assistant’s inference path (llama.cpp serving a quantized Qwen model) is independent of the Python framework stack you would use for training or general tensor compute. Running PyTorch, TensorFlow, or JAX does not interfere with InterGen, and InterGen does not depend on them.

The blessed path

The supported, repeatable way to run these frameworks on InterGenOS is to install them into a project-local Python virtual environment rather than into the system Python. This keeps framework dependencies out of the verified system image, leaves the integrity-checked base untouched, and makes each project reproducible and disposable.

The general shape of the path is:

  1. Confirm your GPU is detected and the acceleration runtime is present (see the other GPU pages in this section).
  2. Create a per-project virtual environment.
  3. Install the framework build that matches your accelerator into that environment.
  4. Verify the framework can see the GPU before you rely on it.

Each framework distributes accelerator-specific builds. You install the build that matches the GPU and runtime on your machine; a CPU-only build will run but will not use the GPU.

GPU detection vs. framework GPU support

A working desktop GPU under GNOME 49 on Wayland (the desktop InterGenOS ships today) is not the same thing as a framework-visible compute device. Verify both layers:

  • The system sees the GPU (kernel driver loaded, device present).
  • The framework sees the GPU (its own device query reports the accelerator).

Use each framework’s built-in device check after installation:

  • PyTorch exposes a runtime check for accelerator availability and device count.
  • TensorFlow can list the physical compute devices it has registered.
  • JAX reports the active backend and the devices bound to it.

If the system sees the GPU but the framework does not, the mismatch is almost always between the framework build you installed and the accelerator runtime on the host. Reinstall the framework build that matches your runtime rather than changing the system.

Staying inside the security posture

Two things to keep in mind:

  • Frameworks pull large dependency trees from external indexes. That is content crossing into your machine. Treat third-party model weights and packages with the same care as any other untrusted download, and prefer pinned, hash-checked installs inside the project environment.
  • InterGen’s safety classifier still applies to assistant-driven actions. If you ask the local assistant to set up or manage a framework environment, state-changing steps are surfaced for confirmation and destructive operations are refused outright. Having the assistant manage your environment does not weaken that boundary.