ling-llm

# ling-llm

A freestanding Qwen3 inference engine: `no_std` + `alloc` + `libm`, no operating system, no GPU. It reads the model, its tokenizer and its
vocabulary out of one GGUF file, runs it on the CPU, and streams text. It exists so **LingOS can ask a language model a question with
nothing else installed**, but it runs anywhere Rust does (it is also what the host tests compare against `candle`).

* `gguf`   the container, read through a `Source` trait (a file, a disk image, bytes in memory)
* `quant`  F32 / F16 / Q8_0 / Q4_K / Q6_K tensors, held in chunks of at most 8 MiB, and the matrix products (Q8_0 in integers against
           Q8_0 activations, the way llama.cpp and candle do it; K-quants dequantized a block at a time)
* `tok`    byte-level BPE built from the GGUF's own tokens and merges, with Qwen's pre-tokenizer written out as a scanner
* `model`  the Qwen3 transformer with an f16 KV cache; prompts go through in batches so each weight row is read once per batch
* `sample` greedy and temperature / top-p
* `chat`   the ChatML prompt (the same one `ling-ai` builds) and the generation loop

## Checked against the real thing

`cargo test -p ling-llm --features std --release --test vs_candle -- --nocapture` loads `assets/models/Qwen3-0.6B-Q8_0.gguf` and compares:
the tokenizer with the HF `tokenizers` crate on 14 awkward strings (Unicode, specials, contractions, digits, tabs) - identical; the first
logits against candle's quantized Qwen3 - cosine 0.9985; ten greedy tokens - identical ("1. Blue / 2. Red / 3."). On a desktop CPU,
built for generic x86-64, it does about 16 tokens/s with the 0.6B model.

`cargo check -p ling-llm --target x86_64-unknown-none` builds it bare-metal.

## In LingOS

`crates/ling-kernel/src/drivers/llm.rs` loads the weights from a raw second IDE disk and runs generation in a background kernel task;
the Terminal gets `ask <question>` and `llm [status|load|stop]`. See LingOS's README.