AI and language models
Running and talking to language models.
Model files and tokens
gguf: GGUF model files read and written over a byte slice.safetensors: Hugging Face safetensors files read.bpetokenize: Byte-level byte-pair-encoding tokenization, as in GPT-2 and Llama 3.
Running a model
llmrun: A language model from a GGUF file running on the GPU: Qwen3 or a Llama-layout model, with a KV cache and sampling.llmkernels: The GPU compute kernels for a Llama-family transformer, each checked against a CPU twin.llmreference: The CPU side: weight block layouts, decoders, and CPU twins of the kernels.
Talking to a model
chatapi: A client for the OpenAI-compatible Chat Completions API, over json, sse and actorhttp.