When organizations deploy artificial intelligence in regulated or air-gapped environments, sending sensitive queries over the internet to hosted API endpoints is simply not an option. Local LLM inference provides the definitive solution by running complete language model workloads directly on local infrastructure. Palimpsests offers an open-source, fully reproducible local inference engine built on a performant llama.cpp backend. Designed for Python 3.11+ environments, it allows companies to execute models locally, keep the attention state on-site, and enforce total network isolation.

Performance efficiency is built directly into the engine's architecture. Rather than re-reading system prompts or context windows on every call, Palimpsests holds shared prefixes warm in memory and executes multi-step tool calls seamlessly without re-prefilling context. Its KV persistence capability further allows computed attention states to be saved to disk and restored across process restarts. By pairing high-speed local processing with a cryptographic, offline-verifiable audit trail, this local inference framework empowers enterprises to harness powerful AI capabilities while maintaining strict compliance, zero network egress, and absolute data privacy.