LLMRabbi

Feeding a book's raw text into training gives a model a better memory of the book. It doesn't change how the model reasons, converses or applies the book's ideas. LLMRabbi takes a different route.

The idea

Philosophy teachers sometimes skip the essay and have students roleplay a conversation from inside a worldview. To hold up their end, students have to internalize the ideas well enough to apply them to questions the text never addresses. LLMRabbi does the same for a model. It reads a section of a book, raises topics the book has a perspective on, and has an LLM write the assistant's side of the exchange grounded in that passage. Fine-tuning on those exchanges shifts how the model weighs things by default.

That's also why a system prompt like "answer as a Stoic" falls short. It reaches for the stereotype, the most quotable lines, and collapses when the conversation drifts somewhere the stereotype doesn't cover. Training on grounded dialogue changes the reasoning underneath. We've used the approach on self-help, philosophy, religious and business books.

How it works

• Parsing and chunking. ePub and PDF books are split into ordered units with chapter and page awareness, chunked by pages or chapters, with a minimum word count to skip title pages and section breaks.
• Local inference. Any GGUF model runs through llama.cpp with full control of sampling, and Anthropic, OpenAI or Ollama can be used instead.
• Constrained output. GBNF grammars or JSON Schema keep every sample in a usable format, and several samples per chunk add variety.
• Resumable runs. State is checkpointed after every inference pass and validated against a hash of the source book, so a long run can survive an interruption.

The app pairs a FastAPI backend with a React and Vite front end, and ships as GPU and CPU Docker images.