A getting-started guide for the ultra-lightweight 1B-parameter local LLM Model page: https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF License: Apache-2.0 (inherited from the base model MiniCPM5-1B)

1. Model Overview

Item Details
Parameter scale 1B (1 billion)
Base model openbmb/MiniCPM5-1B
Fine-tuning data Fable 5 data (post-training)
Distribution format GGUF (quantized build for llama.cpp-family runtimes)
Max context 128K tokens (131,072 per upstream config.json)
Architecture llama
Chat template MiniCPM5's native template embedded in GGUF metadata
Supported languages English, Chinese
Specializations Code generation/debugging, instruction following, tool calling

This model is an ultra-lightweight model that can run on CPU alone or on a low-spec GPU environment, usable across the entire GGUF-compatible runtime ecosystem — llama.cpp, Ollama, LM Studio, jan, KoboldCpp, and more. It features a hybrid reasoning structure that can switch between "Thinking" mode (chain-of-thought reasoning) and "No Think" mode (fast response).

Note: "Fable 5" in the name refers to the source of the training data, and this is an unrelated open-source community model — not Anthropic's commercial closed model "Claude Fable 5."

2. Available Files (Choosing a Quantization)

File Quantization Size Notes
...-Q4_K_M.gguf Q4_K_M ~657 MB Minimum footprint, for low-memory environments
...-Q5_K_M.gguf Q5_K_M ~751 MB Balance of quality and size
...-Q8_0.gguf Q8_0 ~1.1 GB Recommended default
...-F16.gguf F16 ~2.1 GB Full-precision converted original

Selection guide: 1B models are relatively sensitive to quantization loss, so if you have memory to spare, using Q8_0 as your default is recommended. Only consider Q4_K_M in extremely constrained environments with under 2GB of RAM.

3. Installation and Usage

3.1 llama.cpp (CLI)

macOS / Linux installation:

curl -LsSf https://llama.app/install.sh | sh

Windows (WinGet):

winget install llama.cpp

Direct inference from the terminal:

llama cli -hf GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF:Q4_K_M

Running directly from a local file (Q8_0 example):

llama-cli \
  -m MiniCPM5-1B-Claude-Opus-Fable5-Thinking-Q8_0.gguf \
  -p "Write a Python function to merge two sorted lists." \
  -n 512 \
  --temp 0.9 --top-p 0.95 \
  -c 8192

The context length (-c) supports up to 131,072, but the actually usable length must be tuned based on your VRAM/RAM.

3.2 llama.cpp Server (OpenAI-Compatible API)

llama-server \
  -m MiniCPM5-1B-Claude-Opus-Fable5-Thinking-Q8_0.gguf \
  -c 8192 --port 8080

Once the server starts, you can use the web UI and the OpenAI-compatible /v1/chat/completions endpoint at http://localhost:8080. Existing code built on the OpenAI SDK integrates as-is — just change the base_url.

3.3 Ollama

ollama run hf.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF:Q4_K_M

This pulls and runs the model directly from the HuggingFace repository, with no need to write a separate Modelfile.

3.4 LM Studio / jan / KoboldCpp

Just download the .gguf file from the repository and load it. Since the MiniCPM5 chat template is embedded in the GGUF metadata, no manual template configuration is needed.

  • LM Studio: type GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF in the search bar and download the quantization version you want

3.5 llama-cpp-python (Python Integration)

pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF",
    filename="MiniCPM5-1B-Claude-Opus-Fable5-Thinking-Q8_0.gguf",
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Write a Python function to merge two sorted lists."}
    ]
)
print(response["choices"][0]["message"]["content"])

3.6 vLLM

pip install vllm
vllm serve "GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF"

Calling the OpenAI-compatible API:

curl -X POST "http://localhost:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

3.7 Docker Model Runner

docker model run hf.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF:Q4_K_M

3.8 Coding Agent Integration (Pi / Hermes / OpenClaw)

The approach is to run a llama.cpp server as the backend, then register its OpenAI-compatible endpoint (http://localhost:8080/v1) as a custom provider in each agent. This is useful for local coding agent experimentation.

4. Recommended Sampling Parameters

Inherits the base model's (MiniCPM5-1B) default generation settings.

Mode Parameters
Think (default) temperature=0.9, top_p=0.95
No Think temperature=0.7, top_p=0.95, enable_thinking=False
  • Think mode: outputs an internal reasoning block before the final answer. Suited to complex coding/reasoning tasks, but requires post-processing logic to parse and strip the reasoning block when integrating into a pipeline.
  • No Think mode: answers immediately with no reasoning process. Suited to latency-sensitive chatbot/classification tasks.

5. Performance Benchmarks

Tool-calling performance was significantly improved in the V2 version (per figures published by the model's creators).

Model BFCL (non_live) BFCL (live) API-Bank
MiniCPM5-1B (Base) 41.51% 60.24% 7.30%
V2 Thinking model 43.06% 63.33% 22.10%

The API-Bank score in particular rose roughly 3x, from 7.30% to 22.10%, demonstrating the effect of tool-calling-specialized training. A derivative model further specialized for tool use, MiniCPM5-Claude-Toolusage, is also provided separately.

6. Strengths

  • Ultra-lightweight local execution: at as little as 657MB (Q4_K_M), it can run on CPU alone, Raspberry Pi-class SBCs, and older laptops
  • 128K long context: an unusually long context window for a 1B-class model, usable for analyzing large codebases and long documents
  • Hybrid reasoning: switch between Think/No Think modes to choose quality vs. speed per task
  • Enhanced tool calling: designed with the aim of SOTA tool-calling performance among comparable 1B open-source models
  • Broad runtime compatibility: supports virtually the entire GGUF ecosystem — llama.cpp, Ollama, LM Studio, vLLM, Docker, and more
  • Apache-2.0 license: few restrictions on commercial use and redistribution
  • Embedded template: the chat template is included in the GGUF, so it's ready to use immediately without separate configuration

7. Weaknesses and Limitations

  • Fundamental limits of a 1B model: a significant gap remains vs. frontier-class models (GPT-4, Claude, etc.) on complex general reasoning, multi-step logic, and broad world knowledge. It's more realistic to use it for specific tasks (coding assistance, tool-call routing, classification) than as a general-purpose assistant
  • Extra output from Thinking mode: since the reasoning block is output before the final answer, application integration requires additional parsing logic. Token consumption and latency also increase proportionally to the reasoning block
  • Effective context constraints: 128K is a theoretical maximum, and the actually usable length depends on the runtime and hardware (RAM/VRAM). A setting around 8K is more realistic on low-spec hardware
  • Quantization sensitivity: due to its small size, quality degradation can be relatively pronounced at Q4 and below quantization (the reason Q8_0 is recommended)
  • Language coverage: officially supported languages are centered on English and Chinese. Korean performance needs separate verification
  • No inference provider support: not currently deployed on HuggingFace Inference Providers, so it can't be used as a cloud API (local execution only)

8. Suggested Use-Case Scenarios

Scenario Recommended configuration
Local coding assistant (offline) Q8_0 + Think mode, llama-server integrated with an editor
On-device tool-call router Q8_0 + Think mode (leveraging BFCL/API-Bank strengths)
Low-latency chatbot / classifier Q5_K_M + No Think mode
Large document/codebase summarization F16 or Q8_0 + a long -c setting (sufficient RAM required)
Edge device experimentation Q4_K_M + context of 4K or less

9. Reference Links