Bugs
Report problems with a model on that model's Community tab on Hugging Face — it is attached to the artifact itself, so the report stays next to the thing it describes. Go to the model page from the model index and open Community.
For problems with this website, or anything that does not belong to a single model, use the contact page.
What makes a useful report
- The exact model repository, including whether it is the base, instruct, or GGUF variant, and which quantization.
- How you ran it — Transformers version, llama.cpp build, Ollama, LM Studio.
- The prompt, verbatim, and the sampling parameters. Temperature and
top_pmatter enormously at this scale. - What you expected and what you got.
Things that are not bugs
These are 1M to 250M parameter models with no alignment tuning. The following are expected behavior and documented on every model card:
- Factually wrong answers, confidently stated.
- Losing the thread over a long conversation. The context window is 1024 tokens for Boris and 512 for littlerock.
- Repetition loops under greedy decoding. Sample instead — see Usage.
- Base models ignoring instructions. Base models continue text; use an Instruct variant.
- Offensive output. There has been no safety tuning.
A benchmark reproduction that disagrees with our published numbers, though, is a bug report and we want it.