About OpenCerebral
OpenCerebral is a non-profit research project that trains small language models from scratch and publishes the complete recipe. It is the successor to New Millennium Artificial Intelligence (NMAI), a division of the Klondike Software Project, and it inherits that project's position that software you cannot inspect is software you do not control.
What we actually do
We pretrain decoder-only language models between 1M and 250M parameters on public web corpora, evaluate them on standard benchmarks, and release the weights. There is no proprietary data, no undisclosed pretraining mix, and no held-back checkpoint. Model cards carry token counts, wall-clock times, loss values, and the passes that regressed as well as the ones that helped.
One GPU
Every OpenCerebral model to date was trained on a single NVIDIA RTX 3060. That constraint is the point rather than an apology for it. A 2.50B-token pretraining run at 125M parameters takes about 33 and a half hours on that card; a 1M-parameter model takes under two. Both are reproducible by anyone with a mid-range desktop GPU and patience, which is the only way the published recipe means anything.
Principles
- Open weights, open recipe. A model whose training data and procedure are secret is a black box no matter how permissive its license is.
- Publish the failures. The littlerock-1M-arithmax card exists because a negative result that was hard to detect is more useful than another incremental win.
- No benchmark laundering. No evaluation data is trained on and no checkpoint is selected on a benchmark. Where our own held-out estimates turned out to be wrong, we say so and show by how much.
- Permissive licensing. Everything is Apache 2.0. No research-only clauses, no acceptable-use riders, no field-of-use restrictions.
Honest limitations
These are small base models. They produce text that is frequently inaccurate, inconsistent, or offensive. None has received alignment or safety tuning. They are useful for research, for education, for embedded and offline experiments, and for understanding what capability actually emerges at a given scale. They are not useful as a factual reference and should not be deployed without supervision.
People
| Name | Role |
|---|---|
Joseph Jones (KlondikeDev) |
Training, evaluation, releases |
OpenCerebral is small and open to contribution — see Get Involved.