About OpenCerebral
OpenCerebral is a non-profit research project that trains small language models from scratch and publishes the complete recipe. It is the successor to New Millennium Artificial Intelligence (NMAI), a division of the Klondike Software Project, and it inherits that project's position that software you cannot inspect is software you do not control.
What we actually do
We pretrain decoder-only language models between 1M and 250M parameters on public web corpora, evaluate them on standard benchmarks, and release the weights, along with chat and reasoning fine-tunes of the larger ones. Alongside them we train image diffusion models from scratch at 128×128 — the Lucy family. There is no proprietary data, no undisclosed pretraining mix, and no held-back checkpoint. Model cards carry token counts, wall-clock times, loss values, and the passes that regressed as well as the ones that helped.
One GPU
Every OpenCerebral model through Boris-1.7 was trained on a single NVIDIA RTX 3060, and Boris-2 and its fine-tunes were trained on a single NVIDIA V100 32GB. A 2.50B-token pretraining run at 125M parameters takes about 33 and a half hours on that card; a 1M-parameter model takes under two. Boris-2's 200B-token run took weeks on the V100. That has a real virtue: the smaller runs are reproducible by anyone with a mid-range desktop GPU and patience, which is the only way a published recipe means anything.
It is worth being exact about what the constraint is. One GPU at a time is what we have, not a limit we have picked out of principle. The sizes here reflect a compute budget, and the budget is the thing we would most like to change — larger models are intended, and the recipe scales before the hardware does. If you have compute to offer, that is the contribution that moves this furthest; see Get Involved.
Principles
- Open weights, open recipe. A model whose training data and procedure are secret is a black box no matter how permissive its license is.
- Publish the failures. Model cards report the passes that regressed alongside the ones that helped, because a negative result is more useful than another incremental win.
- No benchmark laundering. No evaluation data is trained on. Where a released checkpoint was chosen by benchmark score, as Boris-2's short final stage was, the card says so, says how many candidates were tried, and says how much of the margin could be noise. Where our own held-out estimates turned out to be wrong, we say so and show by how much.
- Permissive licensing. Apache 2.0 wherever the training data allows it — no acceptable-use riders, no field-of-use restrictions. The one exception is Lucy-2, which is research-only because CelebA is, and we say so plainly rather than quietly relicensing upstream terms.
Honest limitations
These are small models. They produce text that is frequently inaccurate, inconsistent, or offensive. None has received alignment or safety tuning. They are useful for research, for education, for embedded and offline experiments, and for understanding what capability actually emerges at a given scale. They are not useful as a factual reference and should not be deployed without supervision.
People
| Name | Role |
|---|---|
Joseph Jones (KlondikeDev) |
Training, evaluation, releases |
OpenCerebral is small and open to contribution — see Get Involved.
