About OpenCerebral
OpenCerebral is a non-profit research project that trains small language models from scratch and publishes the complete recipe. It is the successor to New Millennium Artificial Intelligence (NMAI), a division of the Klondike Software Project, and it inherits that project's position that software you cannot inspect is software you do not control.
What we actually do
We pretrain decoder-only language models between 1M and 250M parameters on public web corpora, evaluate them on standard benchmarks, and release the weights. Alongside them we train image diffusion models from scratch at 128×128 — the Lucy family. There is no proprietary data, no undisclosed pretraining mix, and no held-back checkpoint. Model cards carry token counts, wall-clock times, loss values, and the passes that regressed as well as the ones that helped.
One GPU
Every OpenCerebral model to date was trained on a single NVIDIA RTX 3060. A 2.50B-token pretraining run at 125M parameters takes about 33 and a half hours on that card; a 1M-parameter model takes under two. That has a real virtue: both are reproducible by anyone with a mid-range desktop GPU and patience, which is the only way a published recipe means anything.
It is worth being exact about what the constraint is. Consumer GPUs are what we have, not a limit we have picked out of principle. The sizes here reflect a compute budget, and the budget is the thing we would most like to change — larger models are intended, and the recipe scales before the hardware does. If you have compute to offer, that is the contribution that moves this furthest; see Get Involved.
Principles
- Open weights, open recipe. A model whose training data and procedure are secret is a black box no matter how permissive its license is.
- Publish the failures. The littlerock-1M-arithmax card exists because a negative result that was hard to detect is more useful than another incremental win.
- No benchmark laundering. No evaluation data is trained on and no checkpoint is selected on a benchmark. Where our own held-out estimates turned out to be wrong, we say so and show by how much.
- Permissive licensing. Apache 2.0 wherever the training data allows it — no acceptable-use riders, no field-of-use restrictions. The one exception is Lucy-2, which is research-only because CelebA is, and we say so plainly rather than quietly relicensing upstream terms.
Honest limitations
These are small base models. They produce text that is frequently inaccurate, inconsistent, or offensive. None has received alignment or safety tuning. They are useful for research, for education, for embedded and offline experiments, and for understanding what capability actually emerges at a given scale. They are not useful as a factual reference and should not be deployed without supervision.
People
| Name | Role |
|---|---|
Joseph Jones (KlondikeDev) |
Training, evaluation, releases |
OpenCerebral is small and open to contribution — see Get Involved.