Usage

Transformers — base model

Base models continue text. They do not follow instructions.

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("opencerebral/Boris-1.3-125M")
model = AutoModelForCausalLM.from_pretrained("opencerebral/Boris-1.3-125M")

ids = tok("The ocean is", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=40, do_sample=True, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

Transformers — Boris-2 previews

Boris-2 ships its own modeling code, so pass trust_remote_code=True. It has no KV cache, so generation recomputes the prefix each step. Run it in float32 or bfloat16, not float16.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "opencerebral/Boris-2-Preview-2"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)

x = tok("The water cycle begins when", return_tensors="pt")
y = model.generate(**x, max_new_tokens=40, do_sample=True, top_p=0.95)
print(tok.decode(y[0], skip_special_tokens=True))

Transformers — instruct model

Instruct models ship a chat template, so apply_chat_template() works directly.

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("opencerebral/Boris-1.3-125M-Instruct")
model = AutoModelForCausalLM.from_pretrained("opencerebral/Boris-1.3-125M-Instruct")

messages = [{"role": "user", "content": "What's a good way to start learning C?"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=120, do_sample=True, top_p=0.95)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

llama.cpp — GGUF

The GGUF builds run on CPU with no Python involved.

# interactive chat, pulled straight from Hugging Face
llama-cli -hf opencerebral/Boris-1.3-125M-Instruct-GGUF -cnv

# or an OpenAI-compatible local server
llama-server -hf opencerebral/Boris-1.3-125M-Instruct-GGUF --port 8080

See GGUF Downloads for the available quantizations.

Sampling notes

Small models degrade quickly with greedy decoding — they fall into repetition loops. Sample. top_p=0.95 with a temperature near 0.8 is a reasonable starting point. Keep prompts short: the context window is 1024 tokens for Boris 1.x, 2048 for Boris-2, and 512 for littlerock, and quality drops well before the limit.

Before you deploy anything

These models have received no alignment or safety tuning and will produce inaccurate, inconsistent, or offensive text. Do not use them as a factual reference or run them unsupervised in front of users.