Trained from scratch · single H100 · one epoch

A language model that only ever read case law, SEC filings, and a slice of the open web.

125.8M parameters. No pretrained checkpoint, no distillation — every weight was initialized fresh and trained on 2.04 billion tokens cleaned, deduplicated, and decontaminated from three public sources. This page is the full paper trail.

125,847,552 params 16,384 vocab · custom BPE 2.04B train tokens val loss 2.326 · ppl ≈10.2
base model · ready ankitw497/slm-125m-base
40 tokens
base model — raw completion
// output will appear here — one continuous, unedited sample from the model

This is the raw base checkpoint — one epoch, no instruction-tuning, no RLHF, no safety filtering beyond what the training data itself excluded. It completes text the way a language model from 2019–2020 would; it does not follow instructions or answer questions directly. Numbers and case names it produces are learned style, not retrieved fact — treat everything it writes as fabricated. Served live from a CPU container on Modal; first request after idle may take 10–20s to cold-start.

Architecture

A standard LLaMA, scaled all the way down.

No architectural novelty here on purpose — a plain decoder-only transformer, so every result below is attributable to the data and training recipe, not a clever architecture.

Total parameters125,847,552 (~125.8M)
ArchitectureLLaMA · transformers.LlamaForCausalLM
Layers12
Hidden size768
Intermediate size3,072 (SwiGLU)
Attention heads12 · head dim 64
KV heads12 (MHA, not GQA)
Vocabulary16,384 · custom byte-level BPE
Context length1,024 tokens
Positional encodingRoPE · θ = 10,000
NormalizationRMSNorm · ε = 1e-5
ActivationSiLU (SwiGLU)
EmbeddingsTied input/output
Attention biasNone
Data pipeline

Legal-first, not 70/20/10.

The two legal sources only contain about 2 billion clean tokens between them — nowhere near enough for a 70% share of a 10B-token corpus. So the recipe takes all of both legal sources and tops up with a capped slice of web text.

SourceRowsFieldEst. availableToken cap
US case law HFforLegal/case-law282,390document~0.81B1.0B
SEC filings PleIAs/SEC48,543text~1.16B1.3B
fineweb-edu sample-10BT9,670,000text~11.67B0.5B
01 · Clean

Line filters, boilerplate, OCR gate

Line-level noise filtering, boilerplate stripping (10-K headers, page numbers, SEC letterhead), 4-gram repetition detection, ASCII/langdetect English gate, and a dictionary-based OCR-garble gate for scanned case law.

2.68B proxy tokens kept · ~97%
02 · Dedup + decontaminate

Exact + near-dup + eval leakage

BLAKE2 exact-hash dedup, MinHash/LSH near-duplicate removal on case law (threshold 0.8), and a 13-gram contamination strip against the CaseHOLD / LexGLUE eval sets.

2.40B tokens · 24k contaminated docs removed
03 · Tokenize + pack

Custom 16K BPE, 99/1 split

A fresh byte-level BPE tokenizer trained on the cleaned corpus, then packed into 1,024-token windows with an <|eos|> between documents, every 100th window routed to validation.

2.04B train tokens · 20.6M val
Realized mix
case law · ~712M tok SEC filings · ~860M tok fineweb-edu · ~464M tok
Training run

2.85 hours on one H100, start to finish.

8× fewer GPUs than the initial default — at 125M parameters the model can't keep a larger cluster fed, so the extra GPUs would mostly sit idle at higher cost.

Sequence length1,024
Micro batch size32
Global batch524,288 tok · 16 accum steps
OptimizerAdamW · β(0.9, 0.95) · wd 0.1
LR schedulecosine · 6e-4 → 6e-5
Warmup200M tok · ~381 steps
Precisionfp32 weights · bf16 autocast
GPU1× H100 · $4.33/hr effective

Training loss — all 3,889 steps

9.87 (ppl 19.4k) → 2.263 (ppl 9.6) nats
3,889
optimizer steps
2.04B
tokens seen · 1 epoch
2.85hr
wall-clock compute
$15.54
total spend, all 6 phases
Results

Where the base model landed.

Final validation loss
2.326
Perplexity
≈ 10.2
Random-init baseline
9.87 = ln(16384)
huggingface.co/ankitw497/slm-125m-base

Built with Modal (CPU data pipeline, GPU training, and the CPU endpoint powering the demo above), weights hosted on Hugging Face, this page on Vercel. Tokens seen ≈ 16.2× parameter count — just under the ~20× Chinchilla-optimal ratio for a single epoch over this corpus.

Fine-tuning · Phase 2

Teaching the base model to answer, not just continue.

A full fine-tune of the base checkpoint above on 10,000 grounded question-answer pairs — every weight updated, no adapters. Below: the live model, exactly how its training data was built, and what actually happened when it trained (including a real overfitting bug caught mid-run).

QA model · ready ankitw497/slm-125m-qa
QA fine-tune — step 300 checkpoint
// answer will appear here

This model was fine-tuned closed-book — it never sees the source passage, only the question, at training or inference time. At 125.8M parameters it doesn't have the capacity to reliably memorize facts from a 2.4B-token corpus, so treat specific names, dates, and figures as unverified — the improvement here is in domain style and answer structure, not factual recall. Served live from a second CPU container on Modal; first request after idle may take 10–20s.

How the dataset was built

10,000 grounded pairs, synthesized from the same corpus.

Not new data — the same cleaned, deduped corpus from the pretraining pipeline, turned into question-answer pairs by an LLM teacher, then filtered hard.

01 · Sample chunks

4,761 passages, same mix as pretraining

~300-character passages sampled from the corpus, proportional to the realized pretraining mix (case law 35% / SEC 42% / web 23%), so the QA data reflects the same domain balance.

4,761 passages · 1,666 / 2,000 / 1,095 by source
02 · Generate

Gemini 3.1 Flash Lite, thinking off

3 grounded QA pairs requested per passage, run as parallel real-time calls fanned out across Modal workers (a Batch API attempt stalled with zero visible progress for over an hour and was cancelled). 100% success rate, 0 failures across 4,761 chunks.

14,283 raw pairs · exactly 3/chunk, every time
03 · Filter + dedup

Grounding check, exact + near-dup removal

Word-overlap grounding check against the source passage, exact-match dedup on normalized questions, then embedding-similarity near-dup removal (cosine ≥ 0.92) via local sentence-transformers — no extra API cost.

13,261 survive · 92.9% pass rate
Filter stepRemovedRemaining
Parsed from teacher output14,283
Not grounded in source passage−1814,265
Exact-duplicate questions−45413,811
Near-duplicate (embedding sim ≥ 0.92)−55013,261
Randomly trimmed to target size−3,26110,000
Final dataset mix
SEC filings · 4,184 pairs case law · 3,379 pairs fineweb-edu · 2,437 pairs
The training run

4.5 minutes on a T4 — and a real bug worth showing.

Full fine-tune, not LoRA — at 125.8M params there's no memory problem adapters need to solve. The interesting part isn't the setup, it's what the validation loss revealed mid-run.

MethodFull fine-tune · all 125.8M params
Base checkpointankitw497/slm-125m-base
Train / val examples9,800 / 200
Format<|user|>Q<|assistant|>A<|eos|>
Loss maskinganswer span only
LR schedulecosine · 3e-5 → 3e-6
Batch32 examples/step · dynamic padding
GPU1× T4 · fp16 + grad scaling

Train vs. validation loss — the overfitting is visible

best: step 300 · val 2.12
What actually happened

The run was configured for 3 epochs (918 steps). Train loss fell smoothly the whole way, down to ~1.0 — which looked great until the validation curve told a different story: val loss bottomed out at step 300, under one full epoch, then rose for the remaining two-thirds of training as the model started memorizing the training set instead of generalizing. The first run shipped the last-step checkpoint by default and would have deployed the overfit model. The code was changed to track and save whichever checkpoint had the lowest validation loss during training, and the run was repeated — the shipped model above is from step 300, not step 918.

300/918
best checkpoint step
2.12
best val loss · ppl ≈8.3
3.13
baseline val loss · ppl ≈22.9
4.5min
wall-clock, 1× T4
Real output, unedited

What the shipped checkpoint actually says.

Five held-out validation questions the model never trained on — expected answer (from the Gemini teacher) next to what the fine-tuned model actually generates.

QuestionExpectedGenerated
Best validation loss
2.1224
Perplexity
≈ 8.3
vs. pre-finetune baseline
3.13 → 2.12
huggingface.co/ankitw497/slm-125m-qa