Tiny Web LLM Lab
Train a small next-token model directly in the browser and generate text.
This page demonstrates the core mechanics of language modeling: tokenization, context windows, logits, softmax, and gradient-based updates.
It is intentionally tiny and educational. You can inspect training loss and compare decoding strategies such as Greedy, Sampling, and Top-k.
Tokens are space-split words. The model averages token embeddings in the current context and projects them to vocabulary logits.
$$ h = \frac{1}{K}\sum E(t_i),\quad z = W^Th + b,\quad p = softmax(z) $$
Training uses per-token cross-entropy with simple SGD updates.
1) Training Corpus
2) Hyperparameters
3) Training Run
0%