A recap of the conversation — from the core mechanism through training and tokenization to a hands-on fine-tuning example and deployment.
An LLM always predicts exactly one thing: which token (a word or part of a word) is most likely to come next. This prediction is repeated token by token until a complete response emerges.
Each step produces a probability over all ~50,000 possible tokens; the most likely one is appended, then the cycle begins again.
For the weights to be able to compute these probabilities sensibly, they first have to be trained. That happens in a constantly repeating, four-step cycle:
Training means: the weights themselves are changed step by step, based on a recurring four-step cycle.
Step one, the prediction (forward pass): You give the model a piece of text, for example "The sky is," and have it predict which word comes next. With the current (initially random) weights, this produces some probability distribution, at first usually complete nonsense.
Step two, the error (loss): From the real training text you know the actual next word (here, for example, "blue"). You compare the model's prediction with this real word and compute a numeric value for how wrong the prediction was — this is called the loss.
Step three, tracing back (backpropagation): Now comes the actual mathematical work. The procedure calculates, for each of the billions of weights in the network, how strongly and in which direction that one weight contributed to the measured error — by systematically computing backward through the network, hence the name.
Step four, the adjustment (gradient descent): Based on this information, each weight is nudged a small amount in the direction that would make the error a bit smaller next time — just a small step, not the full correction all at once.
No human prescribes "correct answers" here: in the first training phase, the training text itself supplies the solution — the next word is hidden and the model has to predict it. This is called self-supervised learning, because the data essentially corrects itself, without a human having to evaluate every single answer in advance.
A finished chat model like this one doesn't come into being in a single step, but through three successive phases:
The number of weights (parameters) determines a model's capacity — how many patterns it could theoretically represent. Whether this capacity is used well depends on the quality of the training data.
More weights = more capacity (the shelf is bigger). But what determines whether the result is good is what gets put on it — not how much space is available.
The math of neural networks consists almost entirely of matrix multiplications — many small, mutually independent computations that can run simultaneously instead of one after another. That's exactly what GPUs were built for.
Because the same computation (multiply + add) has to happen millions of times in parallel for every weight simultaneously, a GPU is often up to a hundred times faster here than a CPU — the original purpose (rendering images, many pixels at once) happens to fit ideally with training neural networks.
Before a sentence is processed, it's broken down into tokens — building blocks from a fixed, pre-built list (the vocabulary, usually 30,000–100,000 entries). The technique behind this is called Byte Pair Encoding (BPE).
Using a small, self-trained BPE tokenizer (training text: a few sentences about capital cities), the sentence "What is the capital of France" was split like this:
The token list is fixed once and doesn't grow automatically afterward. New words are assembled from existing smaller building blocks; the "knowledge" about them arises in the weights, not as a new vocabulary entry.
Only the very first step can be shown concretely: every token has a fixed ID, which corresponds to a row in the so-called embedding table (part of the weights). Everything after that is spread across billions of shared weights and can no longer be attributed to a single token or concept.
At a fixed model size, all capacity is eventually "spoken for." If you train a finished model heavily on new content, it can lose old capabilities.
This is measured with fixed benchmark test sets, compared before and after post-training — not by repeatedly asking the same question.
| Technique | Effect |
|---|---|
| Replay / rehearsal | keep mixing in old training data |
| Low learning rate | only gentle, small weight adjustments |
| LoRA / adapters | base weights stay completely frozen |
| Elastic Weight Consolidation | important old weights are specifically protected |
Instead of all the billions of weights, LoRA only trains small, additional weight matrices. The base model stays completely frozen.
Advantage: low computational cost (often a single GPU instead of a data center) and low risk of catastrophic forgetting, because the original stays untouched. With QLoRA, the base model is additionally quantized to 4-bit to save memory.
The complete practical path we walked through:
Training example (format as in training_data_example.jsonl):
The training script uses transformers, peft (LoRA) and
trl, and runs through exactly the learning cycle described above for
each example. Rule of thumb: a few hundred to a few thousand carefully reviewed
examples are usually enough for noticeable improvements.
The complete, commented finetune.py script from our
conversation — uses transformers, peft (LoRA/QLoRA) and
trl, loads the base model in 4-bit, configures LoRA, trains with
training_data_example.jsonl and tests the result directly at the end.
"""
Example script: fine-tuning an open LLM (e.g. Mistral-7B or Llama-3-8B)
with LoRA (parameter-efficient fine-tuning) for a company-specific
use case (here: IT support / customer communication for a software company).
Required libraries (see README.md for installation):
pip install transformers peft trl datasets accelerate bitsandbytes torch
Prerequisite: a GPU with sufficient VRAM (see README.md for guidance).
For a 7B model with 4-bit quantization (QLoRA), 16-24 GB VRAM is often
enough (e.g. an RTX 4090 or a cloud GPU like A10/A100).
This script is deliberately kept simple to show the basic flow. For
production projects you should additionally add validation splits,
logging (e.g. Weights & Biases) and hyperparameter search.
"""
import torch
from datasets import load_dataset
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
TrainingArguments,
)
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from trl import SFTTrainer, SFTConfig
# ---------------------------------------------------------------------------
# 1. Choose base model
# ---------------------------------------------------------------------------
# A freely available, open model from Hugging Face. For real-world use
# you may have to request access from the provider once (e.g. Meta for
# Llama); Mistral models are usually usable without approval.
BASE_MODEL = "mistralai/Mistral-7B-Instruct-v0.3"
# ---------------------------------------------------------------------------
# 2. Load model in 4-bit (QLoRA) -> massively saves GPU memory
# ---------------------------------------------------------------------------
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
quantization_config=bnb_config,
device_map="auto",
)
model = prepare_model_for_kbit_training(model)
# ---------------------------------------------------------------------------
# 3. Configure LoRA
# ---------------------------------------------------------------------------
# Instead of all the billions of weights, only small, additional
# weight matrices are trained ("r" determines their size/capacity).
# The original weights stay frozen -> low risk of catastrophic
# forgetting, low computational cost.
lora_config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Example output: "trainable params: 20,971,520 || all params: 7,262,703,616
# || trainable%: 0.29 %"
# -> Only about 0.3 % of all weights are actually changed.
# ---------------------------------------------------------------------------
# 4. Load training data
# ---------------------------------------------------------------------------
# Expected format: a JSON Lines file, one line = one training example
# with the fields "instruction", "input" (optional) and "output".
# See training_data_example.jsonl for a concrete example.
dataset = load_dataset("json", data_files="training_data_example.jsonl", split="train")
def format_prompt(example):
"""Turns a training example into the model's chat/prompt format."""
if example.get("input"):
prompt = f"{example['instruction']}\n\n{example['input']}"
else:
prompt = example["instruction"]
text = (
f"<s>[INST] {prompt} [/INST] {example['output']}</s>"
)
return {"text": text}
dataset = dataset.map(format_prompt)
# ---------------------------------------------------------------------------
# 5. Training configuration
# ---------------------------------------------------------------------------
sft_config = SFTConfig(
output_dir="./result-finetuned-model",
num_train_epochs=3,
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
learning_rate=2e-4, # deliberately low, see the chapter on
# catastrophic forgetting in the conversation
logging_steps=5,
save_strategy="epoch",
bf16=True,
max_seq_length=1024,
dataset_text_field="text",
report_to="none", # alternatively e.g. "wandb" for a logging dashboard
)
# ---------------------------------------------------------------------------
# 6. Start training
# ---------------------------------------------------------------------------
trainer = SFTTrainer(
model=model,
args=sft_config,
train_dataset=dataset,
)
trainer.train()
# ---------------------------------------------------------------------------
# 7. Save result (only the small LoRA weights, a few MB)
# ---------------------------------------------------------------------------
trainer.save_model("./result-finetuned-model")
tokenizer.save_pretrained("./result-finetuned-model")
print("Done. The fine-tuned model (LoRA adapter) is located at "
"./result-finetuned-model")
# ---------------------------------------------------------------------------
# 8. Quick test after training
# ---------------------------------------------------------------------------
question = "A customer asks: 'What does an additional user license cost?'"
input_ids = tokenizer(f"<s>[INST] {question} [/INST]", return_tensors="pt").to(model.device)
output = model.generate(**input_ids, max_new_tokens=150)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Recurring sentence structures ("Do you support X?") don't have to be duplicated by hand. A template plus a list of topics/facts generates the JSONL lines automatically.
You don't need to artificially insert general typos — the base model's pretraining already covers that. What's worth doing deliberately, though, is targeting typos that happen to form another real word (e.g. "DATE interface" instead of "DATEV interface") — a specifically added training example helps avoid genuine misunderstandings here.
A model doesn't necessarily need to fit entirely into GPU memory.
device_map="auto" distributes it intelligently:
Other levers: stronger quantization (4-bit, 3-bit), a smaller model, gradient checkpointing, smaller batch size/sequence length. For weaker laptops, tools like Unsloth (significantly lower memory footprint) are also useful, or on Apple Silicon machines, MLX/llama.cpp with Metal support.
| RAG | Fine-tuning | |
|---|---|---|
| Changes the weights? | No | Yes |
| Good for | current, frequently changing facts | tone, format, behavior |
| Effort | low (vector database + context) | higher (training data + GPU) |
| Forgetting risk | none (model stays unchanged) | present, but reducible via LoRA |
In practice, often combined: a lightly fine-tuned model for style and behavior, plus RAG for current, company-specific facts.
With real training, the weights themselves change permanently, through the cycle of predicting, measuring error, and adjusting, repeated over enormous amounts of data and enormous compute time (for large models this takes weeks to months on thousands of specialized chips). With RAG, the model stays exactly as it is — it merely gets additional text supplied at the runtime of a single request, which is immediately "forgotten" again after answering the question, because nothing about the weights was changed. That's why RAG is so well suited for current, constantly changing information, while real training is better suited for fundamental, stable language and factual understanding.
Unlike with a database, there is no connection string when loading a model locally — the weights are loaded once, completely, into your own working memory. With an API connection (e.g. using a cloud provider), the database comparison does apply, though.
Loading locally (from_pretrained) | API usage | |
|---|---|---|
| Where do the computations run? | on your own hardware | at the provider |
| Network connection during use? | none | yes, per request |
| Access to the weights? | yes → fine-tuning possible | no (except via special fine-tuning APIs) |
| Comparable to | loading a large file into memory | a database connection (connection string / API key) |
Ollama is meant for efficiently running finished models (format: GGUF), not for training them. The workflow maps almost one-to-one onto building and deploying a Java application:
Training data contains many typos and colloquial language — yet the model usually answers correctly. Three reasons:
| 1 | Statistical majority: the correct spelling is always the same, while errors are spread across many different, rare variants. |
| 2 | Weighting good sources: edited texts (books, professional articles) are weighted more heavily during training. |
| 3 | The fine-tuning phase: target answers in the training examples are always cleanly formulated — regardless of the quality of the question. |
A common but inaccurate assumption: "The provider analyzes how the model has changed through use." That's technically not correct — an already deployed LLM does not change on its own through use; the weights stay frozen, as already explained in the section on loading locally.
What gets collected (depending on privacy settings and terms of use) is the conversation text itself — input and output as plain text, completely separate from the model and its weights. The model itself is not examined or "dissected" in the process; it's simply recorded what was asked and answered — exactly the same kind of raw material as in the original training, just real conversation transcripts instead of general internet text.
The curation (which conversations are good enough quality, where does the model show recognizable weaknesses, which topics are underrepresented, are there duplicates) happens at the text level, before any new training even begins. Only afterward does this curated selection feed into a new, deliberate training run (additional pretraining material, supervised fine-tuning, or RLHF) — with the same prediction/error/adjustment cycle as right at the beginning. The result is a completely new, standalone model version with newly computed weights, not a version of the model you were just talking to that "grew" in the background.
| Core principle | Token-by-token probability prediction, no text archive |
| Training | Prediction → error → backpropagation → adjust weights, billions of times |
| Quality | Parameter count = capacity; data quality determines what comes of it |
| Tokenization | Fixed building-block list (BPE), frequency determines granularity |
| Knowledge in the model | spread across billions of weights, not localizable |
| Fine-tuning yourself | LoRA/QLoRA on open models, with curated JSONL data |
| Current facts | RAG rather than constant retraining |
| Deployment | maintain the source (HF/LoRA), rebuild to GGUF for Ollama each time |
For quick reference: every technical term and abbreviation used in this document, sorted alphabetically.
| Abbreviation | Meaning |
|---|---|
| API | Application Programming Interface — an interface for using a system (e.g. an LLM) without running the software locally; the request goes out over the internet, and the response comes back from the provider's server. |
| BPE | Byte Pair Encoding — a tokenization method that merges text step by step, based on frequency, into a fixed list of word-building blocks (tokens); the basis of modern tokenizers. |
| CPU | Central Processing Unit — a computer's main processor; few but highly versatile cores, good for sequential tasks but poorly suited to the massively parallel matrix computations involved in LLM training. |
| EWC | Elastic Weight Consolidation — a technique against catastrophic forgetting: weights important for already-learned knowledge are specifically "protected" and changed less during further training. |
| GGUF | GPT-Generated Unified Format — a compressed, quantized file format for model weights, used by llama.cpp and, building on that, Ollama, for efficient running (not training). |
| GPU | Graphics Processing Unit — a graphics card; has thousands of smaller cores that can work simultaneously — ideal for the massively parallel matrix multiplications in training and running LLMs. |
| HF | Hugging Face — a platform and library ecosystem (including transformers, peft, trl, datasets) for downloading, training and sharing models; "safetensors" is the de facto standard format for training purposes. |
| JSONL | JSON Lines — a text format where each line of the file is a standalone, complete JSON object (instead of one large JSON array); a common format for training datasets in fine-tuning. |
| LLM | Large Language Model — a large language model; predicts text token by token based on learned probabilities, without storing text as such. |
| LoRA | Low-Rank Adaptation — a fine-tuning method: the original model weights stay frozen, while small additional low-rank matrices ("adapters") are trained instead — only a fraction of the original parameters. |
| NF4 | 4-bit NormalFloat — a quantization format that compresses weights to 4 bits, specifically tailored to the typical distribution of model weights; a core component of QLoRA. |
| PEFT | Parameter-Efficient Fine-Tuning — an umbrella term for fine-tuning methods (including LoRA) where only a small portion of the model's parameters is trained, instead of all the weights; also the name of the Hugging Face library of the same name. |
| QLoRA | Quantized LoRA — a combination of 4-bit quantization of the frozen base model (see NF4) and LoRA adapters; enables fine-tuning of large models on hardware with significantly less GPU memory. |
| RAG | Retrieval-Augmented Generation — a technique where matching text passages (e.g. from a vector database) are looked up at runtime and supplied to the model as context; it does not change the weights. |
| RLHF | Reinforcement Learning from Human Feedback — a training phase in which a model is further refined based on human ratings (which answer is better?), usually after SFT. |
| SFT | Supervised Fine-Tuning — the training phase after pretraining, in which the model learns to answer in a desired format (e.g. as a helpful assistant) using concrete example question/answer pairs. |
| VRAM | Video RAM — a graphics card's memory; limits how large a model (or how large a quantized version of it) can be loaded and trained on that GPU. |
Note on brand and product names: The product, company and brand names mentioned in this document — including DATEV, Ollama, Meta Llama, Mistral AI, Hugging Face, OpenAI/GPT, Python, Java, Microsoft PowerPoint, Google Colab, RunPod, Apple/MLX — are trademarks or registered trademarks of their respective rights holders. They are used here solely for illustrative and explanatory purposes; no affiliation, endorsement, or partnership with the respective companies is implied.