Fine-Tuning with PEFT & TRL

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written against peft 0.21.1, trl 1.14.1, transformers 5.17.0, datasets 5.0.1, bitsandbytes 0.50.2, sentence-transformers 6.1.0 and setfit 1.2.0, as checked on 2026-09-30. Fine-tuning adapts a Hub model to your task, your style or your data. It is the last resort on the ladder prompting, retrieval, fine-tuning, because it adds a training pipeline, a dataset to maintain and a model to re-evaluate every time the base model changes. This page covers when it pays off, how parameter-efficient methods make it fit on one GPU, and how the result gets back to a phone or a laptop. Serving the result to many users is covered by vLLM and SGLang and Running Local LLMs in Production; formats are explained on Model Formats; downloading and licences are on Hub Essentials and Choosing Models and Licences. Training from scratch is out of scope.

When to fine-tune, and when not to

Start from the failure you observe, not from the technique you want to try. Each lever changes a different thing:

Symptom	Usually fixed by	Why
The model ignores or misreads instructions	Better prompt, examples in the prompt (few-shot), structured output	Cheapest change and reversible in seconds; see LLM Foundations & Prompting
The model lacks facts: private documents, fresh data, your product catalogue	Retrieval (RAG)	Knowledge changes daily; weights do not. Facts are cited and deleted by editing the index, see Generating Embeddings in Practice
Output must follow a style, format, tone or domain vocabulary consistently, and prompts get long	Supervised fine-tuning (SFT)	Behaviour is baked into the weights instead of repeated in every prompt
Two acceptable answers exist and you know which one users prefer	Preference tuning (DPO)	Learns from ranked pairs, not from single gold answers
A small on-device model must match a large one on one narrow task	SFT or distillation on outputs of a larger model (check the larger model’s licence permits it)	Narrow tasks need far fewer parameters than general chat
Text must be classified or ranked and you have tens to thousands of labels	Fine-tuned encoder, SetFit, or a fine-tuned embedding model	Smaller, faster and cheaper than prompting an LLM per item

Symptom

Usually fixed by

Why

The model ignores or misreads instructions

Better prompt, examples in the prompt (few-shot), structured output

Cheapest change and reversible in seconds; see LLM Foundations & Prompting

The model lacks facts: private documents, fresh data, your product catalogue

Retrieval (RAG)

Knowledge changes daily; weights do not. Facts are cited and deleted by editing the index, see Generating Embeddings in Practice

Output must follow a style, format, tone or domain vocabulary consistently, and prompts get long

Supervised fine-tuning (SFT)

Behaviour is baked into the weights instead of repeated in every prompt

Two acceptable answers exist and you know which one users prefer

Preference tuning (DPO)

Learns from ranked pairs, not from single gold answers

A small on-device model must match a large one on one narrow task

SFT or distillation on outputs of a larger model (check the larger model’s licence permits it)

Narrow tasks need far fewer parameters than general chat

Text must be classified or ranked and you have tens to thousands of labels

Fine-tuned encoder, SetFit, or a fine-tuned embedding model

Smaller, faster and cheaper than prompting an LLM per item

Fine-tuning is a poor way to teach facts: a model can memorize a sentence yet still fail to use it when phrased differently, and updates require retraining. Combine the levers instead: a fine-tuned model that follows your output format, fed by retrieved context. Before investing, write the evaluation set first (even 50 to 200 labelled examples); if a good prompt already reaches the target on it, stop. The evaluation set is also how you detect that fine-tuning made the model worse on everything else (catastrophic forgetting), see LLM Foundations & Prompting for evaluation habits.

flowchart TD A["Observed failure on your eval set"] --> B{"Missing facts or fresh data?"} B -- yes --> C["RAG: index the documents"] B -- no --> D{"Prompt and few-shot examples enough?"} D -- yes --> E["Ship the prompt"] D -- no --> F{"Style, format or narrow skill?"} F -- yes --> G["SFT with LoRA or QLoRA"] F -- no --> H{"Ranked preference pairs available?"} H -- yes --> I["DPO on top of SFT"] H -- no --> J["Re-examine task and data"] G --> K["Evaluate, merge, export"] I --> K

Figure 1. The decision as a flowchart

The model card of a base model states its licence, which decides whether you may fine-tune and redistribute (see Choosing Models and Licences). Official reference for the methods below: PEFT documentation and TRL documentation.

LoRA: the update and the parameter savings

Full fine-tuning updates every weight matrix . LoRA (Low-Rank Adaptation) freezes and learns a low-rank correction instead:

The forward pass of an adapted layer is . Initialization matters: gets a small random value and starts at zero, so at step 0 and training begins exactly at the base model. Only and receive gradients. The scalar rescales the update, so lora_alpha and r are tuned together: doubling r with lora_alpha fixed halves the per-direction scale. PEFT also offers use_rslora=True, which scales by so that larger ranks do not shrink the update, and use_dora=True for DoRA (weight-decomposed LoRA); both are optional variants of the same idea.

The trainable parameters for one adapted matrix are

so the fraction trained is . For a square projection with : trainable values against , about 0.78 percent of that matrix. Adapters on all linear layers of a model are still typically well below a few percent of the model, which is why the optimizer state (the dominant training memory cost, see the estimates below) nearly disappears and why an adapter file is megabytes, not gigabytes.

LoRA with PEFT: wrap a model, count trainable parameters

from transformers import AutoModelForCausalLM
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", dtype="auto")

config = LoraConfig(
    r=16,
    lora_alpha=32,            # scale = lora_alpha / r = 2
    lora_dropout=0.05,
    target_modules="all-linear",
    task_type="CAUSAL_LM",
)
peft_model = get_peft_model(model, config)
peft_model.print_trainable_parameters()   # trainable params vs. all params

target_modules can be a list of module names (for example ["q_proj", "v_proj"]; names differ per architecture) or "all-linear", which adapts every linear layer except the output head. Qwen/Qwen3-0.6B is Apache-2.0 and not gated on the Hub at the time of writing. Official reference: PEFT — LoRA developer guide.

Rules of thumb, all worth validating on your evaluation set rather than trusting: a rank between 8 and 64 covers most narrow tasks; adapting all linear layers usually helps more than raising the rank on attention projections only; and because only new parameters are learned, adapter training uses a higher learning rate than full fine-tuning (TRL’s own guide suggests about 1e-4 for SFT adapters against the 2e-5 default of SFTConfig).

QLoRA and NF4

QLoRA keeps the frozen base model in 4-bit precision and trains LoRA adapters in higher precision on top of it. Gradients flow through the dequantized weights into the adapters. The 4-bit type is NF4 (NormalFloat4), whose 16 levels are placed at the quantiles of a standard normal distribution, which suits the roughly normal distribution of pretrained weights; blockwise scales are themselves quantized ("double quantization") to save a further fraction of a bit per weight. The construction is described in the QLoRA paper and summarised on Quantization for Inference. The bitsandbytes library provides the kernels; transformers exposes them through BitsAndBytesConfig. Expect a CUDA GPU for bitsandbytes (backend support for other hardware varies by release; check its installation page).

QLoRA: a 4-bit NF4 base plus LoRA adapters

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", quantization_config=bnb)
model = prepare_model_for_kbit_training(model)

lora = LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", task_type="CAUSAL_LM")
model = get_peft_model(model, lora)

Memory for the frozen weights falls from about bytes (bf16) to about bytes plus quantization constants, where is the parameter count; the estimates section turns this into a calculator. Two caveats: QLoRA trades memory for speed, because every step dequantizes weights on the fly; and an adapter trained against a 4-bit base is best merged into a full-precision copy of that base rather than into the quantized weights (see the merging section). Official reference: PEFT — Quantization, and the paper QLoRA: Efficient Finetuning of Quantized LLMs.

Supervised fine-tuning with TRL

TRL (Transformer Reinforcement Learning) provides trainers on top of transformers.Trainer. SFTTrainer with SFTConfig is the standard route for instruction and chat tuning. The trainer accepts a model id or a loaded model, args=SFTConfig(…), a train_dataset (and optional eval_dataset), a peft_config to train an adapter, a quantization_config to load the model in 4-bit from an id, and a processing_class (the tokenizer or processor; in current TRL this name replaced the former tokenizer= argument, and if omitted it is loaded from the model id).

SFT of a LoRA adapter with TRL

from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

dataset = load_dataset("trl-lib/Capybara", split="train")   # conversational "messages" format

args = SFTConfig(
    output_dir="qwen3-0.6b-capybara-lora",
    learning_rate=1e-4,             # adapters tolerate a higher rate than full fine-tuning
    num_train_epochs=1,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    max_length=2048,                # truncation length (formerly max_seq_length)
    packing=True,                   # pack short examples into full-length sequences
    assistant_only_loss=True,       # loss on assistant turns only (needs a generation-aware chat template)
    logging_steps=10,
)

trainer = SFTTrainer(
    model="Qwen/Qwen3-0.6B",
    args=args,
    train_dataset=dataset,
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", task_type="CAUSAL_LM"),
)
trainer.train()
trainer.save_model()                # saves the adapter, not the merged model

Arguments worth knowing (all confirmed in the TRL 1.14.1 SFTConfig reference):

max_length (default 1024) truncates tokenized sequences; with packing it sets the packed sequence length.
packing and packing_strategy (bfd by default, plus bfd_split and wrapped) raise throughput by removing padding.
completion_only_loss defaults to loss on the completion for prompt-completion datasets and on the whole sequence for plain text; assistant_only_loss=True restricts loss to assistant messages in conversational data and requires the chat template to mark assistant spans (TRL patches the templates of some known model families automatically and documents the generation markers for others).
dataset_text_field (default "text") and formatting_func select or build the text to train on.
chat_template_path and eos_token align a base model with a chat template and stop token.
model_init_kwargs forwards arguments such as dtype to from_pretrained; when the trainer loads the model from an id the default dtype is float32 unless you set it, which differs from from_pretrained in transformers v5.
loss_type defaults to a chunked negative log-likelihood that lowers peak memory; use_liger_kernel switches it to the plain variant.
Gradient checkpointing is on by default and bf16 is enabled automatically when fp16 is not set.

Because SFTConfig extends TrainingArguments, all standard options (eval_strategy, save_steps, report_to, push_to_hub, hub_model_id) apply. Pushing needs a write token read from the environment ($HF_TOKEN or hf auth login), never hard-coded, see Hub Essentials. Official reference: TRL — SFT Trainer.

Preference tuning with DPO

After SFT, Direct Preference Optimization aligns the model with pairs of a preferred (chosen) and a dispreferred (rejected) answer to the same prompt, without training a separate reward model. With a trainable policy , a frozen reference (the model before DPO), a preferred completion , a rejected one , the sigmoid and the strength parameter , the loss is

so it widens the margin between the two implicit rewards while beta (default 0.1 in DPOConfig; higher means staying closer to the reference) limits drift. When you pass no ref_model, TRL uses the initial policy (the model before DPO starts) as the reference; with a PEFT adapter that is the base model with the adapter switched off, so no second full copy of the weights is needed.

DPO of a LoRA adapter with TRL

from datasets import load_dataset
from peft import LoraConfig
from trl import DPOConfig, DPOTrainer

dataset = load_dataset("trl-lib/ultrafeedback_binarized", split="train")   # prompt / chosen / rejected

args = DPOConfig(
    output_dir="qwen3-0.6b-dpo-lora",
    beta=0.1,
    learning_rate=1e-5,            # TRL suggests about 1e-5 for adapters; default is 1e-6
    max_length=1024,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
)

trainer = DPOTrainer(
    model="Qwen/Qwen3-0.6B",
    args=args,
    train_dataset=dataset,
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", task_type="CAUSAL_LM"),
)
trainer.train()

The preference dataset holds an explicit prompt plus chosen and rejected, either as plain strings or as lists of chat messages (TRL applies the chat template for conversational data). DPOConfig.loss_type defaults to ["sigmoid"] and accepts other published variants (for example hinge, ipo, sigmoid_norm for length normalization) and weighted combinations through loss_weights. When passing your own processing_class to DPOTrainer, its padding side must be left. Start DPO from an SFT checkpoint of the same model (merge the SFT adapter first, or continue training it as a PeftModel); applying DPO to a base model that never saw your format rarely works. Official reference: TRL — DPO Trainer and the paper Direct Preference Optimization.

Preparing datasets

Data quality dominates. A few hundred clean, diverse, consistently formatted examples routinely beat tens of thousands of noisy ones for narrow tasks; pick the size by watching the evaluation set. A preparation checklist:

Pick the TRL format. messages (chat turns), prompt plus completion, or plain text for SFT; prompt, chosen, rejected for DPO. Let TRL apply the chat template (conversational formats) rather than concatenating strings by hand, and train with the same template you will serve with, or responses will not terminate properly.
Deduplicate and filter. Remove near-duplicates, empty or truncated answers, and examples that leak your evaluation set. Decide how you treat personal data: fine-tuned weights can reproduce training text.
Split before you tune. Keep a held-out validation set and a separate final test set; never tune on the test set.
Check licences. Dataset cards carry licences and terms just like models; outputs of a hosted model may be restricted as training data by that provider’s terms.
Watch length. Examples longer than max_length are truncated, which can cut off the answer you want to teach.

Converting a raw dataset to TRL’s conversational prompt-completion format

from datasets import load_dataset

raw = load_dataset("json", data_files="support_tickets.jsonl", split="train")
# each row: {"question": "...", "answer": "..."}

def to_messages(row):
    return {
        "prompt": [{"role": "user", "content": row["question"]}],
        "completion": [{"role": "assistant", "content": row["answer"]}],
    }

ds = raw.map(to_messages, remove_columns=raw.column_names)
ds = ds.shuffle(seed=42).train_test_split(test_size=0.1, seed=42)
train_ds, eval_ds = ds["train"], ds["test"]

With a prompt-completion dataset, SFT computes loss on the completion only by default. Official reference: TRL — Dataset formats and datasets documentation.

Merging adapters

A LoRA adapter is a small pair of files (adapter_config.json and the weights, in safetensors) that only makes sense next to its base model. You have two deployment choices:

Keep it separate. Load the base once and swap adapters per task. Runtimes that support adapters (vLLM, llama.cpp) can do this; the cost is a little extra compute per token.
Merge it. Fold into once, giving a plain model with no extra latency and no dependency on PEFT at inference time. Merge into a full-precision copy of the base: merging into already-quantized weights rounds the result and is unsupported for some quantization schemes, so after QLoRA reload the base in bf16, attach the adapter, merge, then quantize the merged model for the target runtime.

Merging an adapter into a bf16 base and saving a normal model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "qwen3-0.6b-capybara-lora")
merged = model.merge_and_unload()            # returns the merged model; not in place

merged.save_pretrained("qwen3-0.6b-merged")  # safetensors weights and config
AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B").save_pretrained("qwen3-0.6b-merged")

Merged output is a derivative of the base model: keep the base model’s licence obligations and attribution when you publish it (see Choosing Models and Licences). Official reference: PEFT — LoRA developer guide (merge_and_unload).

Exporting to GGUF and ONNX for on-device use

The merged directory is an ordinary Hugging Face checkpoint, so the conversion tools from Conversion & Quantization apply to it (formats are explained on Model Formats).

GGUF (llama.cpp, Ollama, LM Studio). llama.cpp ships convert_hf_to_gguf.py for full models and convert_lora_to_gguf.py for PEFT adapters; the latter takes the adapter directory and a --base directory with the base model’s config and tokenizer, and writes a GGUF adapter that llama.cpp loads with its --lora option at run time. Quantizing is a separate step with llama-quantize, covered on Quantization for Inference and llama.cpp.

Merged model to a quantized GGUF, or adapter to a GGUF LoRA (run from a llama.cpp checkout)

# Option A: merged model -> f16 GGUF -> quantized GGUF
python convert_hf_to_gguf.py ./qwen3-0.6b-merged --outfile qwen3-0.6b-merged-f16.gguf --outtype f16
llama-quantize qwen3-0.6b-merged-f16.gguf qwen3-0.6b-merged-Q4_K_M.gguf Q4_K_M

# Option B: keep the adapter separate; base GGUF plus a GGUF adapter
python convert_hf_to_gguf.py ./Qwen3-0.6B --outfile Qwen3-0.6B-f16.gguf --outtype f16
python convert_lora_to_gguf.py ./qwen3-0.6b-capybara-lora --base ./Qwen3-0.6B --outtype f16 \
  --outfile qwen3-0.6b-capybara-lora-f16.gguf
llama-server -m Qwen3-0.6B-f16.gguf --lora qwen3-0.6b-capybara-lora-f16.gguf

Without --outfile, convert_lora_to_gguf.py writes its default-named file next to the adapter, so pass --outfile to control where the adapter GGUF lands (here ./Qwen3-0.6B must hold the base model’s downloaded files). Only architectures supported by llama.cpp convert, and flags evolve, so run each script with --help. Official references: convert_lora_to_gguf.py and llama.cpp.

ONNX (ONNX Runtime, Transformers.js, Windows, Android). Export the merged model with Optimum ONNX; for text generation the text-generation-with-past task keeps the key-value cache inside the graph. For on-device LLMs the ONNX Runtime GenAI model builder produces a runtime-ready folder, including int4 weights; its README documents the current command-line flags, which this page does not repeat because they change between releases. Classifier and embedding models export directly as encoders.

Merged model to ONNX with optimum-cli

pip install "optimum-onnx[onnxruntime]"
optimum-cli export onnx --model ./qwen3-0.6b-merged --task text-generation-with-past qwen3-0.6b-onnx/

optimum-onnx 0.1.0, the latest release on PyPI on 2026-09-30, pins optimum~=2.1.0 and transformers>=4.36,<4.58, so installing it downgrades a transformers 5 environment; run the export in a separate virtual environment (for example python -m venv .venv-onnx) and keep the transformers 5 baseline in your main one.

Expect to re-check accuracy on your evaluation set after every quantization or export step; each one changes the weights. Core ML, LiteRT, MLX and ExecuTorch routes start from the same merged checkpoint and are listed on Model Formats. Official references: Optimum ONNX and ONNX Runtime GenAI.

Fine-tuning embedding models with sentence-transformers

When a retrieval pipeline underperforms on your domain (legal clauses, part numbers, internal jargon), fine-tuning the embedding model often beats a larger general one and stays cheap: encoder models with tens of millions of parameters train in minutes. sentence-transformers provides SentenceTransformerTrainer, a Trainer subclass, with losses that take pairs or triplets. The common starting point is MultipleNegativesRankingLoss: each batch gives (anchor, positive) pairs and every other positive in the batch acts as a negative, so larger batches help and duplicates in a batch hurt, hence BatchSamplers.NO_DUPLICATES.

Fine-tuning an embedding model on (query, relevant passage) pairs

from datasets import Dataset
from sentence_transformers import (
    SentenceTransformer,
    SentenceTransformerTrainer,
    SentenceTransformerTrainingArguments,
)
from sentence_transformers.losses import MultipleNegativesRankingLoss
from sentence_transformers.sampler import BatchSamplers

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")   # Apache-2.0

train = Dataset.from_dict({
    "anchor":   ["How do I reset my password?", "What is the refund window?"],
    "positive": ["To reset your password open Settings > Security ...", "Refunds are accepted within ..."],
})

args = SentenceTransformerTrainingArguments(
    output_dir="minilm-support",
    num_train_epochs=1,
    per_device_train_batch_size=32,
    learning_rate=2e-5,
    warmup_ratio=0.1,
    batch_sampler=BatchSamplers.NO_DUPLICATES,
)
trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train,
    loss=MultipleNegativesRankingLoss(model),
)
trainer.train()
model.save_pretrained("minilm-support/final")

Measure with a held-out set of queries and relevant documents (recall at k, MRR) using the library’s evaluators, and re-embed your whole corpus after changing the model: vectors from different models, or before and after tuning, are not comparable. Dimension and index implications are covered on Generating Embeddings in Practice. Hard negatives (passages that look relevant but are not) improve results the most and can be mined with the current model. Official reference: Sentence Transformers — Training overview.

Fine-tuning classifiers with SetFit

SetFit trains a text classifier from a handful of labelled examples per class (often 8 to 64). It first fine-tunes a sentence-transformer contrastively on pairs generated from the labelled texts, then fits a small classification head on the resulting embeddings. The result is a small model that runs on CPU and is easy to export. For larger labelled sets, fine-tuning an encoder such as BERT with Trainer remains a solid alternative and is worth trying against SetFit on your evaluation set. Check the project’s repository for recent activity before depending on it in a long-lived product.

Few-shot classifier with SetFit

from datasets import load_dataset
from setfit import SetFitModel, Trainer, TrainingArguments, sample_dataset

dataset = load_dataset("SetFit/sst2")
train_ds = sample_dataset(dataset["train"], label_column="label", num_samples=8)   # 8 per class
test_ds = dataset["test"]

model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5")   # MIT-licensed embedding body
args = TrainingArguments(batch_size=32, num_epochs=1)
trainer = Trainer(model=model, args=args, train_dataset=train_ds)
trainer.train()
print(trainer.evaluate(test_ds))

model.save_pretrained("setfit-sst2-8-shot")
print(model.predict(["A charming and affecting journey.", "It is slow -- very, very slow."]))

Datasets need text and label columns unless you pass column_mapping. Official reference: SetFit — Quickstart. For the task-head route see the transformers fine-tuning guide.

Compute estimates

Estimate before renting or buying hardware. Let be the parameter count of the base model, the bytes per stored weight (2 for bf16, about 0.5 for 4-bit), the number of training tokens processed, and the number of trainable parameters.

Memory. Training memory is the sum of weights, gradients, optimizer state and activations:

For full fine-tuning with mixed precision and AdamW, , gradients are kept at 2 or 4 bytes and the optimizer keeps two moment tensors (and often an fp32 master copy of the weights), so a common planning figure is on the order of 16 bytes per parameter before activations. With LoRA, is tiny, so the last term collapses and ; with QLoRA, plus quantization constants. grows with batch size and sequence length and shrinks with gradient checkpointing (on by default in TRL) at the cost of an extra forward pass. The inference-side counterpart (KV cache) is on Hardware Sizing.

Time. A transformer forward pass costs about floating-point operations per token and a full training step about (forward plus backward). LoRA skips the weight-gradient part of the backward pass, so it is cheaper than full fine-tuning but not free, and gradient checkpointing adds a recomputed forward pass. With a sustained throughput (peak FLOP/s of your accelerator times the fraction you actually reach, typically well below one):

Read your accelerator’s peak figure from its datasheet and measure the efficiency fraction with a 50-step pilot run; this page does not quote vendor numbers because they change with every generation and depend on precision and sparsity. Token count examples average tokens per example epochs (with packing, padding does not inflate it).

A sizing calculator: memory and time from the formulas above

def train_estimate(params_b, *, bytes_per_weight, trainable_params, tokens,
                   peak_flops, efficiency=0.3, c=5.0, act_gb=4.0,
                   grad_opt_bytes_per_trainable=12):
    """params_b in billions; peak_flops in FLOP/s; efficiency is the fraction sustained."""
    P = params_b * 1e9
    mem_gb = (P * bytes_per_weight + trainable_params * grad_opt_bytes_per_trainable) / 1e9 + act_gb
    hours = c * P * tokens / (peak_flops * efficiency) / 3600
    return mem_gb, hours

# LoRA r=16 on all linear layers of a 7B model is on the order of tens of millions of parameters
lora = train_estimate(7, bytes_per_weight=2, trainable_params=40e6, tokens=5e6, peak_flops=1e14)
qlora = train_estimate(7, bytes_per_weight=0.55, trainable_params=40e6, tokens=5e6, peak_flops=1e14)
full = train_estimate(7, bytes_per_weight=2, trainable_params=7e9, tokens=5e6, peak_flops=1e14, c=6)
print(lora, qlora, full)   # (GB, hours) for each setup; peak_flops here is a placeholder, use your datasheet

The calculator is a planning tool with explicit assumptions (the 12 bytes per trainable parameter cover fp32 gradients and two fp32 Adam moments; change it for your optimizer); it ignores fragmentation and multi-GPU overheads, so leave headroom and confirm with a short run. trainer.train() logs real throughput, and num_tokens and mean_token_accuracy are logged by the TRL trainers for tracking.