Skip to content

Training

A run is a YAML file under configs/train/ plus key=value overrides. A typo or an invalid value fails at startup. The resolved config is saved to <run.out>/config.yaml.

make train-encoder CONFIG=configs/train/ultrabert_380m_sigmoid_norm.yamlGPUS=0
make train-decoder CONFIG=configs/train/qwen3_0.6b_lora.yamlGPUS=0,1,2,3 ARGS="optim.lr=5e-5 run.out=runs/x"
scripts/train.shconfigs/train/qwen3_0.6b_full.yaml0 run.seed=1 # same, in the background, log in <run.out>.log

GPUS sets the processes: DDP for encoders, FSDP2 for decoders. optim.batch_size is the total across all GPUs.

config model loss
ultrabert_380m_softmax_yes.yaml UltraBERT-380M softmax over ℓ(yes)
ultrabert_380m_softmax_yesno.yaml UltraBERT-380M softmax over ℓ(yes) − ℓ(no)
ultrabert_380m_sigmoid_norm.yaml UltraBERT-380M σ(s) normalised over the options
ultrabert_380m_sqrt_softplus.yaml UltraBERT-380M √softplus(s) normalised over the options
ettin_encoder_400m.yaml Ettin-400M sigmoid-norm
qwen3_0.6b_lora.yaml, qwen3_4b_lora.yaml Qwen3, LoRA r16 L_tree + KL anchors
qwen3_0.6b_full.yaml Qwen3-0.6B, full fine-tuning L_tree + KL anchors

Arguments

Shared by both families

key default meaning
data.source data/splits/reason_long_nimble local folder of train/eval.parquet, or a Hub id (Athekunal/llm-jev-reason-long)
data.max_tokens 16384 drop longer training rows
data.eval_per 100 dev rows per dataset evaluated before and after training (at most 100; never test rows)
optim.lr 1e-4 peak learning rate
optim.batch_size 32 rows per optimizer step, total across all GPUs
optim.warmup_steps / warmup_ratio None / 0.05 linear warmup, then linear decay
optim.min_lr_ratio 0.0 final lr / peak
optim.weight_decay, betas, eps, max_grad_norm 0.01, [0.9, 0.999], 1e-8, 1.0 AdamW and clipping
run.out required run directory
run.seed 20260922 option shuffles and visiting order
run.epochs 1 passes over the training rows
run.max_steps -1 stop early, e.g. 10 for a smoke test
run.eval_every 0 also evaluate and checkpoint every N steps
run.skip_base_eval false skip the evaluation before training
run.mixed_precision "no" bf16 in every config; fp16 on pre-Ampere GPUs (quote "no" in YAML)

Encoder (model.*, format.*, optim.loss)

key default meaning
model.id required Hub id or path of the masked LM
model.revision None Hub commit
model.attention varlen varlen (FlashAttention-2, packs rows, Ampere or newer) or sdpa
model.pack_train true pack each GPU's rows into one varlen forward, about 4× faster; false, or attention: sdpa (Ettin), trains one row per forward
model.pack_tokens 32768 tokens per packed forward
model.gradient_checkpointing true recompute activations in the backward pass
format.template decision_mlm prompts/<name>.j2, one mask per option
format.yes_token " yes" s_k = ℓ_k(yes)
format.no_token None set " no": s_k = ℓ_k(yes) − ℓ_k(no)
optim.loss listwise listwise (softmax), sigmoid_norm, sqrt_softplus (the last two need format.no_token)

Decoder (model.*, objective.*)

key default meaning
model.id required Hub id or path of the causal LM (Qwen3 / Qwen3.5 chat template)
model.finetune lora lora or full (fp32 weights, frozen bf16 reference model)
model.lora_rank 16 LoRA rank (alpha = 2 × rank)
model.batch_tokens, batch_rows 32768, 64 scoring batch limits (evaluation)
objective.anchors [mass, out, pos] KL anchors added to L_tree, e.g. "objective.anchors=[out,pos]"
objective.lam 1.0 anchor weight
data.gen_per 0 dev rows per dataset also answered as free text

Qwen3.5 needs the qwen35 extra (uv sync --extra qwen35, or the qwen35 Docker target): ARGS="model.id=Qwen/Qwen3.5-4B run.out=runs/qwen35_4b".

Outputs

runs/x/
  config.yaml          resolved config
  train.log  tb/       per-step loss, accuracy, lr, grad norm (TensorBoard)
  eval_base.json       dev accuracy before training, per dataset and ALL
  eval_final.json      after training
  model/               encoder weights + tokenizer; full fine-tuned decoder
  adapter/             decoder LoRA adapter
  checkpoints/step-N/  with run.eval_every

Evaluate

scripts/evaluate.sh encoder runs/x 0 # dev split + external benchmarks
decision-models-evaluate --configconfigs/evaluate/encoder.yamlmodel.path=runs/x rows.part=test tag=test
decision-models-evaluate --configconfigs/evaluate/decoder.yamlmodel.path=Qwen/Qwen3-0.6B model.adapter=runs/q/adapter out=runs/q
decision-models-benchmarks --configconfigs/evaluate/benchmarks.yamlmodel.kind=encoder model.path=runs/x out=runs/x

key meaning
model.kind, model.path encoder + run directory, or decoder + Hub id/path
model.adapter decoder LoRA adapter
rows.part dev (tune here), test (report here), all
rows.datasets, rows.per subset of datasets, rows per dataset
out, tag writes <out>/eval_<tag>.json

From Python

configs/train/ultrabert_380m_sigmoid_norm.yaml

from decision_models import config
from decision_models.training import encoder, trainer

cfg, raw = config.from_command_line(config.EncoderTrainConfig, argv=[
    "--config", "configs/train/ultrabert_380m_sigmoid_norm.yaml", "run.out=runs/x", "run.max_steps=10"])
trainer.launch(cfg, raw, lambda device: encoder.build(cfg, device))