Skip to content

Docker

docker/Dockerfile has one target per model family. Each image holds the code at /root/decision-models with its .venv: CUDA 12.8, torch 2.9.0, Python 3.11.

target trains extras
encoder UltraBERT, Ettin (make train-encoder) dev
decoder Qwen3 with LLM-as-Jev (make train-decoder) dev
qwen35 Qwen3.5 decoders dev, qwen35 (builds causal-conv1d)

You need an NVIDIA driver r570 or newer and the NVIDIA Container Toolkit.

Build

make docker-build TARGET=encoder        # or decoder, qwen35
# = docker build -f docker/Dockerfile --target encoder -t decision-models:encoder .
make docker-build TARGET=qwen35 DOCKER_ARGS="--build-arg TORCH_CUDA_ARCH_LIST=9.0"   # one GPU generation: faster compile
make docker-build TARGET=encoder DOCKER_ARGS="--platform linux/arm64"                # Grace Hopper / Grace Blackwell

Run

make docker-run TARGET=encoder
# = docker run --rm -it --gpus all --ipc=host --shm-size=32g \
#     --ulimit memlock=-1 --ulimit stack=67108864 \
#     -v $HF_HOME:/root/.cache/huggingface -v $PWD/runs:/root/decision-models/runs \
#     decision-models:encoder

Inside the container, train the same way as on the host:

make train-encoder CONFIG=configs/train/ultrabert_380m_sigmoid_norm.yamlGPUS=0
make train-decoder CONFIG=configs/train/qwen3_0.6b_lora.yamlGPUS=0,1,2,3 ARGS="data.source=Athekunal/llm-jev-reason-long"

flag why
--ipc=host --shm-size=32g shared memory for NCCL (multi-GPU) and DataLoader workers
-v $HF_CACHE:/root/.cache/huggingface keeps weights, datasets and the FA2 kernel between containers (HF_CACHE defaults to $HF_HOME, else ~/.cache/huggingface)
-v runs:... checkpoints and logs outlive the container
DOCKER_ARGS="-e HF_TOKEN" gated models, pushing to the Hub

GPU generations

The configs assume Ampere or newer: bf16 autocast, plus FlashAttention-2 varlen for the encoders. On older GPUs, override both settings:

GPU compute capability settings
Blackwell (B200, RTX 50) 10.0, 12.0 configs as they are
Hopper (H100, H200, GH200) 9.0 configs as they are
Ada (L4, L40S, RTX 4090) 8.9 configs as they are
Ampere (A100, A10, A6000, RTX 3090) 8.0, 8.6 configs as they are (the tested setup)
Turing (T4, RTX 2080) 7.5 model.attention=sdpa model.pack_train=false run.mixed_precision=fp16 (encoders), run.mixed_precision=fp16 (decoders)

The fp16 settings on Turing have not been tested. On a GPU older than Ampere, attention=varlen stops at startup with an error that says to use sdpa. Only the A100 setup has been trained end to end. The other rows use the same torch and kernel builds.