Configuration Templates
Copy these templates and customize for your use case.LLM Training Templates
Quick SFT (Small Model)
Best for testing and small datasets:task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: quick-sft
data:
path: ./data.jsonl
train_split: train
valid_split: null
chat_template: tokenizer
column_mapping:
text_column: text
log: wandb
params:
epochs: 3
batch_size: 4
lr: 3e-5
Production SFT (7B Model)
For full fine-tuning with LoRA:task: llm-sft
backend: local
base_model: meta-llama/Llama-3.2-8B
project_name: production-sft
data:
path: ./conversations.jsonl
train_split: train
valid_split: validation
chat_template: tokenizer
column_mapping:
text_column: text
log: wandb
hub:
push_to_hub: false
params:
epochs: 3
batch_size: 2
gradient_accumulation: 8
lr: 3e-5
warmup_ratio: 0.1
mixed_precision: bf16
peft: true
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
use_flash_attention_2: true
packing: true
save_strategy: steps
save_steps: 500
save_total_limit: 1
DPO Preference Training
For preference alignment:task: llm-dpo
backend: local
base_model: meta-llama/Llama-3.2-1B
project_name: dpo-aligned
data:
path: ./preferences.jsonl
train_split: train
valid_split: null
chat_template: tokenizer
column_mapping:
prompt_text_column: prompt
text_column: chosen
rejected_text_column: rejected
log: wandb
params:
dpo_beta: 0.1
max_prompt_length: 128
max_completion_length: null
epochs: 1
batch_size: 2
gradient_accumulation: 4
lr: 5e-6
peft: true
lora_r: 16
lora_alpha: 32
ORPO Training
Combined SFT + preference optimization:task: llm-orpo
backend: local
base_model: google/gemma-2-2b
project_name: gemma-orpo
data:
path: ./preferences.jsonl
train_split: train
valid_split: null
chat_template: tokenizer
column_mapping:
prompt_text_column: prompt
text_column: chosen
rejected_text_column: rejected
log: wandb
params:
dpo_beta: 0.1
max_prompt_length: 128
max_completion_length: null
epochs: 3
batch_size: 2
lr: 5e-5
peft: true
lora_r: 16
Knowledge Distillation
For compressing larger models:task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: distilled-gemma
data:
path: ./prompts.jsonl
train_split: train
valid_split: null
chat_template: tokenizer
column_mapping:
text_column: text
log: wandb
params:
use_distillation: true
teacher_model: google/gemma-2-2b
distill_temperature: 3.0
distill_alpha: 0.7
epochs: 5
batch_size: 8
lr: 1e-4
Classification Templates
Text Classification
task: text-classification
backend: local
base_model: bert-base-uncased
project_name: sentiment
data:
path: ./reviews.csv
train_split: train
valid_split: null
column_mapping:
text_column: text
target_column: target
log: wandb
params:
epochs: 5
batch_size: 16
lr: 5e-5
Multi-Class Classification
task: text-classification
backend: local
base_model: microsoft/deberta-v3-base
project_name: categorizer
data:
path: ./categories.csv
train_split: train
valid_split: validation
column_mapping:
text_column: content
target_column: target
log: wandb
params:
epochs: 10
batch_size: 8
lr: 1e-5
warmup_ratio: 0.1
Token Classification (NER)
task: token-classification
backend: local
base_model: bert-base-cased
project_name: entity-extractor
data:
path: ./ner_data.json
train_split: train
valid_split: null
column_mapping:
tokens_column: tokens
tags_column: tags
log: wandb
params:
epochs: 5
batch_size: 16
lr: 5e-5
Vision Templates
Image Classification
task: image-classification
backend: local
base_model: google/vit-base-patch16-224
project_name: image-classifier
data:
path: ./images/
train_split: train
valid_split: null
column_mapping:
image_column: image
target_column: target
log: wandb
params:
epochs: 10
batch_size: 32
lr: 5e-5
Object Detection
task: object-detection
backend: local
base_model: facebook/detr-resnet-50
project_name: detector
data:
path: ./coco_format/
train_split: train
valid_split: null
column_mapping:
image_column: image
objects_column: objects
log: wandb
params:
epochs: 20
batch_size: 8
lr: 1e-4
Vision-Language Model (VQA)
task: vlm:vqa
backend: local
base_model: google/paligemma-3b-pt-224
project_name: vlm-vqa
data:
path: ./vqa_data.jsonl
train_split: train
valid_split: null
column_mapping:
image_column: image
text_column: text
prompt_text_column: prompt
log: wandb
params:
epochs: 3
batch_size: 2
lr: 5e-5
gradient_accumulation: 4
peft: true
lora_r: 16
lora_alpha: 32
Advanced Templates
Hyperparameter Sweep
task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: sweep-experiment
data:
path: ./data.jsonl
train_split: train
valid_split: validation
chat_template: tokenizer
column_mapping:
text_column: text
log: wandb
params:
use_sweep: true
sweep_backend: optuna
sweep_n_trials: 20
sweep_metric: eval_loss
sweep_direction: minimize
# Base parameters (sweep will vary these)
epochs: 3
batch_size: 4
lr: 3e-5
peft: true
lora_r: 16
Sentence Transformers
task: sentence-transformers:pair_score
backend: local
base_model: sentence-transformers/all-MiniLM-L6-v2
project_name: embeddings
data:
path: ./pairs.csv
train_split: train
valid_split: null
column_mapping:
sentence1_column: sentence1
sentence2_column: sentence2
target_column: score
log: wandb
params:
epochs: 3
batch_size: 8
lr: 3e-5
Usage
Save any template asconfig.yaml and run:
aitraining --config config.yaml
Next Steps
YAML Configs
Config file structure details
LLM Training
Parameter reference