> ## Documentation Index
> Fetch the complete documentation index at: https://docs.monostate.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Training

> Train large language models with the CLI

# LLM Training

The `aitraining llm` command trains large language models with support for multiple trainers and techniques.

## Quick Start

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --trainer sft
```

## Available Trainers

| Trainer                       | Description                                              |
| ----------------------------- | -------------------------------------------------------- |
| `default` / `sft` / `generic` | Supervised fine-tuning                                   |
| `dpo`                         | Direct Preference Optimization                           |
| `orpo`                        | Odds Ratio Preference Optimization                       |
| `ppo`                         | Proximal Policy Optimization                             |
| `grpo`                        | Group Relative Policy Optimization (custom environments) |
| `reward`                      | Reward model training                                    |
| `distillation`                | Knowledge distillation                                   |

<Note>
  `generic` is an alias for `default`. All three (`default`, `sft`, `generic`) produce the same behavior.
</Note>

<Warning>
  **PPO Trainer Requirements**: PPO requires either `--rl-reward-model-path` (path to a trained reward model) or `--model-ref` (reference model for KL divergence). See [PPO Training](/advanced/ppo-rl-training) for full documentation.
</Warning>

<Warning>
  **GRPO Trainer Requirements**: GRPO requires `--rl-env-module` (Python module path) and `--rl-env-class` (class name) for the custom environment. See [GRPO Training](/advanced/grpo-training) for full documentation.
</Warning>

## Parameter Groups

Parameters are organized into logical groups:

### Basic Parameters

| Parameter        | Description             | Default               |
| ---------------- | ----------------------- | --------------------- |
| `--model`        | Base model to fine-tune | `google/gemma-3-270m` |
| `--data-path`    | Path to training data   | `data`                |
| `--project-name` | Output directory name   | `project-name`        |
| `--train-split`  | Training data split     | `train`               |
| `--valid-split`  | Validation data split   | `None`                |

<Note>
  **Always specify these parameters**: While `--model`, `--data-path`, and `--project-name` have defaults, you should always explicitly set them for your use case. The `--project-name` parameter sets the output folder - use a path like `--project-name ./models/my-experiment` to control where the trained model is saved.
</Note>

### Training Configuration

| Parameter                 | Description               | Default       |
| ------------------------- | ------------------------- | ------------- |
| `--trainer`               | Training method           | `default`     |
| `--epochs`                | Number of training epochs | `1`           |
| `--batch-size`            | Training batch size       | `2`           |
| `--lr`                    | Learning rate             | `3e-5`        |
| `--mixed-precision`       | fp16/bf16/None            | `None`        |
| `--gradient-accumulation` | Accumulation steps        | `4`           |
| `--warmup-ratio`          | Warmup ratio              | `0.1`         |
| `--optimizer`             | Optimizer                 | `adamw_torch` |
| `--scheduler`             | LR scheduler              | `linear`      |
| `--weight-decay`          | Weight decay              | `0.0`         |
| `--max-grad-norm`         | Max gradient norm         | `1.0`         |
| `--seed`                  | Random seed               | `42`          |

### Checkpointing & Evaluation

| Parameter                  | Description                                             | Default |
| -------------------------- | ------------------------------------------------------- | ------- |
| `--eval-strategy`          | When to evaluate (`epoch`, `steps`, `no`)               | `epoch` |
| `--save-strategy`          | When to save (`epoch`, `steps`, `no`)                   | `epoch` |
| `--save-steps`             | Save every N steps (if save-strategy=steps)             | `500`   |
| `--save-total-limit`       | Max checkpoints to keep                                 | `1`     |
| `--logging-steps`          | Log every N steps (-1 for auto)                         | `-1`    |
| `--resume-from-checkpoint` | Resume from checkpoint path, or `auto` to detect latest | `None`  |

### Performance & Memory

| Parameter                          | Description                                                           | Default |
| ---------------------------------- | --------------------------------------------------------------------- | ------- |
| `--auto-find-batch-size`           | Automatically find optimal batch size                                 | `False` |
| `--disable-gradient-checkpointing` | Disable memory optimization                                           | `False` |
| `--unsloth`                        | Use Unsloth for faster training (SFT only, llama/mistral/gemma/qwen2) | `False` |
| `--use-sharegpt-mapping`           | Use Unsloth's ShareGPT mapping                                        | `False` |
| `--use-flash-attention-2`          | Use Flash Attention 2 for faster training                             | `False` |
| `--attn-implementation`            | Attention implementation (`eager`, `sdpa`, `flash_attention_2`)       | `None`  |

<Note>
  **Unsloth Requirements**: Unsloth only works with `sft`/`default` trainers and specific model architectures (llama, mistral, gemma, qwen2). See [Unsloth Integration](/advanced/unsloth-integration) for details.
</Note>

### Backend & Distribution

| Parameter               | Description                               | Default |
| ----------------------- | ----------------------------------------- | ------- |
| `--backend`             | Where to run (`local`, `spaces`)          | `local` |
| `--distributed-backend` | Distribution backend (`ddp`, `deepspeed`) | `None`  |
| `--ddp-timeout`         | DDP/NCCL timeout in seconds               | `7200`  |

<Note>
  **Multi-GPU Behavior**: With multiple GPUs and `--distributed-backend` not set, DDP is used automatically. Set `--distributed-backend deepspeed` for DeepSpeed Zero-3 optimization. Training is launched via Accelerate.
</Note>

<Warning>
  **DeepSpeed Checkpointing**: When using DeepSpeed, model saving uses `accelerator.get_state_dict()` and unwraps the model. PEFT adapter saving is handled differently under DeepSpeed.
</Warning>

### PEFT/LoRA Parameters

| Parameter          | Description               | Default      |
| ------------------ | ------------------------- | ------------ |
| `--peft`           | Enable LoRA training      | `False`      |
| `--lora-r`         | LoRA rank                 | `16`         |
| `--lora-alpha`     | LoRA alpha                | `32`         |
| `--lora-dropout`   | LoRA dropout              | `0.05`       |
| `--target-modules` | Modules to target         | `all-linear` |
| `--quantization`   | int4/int8 quantization    | `None`       |
| `--merge-adapter`  | Merge LoRA after training | `True`       |

### Data Processing

| Parameter                | Description                                                 | Default                |
| ------------------------ | ----------------------------------------------------------- | ---------------------- |
| `--text-column`          | Text column name                                            | `text`                 |
| `--block-size`           | Max sequence length                                         | `-1` (model default)   |
| `--model-max-length`     | Maximum model input length                                  | Auto-detect from model |
| `--padding`              | Padding side (`left` or `right`)                            | `right`                |
| `--add-eos-token`        | Append EOS token                                            | `True`                 |
| `--chat-template`        | Chat template to use                                        | Auto by trainer        |
| `--packing`              | Enable sequence packing (requires flash attention)          | `None`                 |
| `--auto-convert-dataset` | Auto-detect and convert dataset format                      | `False`                |
| `--max-samples`          | Limit dataset size for testing                              | `None`                 |
| `--save-processed-data`  | Save processed data: `auto`, `local`, `hub`, `both`, `none` | `auto`                 |

<Note>
  **Chat Template Auto-Selection**: SFT/DPO/ORPO/Reward trainers default to `tokenizer` (model's built-in template). Use `--chat-template none` for plain text training.
</Note>

<Note>
  **Processed Data Saving**: By default (`auto`), processed data is saved locally to `{project}/data_processed/`. If the source dataset was from the Hub, it's also pushed as a private dataset. Original columns are renamed to `_original_*` to prevent conflicts.
</Note>

## Training Examples

### SFT with LoRA

```bash theme={null}
aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./conversations.jsonl \
  --project-name llama-sft \
  --trainer sft \
  --peft \
  --lora-r 16 \
  --lora-alpha 32 \
  --epochs 3 \
  --batch-size 4
```

### DPO Training

For DPO, you must specify the column names for prompt, chosen, and rejected responses:

```bash theme={null}
aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./preferences.jsonl \
  --project-name llama-dpo \
  --trainer dpo \
  --prompt-text-column prompt \
  --text-column chosen \
  --rejected-text-column rejected \
  --dpo-beta 0.1 \
  --peft \
  --lora-r 16
```

<Warning>
  DPO and ORPO require `--prompt-text-column` and `--rejected-text-column` to be specified.
</Warning>

### ORPO Training

ORPO combines SFT and preference optimization:

```bash theme={null}
aitraining llm --train \
  --model google/gemma-2-2b \
  --data-path ./preferences.jsonl \
  --project-name gemma-orpo \
  --trainer orpo \
  --prompt-text-column prompt \
  --text-column chosen \
  --rejected-text-column rejected \
  --peft
```

### GRPO Training

Train with Group Relative Policy Optimization using your own reward environment:

```bash theme={null}
aitraining llm --train \
  --model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
  --trainer grpo \
  --rl-env-module my_envs.hotel_env \
  --rl-env-class HotelEnv \
  --rl-num-generations 4 \
  --rl-max-new-tokens 256 \
  --peft \
  --lr 1e-5
```

<Note>
  GRPO generates multiple completions per prompt, scores them via your environment (0-1), and optimizes the policy. See [GRPO Training](/advanced/grpo-training) for environment interface details.
</Note>

### Knowledge Distillation

Train a smaller model to mimic a larger one:

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --teacher-model google/gemma-2-2b \
  --data-path ./prompts.jsonl \
  --project-name distilled-model \
  --use-distillation \
  --distill-temperature 3.0
```

<Note>
  Distillation defaults: `--distill-temperature 3.0`, `--distill-alpha 0.7`, `--distill-max-teacher-length 512`
</Note>

## Logging & Monitoring

### Weights & Biases (Default)

W\&B logging with LEET visualizer is **enabled by default**. The LEET visualizer shows real-time training metrics directly in your terminal.

```bash theme={null}
# W&B is on by default - just run training
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model
```

To disable W\&B or the visualizer:

```bash theme={null}
# Disable W&B logging entirely
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --log none

# Keep W&B but disable terminal visualizer
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --no-wandb-visualizer
```

### TensorBoard

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --log tensorboard
```

## Push to Hugging Face Hub

Upload your trained model:

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --push-to-hub \
  --username your-username \
  --token $HF_TOKEN
```

<Note>
  The repository is created as **private** by default. By default, the repo will be named `{username}/{project-name}`.
</Note>

### Custom Repository Name or Organization

Use `--repo-id` to push to a specific repository, useful for:

* Pushing to an **organization** instead of your personal account
* Using a **different repo name** than your local `project-name`

```bash theme={null}
# Push to an organization
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name ./local-training-output \
  --push-to-hub \
  --repo-id my-organization/my-custom-model-name \
  --token $HF_TOKEN

# Push to personal account with different name
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name ./experiment-v3 \
  --push-to-hub \
  --repo-id your-username/production-model \
  --token $HF_TOKEN
```

| Parameter                            | Description                           | Default                     |
| ------------------------------------ | ------------------------------------- | --------------------------- |
| `--push-to-hub`                      | Enable pushing to Hub                 | `False`                     |
| `--hub-private` / `--no-hub-private` | Create repo as private or public      | `True` (private)            |
| `--username`                         | HF username (for default repo naming) | `None`                      |
| `--token`                            | HF API token                          | `None`                      |
| `--repo-id`                          | Full repo ID (e.g., `org/model-name`) | `{username}/{project-name}` |

<Warning>
  When using `--repo-id`, you don't need `--username` since the repo ID already specifies the destination. However, you still need `--token` for authentication.
</Warning>

## Advanced Options

### Hyperparameter Sweeps

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name sweep-experiment \
  --use-sweep \
  --sweep-backend optuna \
  --sweep-n-trials 10
```

### Enhanced Evaluation

```bash theme={null}
aitraining llm --train \
  --model google/gemma-3-270m \
  --data-path ./data \
  --project-name my-model \
  --use-enhanced-eval \
  --eval-metrics "perplexity,bleu"
```

## View All Parameters

See all parameters for a specific trainer:

```bash theme={null}
aitraining llm --trainer sft --help
aitraining llm --trainer dpo --help
```

## Next Steps

<CardGroup cols={2}>
  <Card title="YAML Configs" href="/cli/yaml-configs">
    Use configuration files
  </Card>

  <Card title="DPO Training" href="/advanced/dpo-training">
    Deep dive into DPO
  </Card>

  <Card title="LoRA/PEFT" href="/advanced/lora-peft">
    Efficient fine-tuning
  </Card>

  <Card title="Distillation" href="/advanced/prompt-distillation">
    Knowledge distillation
  </Card>

  <Card title="GRPO Training" href="/advanced/grpo-training">
    RL with custom environments
  </Card>
</CardGroup>