> ## Documentation Index
> Fetch the complete documentation index at: https://docs.monostate.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Unsloth Integration

> Faster LoRA training with Unsloth

# Unsloth Integration

Unsloth provides optimized training for LoRA fine-tuning, significantly reducing training time and memory usage.

## Requirements

| Requirement            | Details                              |
| ---------------------- | ------------------------------------ |
| **Installation**       | `pip install unsloth`                |
| **Supported Trainers** | `default`, `sft` only                |
| **Supported Models**   | `llama`, `mistral`, `gemma`, `qwen2` |
| **Platform**           | Linux recommended                    |

<Warning>
  Unsloth only works with **SFT training** (`--trainer sft` or `--trainer default`). DPO, ORPO, PPO, and other trainers are not supported.
</Warning>

## Supported Model Architectures

Unsloth is optimized for specific model families:

| Architecture | Example Models                         |
| ------------ | -------------------------------------- |
| `llama`      | Llama 2, Llama 3, Llama 3.1, Llama 3.2 |
| `mistral`    | Mistral 7B, Mistral Nemo               |
| `gemma`      | Gemma, Gemma 2                         |
| `qwen2`      | Qwen 2, Qwen 2.5                       |

Other model architectures will fall back to standard training with a warning.

## Quick Start

```bash theme={null}
aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./data \
  --project-name fast-model \
  --trainer sft \
  --unsloth \
  --peft \
  --lora-r 16
```

## Parameters

| Parameter              | CLI Flag                 | Default | Description                                          |
| ---------------------- | ------------------------ | ------- | ---------------------------------------------------- |
| `unsloth`              | `--unsloth`              | `False` | Enable Unsloth for faster training                   |
| `use_sharegpt_mapping` | `--use-sharegpt-mapping` | `False` | Use Unsloth's ShareGPT mapping instead of converting |

## Python API

```python theme={null}
from autotrain.trainers.clm.params import LLMTrainingParams
from autotrain.project import AutoTrainProject

params = LLMTrainingParams(
    model="meta-llama/Llama-3.2-1B",
    data_path="./data",
    project_name="fast-model",

    trainer="sft",
    unsloth=True,

    peft=True,
    lora_r=16,
    lora_alpha=32,

    epochs=3,
    batch_size=4,
)

project = AutoTrainProject(params=params, backend="local", process=True)
project.create()
```

## With Quantization

Unsloth works with int4 and int8 quantization for reduced memory usage:

```bash theme={null}
aitraining llm --train \
  --model meta-llama/Llama-3.2-3B \
  --data-path ./data \
  --project-name quantized-model \
  --trainer sft \
  --unsloth \
  --peft \
  --quantization int4 \
  --lora-r 16
```

## How It Works

When Unsloth is enabled and requirements are met:

1. Uses `FastLanguageModel` from Unsloth library for optimized model loading
2. Applies optimized gradient checkpointing (`use_gradient_checkpointing="unsloth"`)
3. Automatically configures LoRA target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
4. Integrates with PEFT for efficient adapter training

## Fallback Behavior

If Unsloth cannot be used, training continues with standard transformers/PEFT:

* **Unsloth not installed**: Warning logged, continues without Unsloth
* **Unsupported model type**: Warning logged, continues without Unsloth
* **Unsupported trainer**: Unsloth not applied (only SFT supported)

```
WARNING: Unsloth not available, continuing without it...
```

## ShareGPT Mapping

Use `--use-sharegpt-mapping` to preserve ShareGPT format instead of converting:

```bash theme={null}
aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./sharegpt_data.json \
  --project-name model \
  --trainer sft \
  --unsloth \
  --use-sharegpt-mapping \
  --peft
```

## Next Steps

<CardGroup cols={2}>
  <Card title="LoRA/PEFT" href="/advanced/lora-peft">
    Efficient fine-tuning techniques
  </Card>

  <Card title="Quantization" href="/advanced/quantization">
    Reduce memory with quantization
  </Card>
</CardGroup>