Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.monostate.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Unsloth Integration

Unsloth provides optimized training for LoRA fine-tuning, significantly reducing training time and memory usage.

​
Requirements

RequirementDetails
Installationpip install unsloth
Supported Trainersdefault, sft only
Supported Modelsllama, mistral, gemma, qwen2
PlatformLinux recommended
Unsloth only works with SFT training (--trainer sft or --trainer default). DPO, ORPO, PPO, and other trainers are not supported.

​
Supported Model Architectures

Unsloth is optimized for specific model families:
ArchitectureExample Models
llamaLlama 2, Llama 3, Llama 3.1, Llama 3.2
mistralMistral 7B, Mistral Nemo
gemmaGemma, Gemma 2
qwen2Qwen 2, Qwen 2.5
Other model architectures will fall back to standard training with a warning.

​
Quick Start

aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./data \
  --project-name fast-model \
  --trainer sft \
  --unsloth \
  --peft \
  --lora-r 16

​
Parameters

ParameterCLI FlagDefaultDescription
unsloth--unslothFalseEnable Unsloth for faster training
use_sharegpt_mapping--use-sharegpt-mappingFalseUse Unsloth’s ShareGPT mapping instead of converting

​
Python API

from autotrain.trainers.clm.params import LLMTrainingParams
from autotrain.project import AutoTrainProject

params = LLMTrainingParams(
    model="meta-llama/Llama-3.2-1B",
    data_path="./data",
    project_name="fast-model",

    trainer="sft",
    unsloth=True,

    peft=True,
    lora_r=16,
    lora_alpha=32,

    epochs=3,
    batch_size=4,
)

project = AutoTrainProject(params=params, backend="local", process=True)
project.create()

​
With Quantization

Unsloth works with int4 and int8 quantization for reduced memory usage: 
aitraining llm --train \
  --model meta-llama/Llama-3.2-3B \
  --data-path ./data \
  --project-name quantized-model \
  --trainer sft \
  --unsloth \
  --peft \
  --quantization int4 \
  --lora-r 16

​
How It Works

When Unsloth is enabled and requirements are met:
  1. Uses FastLanguageModel from Unsloth library for optimized model loading
  2. Applies optimized gradient checkpointing (use_gradient_checkpointing="unsloth")
  3. Automatically configures LoRA target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  4. Integrates with PEFT for efficient adapter training

​
Fallback Behavior

If Unsloth cannot be used, training continues with standard transformers/PEFT:
  • Unsloth not installed: Warning logged, continues without Unsloth
  • Unsupported model type: Warning logged, continues without Unsloth
  • Unsupported trainer: Unsloth not applied (only SFT supported)
WARNING: Unsloth not available, continuing without it...

​
ShareGPT Mapping

Use --use-sharegpt-mapping to preserve ShareGPT format instead of converting: 
aitraining llm --train \
  --model meta-llama/Llama-3.2-1B \
  --data-path ./sharegpt_data.json \
  --project-name model \
  --trainer sft \
  --unsloth \
  --use-sharegpt-mapping \
  --peft

​
Next Steps

LoRA/PEFT

Efficient fine-tuning techniques

Quantization

Reduce memory with quantization