Choosing the Right Model
The model you choose dramatically affects training time, quality, and hardware requirements. This guide helps you make the right choice.Model Size vs Hardware
The golden rule: A model needs roughly 2x its parameter count in GB of memory for training. A 7B model needs ~14GB VRAM for full training, or ~8GB with LoRA.
Quick Reference
Memory Estimation Formula
- Full training: 7B × 16 = ~112GB (needs multi-GPU)
- With LoRA: 7B × 2 + 2GB = ~16GB
- With LoRA + int4: 7B × 0.5 + 2GB = ~6GB
Base vs Instruction-Tuned Models
This is one of the most important decisions you’ll make.Base Models (Pretrained)
Examples:google/gemma-2-2b, meta-llama/Llama-3.2-1B
What they are: Trained on raw text to predict the next word. They know language but don’t know how to be helpful.
When to use:
- You have lots of training data (10k+ examples)
- You want full control over the model’s behavior
- You’re training for a specific format (not chat)
- You want to create your own instruction style
Instruction-Tuned Models (IT/Instruct)
Examples:google/gemma-2-2b-it, meta-llama/Llama-3.2-1B-Instruct
What they are: Base models that have already been trained to follow instructions and be helpful.
When to use:
- You have limited training data (100-5k examples)
- You want to refine existing helpful behavior
- You’re building a chatbot or assistant
- You want faster results with less data
Decision Matrix
Model Families
Google Gemma
Versions: Gemma 2, Gemma 3
Strengths: Great for smaller sizes, efficient, good multilingual support
Tip: Add
-it suffix for instruction-tuned versions
Meta Llama
Versions: Llama 3.1, Llama 3.2
Strengths: Excellent quality, strong reasoning, great community support
Note: Requires accepting license on HuggingFace first
Mistral
Strengths: Efficient, fast inference, good at code
Tip: Mistral often punches above its weight class
Qwen (Alibaba)
Strengths: Excellent multilingual, especially Asian languages
Searching for Models
In the wizard, use these commands:Sorting Options
Tips for Choosing
Start small, scale up
Start small, scale up
Always start with a smaller model like
gemma-3-270m. Get your pipeline working, verify your dataset is formatted correctly, then scale up to larger models.Don't chase the biggest model
Don't chase the biggest model
A well-trained 3B model often beats a poorly-trained 7B model. Focus on data quality first, then scale the model.
Match model to data
Match model to data
If you only have 500 examples, a 270M-1B model is plenty. Using a 7B model will just memorize your data instead of learning patterns.
Consider inference costs
Consider inference costs
If you’re deploying the model, remember: larger models cost more to run. A 1B model is 7x cheaper to serve than a 7B model.
Try instruction-tuned first
Try instruction-tuned first
Unless you have 10k+ high-quality examples, start with an instruction-tuned model. You’ll get better results faster.
Validating Your Choice
After selecting a model, the wizard validates it exists:Next Steps
Dataset Guide
Prepare your training data
LoRA for Large Models
Train bigger models on limited hardware