2026 · Python · PyTorch · PEFT / LoRA
LoRA Medical QA Fine-Tune
Parameter-efficient fine-tuning of Llama 3.2 1B for medical Q&A on a single free T4 GPU.
- 2.28 → 1.08
- Eval loss
- −52.6%
- Δ eval loss
- 9.77 → 2.94
- Perplexity
- ~11M (<1%)
- Trainable params
Problem
Full fine-tuning is out of reach on consumer hardware, and most "fine-tune" demos hide the honest before/after. This is a complete, reproducible PEFT workflow — data prep with loss masking through baseline-vs-tuned evaluation — proving a 1B model can adapt to a domain on a single free GPU.
Approach
- 01LoRA adapters on Llama 3.2 1B Instruct — ~11M trainable params, under 1% of the model.
- 02MedQuAD (NIH medical Q&A), with loss masking so only answer tokens contribute to the loss.
- 03T4-optimized: fp16, gradient checkpointing, and accumulation — 2 epochs in ~1h45m on one Kaggle T4.
- 04Config-driven scripts: baseline → train → eval → plot → infer, with fixed seeds and splits.
Architecture
- MedQuAD → preprocessing + loss masking
- Llama 3.2 1B Instruct (frozen base)
- LoRA adapters · ~11M params (<1%)
- fp16 + grad checkpointing on a T4
- Baseline vs. tuned evaluation
Challenges
- Fitting training into 16 GB — fp16, checkpointing, and accumulation instead of a bigger batch.
- Staying honest that gains are largely style/format adaptation, not clinical expertise.
Key features
- LoRA PEFT on Llama 3.2 1B Instruct
- MedQuAD with answer-only loss masking
- Config-driven, reproducible seeds and splits
- Unit + smoke tests
- Before/after side-by-side sample comparisons
The README is explicit that improvement reflects style/format adaptation, not clinical expertise — included here because honest evaluation is the point.