Week 3 Plan: Semi-hands-on Fine-Tuning, Evaluation, and Agent Orchestration

[Blue-No1]

Week 3 Plan

This issue will track goals and tasks for Week 3.
Mục tiêu tuần 3: thực hành semi-hands-on với LoRA/QLoRA, thiết lập repo evaluation, agent orchestration, và mở rộng fine-tuning configs.

🎯 Goals

• Semi-hands-on LoRA/QLoRA experiments on open-weight LLMs.
• Prepare evaluation scripts & probes for base vs LoRA models.
• Minimal agent orchestration loop with mock tools & planner.
• Extend fine-tuning experiments with QLoRA (8-bit) configs.
• Log example outputs (training logs, eval results) for reproducibility.

✅ Tasks

• Day 1: fine-tuning-experiments repo
• README.md, notes.md
• configs/lora-llama3-8b.yaml
• scripts/train_lora.py
• data/demo.jsonl, logs/train.sample.txt
• Day 2: evaluation-metrics repo
• README.md, harness_commands.md
• probes/simple_math.jsonl
• scripts/score_exact_match.py
• Day 3: agent-orchestration-lab repo
• README.md, notes/orchestration_notes.md
• scripts/agent_loop.py (mock tools + planner)
• Day 4: fine-tuning-experiments updates
• configs/qlora-llama3-8b.yaml
• scripts/train_qlora.py
• logs/qlora.sample.txt
• Day 5: evaluation-metrics updates
• results/sample_eval.json
• scripts/merge_results.py

📝 Notes

• Focus on semi-hands-on skeletons; all code/configs should be runnable, even if on tiny datasets.
• Keep GPU memory modest (8–24GB) for LoRA/QloRA.
• Placeholder datasets and logs are fine; the goal is reproducible skeletons.
• Progress to be logged in each repo’s Progress Log in README.md.

---

Feel free to add updates below as comments.