Replies: 3 comments
Beginner Projects1. Tokenizer Comparison ToolCompare BPE, WordPiece, and whitespace tokenizers with visual token splits
2. Attention VisualizerInteractive tool showing attention scores and multi-head patterns
3. Mini Text GeneratorSmall GPT trained on focused dataset (Shakespeare, recipes, code)
Intermediate Projects4. Domain-Specific LLMPretrain on specialized corpus (medical, legal, programming)
5. Multi-Task ClassifierSingle model for sentiment, spam detection, and topic classification
6. Code AssistantFinetune for code explanation, generation, and documentation
Advanced Projects7. Instruction Chatbot with DPOConversational AI with preference learning and alignment
8. Mixture-of-Experts ModelSparse model with expert routing and load balancing
9. Production LLM ServerOptimized serving with quantization, KV-cache, and REST API
10. RAG Question AnsweringDocument retrieval with semantic search and source attribution
Quick Start
|
|
A global LLM localized on everyones PC and laptop, that runs and makes compute available when users are not using their PC's. One could even create a small payment solution for it, that would encourage more people to let their PCs on while not using them. Security must be FortKnox.There are a lot of unused graphic cards and cryptomining hardware that people could plug in for such a tool. |
|
For a project idea: try fine-tuning the book's model on a domain-specific dataset (medical texts, code, etc.) and compare perplexity before/after. Great way to understand how pre-training vs fine-tuning affects LLM behavior. Also sharing: ML Interview Prep — covers LLM fine-tuning strategies (LoRA, RLHF, instruction tuning) with interview-focused explanations. Might be useful as a reference while working through this book! |
Uh oh!
There was an error while loading. Please reload this page.
Suggest any project to build
All reactions