LLM Fine-tuning: Day 0

Fine-tuning a Small/Large Language Model

For the next few weeks, I will be fine-tuning a small language model (e.g. GPT-2 (124M parameters) or DistilBERT) to generate plain English explanations from Python code snippets.

• Week 1: Enviornment setup, data collection, and data pre-processing
• Week 2: Model selection and initial training
• Week 3: Training optimization and evaluation
• Week 4: Model refinement and deployment

The end goal is to publish/share the model on hugging face or deploy it on a web service.

I will locally train the model on my computer, but will consider migrating to Google Colabs if needed