LLM Fine-tuning: Day 7
Full Training on Colabs and Testing Prep
Colabs
Today, I managed to get a free Colabs Pro subscription through their student verification process, meaning that I have access to better GPUs.
Instead of using T4 (which I constant had an issue with), I decided to use A100 to reduce my training duration significantly.
I revised my training script to optimize it for the A100 GPU and utilize most of 40GB of GPU provided. So, I set the script to use 0.95 (95%) of the GPU provided.
Setting Up for Training
Then, I set up the GPT-2 124 parameter model, its tokenizer, and my special tokens to prepare for training.
As usual, I initialzed Weights and Biases to mointor the training loss, validation loss, and learning rate.
After that, I loaded my full tokenized datasets and printed out how many I had for tracking purposes.
With all of that set, I wrote my training arguments to fully utilize the A100 GPU’s capabilities.
- Increased the batch size to 32 and reduced the gradient accumulation steps to 2
- Increased epoch to 5 for better quality results and dataloader workers to 16
- Enhanced learning rate to 5e-4 and set up weight decay to prevent overfitting
- Set up methods for logging training/validation loss
Training Results
I successfully ran the script and the training process only took around 2.5 hours, which is a significant reduction from ~5 hours using the T4 GPU.
Looking at my wandb panels, it’s clear that my training process went well without any issues because you can see that both the training and validation losses increased with more datasets, meaning that model became more accurate. Also, the decay seen in the learning rate showcases improved convergence towards an optimized solution
Moving On
I will work on creating simple test to evaluate the quality of the model’s output and eventually test it on all the codes in my testing dataset.
If the model performs with at least a 80% accuracy, I will deploy it on hugging face and use API calls to make a simple web application for use.