What is an LLM?
Fine tune LLM-
"fine-tuning" usually refers to the process of taking a pre-trained model and further training it on a more specific dataset or for a particular task. This is often done in scenarios where you have a large, general-purpose model that has been trained on vast amounts of data but needs to be adapted for a specific use case.
Here are different ways of fine-tuning in AI:
Transfer Learning: This involves taking a model that has been trained on one task and repurposing it for a different but related task. For instance, a model trained on general image recognition can be fine-tuned for a specific type of image classification.
Learning Rate Adjustment: Adjusting the learning rate during fine-tuning is crucial. A lower learning rate is generally used to ensure that the pre-trained weights are not distorted too quickly and that the model can gradually adapt to the new data.
Layer Freezing: In this method, some layers of the pre-trained model are "frozen" or kept unchanged, while others are allowed to update during the fine-tuning process. This is common in deep learning models like Convolutional Neural Networks (CNNs).
Data Augmentation: Involves creating variations of the training data, which helps in improving the robustness of the model and can be particularly useful if the fine-tuning dataset is small.
Regularization Techniques: Applying regularization methods like dropout or L1/L2 regularization during fine-tuning helps in preventing overfitting to the new dataset.
Hyperparameter Tuning: Fine-tuning also involves adjusting hyperparameters like batch size, number of training epochs, and others to find the best settings for the new task.
Feature Extraction: Sometimes, the lower layers of a pre-trained model are used only for feature extraction, and new layers are added on top for the specific task, which are then trained.
Domain-Specific Fine-Tuning: This involves training the model further on data that is very specific to the domain or industry where it will be applied, to make it more effective in that context.
https://www.youtube.com/watch?v=eC6Hd1hFvos&ab_channel=ShawTalebi
RAG
LORA
Source- https://towardsdatascience.com/fine-tune-your-own-llama-2-model-in-a-colab-notebook-df9823a04a32