Offered By: IBMSkillsNetwork
Advanced Fine-Tuning for Large Language Models (LLMs)
Fine-tune Large Language Models (LLMs) to enhance AI accuracy and optimize performance with cutting-edge skills employers seek
Continue reading Premium
Course
Artificial Intelligence
At a Glance
Fine-tune Large Language Models (LLMs) to enhance AI accuracy and optimize performance with cutting-edge skills employers seek
In the Generative Advanced Fine Tuning for LLMs course you will gain hands-on expertise in fine-tuning large language models (LLMs) and learn how to optimize AI systems for real-world applications. This course will equip you with cutting-edge skills that are highly sought after by top employers.
You will learn:
- In-demand Generative AI engineering skills in fine-tuning LLMs employers are actively looking for in just 2 weeks
- Instruction-tuning and reward modeling with Hugging Face, plus LLMs as policies and RLHF
- Direct preference optimization (DPO) with partition function and Hugging Face and how to create an optimal solution to a DPO problem
- How to use proximal policy optimization (PPO) with Hugging Face to create a scoring function and perform dataset tokenization
Course Overview
Fine-tuning your LLM is crucial for aligning it with specific business needs, enhancing accuracy, and optimizing its performance. In turn, this gives businesses precise, actionable insights that drive efficiency and innovation. This course gives aspiring Generative AI engineers valuable fine-tuning skills employers are actively seeking.
During this intermediate-level course, you’ll explore different approaches to fine-tuning and casual LLMs with human feedback and direct preference. You’ll look at LLMs as policies for probability distributions for generating responses and the concepts of instruction-tuning with Hugging Face. You’ll learn to calculate rewards using human feedback and reward modeling with Hugging Face. Plus, you’ll explore reinforcement learning from human feedback (RLHF), proximal policy optimization (PPO) and PPO Trainer, and optimal solutions for direct preference optimization (DPO) problems.
As you learn, you’ll get valuable hands-on experience in online labs where you’ll work on reward modeling, PPO, and DPO.
If you’re looking to add in-demand capabilities in fine-tuning LLMs to your resume, ENROLL TODAY and build the job-ready skills employers are looking for in just two weeks!
Course Syllabus
Module 0: Welcome
- Video: Course Introduction
- Specialization Overview
- General Information
- Learning Objectives and Syllabus
- Helpful Tips for Course Completion
- Grading Scheme
Module 1: Different Approaches to Fine-Tuning
- Reading: Module Introduction and Learning Objectives
- Video: Basics of Instruction-Tuning
- Video: Instruction-Tuning with Hugging Face
- Reading: Instruction Tuning
- Lab: Instruction Fine-Tuning LLMs
- Video: Reward Modeling: Response Evaluation
- Video: Reward Model Training
- Video: Reward Modeling with Hugging Face
- Reading:Reward Modeling & Response Evaluation
- Lab: Reward Modeling
- Practice Quiz: Different Approaches to Fine-Tuning
- Reading: Module Summary and Highlights
- Graded Quiz: Different Approaches to Fine-Tuning
Module 2: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
- Reading: Module Introduction and Learning Objectives
- Video: Large Language Models (LLMs) as Distributions
- Video: From Distributions to Policies
- Video: Reinforcement Learning from Human Feedback (RLHF)
- Video: Proximal Policy Optimization (PPO)
- Video: PPO with Hugging Face
- Video: PPO Trainer
- Lab: Reinforcement Learning from Human Feedback Using PPO
- Video: DPO – Partition Function
- Video: DPO – Optimal Solution
- Video: From Optimal Policy to DPO
- Video: DPO with Hugging Face
- Lab: Direct Preference Optimization (DPO) Using Hugging Face
- Reading: Fine-Tune LLMs Locally with InstructLab
- Reading: Module Summary and Highlights
- Practice Quiz: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
- Graded Quiz: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
- Reading: Cheat Sheet – Generative AI Advanced Fine-Tuning for LLMs
- Reading: Glossary – Generative AI Advance Fine-Tuning for LLMs
Course Wrap Up
- Reading: Course Conclusion
- Reading: Congratulations and Next Steps
- Reading: Teams and Acknowledgments
- Copyright and Trademarks
Recommended Skills Prior to Taking this Course
Basic knowledge of LLMs, instruction-tuning, and reinforcement learning. Familiarity with machine learning and neural network concepts is also beneficial.
Estimated Effort
8 Hours
Level
Intermediate
Skills You Will Learn
Direct Preference Optimization (DPO), Hugging Face, Instruction-tuning, Proximal Policy Optimization (PPO), Reinforcement Learning
Language
English
Course Code
AI0212EN