Week 7 Outline
- Review Quiz 3
- Discuss experience reports
- GPT-2 architecture (small)
Downloads
Topics
- Overview of code and config table
- Raschka figure 5.17
- Tokenizer review
- wte entry in params
- Analysis of wte token values
- Text generation overview
- Thursday: Revisit next token generation (code for demo)
- Loss function for next token prediction
- Overview of fine tuning for classification
- Spam classification (text) vs. Course description classification
- Development process (see Raschka figure 6.8)
- Architecture (see Raschka figure 6.9)
- Project discussion for next week
Prepare 5 minute overview of project idea(s)
- Project proposals
- Experience report ideas