85 Welcome to Large Language Models
This module introduces Large Language Models (LLMs) and their applications in data science. We’ll explore basic concepts, use cases, ethical considerations, and hands-on applications of LLMs. Eventually, there will be video lectures. Once those videos exist, please watch the videos and work your way through the notes. The videos will eventually start on the next page. You will eventually be able to find the video playlist for this module [here][pl_llm]. The slides used to make the videos in this module will be able to be found in the slides repo.
This module is in active development, as part of an AI teaching workshop sponsored by the Wake Forest University’s Center for Teaching.
85.1 Module Materials
- Slides from Lectures:
- Intro to LLMs,
- LLMs in Data Science,
- [LLM Ethics][d33_llmethics]
- Suggested Readings
- All subchapters of this module, including
- …
- R4DS
- TBD, including
- [TBD]
- [TBD]
- TBD, including
- Key Papers:
- Attention Is All You Need by Vaswani et al., NeurIPS
2017.
- This paper introduces the Transformer model, a fundamental architecture for modern LLMs.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Devlin et al., NAACL 2019.
- BERT introduces bidirectional training and significantly improves NLP tasks.
- Language Models are Few-Shot Learners by Brown et al., NeurIPS 2020.
- This paper details GPT-3 and its few-shot learning capabilities.
- Training language models to follow instructions with human feedback by Ouyang et al., NeurIPS 2022.
- Discusses the integration of human feedback to improve LLM performance and ethical considerations.
- Attention Is All You Need by Vaswani et al., NeurIPS
- All subchapters of this module, including
- Activities
- Exploring LLM Capabilities with OpenAI’s API [TBD]
- Lab