Book contents
- Frontmatter
- Contents
- List of Figures
- List of Tables
- Preface
- 1 Introduction
- 2 The Perceptron
- 3 Logistic Regression
- 4 Implementing Text Classification Using Perceptron and Logistic Regression
- 5 Feed-Forward Neural Networks
- 6 Best Practices in Deep Learning
- 7 Implementing Text Classification with Feed-Forward Networks
- 8 Distributional Hypothesis and Representation Learning
- 9 Implementing Text Classification Using Word Embeddings
- 10 Recurrent Neural Networks
- 11 Implementing Part-of-Speech Tagging Using Recurrent Neural Networks
- 12 Contextualized Embeddings and Transformer Networks
- 13 Using Transformers with the Hugging Face Library
- 14 Encoder-Decoder Methods
- 15 Implementing Encoder-Decoder Methods
- 16 Neural Architectures for Natural Language Processing Applications
- Appendix A Overview of the Python Language and Key Libraries
- Appendix B Character Encodings: ASCII and Unicode
- References
- Index
13 - Using Transformers with the Hugging Face Library
Published online by Cambridge University Press: 01 February 2024
- Frontmatter
- Contents
- List of Figures
- List of Tables
- Preface
- 1 Introduction
- 2 The Perceptron
- 3 Logistic Regression
- 4 Implementing Text Classification Using Perceptron and Logistic Regression
- 5 Feed-Forward Neural Networks
- 6 Best Practices in Deep Learning
- 7 Implementing Text Classification with Feed-Forward Networks
- 8 Distributional Hypothesis and Representation Learning
- 9 Implementing Text Classification Using Word Embeddings
- 10 Recurrent Neural Networks
- 11 Implementing Part-of-Speech Tagging Using Recurrent Neural Networks
- 12 Contextualized Embeddings and Transformer Networks
- 13 Using Transformers with the Hugging Face Library
- 14 Encoder-Decoder Methods
- 15 Implementing Encoder-Decoder Methods
- 16 Neural Architectures for Natural Language Processing Applications
- Appendix A Overview of the Python Language and Key Libraries
- Appendix B Character Encodings: ASCII and Unicode
- References
- Index
Summary
One of the key advantages of transformer networks is the ability to take a model that was pretrained over vast quantities of text and fine-tune it for the task at hand. Intuitively, this strategy allows transformer networks to achieve higher performance on smaller datasets by relying on statistics acquired at scale in an unsupervised way (e.g., through the masked language model training objective). To this end, in this chapter, we will use the Hugging Face library, which has a rich repository of datasets and pretrained models, as well as helper methods and classes that make it easy to target downstream tasks. Using pretrained transformer encoders, we will implement the two tasks that served as use cases in the previous chapters: text classification and part-of-speech tagging.
- Type
- Chapter
- Information
- Deep Learning for Natural Language ProcessingA Gentle Introduction, pp. 194 - 215Publisher: Cambridge University PressPrint publication year: 2024