Book contents
- Frontmatter
- Contents
- List of Figures
- List of Tables
- Preface
- 1 Introduction
- 2 The Perceptron
- 3 Logistic Regression
- 4 Implementing Text Classification Using Perceptron and Logistic Regression
- 5 Feed-Forward Neural Networks
- 6 Best Practices in Deep Learning
- 7 Implementing Text Classification with Feed-Forward Networks
- 8 Distributional Hypothesis and Representation Learning
- 9 Implementing Text Classification Using Word Embeddings
- 10 Recurrent Neural Networks
- 11 Implementing Part-of-Speech Tagging Using Recurrent Neural Networks
- 12 Contextualized Embeddings and Transformer Networks
- 13 Using Transformers with the Hugging Face Library
- 14 Encoder-Decoder Methods
- 15 Implementing Encoder-Decoder Methods
- 16 Neural Architectures for Natural Language Processing Applications
- Appendix A Overview of the Python Language and Key Libraries
- Appendix B Character Encodings: ASCII and Unicode
- References
- Index
7 - Implementing Text Classification with Feed-Forward Networks
Published online by Cambridge University Press: 01 February 2024
- Frontmatter
- Contents
- List of Figures
- List of Tables
- Preface
- 1 Introduction
- 2 The Perceptron
- 3 Logistic Regression
- 4 Implementing Text Classification Using Perceptron and Logistic Regression
- 5 Feed-Forward Neural Networks
- 6 Best Practices in Deep Learning
- 7 Implementing Text Classification with Feed-Forward Networks
- 8 Distributional Hypothesis and Representation Learning
- 9 Implementing Text Classification Using Word Embeddings
- 10 Recurrent Neural Networks
- 11 Implementing Part-of-Speech Tagging Using Recurrent Neural Networks
- 12 Contextualized Embeddings and Transformer Networks
- 13 Using Transformers with the Hugging Face Library
- 14 Encoder-Decoder Methods
- 15 Implementing Encoder-Decoder Methods
- 16 Neural Architectures for Natural Language Processing Applications
- Appendix A Overview of the Python Language and Key Libraries
- Appendix B Character Encodings: ASCII and Unicode
- References
- Index
Summary
In this chapter, we provide an implementation of the multilayer neural network described in Chapter 5, along with several of the best practices discussed in Chapter 6. Still keeping things fairly simple, our network will consist of two fully connected layers: a hidden layer and an output layer. Between these layers, we will include dropout and a nonlinearity. Further, we make use of two PyTorch classes: a Dataset and a DataLoader. The advantage of using these classes is that they make several things easy, including data shuffling and batching. Last, since the classifier’s architecture has become more complex, for optimization we transition from stochastic gradient descent to the Adam optimizer to take advantage of its additional features such as momentum and L2 regularization.
- Type
- Chapter
- Information
- Deep Learning for Natural Language ProcessingA Gentle Introduction, pp. 107 - 116Publisher: Cambridge University PressPrint publication year: 2024