Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Pattern Recognition and Machine Learning
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

2190 results in Pattern Recognition and Machine Learning

Page 72 of 110

  • Preface

  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp xi-xiv

  • Summary

    In the last decade the user base of machine learning has grown dramatically. From a relatively small circle in computer science, engineering, and mathematics departments the users of machine learning now include students and researchers from every corner of the academic universe, as well as members of industry, data scientists, entrepreneurs, and machine learning enthusiasts. The book before you is the result of a complete tearing down of the standard curriculum of machine learning into its most basic components, and a curated reassembly of those pieces (painstakingly polished and organized) that we feel will most benefit this broadening audience of learners. It contains fresh and intuitive yet rigorous descriptions of the most fundamental concepts necessary to conduct research, build products, tinker, and play.

    Intended audience and book pedagogy

    This book was written for readers interested in understanding the core concepts of machine learning from first principles to practical implementation. To make full use of the text one only needs a basic understanding of linear algebra and calculus (i.e., vector and matrix operations as well as the ability to compute the gradient and Hessian of a multivariate function), plus some prior exposure to fundamental concepts of computer programming (i.e., conditional and looping structures). It was written for first time learners of the subject, as well as for more knowledgeable readers who yearn for a more intuitive and serviceable treatment than what is currently available today.

    To this end, throughout the text, in describing the fundamentals of each concept, we defer the use of probabilistic, statistical, and neurological views of the material in favor of a fresh and consistent geometric perspective. We believe that this not only permits a more intuitive understanding of many core concepts, but helps establish revealing connections between ideas often regarded as fundamentally distinct (e.g., the logistic regression and support vector machine classifiers, kernels, and feed-forward neural networks). We also place significant emphasis on the design and implementation of algorithms, and include many coding exercises for the reader to practice at the end of each chapter. This is because we strongly believe that the bulk of learning this subject takes place when learners “get their hands dirty” and code things up for themselves.

  • Part II - Tools for fully data-driven machine learning

  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 129-130

  • Summary

    In Sections 3.2 and 4.5 we have discussed how understanding of regression and classification datasets can be used to forge useful features in particular instances. With regression we saw that by visualizing low-dimensional data we could form excellent features for particular datasets like e.g., data from Galileo's classic ramp experiment. Later, when discussing classification, we also saw how basic features can be designed for e.g., image data using our understanding of natural signals and the mammalian visual processing system. Unfortunately, due to our general ignorance regarding most types of phenomena in the universe, instances such as these are rare and we often have no knowledge on which to construct reasonable features at all. However, we can, as described in the next three chapters, automate the process of feature design itself by leveraging what we know strong features should accomplish for regression/classification tasks.

  • 5 - Automatic feature design for regression

  • from Part II - Tools for fully data-driven machine learning
  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 131-165

  • Summary

    As discussed in the end of Section 3.2, rarely can we design perfect or even strongly performing features for the general regression problem by completely relying on our understanding of a given dataset. In this chapter we describe tools for automatically designing proper features for the general regression problem, without the explicit incorporation of human knowledge gained from e.g., visualization of the data, philosophical reflection, or domain expertise.

    We begin by introducing the tools used to perform regression in the ideal but extremely unrealistic scenario where we have complete and noiseless access to all possible input feature/output pairs of a regression phenomenon, i.e., a continuous function (as first discussed in Section 3.2). Here we will see how, in the case where we have such unfettered access to regression data, perfect features can be designed automatically by combining elements from a set of basic feature transformations. We then see how this process for building features translates, albeit imperfectly, to the general instance of regression where we have access to only noisy samples of a regression relationship. Following this we describe cross-validation, a crucial procedure to employing automatic feature design in practice. Finally we discuss several issues pertaining to the best choice of primary features for automatic feature design in practice.

    Automatic feature design for the ideal regression scenario

    In Fig. 5.1 we illustrate a prototypical dataset on which we perform regression, where our input feature and output have some sort of clear nonlinear relationship. Recall from Section 3.2 that at the heart of feature design for regression is the tacit assumption that the data we receive are in fact noisy samples of some underlying continuous function (shown in dashed black in Fig. 5.1). Our goal in solving the general regression problem is then, using the data at our disposal (which we may think of as noisy glimpses of the underlying function), to approximate this data-generating function as well as we can.

    In this section we will assume the impossible: that we have complete access to a clean version of every input feature/output pair of a regression phenomenon, or in other words that our data completely traces out a continuous function y (x).

  • 1 - Introduction

  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 1-18

  • Summary

    Machine learning is a rapidly growing field of study whose primary concern is the design and analysis of algorithms which enable computers to learn. While still a young discipline, with much more awaiting to be discovered than is currently known, today machine learning can be used to teach computers to perform a wide array of useful tasks. This includes tasks like the automatic detection of objects in images (a crucial component of driver-assisted and self-driving cars), speech recognition (which powers voice command technology), knowledge discovery in the medical sciences (used to improve our understanding of complex diseases), and predictive analytics (leveraged for sales and economic forecasting). In this chapter we give a high level introduction to the field of machine learning and the contents of this textbook. To get a big picture sense of how machine learning works we begin by discussing a simple toy machine learning problem: teaching a computer how to distinguish between pictures of cats from those with dogs. This will allow us to informally describe the procedures used to solve machine learning problems in general.

    Teaching a computer to distinguish cats from dogs

    To teach a child the difference between “cat” versus “dog”, parents (almost!) never give their children some kind of formal scientific definition to distinguish the two; i.e., that a dog is a member of Canis Familiaris species from the broader class of Mammalia, and that a cat while being from the same class belongs to another species known as Felis Catus. No, instead the child is naturally presented with many images of what they are told are either “dogs” or “cats” until they fully grasp the two concepts. How do we know when a child can successfully distinguish between cats and dogs? Intuitively, when they encounter new (images of) cats and dogs, and can correctly identify each new example. Like human beings, computers can be taught how to perform this sort of task in a similar manner. This kind of task, where we aim to teach a computer to distinguish between different types of things, is referred to as a classification problem in machine learning.

  • Part III - Methods for large scale machine learning

  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 217-218

  • Summary

    In the following two chapters we describe two general sets of tools for dealing with large scale data. First, in Chapter 8, we introduce advanced gradient methods which extend the standard gradient descent scheme first described in Chapter 2. These techniques help us to deal with very large datasets directly by making use of more efficient algorithms. In Chapter 9, on the other hand, we introduce general techniques for significantly reducing the size of datasets prior to performing regression or classification.

  • 9 - Dimension reduction techniques

  • from Part III - Methods for large scale machine learning
  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 245-262

  • Summary

    Large datasets, as well as data consisting of a large number of features, present computational problems in the training of predictive models. In this chapter we discuss several useful techniques for reducing the dimension of a given dataset, that is reducing the number of data points or number of features, often employed in order to make predictive learning methods scale to larger datasets. More specifically, we discuss widely used methods for reducing the data dimension, that is the number of data points, of a dataset including random subsampling and K-means clustering. We then detail a common way of reducing the feature dimension, or number features, of a dataset as explained in Fig. 9.1. A classical approach for feature dimension reduction, principal component analysis (PCA), while often used for general data analysis is a relatively poor tool for reducing the feature dimension of predictive modeling data. However, PCA presents a fundamental mathematical archetype, the matrix factorization, that provides a very useful way of organizing our thinking about a wide array of important learning models (including linear regression, K-means, recommender systems – introduced after detailing PCA in this chapter), all of which may be thought of as variations of the simple theme of matrix factorization.

    Techniques for data dimension reduction

    In this section we detail two common ways of reducing the data dimension of a dataset: random subsampling and K-means clustering.

    Random subsampling

    Random subsampling is a simple and intuitive way of reducing the data dimension of a dataset, and is often the first approach employed when performing regression/ classification on datasets too large for available computational resources. Given a set of P points we keep a random subsample of S < P of the entire set. Clearly the smaller we choose S the larger the chance we may loose an important structural characteristic of the underlying dataset (for example the geometry of the separating boundary between two classes of data). While there is no formula or hard rule for how large S should be, a simple guideline used in practice is to choose S as large as possible given the computational resources available so as to minimize this risk.

  • Part I - Fundamental tools and concepts

  • Jeremy Watt, Northwestern University, Illinois, Reza Borhani, Northwestern University, Illinois, Aggelos K. Katsaggelos, Northwestern University, Illinois
  • Book:
    Machine Learning Refined
    Published online:
    05 September 2016
    Print publication:
    08 September 2016, pp 19-20

  • Summary

    In the three chapters that follow we describe in significant detail the basic concepts of machine learning introduced in Chapter 1, beginning with an introduction to several fundamental tools of numerical optimization in Chapter 2. This includes a thorough description of calculus-defined optimality, as well as the widely used gradient descent and Newton's method algorithms. Discussing these essential tools first will enable us to immediately and effectively deal with all of the formal learning problems we will see throughout the entirety of the text. Chapters 3 and 4 then introduce linear regression and classification respectively, the two predictive learning problems which form the bedrock of modern machine learning. We motivate both problems naturally, letting illustrations and geometric intuition guide our formal derivations, and in each case describe (through a variety of examples) how knowledge is used to forge effective features.

Machine Learning Refined
  • Foundations, Algorithms, and Applications
  • Jeremy Watt, Reza Borhani, Aggelos K. Katsaggelos
  • Published online:
    05 September 2016
    Print publication:
    08 September 2016
  • Providing a unique approach to machine learning, this text contains fresh and intuitive, yet rigorous, descriptions of all fundamental concepts necessary to conduct research, build products, tinker, and play. By prioritizing geometric intuition, algorithmic thinking, and practical real world applications in disciplines including computer vision, natural language processing, economics, neuroscience, recommender systems, physics, and biology, this text provides readers with both a lucid understanding of foundational material as well as the practical tools needed to solve real-world problems. With in-depth Python and MATLAB/OCTAVE-based computational exercises and a complete treatment of cutting edge numerical optimization techniques, this is an essential resource for students and an ideal reference for researchers and practitioners working in machine learning, computer science, electrical engineering, signal processing, and numerical optimization.

  • 13 - Objective Bayes Inference and MCMC

  • from Part II - Early Computer-Age Methods
  • Bradley Efron, Stanford University, California, Trevor Hastie, Stanford University, California
  • Book:
    Computer Age Statistical Inference
    Published online:
    05 July 2016
    Print publication:
    21 July 2016, pp 233-263

  • Summary

    From its very beginnings, Bayesian inference exerted a powerful influence on statistical thinking. The notion of a single coherent methodology employing only the rules of probability to go from assumption to conclusion was and is immensely attractive. For 200 years, however, two impediments stood between Bayesian theory's philosophical attraction and its practical application.

  • In the absence of relevant past experience, the choice of a prior distribution introduces an unwanted subjective element into scientific inference.

  • Bayes’ rule (3.5) looks simple enough, but carrying out the numerical calculation of a posterior distribution often involves intricate higherdimensional integrals.

    • The two impediments fit neatly into the dichotomy of Chapter 1, the first being inferential and the second algorithmic.

    A renewed cycle of Bayesian enthusiasm took hold in the 1960s, at first concerned mainly with coherent inference. Building on work by Bruno de Finetti and L. J. Savage, a principled theory of subjective probability was constructed: the Bayesian statistician, by the careful elicitation of prior knowledge, utility, and belief, arrives at the correct subjective prior distribution for the problem at hand. Subjective Bayesianism is particularly appropriate for individual decision making, say for the business executive trying to choose the best investment in the face of uncertain information.

    It is less appropriate for scientific inference, where the sometimes skeptical world of science puts a premium on objectivity. An answer came from the school of objective Bayes inference. Following the approach of Laplace and Jeffreys, as discussed in Section 3.2, their goal was to fashion objective, or “uninformative,” prior distributions that in some sense were unbiased in their effects upon the data analysis.

    In what came as a surprise to the Bayes community, the objective school has been the most successful in bringing Bayesian ideas to bear on scientific data analysis. Of the 24 articles in the December 2014 issue of the Annals of Applied Statistics, 8 employed Bayesian analysis, predominantly based on objective priors.

    This is where electronic computation enters the story. Commencing in the 1980s, dramatic steps forward were made in the numerical calculation of high-dimensional Bayes posterior distributions. Markov chain Monte Carlo (MCMC) is the generic name for modern posterior computation algorithms.

Page 72 of 110