We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
from Part III - Big Data Applications
Published online by Cambridge University Press: 18 May 2017
In this chapter, we discuss distributed algorithms for solving large-scale optimization problems over networks. In particular, the optimization problem is defined over a network of nodes/agents where each of them may have a piece of data, or may be in charge of updating a subset of variables, and they are required to collaboratively perform the desired optimization. The key question to be addressed is, under various problem settings, what protocols should the agents follow so as to efficiently optimize the overall problem. These types of distributed optimizing algorithms have recently become increasingly important in various big data problems, with applications ranging from distributed machine learning, signal processing, networking, multi-agent systems, parallel optimization, etc. There are a few driving forces behind these new trends, as we briefly list below:
1. The advances in sensor technologies makes distributed data acquisition ubiquitous.
2. The increasing size of optimization problems in big data era calls for distributed data storage.
3. Privacy issues in big data problem prevents direct data exchange among different (possibly geographically distributed) data sources.
4. Using multi-core high-performance computing systems (with either shared or distributed memory) to solve big data optimization problems can significantly reduce the solution time.
In this chapter, we present a few recent algorithms for solving such distributed optimization problems. We will cover a wide range of algorithms and discuss their convergence behavior, as well as their performance in terms of scaling with network/problem sizes.
The chapter is organized as follows. In Sections 9.1–9.4, we present a general problem formulation and a few popular first-order methods. In Section 9.5, we discuss a special cases of the general formulation (i.e., the global consensus problem). In Section 9.6, we compare various algorithms presented in this chapter.
Background
Problem Formulation
The Optimization Problem
Consider a network with N distributed agents, each capable of communicating with its immediate neighbors; see Figure 9.1 for an illustration.
To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Find out more about the Kindle Personal Document Service.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.
