Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Computer Hardware, Architecture and Distributed Computing
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

705 results in Computer Hardware, Architecture and Distributed Computing

Page 9 of 36

  • 6 - Message-passing Programming

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp 214-242

  • Summary

    INTRODUCTION

    One of the types of parallel processing is distributed computing. This type of processing can be conducted in integrated computers with distributed memory, or in clusters, which are systems of homogeneous or heterogeneous networked computers. In distributed computing the tasks communicate via communication channels (or links). The channels form an interconnection network between processors or computers. Processors or computers that are vertices of a network perform computing tasks, as well as send and receive messages.

    In this chapter we explore how to implement parallel programs that consist of tasks cooperating with each other using message passing. Parallel programs should be written in a suitable programming language. Probably the only language specially developed to describe parallel computing with message passing was occam. This language proposed by May et al. in Inmos company was based on the CSP notation (acronym for Communicating Sequential Processes) defined by Hoare. In the 1980s occam was used as the programming language for transputers—systems of large-scale integration, each combining a processor and four communication channels. Along with development of computer hardware, it turned out that occam due to certain weaknesses and restrictions is insufficient to describe distributed computing. Nowadays, these are often carried out using C or Fortran languages augmented by functions intended for cooperation of parallel processes. The most popular libraries of such functions are PVM (Parallel Virtual Machine) and MPI (Message Passing Interface).

    The PVM library was developed in Oak Ridge National Laboratory. It permits the creation and execution of parallel programs in heterogeneous networks consisting of sequential and parallel computers. Another popular and largely universal library that is used to build distributed programs is MPI. It can be applied together with the OpenMP interface in computers with distributed memory (see Section 5.4.3), in particular in clusters composed of multicore processors or SMP nodes (see Sections 5.4.2 and 5.4.3, and Chapter 7). The library is highly portably enabling to build scalable programs for applications where achieving high computational performance is essential.

  • 2 - Basic Models of Parallel Computation

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp 35-62

  • Summary

    THE SHARED MEMORY MODEL

    A model is a theoretical or physical object whose analysis or observation allows for exploration of another real object or process. A model represents an explored object in a simplified manner by taking into account only its basic features. Due to simplifications models are easier to analyze than the corresponding real objects.

    The subject of our interest are models of computers enabling the study of computational processes executed within those computers. The models, called models of computation, are helpful when analyzing and designing algorithms, as well as in determining performance metrics used for evaluation of algorithms (see Section 3.1). Models of computation should not be associated with a specific computer architecture, or with a class of such architectures. In other words, their independence from hardware is essential. Another essential feature should be versatility that ensures that algorithms developed adopting these models can be implemented and run on computers with different architectures. It is particularly important in the field of parallel computing where diversity of architectures is high. As a result of this diversity several models have been advanced. Unfortunately, due to relatively large number of requirements that a model should satisfy, partly in conflict with each other, none of the models developed so far has become a generally accepted model of parallel computation. The frequently used are the shared memory model (or parallel random access machine model, PRAM) and the network model. They correspond to parallel computation conducted with the use of shared memory and by sending messages over some communication network. Before discussing these models, we will present a sequential model of computation underlying the PRAM model.

    2.1.1 The RAM model

    A widely accepted model of sequential computation is the machine with random access memory (RAM). The model consists of a processor and memory containing a potentially infinite number of cells Mi for i = 1, 2, 3, … (Figure 2.1). In each memory cell identified by its address i, a finite value expressed in binary, perhaps very large, can be stored. The model assumes that the time to read (write) a value from (in) a cell Mi is constant and equal to unit time, regardless of a cell address.

  • Preface

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp xxi-xxviii

  • Summary

    Solving contemporary scientific and technology problems requires the use of computers with a high speed of computation. Over the last 60 years the rate of this speed has increased 16 trillion (1012) times. In the 1950s the speed of computation of a Univac 1 computer was about 1 kflop/s (flop denotes a floating-point operation), and in 2015 China's supercomputer Tianhe-2 (Milky Way-2), which contained 3 120 000 cores working in parallel, achieved the computation speed of more than 33 Pflop/s (petaflop stands for one quadrillion, 1015, floating-point operations). Despite a significant increase in computational capabilities, researchers simplify models of considered problems because their numerical simulation takes too long. The demand for more and more computing power has increased and it is believed that this trend will continue in the future. There are several reasons behind this trend. Models of investigated phenomena and processes have become more complex and larger amounts of data are being processed. The requirements regarding accuracy of results also grow, which entails a higher resolution of models being developed. The fields in which, through large computing power, significant results have been achieved include: aeronautics, astrophysics, bioinformatics, chemistry, economics and trade, energy, geology and geophysics, materials science, climatology, cosmology, medicine, meteorology, nanotechnology, defense, and advanced engineering.

    For example, in the U.S. National Aeronautics and Space Agency (NASA), simulation problems related to research missions conducted by space shuttles have been investigated [50]. A parallel computer SGIAltix with 10 240 processors, consisting of 20 nodes each holding 512 processors, installed in the J. Ames Center allowed for simulation of a pressure distribution around a space shuttle during its flight. A package of computational fluid dynamics used for this goal was a tool for designing geometry of the parts of a space shuttle, that is, launchers and orbital units. Another group of issues resolved in the NASA research centers concerned jet drive units. One of the tasks was to simulate a flow of liquid fuel supplied to a space shuttle main engine by a turbine pump ([31], sect. 2.4).

    In order to improve aircraft performance and safety, NASA conducts research in new aircraft technologies. One of the objectives is the accurate prediction of aerodynamic and structural performance for rotorcraft designed for civil and military applications.

  • 5 - Architectures of Parallel Computers

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp 175-213

  • Summary

    Classification of Architectures

    Roughly speaking, computer architecture is a structure of computer system components. Architecture, in addition to manufacturing technology, is a major factor determining the speed of a computer. Therefore designers devote a great deal of attention to improving computer architectures. One of architecture classifications is Flynn's taxonomy, which is based on the concepts of instruction stream and data stream. An instruction stream is a sequence of instructions executed by a processor, and a data stream is a sequence of data processed by an instruction stream. Depending on the multiplicity of instruction and data streams occurring on a computer, Flynn has distinguished four classes of architectures (Figure 5.1).

    Computers of SISD architecture, in brief SISD computers, are conventional computers wherein a processor executes a single instruction stream processing a single data stream. In modern processors, regularly more than one instruction is executed within a single clock cycle. Processors are equipped with a certain number of functional units enabling implementation of instruction in a pipelined fashion. Processors with multiple functional units are called superscalar.

    Suppose that the process of executing an instruction consists of six sequentially performed microoperations (also termed microinstructions): fetch instruction (FI), decode instruction (DI), calculate operand address (CA), fetch operand (FO), execute instruction (EI), write result (WR). A sequence of microoperations making up the process of implementing an instruction is called pipeline. Each microoperation in the sequence is also called stage of a pipeline, so in our example we have the 6-stage pipeline. Assume that the separate functional units (hardware circuitry) J1, J2, …, J6 have been implemented in a processor to perform particular microoperations. Pipelined execution of an instruction stream i1, i2, …, i8, … is illustrated in Figure 5.2. In the first stage (clock cycle 1), unit J1 fetches instruction i1 from program memory and forwards it to unit J2 for decoding. In the second stage (clock cycle 2) unit J2 decodes instruction i1, and unit J1 fetches instruction i2. At the end of stage 2, units J1 and J2 send results of their operation to units J2 and J3, respectively, and so on.

  • 7 - Shared-memory Programming

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp 243-282

  • Summary

    INTRODUCTION

    Another basic type of parallel processing, next to message-passing computing, is shared-memory computing. As the name indicates, this type of computing assumes that programs have access to shared memory covering the whole or a part of the operating memory of a parallel computer. A single program is executed by one or more collaborating threads created and deleted dynamically. The threads executed in parallel can operate on both the shared data and private data.

    In this chapter we concentrate on the principles of shared-memory programming with the use of OpenMP API (Open Multi-Processing Application Programming Interface) that works in conjunction with C/C++ and Fortran languages. The interface is defined by the collection of compiler directives, library functions, and environment variables. These elements allow users to indicate to a compiler the parts of a sequential program that can be executed in parallel. Basically, the interface is intended to create programs on shared-memory architectures (see Section 5.3). However, supported by MPI it can also be used to write parallel programs on clusters equipped with shared as well as distributed memory (see Sections 5.4.1 and

    5.4.2).

    The first version of the interface was developed by the OpenMP Architecture Review Board (ARB) in 1997. The board, which is still active, includes representatives of computer and software manufacturers, as well as a wide range of experts from various research institutions and universities.2 What is important, from the very beginning OpenMP received support from the leading computer companies such as Compaq, Digital, IBM, Intel, and Silicon Graphics. The producer group supporting OpenMP is constantly growing, as evidenced by creation of compilers for C,C++ and Fortran that accept the interface. The OpenMP 2.5 version (May 2005) has become a kind of standard. The work on version 3.0 was completed in May 2008, and on version 3.1 in July 2011.

  • 4 - Designing Parallel Algorithms

  • Zbigniew J. Czech
  • Book:
    Introduction to Parallel Computing
    Published online:
    06 January 2017
    Print publication:
    11 January 2017, pp 125-174

  • Summary

    STEPS OF DESIGNING

    As we stated in Section 3.1, a parallel algorithm is composed of a number of algorithm components. These components are intended to solve the subproblems into which the original problem has been divided. Normally, the components cooperate with each other using intermediate results of computation as well as synchronize their action. The desired output of the parallel algorithm is a composition of results computed by the algorithm components. Since the subproblems are solved by the components of a parallel program called tasks, we will use the terms subproblem and task interchangeably.

    The process of designing a parallel algorithm consists of four steps:

    1. □ decomposition of a computational problem into tasks that can be executed simultaneously, and development of sequential algorithms for individual tasks;

    2. □ analysis of computation granularity;

    3. □ minimizing the cost of the parallel algorithm;

    4. □ assigning tasks to processors executing the parallel algorithm.

    The following sections characterize these activities in more detail. In addition, we present some general methods for designing parallel algorithms, such as the method of data parallelism, the method of functional parallelism, the method of task pool, the method of master and slaves, and the method of pipelining. These methods differ from each other with respect to ways of exploiting potential parallelism in the computation that solve the problem, as well as with respect to the ways of assigning processors the tasks to be executed.

    PROBLEM DECOMPOSITION

    4.2.1 Types of Decomposition

    Dividing a computational problem into tasks, called decomposition or partitioning, can be done in several ways. We will discuss them on examples

    Example 4.1 Evaluation of test results

    Let us examine a problem of evaluation of multiple-choice test results. Suppose that the test results are written on n sheets, where on each sheet there are m questions with marked answers.

Introduction to Parallel Computing
  • Zbigniew J. Czech
  • Published online:
    06 January 2017
    Print publication:
    11 January 2017
  • The constantly increasing demand for more computing power can seem impossible to keep up with. However, multicore processors capable of performing computations in parallel allow computers to tackle ever larger problems in a wide variety of applications. This book provides a comprehensive introduction to parallel computing, discussing theoretical issues such as the fundamentals of concurrent processes, models of parallel and distributed computing, and metrics for evaluating and comparing parallel algorithms, as well as practical issues, including methods of designing and implementing shared- and distributed-memory programs, and standards for parallel program implementation, in particular MPI and OpenMP interfaces. Each chapter presents the basics in one place followed by advanced topics, allowing novices and experienced practitioners to quickly find what they need. A glossary and more than 80 exercises with selected solutions aid comprehension. The book is recommended as a text for advanced undergraduate or graduate students and as a reference for practitioners.

  • 17 - Exception Handling in C++

  • from PART-II - Object Oriented Programming
  • Kunal Pimparkhede
  • Book:
    Computer Programming with C++
    Published online:
    28 May 2018
    Print publication:
    18 October 2016, pp 969-993

  • Summary

    Overview

    Exception is an abnormal situation in the program that can terminate the program if it is not handled by the programmer. Some examples of exception are as follows:

  • 1. If you accidently perform division by zero in the program, then it is called division by zero exception

  • 2. If you access array elements beyond the specific boundaries of an array index, then it is called array index out of bounds exception. For example, if the size of array a is 5 and if you accidently try to access a[7] that does not exist, then it could give logically incorrect results, which is an exception

  • 3. If your program is required to take salary of an employee as input from the user and by mistake the user inputs a negative number, then it is called an invalid input exception

  • 4. If your program is writing to a file but the file is read-only, then there is an access rights violation in performing the output operation, which is also an example of exception

  • 5. In network communication, if you are writing your own C++ client to communicate with the server on the same network and if server is down when you are sending messages through the client program, then it is called ‘Resource Not available’ type of exception

    • There are many more specific types of exceptions that may occur in your program from case to case. What is important is to understand what exceptions are and how effectively we can handle them to ensure that the program does not terminate abruptly when exception occurs. It is the programmer's responsibility to create robust programs that do not fail abruptly even if any abnormal situation occurs.

    NOTES

    There is a difference between an exception and an error. If there is an error in the program, the program will certainly fail. For example, a semicolon missing after a statement is an example of syntax error; program going out of memory while running a large loop is an example of memory error and so on. Errors cannot be handled and hence the program will certainly fail when an error occurs. Whereas, exception is something that can be handled in the program thereby avoiding the abrupt failure of the program. Exception handlers in C++ are used to handle exceptions (not errors).

  • 5 - Iterative Control Statements: Loops

  • from PART-I - Structured Programming
  • Kunal Pimparkhede
  • Book:
    Computer Programming with C++
    Published online:
    28 May 2018
    Print publication:
    18 October 2016, pp 146-215

  • Summary

    Introduction

    Often we come across requirements where same set of operations need to be executed multiple times to generate the required output. For example, we need to repeatedly perform multiplication to calculate factorial of a number; we need to repeatedly draw lines to construct a polygon and so on. Each time the operation may use different data sets for evaluation; hence, results of each of the individual operations may be different, but the expression that defines the operation is always the same. To give an example, although lines are drawn repeatedly to construct a polygon, the [x,y] coordinates (data values) giving start and end of a line may be different each time. Immaterial of the data used in each stage, we will still consider this as an repetitive execution of the same operation. We can form an expression to draw line from (x1, y1) to (x2, y2) in terms of variables x1, y1, x2 and y2. We can change the value of these variables each time before repeating draw line operation so as to ensure that each line is created at a different position and the consolidated result is a polygon.

    To elaborate what we just said, let us try to construct a triangle using a series of drawLine operations. A triangle can be visualized as a consolidated effect of three lines drawn at correct positions. Hence, the execution of the drawLine operation must be repeated thrice. Before we proceed further, readers are requested to note that the detailed study of computer graphics is out of the scope of this textbook, however, we still give an example to draw a triangle using some abstract operation drawLine to understand the concept of loops in C/C++.

    Let us assume that the expression to draw a single line from point (x1, y1) to point (x2,y2) be as shown below:

    drawLine[(x1,y1),(x2,y2)]

    Drawing a triangle using this operation will now be a three-step process, which is elaborated as below:

    Stage 1: Initialize variables x1=50, y1=0, x2=0, y2=50 and execute the operation

    drawLine[(x1,y1),(x2,y2)]

    The line drawn in this step is shown in Figure 5.1.

  • 15 - Input and Output Streams in C++

  • from PART-II - Object Oriented Programming
  • Kunal Pimparkhede
  • Book:
    Computer Programming with C++
    Published online:
    28 May 2018
    Print publication:
    18 October 2016, pp 881-927

  • Summary

    Overview

    A program is a set of instructions that perform operations on the data to generate a desired output. The data is generally provided to the program from external sources. These sources are called inputs, which are given to the program to attain the required results. On receiving the input data, the C++ code must perform necessary computations to generate the results. These results are called the outputs, which are either to be stored somewhere or rendered to the user.

    It is important to note that a program can take inputs from multiple sources like keyboard, mouse, files, database, and so on. Along the same lines, a program can distribute its output to multiple destinations like computer screen (for rendering), files, database, speaker, and so on. The selection of input sources and output destinations purely depends upon the business requirements on the basis of which the code is designed.

    Given a variety of inputs and outputs possible in today's world, it is important that the programming language makes it easy for the programmer to write a program that can take input from any required source(s) and distribute output to any required destination(s) using a consistent approach. What this means is that, a programming language should offer a layer of abstraction to the programmer, so as to hide the complexities of differences in each of the input/output devices. The programmer's job is to only think towards the core business logic of the program and not to worry about the communication with I/O devices. The only way to make this possible is, the programming language must provide a rich application programming interface (API) that can handle inputs and outputs in the same way for any I/O device. This API is called a stream. An input stream is the one that takes data from any of the input devices and transfers it to the program. Whereas, the output stream is the one that transfers data/results generated by the program to any of the output devices. By definition, stream is a series of bytes that are transferred between the program and the I/O device. Figure 15.1 shows the role of input and output streams to transfer data between program and external devices.

Page 9 of 36