3 - Programming Parallel Computers

Summary

In order to make sense of the way in which users control parallel computers, we shall have to adopt a rather wider definition of the idea of programming than that which is usually taken. This is principally because of the existence of systems which are trained rather than programmed. Later in this chapter I shall attempt to show the equivalence between the two approaches but it is probably best to begin by considering how the conventional idea of programming is applied to parallel systems.

Parallel programming

There are three factors which we should take into account in order to arrive at a proper understanding of the differences between one type of parallel language and another and to appreciate where the use of each type is appropriate. These are whether the parallelism is hidden or explicit, which paradigm is employed and the level of the language. Although there is, inevitably, a certain amount of overlap between these factors, I shall treat them as though they were independent.

The embodiment of parallelism

There is really only one fundamental choice to be made here – should the parallelism embodied in a language be implicit or explicit? That is, should the parallelism be hidden from the programmers, or should it be specified by them? The first alternative is usually achieved by allowing the use of data types, in a program, which themselves comprise multiple data entities.