Scenario

Recent history of the electronic and computer industry can be viewed as three waves of revolutionary processes. The first revolution, making cheap computing power available via microprocessors in the 1970s, led to the PC industry of the 1980s. The cheap laser and fiber optics, which resulted in cheap bandwidth at the end of the 1980s, led to the Internet industry of the 1990s. The third wave, the sensor revolution at the end of the 1990s, will also provide for a new industry. Sensor revolution means that cheap sensor and MEMS (micro-electro-mechanical system) arrays are proliferating in almost all the conceivable forms. Artificial eyes, nose, ears, taste, and somatosensory devices as well as sensing all physical, chemical, and biological parameters, together with microactuators, etc. are becoming commodities. Thousands and millions of generically analog signals are produced waiting for processing. A new computing paradigm is needed. The cited technology assessment reads:

For processing analog array signals, the revolutionary Analogic Cellular Computer paradigm is a major candidate. The core of this computer is a Cellular Nonlinear/neural network (CNN), an array of analog dynamic processors or cells. The computer architecture is the CNN Universal Machine, with its various physical implementations. At the same time, Analogic CNN computers mimic the anatomy and physiology of many sensory and processing organs with an additional capability of stored programmability.

Appendix A: TEMLIB, a CNN Template Library

Under the name TEMLIB, within the Software Library for analogic cellular (CNN) computers, a set of fairly standard types of CNN template data are contained. The template names in TEMLIB can be used in the template and algorithm simulators defined in Appendix B.

Appendix B: TEMPO, template optimization tools

Under the name TEMMASTER, a student version of a program for template optimization and design is available. It is used mainly for Boolean CNN and for robust template design.

Appendix C: CANDY, a simulator for CNN templates and analogic CNN algorithms

Under the name CANDY (CNN Analogic Dynamics), a student version of a software simulator system is available. Multi-layer CNN templates as well as analogic CNN algorithms (defined on the CNN Universal Machine having a one layer, first-order dynamics CNN core) can be simulated. An easy to use Template Runner program as well as a high-level language compiler (Alpha) help the user to analyze complex spatial-temporal dynamics easily and with expressive visualization tools.

During the first years after the introduction of the CNN paradigm, many templates were designed by cut-and-try techniques, playing with a few nonzero template elements, and using a simulator to calculate the CNN dynamics. After a while, some systematic design methodologies emerged. Today several methods are available for generating CNN templates or algorithms, even for complex tasks.

Various design techniques

The main classes of design techniques are as follows:

• systematic methods for binary I/O function via Boolean description and decomposition techniques using uncoupled CNN (see Chapters 5, 6, 7)

• systematic methods for binary I/O function using coupled CNN (see also Chapter 12)

• global optimization techniques as parameter optimization

• genetic algorithms for designing the template elements/synaptic weights

• matching with the spatially discrete representations of partial differential equations (PDEs)

• matching with some neuromorphic models of a living organism, typically the nervous system, in particular the visual pathway of vertebrates (see Chapter 16)

• fuzzy design techniques

• neural network techniques

• matching with existing 2D or 3D algorithms, including techniques in signal processing, telecommunications, adaptive control, nonlinear spatio-temporal dynamical systems, etc.

We have to emphasize, however, that, in spite of the many design techniques, new methods are emerging day by day based on the intuition and skill of the designers. A good example for this is a recent method using active waves applied for a while and combining/colliding with other waves, as well as a method in which a wave metric is used for complex pattern recognition tasks.

The complete stability phenomenon

Our main objective in this section is to derive and prove a general theorem which unifies all of the CNN templates presented in the preceding chapter, and numerous others, under one umbrella. In particular, the mathematical analyses presented for all templates in Chapter 5 could be greatly simplified. We did not present this fundamental theorem earlier for pedagogical reasons: it is essential for the uninitiated students of CNN to understand and appreciate the fundamental notion and power of the shifted DP plots and their dynamic routes.

We have been thoroughly exposed to these rather simple concepts and have learned to exploit the dynamic routes not only for the analysis of the nonlinear dynamics (i.e., transient and asymptotic behaviors), but also as a highly intuitive and potent design tool for deriving optimum and robust CNN templates.

The alert students who have gone over the previous chapter would have recognized that, except for a degenerate case, no matter what the initial conditions are, the solution always converges to a globally asymptotically stable and hence unique equilibrium point (monostable case), or to one of two locally stable equilibrium points (bistable case). Although this latter “convergence” property is rather unusual for nonlinear dynamic circuits having multiple equilibria, it is a gift that nature (i.e., the physical laws) has endowed upon an important class of CNNs on which all current nonlinear information processing applications are based.

Chapter 2

Exercise 2.1 (Simple morph)

Given: two gray-scale images: P1 and P2

Input: U(t) = P1

Initial state: X(0) = P2

Boundary conditions: white frame

Output: Y(t) = a transition from P2 to P1.

Task

Design a single template, which implements this transition.

Example

Exercise 2.2 (Hexagonal neighborhood)

The standard CNN definition specifies that the cells form a rectangular grid. Anther feasible form could be a hexagonal grid.

Task

Give a formula for the side length and the area of a hexagon (measured in cells) in the case of a hexagonal cell grid, when the sphere of influence equals r.

Exercise 2.3 (Triangular neighborhood)

The standard CNN definition specifies that the cells form a rectangular grid. There are only three possibilities to cover the plane. These are rectangular, hexagonal, and triangular.

Task

Give a formula for the area of a triangle in the case of a triangular cell grid, when the sphere of influence equals r.

Chapter 3

Exercise 3.1 (Separate connected objects)

The problem to be solved is to separate connected objects. The example shows a test image where objects are all similar in size. All objects should be separated but their sizes must be preserved.

Digital technology has the key advantage that if a few building blocks are implemented then any complex system can be built from these by

• wiring and

• programming.

Moreover, most of the digital building blocks are placed in a regular arrangement: a simple block is repeated many times in a matrix arrangement (e.g. memories, PLAs, etc.).

The CNN core and the CNN Universal Machine architecture, containing also analog building blocks, possess the very same properties. Due to their special nature, however, they have orders of magnitude advantages in speed, power, and area (SPA) in some standard physical implementations. In many applications, like image flow computing, this advantage might be mission critical.

As a revolutionary feature, stored programmability can be introduced in the analog domain as well. This makes it possible to fabricate visual microprocessors.

In what follows, first, we show the building blocks and their simple CMOS implementation examples, without going into the details of their design issues. The emulated digital implementation will be only briefly reviewed. As to this and the optical implementation, we refer to the literature.

As a summary: using only six simple circuit building blocks, namely:

• resistor,

• capacitor,

• switch,

• VCCS (Voltage Controlled Current Source),

• logic register, and

• logic gate,

the most complex CNN array computer chip can be built in a VLSI friendly, regular structure.

Next, the visual microprocessor and its computational infrastructure is described.

There is an on-going quest by engineers and specialists: compete with and imitate nature, especially some “smart” animals. Vision is one particular area computer engineers are interested in. Terms like “machine vision” and “computer vision” demonstrate this interest. Recently, modeling the living visual system has become a focus in science and technology. As the anatomy and physiology of the eye and other elements of the visual pathway are becoming more and more known, especially in the retinatopic part (Retina, Lateral Geniculate Nucleus (LGN), and the Visual Cortex), engineers have been trying to imitate these models. These studies have led to a better understanding of vision, overcoming the clear deficiencies of earlier, though useful, principles of computer vision before the mid 1980s.

Based on the ground-breaking studies of Barlow, Dowling, and Werblin on vertebrate retinas, a very simple model of the retina, a resistive grid, was implemented on silicon and demonstrated by simulation studies. In spite of the many “silicon retinas” built on this simple resistive grid model, it became clear that these models are too simple to explain even some practical qualitative effects related to higher-order spatio-temporal interactions in the retina. Attempts to address the more sophisticated retinal models led to descriptive and network type models. In the latter case, not only the retina, but a lot of other parts in the visual pathway had been first modeled by using a single paradigm: cellular neural networks.