Hybrid optoelectronic adaptive resonance theory neural...

Hybrid optoelectronic adaptive resonance theoryneural processor, ART1

Thomas P. Caudell

For industrial use, adaptive resonance theory (ART) neural networks have the potential of becoming animportant component in a variety of commercial and military systems. Efficient software emulations ofthese networks are adequate in many of today's low-end applications such as information retrieval orgroup technology. But for larger applications, special-purpose hardware is required to achieve theexpected performance requirements. Direct electronic implementation of this network model hasproven difficult to scale to large-input dimensionality owing to the high degree of interconnectivitybetween layers. Here, a new hardware implementation design of ART1 is proposed that handles inputdimensions of practical size. It efficiently combines the advantages of optical and electronic devices toproduce a stand-alone ARTi processor. Parallel computations are relegated to free-space optics, whileserial operations are performed in VLSI electronics. One possible physical realization of this architec-ture is proposed. No hardware has as yet been built.

Key words: Free-space optical processing, neural networks, adaptive resonance theory, ferroelectricliquid-crystal spatial light modulators, VLSI analog processing.

1. Introduction

Adaptive resonance theory (ART) networks, a class ofneural networks developed by Carpenter and Gross-berg' in 1987, are used to learn stably the classifica-tion codes for streams of input patterns. Clusteringis the basic function performed by an ART network.2These networks generate clusters in a self-organizingmanner, detect and treat novel patterns, and learngeneralizations of the patterns in a relatively smallnumber of presentation cycles.3 4 In Ref. 4 the theo-retical upper limits for the number of presentationepochs necessary to stabilize the learning of ART areall large compared with those observed in practice.For example, an operational system with over 1000distinct complex input patterns always converges infewer than ten epoches. We focus on the specificform of ART that handles binary-input patternsknown as ART1.

Currently ARTi networks are beginning to be usedin a growing number of industrial applications. AtThe Boeing Company, Caudell et al.5 used ARTinetworks in an information retrieval system for the

The author is with Research and Technology, Boeing ComputerServices, The Boeing Company, P.O. Box 24346, Seattle, Washing-ton 98124-0346.

Received 20 August 1991.0003-6935/92/296220-10$05.00/0.

© 1992 Optical Socioty of America,

search and recall of similar engineering designs,which is called the group-technology problem. Ba-loch and Waxman6 used an ARTI network in amodular way to control the behavior of mobile robots.

In all these applications, ART networks are imple-mented in software simulations on conventional com-puter systems, either in serial or in parallel. Interms of execution time, the performance of thesesimulations is acceptable either when small inputpatterns are used or when the applications have norestrictive turnaround-time requirements. The tem-poral performance is usually not an issue whenclassification or generalization performance is thefocus of research. However, when the simulationsgrow to meet the requirements of problems found inthe real world, the brick wall of simulator perfor-mance is quickly encountered. Thus we need toconsider physical implementations of these networkmodels.

Such an idea is not new. In the recent past, pureelectronic implementations of ARTi components andsystems have been proposed. Kaburlasos et al.7 haveconstructed a microcontroller-based ARTi coproces-sor for IBM PC's. Two parallel microcontrollerssimulate the solutions to the nonlinear differentialequations for two of the layers in the network.Software was necessary to complete the functioningARTi network. The system is limited to 64 inputand 40 output neurodes. Tsay and Newcomb8 pre-

6220 APPLIED OPTICS / Vol. 31, No. 29 / 10 October 1992

sented a VLSI implementation of the two majorsubsystems in the ARTi network: the short-termand the long-term memories. The nonlinear differ-ential equations are implemented by using comple-mentary metal-oxide semiconductor (CMOS) devicesand circuits. A small system with four inputs andfour outputs was layed out and simulated by usingMagic, a VLSI simulation and analysis software tool.The high interconnection density needed for theARTi model and the diversity of synapses within thearchitecture limit the size of the input vector inpurely electronic implementations that use parallism.

Optics and electronics have also been previouslycombined to address this interconnection problem.Many of the limitations found in pure electronicimplementations of neural networks were thus re-moved. Psaltis et al.9 implemented an optoelec-tronic version of the Hopfield network by using anoptical matrix-vector multiplier. Caulfield andArmitage 0 implemented a pseudo-optical version ofthe ARTi network that used the optical associativememory model; pseudo in that it does not learn in realtime. Recently Wunsch et al." proposed and testedthe first hybrid optoelectronic ARTi system that useda VanderLugt, or 4F, optical correlator.12 In Ref. 11an algorithmic formulation of the ARTi network wasimplemented. The optics performed the bulk of thecomputations needed for the algorithm by using theintensity of the zero-shift correlation peak to repre-sent vector dot products. In this system, all othercalculations needed for the network algorithm areperformed in a separate digital computer.

The following study takes the concept of an algorith-mic implementation several steps further. After thedetails of the ARTi neural network algorithm arereviewed in Section 2, the design of an optoelectronicarchitecture that implements the algorithm com-pletely, without requiring a separate digital com-puter, is presented in Section 3. The design of onepossible physical realization of this optoelectronicarchitecture is then presented in Section 4, and amethod for using this device as a part of a largermacrocircuit of ARTi network modules is discussedin Section 5.

2. Algorithmic Form of ART1

The ARTi neural network model is canonically repre-sented by a coupled set of ordinary nonlinear differen-tial equations.' If appropriate restrictions are madeon the relationship between the dynamical timeconstant, the learning rate, and the length of time theinput pattern is stable on the network input neu-rodes, then this system of equations reduces to aprocedural algorithm.2 The dynamical time con-stants are required to be much smaller than thelearning rate, which in turn are much smaller thanthe stable presentation time. These restrictions leadto a fast learning mode of operation in which thelearned weights in the system reach asymptotic stabil-ity before a new input pattern is presented. Theimpact of these restrictions on the implementation of

ART1 is enormous: the computational steps of thealgorithm can now be mapped directly onto an algo-rithmic processor. Under these special conditionsfor this model, we need not become embroiled in theanalog implementation of a complex dynamical sys-tem. The mapping of the ARTI algorithm onto aspecific processor depends on many details, not theleast of which is the individual computational stepsrequired in the algorithm. To clarify these state-ments, the remainder of this section describes theARTi algorithm. Section 3 discusses the partition-ing of this algorithm between the optical and theelectronic components.

Figure 1 is a flow chart for the ARTi algorithm andincludes the main pattern presentation loop. Table1 lists the functional operations. The network istrained through repetitive presentation of a set oftraining patterns. This algorithm autonomously

Fig. 1. Flowchart that embodies the processing of an ARTineural network: i is the index of the current input vector I beingpresented to the network; arg max is a function that returns theindex of the maximum element in its list argument, effectivelyimplementing the competition within the F2 layer; Tk is a typicaltemplate in the set of all n, templates; Ak is a list of binary flags thatdetermine which templates participate in the competition; P is asmall constant that biases the search process toward smallertemplates; k* is the current index of the winning F2 neurode; N isthe total number of F1 neurodes and is the dimensionality of theinput vector, p is the vigilance factor, ranging between 0 and 1; na isthe number of F2 neurodes still active in the competition; p is adistinguishing name or label for I; Ck* is the membership roster forcluster k* containing the names of all of the input vectors that wereattracted to Tk*; 1 is the unit vector.

10 October 1992 / Vol. 31, No. 29 / APPLIED OPTICS 6221

- -

Table 1. List of Principal Computational Operations Required toPerforming the ART1 Algorithma

No. Operation Electronics Optics %

lb I2b Tk V3c Tk-I v 914c ITkI V 85c Tk-=Tk*nI v <16c III <0.17c Tn = I v <0.18 arg max } ) <0.019 {Ai = 1; ,ni } V <0.01

10 Ck <-p V <0.0111 > V <0.0112 X V <0.0113 / V <0.0114 + V <0.01

aThese operations can be partitioned naturally between elec-tronic and optical implementations, as indicated by the checkmarks. The last column gives an approximate percentage ofexecution time in the ARTi algorithm.

bRepresentation of information as a light intensity or transmis-sivity.

cAssuming that Nbit = 1024 and n, = 100.

places binary input patterns (vectors) into clusters.Each cluster is represented in neural memory as atemplate prototype. A template is an abstraction ofthe patterns in a cluster, and it is formed andmodified during the learning (training) process. Thenumber of clusters discovered by the network isdetermined by the underlying structure or semanticsof the training set of input patterns, by the order ofpresentation of input patterns, and by a system-levelthreshold called the vigilance factor. During testingof the trained network, neural memories or templatesare directly recalled when examples from knownclusters are presented. On the other hand, thelearning process remains forever plastic in this model,even after training, which permits new or novel inputpatterns to be learned without disturbing old memo-ries.

The algorithm proceeds as shown in Fig. 1. Abinary input pattern I is presented to the system.If this is the first presentation of the first pattern,then this pattern becomes the first template, T1.That is, I gets copied into T, and the number ofclusters n is set to 1; Tk* - I, where k* = 1 (Table 1,operation 7). After this step, the system is ready forthe next input pattern.

Otherwise, a search must determine if I is closeenough to any one of the existing templates. Abest-first search is performed in the order of descend-ing value of a normalized dot product (I Tk)/(I + I01), where k* is the index of the templateunder current consideration in the search (Table 1,operations 3, 4, 13, and 14). (The factor is a smallconstant to help bias the choice toward smallertemplates; it is numerically equal to the reciprocal ofN, the number of binary components of the input

vector.) The index k* is obtained by

[Ak(I . Tk)arg maxI

kE=1,nc, [ 3 + ITk

where the set of flags Ak is used to select thetemplates that are permitted to compete for a matchwith the input pattern; Ak 1 for active templates,while Ak = 0 for inactive templates (Table 1, opera-tions 8 and 12). The arg max function returns theindex of the maximum element in the argument list.Within the presentation time of a pattern, all tem-plates start off active, but become labeled inactive if,after being selected in the best-first search, they failthe close-enough test to I. This test requires that Ifall within a region of feature space around thetemplate Tk* to be considered a member of cluster k*.This region is define by the inequality (I Tk*)/ III p, where p E [0, 1] is the vigilance factor for thenetwork (Table 1, operations 6, 11, and 13). A pvalue of approximately 1 gives a network with highdiscrimination, which forms many clusters. A valueof approximately 0 gives a network with low discrimi-nation, which forms few clusters. If the vigilancetest is not satisfied for the cluster k*, then a resetoccurs; Ak* = 0, and the search continues.

If the vigilance test is satisfied, then the network isin a state of resonance and the winning template Tk*is updated through a simple learning process. Thenew template gets the logical bit-wise AND of the oldtemplate and the input pattern; Tk* <- Tk* n I (Table1, operation 5). This learning model in ARTi is aform of Hebbian learning,13 which is found in many oftoday's self-organizing neural models. In additionto this operation, the input pattern name is added tothe membership set for the k* cluster, and the activecluster flags {AkJ are restored to unity (Table 1,operations 9 and 10). That is, Ck* <-p and (Ak = 1,1 < k < n, , wherep is the name of the current inputpattern and Ck* is the set of names of input patterns,or the roster, of the members of the k*th cluster.The name can be a number or an ASCII sequence.Constructing the rosters for clusters provides informa-tion useful in many practical applications. After thisstep, the system is ready for the next input pattern.

If none of the existing clusters pass the vigilancetest for input pattern I, then a new cluster is formed.That is, the novel pattern I gets copied into Tknew, andthe number of clusters n is incremented by one;Tknew <- I, where knew = nc + 1 (Table 1, operation 7).The novel input pattern name is added to a new rosterfor the knewth cluster, and the active cluster flags Akare restored to unity (Table 1, operations 9 and 10).That is, Cknew - p and Ak = 1, 1 < k < n}. Afterthis step, the system is again ready for the next inputpattern.

As is evident in the above discussion, the ART1algorithm has a mixture of both serial and paralleloperations. The vigilance test and the search pro-cess remain serial-operations independent of anyparticular hardware implementation scheme. The


best that hardware can do for ARTi is to drive the netexecution time in the parallel portions of the algo-rithm to as near zero as possible. Section 3 discussesthe partitioning of these operations between elec-tronic and optical hardware to achieve high perfor-mance.

3. Optoelectronic Architecture

The second column of Table 1 summarizes the math-ematical operations found in the ARTi algorithm.As we can see, several of these operations are matrix-vector multiplications. In fact, the major computa-tional load arises from the computation of the vectordot products and norms. Fortunately, optical com-puting systems can perform these types of operationsefficiently, as is indicated in the fourth column of thetable. It is shown in this section that all vector andmatrix-vector operations can be performed in paral-lel by using a modification of a well-known optoelec-tronic processor. To date, no hardware prototypesof this system have been constructed to our knowl-edge, although research is progressing in that direc-tion.

A. Optical Matrix-Vector Multiplier

In 1985 Farhat and Psaltis' 4 proposed and demon-strated the use of the optical matrix-vector multipli-cation processor (OMVMP) in a neural network sys-tem. Figure 2 shows the basic components of thisconceptually simple system. The processor per-forms a multiplication between a vector whose compo-nents are represented by the intensity of the emitterarray and a matrix whose components are repre-sented by the transmission coefficients of the pixels inthe two-dimensional (2-D) spatial light modulator(SLM). Lens 1 is symbolic of a cylindrical lenssystem that distributes each of the vector's compo-nents along the columns of the matrix. The multipli-cation is performed componentwise as the light fromthe linear array is differently attenuated through

each of the SLM pixel cells. The second cylindricallens system (Lens 2) collects the exiting light alongthe row direction and integrates the sums on an arrayof detectors oriented perpendicularly to the emitterarray. The output of the detectors is the resultingmatrix-vector product. In this system the compo-nents of the vector and the matrix can be eitherbinary or analog; the resulting accuracy in both casesdepends on the specific devices used.

As is also shown in Fig. 2, the OMVMP is typicallycontrolled by a digital computer, which contains in itsmemory images of the vector and the matrix to bemultiplied, as well as the controlling program. Thevector and the matrix are loaded into the opticalprocessor through electronic interface devices, andthe resulting product is read back into memory from acontroller for the detector array. In a sense thisconfiguration uses the optical processor as a coproces-sor to a digital computer.

To see how this system can be used as a coprocessorin the execution of the ARTi algorithm, refer to Fig.3. In the figure it is assumed that the input patternI is a 2-D binary image, and one possible way that itcan be mapped onto the one-dimensional (1-D) emit-ter array is shown. We also see how the first mem-ory template T1 is mapped onto the 2-D SLM device.With this assignment, the OMVMP will compute thedot products [(I T), 1 < k < ni. These dot prod-ucts are read from the detector into memory, and theremainder of the algorithm is executed within thedigital computer. This is a mode of operation simi-lar to that used by Wunsch et al.1 for the 4F opticalcorrelator implementation of an ART 1 network.

1- D Emitter Array

Ii

0=1L=°

First Template T,

1 T I I

_ Ti I...I

1-'rogr

f�h�tI

�TTTTT

�-�-J

Fig. 2. One possible way to use an optical matrix-vector multi-plier as a coprocessor to compute several of the parallel operationsrequired for the ARTi algorithm.

To 1 1 1 1 1 1 1 1 1 ] 1T1 T1 1 T 2 T 1 5* ______ ..._________________ TIN____

T2'J T21 T22*o 1_

T3 T3. _ ___ __; 2-D Spatial Light Modulator

(SLM)

Fig. 3. How binary images and templates can be loaded into theoptical processor arrays. Note that the To all-transmission row isused to compute the input vector norms. T-T 3 , memory tem-plates; Ij, input pattern.


- - |

A simple change in the use of the basic OMVMPpermits this processor to compute the vector normsIII and Tk 1, 1 < k < nJ required in the ARTIalgorithm. Figure 3 shows how I I can be computedby providing an extra row on top of the 2-D SLM thatis fully transmissive. This is equivalent to loadingthe unit vector 1 into the extra row on the SLM.The first detector in the focal plane of Lens 2 thenintegrates (I 1), which is identical to III. To com-pute the norms of the templates, we must take asecond sequential step. The input emitter arraymust be set to the unit vector. With this vector onthe input, the detectors then integrate (1 Tk), whichis again identically equal to the norms I Tk,1 k n.

Thus in two steps, most of the parallel operationslisted under Optics in Table 1 can be computed by thiscoprocessor. The remaining operations are (1) newtemplate copy and (2) existing template update.Since in the OMVMP the input vector and thetemplates must also be stored in the computer'smemory, both of these operations could be performedin software by a serial computer. Amdahl's law,15

which states that the maximum speedup possiblethrough parallelism in an algorithm is set by theremaining serial operations, advises us to seek paral-lel implementations for these remaining ARTI opera-tions. Subsection 3.B describes further modifica-tions to the OMVMP that make these calculationspossible within an optoelectronic processor.

B. Augmented Optical Matrix-Vector Processor

Several major modifications to the basic componentsof the OMVMP are shown in Fig. 4. With theaddition of VLSI analog processing, these compo-nents become relabeled as (1) a smart-input array (SIarray), (2) a smart SLM array (SSLM), and (3) asmart-output array (SO array). These new devicespermit the copy and update operations to be per-formed optoelectronically and thus eliminate the

IndexOutput

Fig. 4. Schematic architecture that uses optics to compute all theoperations checked for optics in Table 1. The remainder of thecomputation is performed in analog electronics. Note the use of asmart SLM that is controlled by light signals in the forward andreverse directions. The mode control signals pathway is the onlynonoptical communications between chips and could be imple-mented as a small cable. The output of the system is the index ofthe resonating cluster.

need for a digital computer. Let us consider eachmodification separately.

The SO array, shown in Fig. 5, replaces most of thefunctionality of the digital computer by a single VLSICMOS chip. This chip incorporates a linear array ofdetectors, a linear array of emitters, an array ofanalog storage registers, analog scalar arithmeticoperators, and a winner-take-all network. The lat-ter implements the arg max function required in thealgorithm. The registers store the input norm IIIcomputed through the first transparent row in theSLM and the template norms I Tk that are computedby loading the unit vector on the SI array. Thesevalues are used at various stages in the ARTi algo-rithm and are combined with the dot products byusing analog scalar arithmetic operators to feed thewinner-take-all network, which is also implementedin analog circuits. Specialized control circuitry regu-lates the reset and search processes and controls thelogic of the comparison operations. In addition, asingle mode control signal, discussed below, is sentback to the SI array along a small cable.

Control of the SSLM is performed by the SO arrayindirectly through modulation of its emitters Bk,which shine light back through Lens 2 to illuminate arow on the modulator. Figure 6 shows a diagram ofa cell on the SSLM. The reverse illumination helpsperform the conjunction of the input pattern and thewinning template, and it helps the copying of an inputpattern into a new template. These operations oc-cur in the cells of the SSLM when the reverseillumination highlights a template row and the SIarray is in the appropriate mode of operation.

A blowup diagram of the SI array is given in Fig. 7,in which we see the four modes of operation con-

I TI I .T2 IeT 3 I 1T41 V1 V B2 TB3 B4

detectos Q 1 ; |eitr

vigil..new

Factor P (03 N)

Analog Winner-Take-All Circuitry_

Control Circuitr

Mode Signal Externalto Smart Input Reset Input

Ary _ __

Cluster Indices Output

ldetector 0 terminalM em.iter analoreiter

-sitch _ cotrl line

Fig. 5. Block diagram of the smart-output (SO) array chip designthat would replace the digital computer and control the opticalprocessor. Since this device simulates a winner-take-all neuralnetwork, only one output is on at a time. If the output is coded inbinary, for example, then the number of pins on the chip packagewill not limit the maximum number of clusters. B-B 4, emitteroutputs; 0, threshold.


2-Blink tow

Tl

Fig. 6. Close-up of a reflective smart pixel on the SSLM imple-mented with ferroelectric liquid crystals (FLC's). Note that theflip-flop memory element has a set ON power-up circuit and that athreshold 0 is globally bussed to each pixel. The summing circuitcombines and thresholds the optical signals from the SI array andthe SO array, along with an electrical signal representing the stateof the flip flop. If the threshold is exceeded, a single clock pulsetoggles the state.

trolled by the signal from the SO array. During thecomputation of the input-pattern-template dot prod-ucts and the norm of the input pattern, the SI array isoperated in mode 0. In this mode the emitter arrayilluminates the columns of the SSLM with the binary-input pattern previously clocked into the array's localbuffers by means of a rapid shift register action.When operated in mode 3, all emitters are turned onto compute en masse the norms of the templates.Note that the SO array is not reverse illuminating theSSLM during these two modes of operation.

During the copy and update operations, the SOarray will have selected a template row on the SSLMfor modification, owing to either a match with anexisting template or a novel pattern detection. Thisselection manifests itself through the reverse illumi-nation of a template row on the SSLM. Referringagain to Fig. 6, we see that the state (i.e., pass/on orblock/off ) of each pixel on the SSLM is determined bythe state of a local flip-flop memory element. Atpower-up time, each state is set to logical ON by a

ModeControl Line

0-0-

Serial DataInput

1 2---

Fig. 7. Schematic details of the SI array. Note that the binaryinput pattern is clocked through a shift register to reach eachemitter pixel. The four modes of operation are given in the insetand are controlled by the mode control lines.

reset circuit. The state of the pixel is toggled when athreshold amount of light is received on the pixel'sphotodetector. The threshold is adjusted so that nostate change occurs when only the forward or thereverse illumination is present alone. Both must besimultaneously active before a toggle of a SSLM cellstate can occur.

To copy an input pattern into a row on the SSLM,the SO array illuminates the correct row, while the SIarray in mode 1 illuminates the columns to be toggledwith the negation of the input pattern. Since theinitial state of each pixel in an unused row is ON, thetemplate pixels in the selected row and selectedcolumns are toggled to the OFF state if the templatepixel was initially ON and the input pixel OFF. Atruth table of the state transitions is shown in Table2. To perform the conjunctive update operation,the SO array illuminates the correct row, while the SIarray, again in mode 1, illuminates the columns to betoggled with the negation of the input pattern. Thisoperation is illustrated in Table 3, which shows theperformance of an in-place conjunction of the tem-plate and the input pattern. The tables also showthat the binary information stored on the SSLM doesnot change when the reverse illumination is removed,as indicated by the rows with Bk = 0-

The designs of these three devices, combined withthe two cylindrical lens systems, constitute a proces-sor that could efficiently implement the entire ARTialgorithm. Table 4 reaffirms this by presenting thesteps in the algorithm that would be used in theoptoelectronic processor. Note that the copy andupdate operations differ only in the cluster index usedto backilluminate the SSLM. Section 4 presents theconcept of a physical implementation of this systemby using ferroelectric liquid-crystal spatial light mod-ulators (FLC-SLM's).

4. Design for Optoelectronic Implementation

FLC's act to rotate the plane of polarization oftransiting light. The degree of rotation depends onthe strength of the longitudinal electric field and thelength of the path through the material. The correctcombination of thickness and field strength can pro-duce a switchable mirror or a SLM. Devices now

Table 2. Truth Table Showing the Copying Operation Required toGenerate a New Cluster Templatea

Tk ` I

Bk I,, jb Tkinitial c Tkfinal

1 1 0 1 2 11 0 1 ld 3d od

o 1 o 1 1 1

0 0 1 1 2 1

aBk is the output of the kth emitter on the SO array, and is ONwhen the kth template is to be created.

bSI array mode = 1, inverted input pattern.cThreshold for toggle of SSLM pixel flip-flip _ 2.5 and ,, = Bk +

In + Tinitial.dValues for the only condition in which a change of state of a

template pixel occurs.


Table 3. Truth Table Showing the Operation Required to Update aCluster Template,

Tk - Tk n I

Bk In, Ib Tkfinitial Ce Tkffinal

1 1 0 1 2 11 0 1 ld 3d od

1 1 0 0 1 01 0 1 0 2 00 1 0 1 1 10 0 1 1 2 10 1 0 0 0 00 0 1 0 1 0

aBk is the output of the kth emitter on the SO array, and is ONwhen the kth template is to be updated.

bSI array mode = 1, inverted input pattern.cThreshold for toggle of SSLM pixel flip-flip _ 2.5 and Y, = Bk +I oinitialdValues for the only condition in which a change of state of a

template pixel occurs.

exist that integrate FLC-SLM's on the surface ofCMOS VLSI integrated ircuits. 16-9 When com-bined with silicon photodetectors, this class of devicehas the potential to implement the active componentsin the aforementioned architecture.

Figure 8 demonstrates how three FLC-SLM hybriddevices could perform the functions of the SI, SO, and

SSLM arrays. The light source floods the FLC-SLM's on the SI and SO arrays with polarized light.This way the devices may act as emitters by reflectingthe light toward the SSLM while modulating thepolarization. Polarizing beam splitters pass rotatedpolarization beams and deflect the unrotated beamsback toward the light source. A third polarizingbeam splitter in front of the SSLM combines thepolarized beams from the two emitting arrays, whichpermits threshold logic to be performed during thetemplate copy and update operations.

This optical system does not require a coherentlight source since no interference phenomenon isused in the processing. A narrow wavelength bandmay be required to achieve reasonable contrast ratioswith the FLC-SLM's. Consequently, high-powerlight-emitting diode arrays may supply the inputillumination.

This processor can potentially occupy a small vol-ume. If the lenses are replaced with solid gradient-index elements, the implementation concept shownin Fig. 8 could occupy a volume of less than 10 cm3.Alignment would be frozen by cementing the ele-ments together into a solid structure. No computerswould be required for operation, although externalcircuitry would be necessary to clock in the inputarray and to make use of the outputs.

Table 4. Mapping of the ARTi Algorithm Into Processing Steps of the Optical Processor

Step SO Array SI Array SSLM

SEARCH AND MATCH operation1. Output mode = 0 to SI array Shift in input, mode a 0 Subthreshold2. Integrate detectors Emitters ON3. Store input norm4. Normalize dot products with template norms5. If no active clusters, then GOTO COPY Mode changing Superthreshold, pixel toggling6. Winner-take-all finds max index7. If FAIL first inequality, then GOTO COPY Mode changing Superthreshold, pixel toggling8. If FAIL second inequality, proceed as follows:

(1) Decrement number of active clusters(2) If none left, then GOTO COPY Mode changing Superthreshold, pixel toggling(3) Otherwise, inhibit winning cluster and

GOTO step 59. Resonance, output cluster

10. GOTO UPDATE Mode changing Superthreshold, pixel toggling11. GOTO step 1

COPY operationa. Output mode = 1 to SI array Invert input, mode = 1 Subthresholdb. Backemitter ON for next available cluster index Emitters ON Pixels togglec. Backemitter OFFd. Output mode = 2 to SI array All ones, mode = 2 Subthresholde. Integrate detectors Emitters ONf. Store template normsg. GOTO step 1

UPDATE operationa. Output mode = 1 to SI array Invert input, mode = 1 Subthresholdb. Backemitter ON for winning cluster index Emitters ON Pixels togglec. Backemitter OFFd. Output mode = 2 to SI array All l's, mode = 2 Subthresholde. Integrate detectors Emitters ONf. Store template normsg. GOTO step 1


-Top View - I

I Output

Array

-Front View -

Lightource

Fig. 8. One possible physical implementation of annetwork by using FLC-SLM CMOS devices. P.Bbeam splitter.

The approximate temporal performarimplementation concept may be calculatTable 1 we conclude that the dominantoperations is generated by the parallel dotrow 3 and by the template norms in row 4.assumption that one 8-bit MULTIPLY oequivalent to approximately ten ADD opelcan make an order-of-magnitude estimnumber of operations necessary to procespattern: Nope = 10 MtemplatesNbits, wherethe maximum number of possible templatis the length of the input vector. For Mtemplates = 100 and Nbit5 = 1024, then theoperations is approximately 1 x 106. Tthat a standard Sun SPARCStation 2 coi- 25 input vectors/s.

The approximate cycle time for the pr(also be calculated. Upon analysis of t1Table 4, we see that the cycle time is dominintegration times of the detectors in theIn fact, the cycle time can be estimated to2Tintegration. This is derived from the facinput pattern requires two parallel detecttions. The integration time Tintegration Ipends on the specific properties of the dettime constants of the analog circuits, thesignal-to-noise ratio required by the algothe specific application in which the netwcSince the total signal level is controlled bysity of the illumination source, a great dealexists in estimating the minimum permitta'tion times. A value in the range 10-5-1(unreasonable for currently available detlight sources. Using this range of timeexample sizes Of Mtemplates = 100 and Nbit5calculate the approximate number of opesecond for this optical processor to be Rop,1011. Although this estimate is based on a

simplifying assumptions, it implies that this proces-sor could potentially cluster upwards of 101/106 =105 input images/s of size 1024 binary pixels.

Operational errors in classification occur in thissystem when either fixed-pattern noise or nonuni-form illumination is too large. To begin to study the

_-_ __ effects of these on system performance, consider aMonte Carlo simulation that was conducted on adigital computer. Figure 9 shows the frequency oferror caused by uniform additive random fixed-

: 7 pattern noise on the SI array, the SSLM, and the SOarray. An error is defined as the case in which aninput pattern is placed in a wrong cluster. Eachcurve in the figure represents the average behavior of

_ a large number of simulations with random inputvectors of a fixed size, in which the range of randomvariation is systematically increased. For each of

SView - these simulations, two templates are matched against- Side~ie- the input pattern. The templates and the inputARTi neural pattern all have the same norms. In truth, the firsts., polarizing template exactly matches the input, while the second

template differs from the first by a preset number ofbits. The difference between the templates is variedto construct the family of smooth curves shown in the

bce of this figure (each curve is labeled with the Hammingted. From distance between templates). These simulation re-number of suits show that ARTi requires approximately ±1%products in uniformity for no errors to occur in the hardest

With the discrimination case (a template Hamming distance ofperation is 1 out of 1024 bits). The number of errors growsrations, we slowly as the size of the nonuniformity increases.ate of the The system behaves more robustly for clusters that3one input are better separated, as indicated by larger HammingMtemplates is distances. It is difficult to generalize these results toes and Nbit5 cases with larger number of clusters owing to theexample, if nature of the ARTI similarity measures.number ofhis impliesild process

ocessor canLe steps inated by theSO array.

be T,le L that eachor integra-trongly de-tectors, thee minimum

rithm, and)rk is used.'the inten-1 of latitudeble integra--4 s is not

ectors ands with the= 1024, werations perS = 1010 to* number of

0C.

VC

1.

.9.

.8.

.7.

No. of Trials I Experiment = 5000_____________________________

N = 1024I I=1 I =IT 2 1=922

0 .2

Fractional H1alf-Range

-I.4.3

Fig. 9. Results of sets of Monte Carlo simulations, each comput-ing the frequency of misclustering as a function of the fractionalhalf-range of additive-fixed-pattern noise on the SI array and theSSLM. The noise was picked from a uniform random distributionover the indicated range. A random input pattern of fixed size issubmitted to an ARTi module that is preset with two clustertemplates, one being identical to the input. The templates arealso of fixed size equal to that of the input pattern. The curves arelabeled with the Hamming distance between the two competingcluster templates.


It

Figure 10 shows the results of a second set ofMonte Carlo simulations that explore the sensitivityof the template update operation to uniform additiverandom fixed-pattern noise. A large number of sim-ulations were performed with an increasing range ofvariation in the noise, which produced the smoothcurve. The vertical axis gives the average number ofbit errors at each variance. A bit error is countedwhen an incorrect toggle occurs or when a correcttoggle does not occur. No errors appear until thehalf-width of the variation exceeds 20%. If thesystem is designed with nonuniformities compatiblewith low classification error, then update errors willnot occur.

5. Discussion

A design for an optical implementation of the ARTineural network has been presented. This design hasthe advantages of a totally optoelectronic implementa-tion: optical interconnects (fan-in and fan-out), VLSIanalog processing, low power consumption, and com-pact monolithic packaging. No software would berequired for the control of the algorithm. The inputpattern is clocked into the input buffer, and after adelay of approximately 2 Tintegration, the classificationcode appears on the output of the SO array. Thedesign has the potential to classify 105 small images/sinto many tens of classes, while remaining resilient tonew or novel input patterns. The technology tobuild the components of this design are within thereach of today's technology.

The major disadvantage of this design is in themapping of a 2-D image onto the 1-D input bufferarray. In many applications, input images will belarger than the 32 x 32 pixel image array used in theexample of Section 4. Typical images of 256 x 256pixels are not unusual in the applications of ARTimentioned in the introduction. This size imagewould require a SLM with 64,000 pixels along oneaxis. With pixel sizes of 10 pum x 10 plm, a slab ofsilicon 65 cm long would be required, clearly notwithin the reach of wafer-scale circuits.

35(

._a)

30C

255

20C

155

105

51

lImi=T 1=1024No. of trials Experimcnt= 10000

)ooI /

Updat-b'3

50I Em Ir /

. .2 .3 .4 5 . .; .8n .9

Fractional Half-Range

Fig. 10. Results of a set of Monte Carlo simulations that com-putes the frequency of toggle errors during a template updateoperation as a function of the fractional half-range of additivefixed-pattern noise on the SSLM and the SO array. The noise waspicked from a uniform random distribution over the indicatedrange. The vertical axis is the average number of bit errors.

- - - - - - - -__ - - - - -Second Layer ofART1 Modules

Fig.11. Illustration of a simple macrocircuit that combines ART1modules into a hierarchy.

As demonstrated by the research of Healy andCaudell,20 the smaller size of the input image is notnecessarily a limitation. Their research shows thatmacrocircuits formed from combinations of ARTimodules can be designed to compute complex logicalfunctions of the input pixel images. The simplestexample of a macrocircuit is a hierarchy of ARTimodules. An example of this hierarchical structureis shown in Fig. 11 for a 128 x 128 pixel image. Theimage is partitioned into small nonoverlappingpatches, each patch being fed to individual 32 x 32ARTI modules. The outputs of this layer of sepa-rate ARTi modules are then funneled into a singleARTi module that combines the lower level results.The vigilance parameters are adjusted so that thisarchitecture acts as a voting machine, with the higherARTi module producing an output code for the most

Second Layer First Layer(1 Module) (one of 16 Modules)

Output

[3 God~~~I..T''I ' ' 9 C7 1Z1!.

Write-with-lightSI Array

CombinerOptics

Fig. 12. Optical processors cascaded together to form a macrocir-cuit of ARTi neural networks. Note that the SI array has beenmodified to include detectors that permit optical writing of theinput vector. This extra circuitry also provides a mechanism forreading the learned templates stored on the SSLM.


Input

G`�

popular interpretation of the input object. Morecomplicated functionality has been designed intonetwork macrocircuits.21

Figure 12 shows how this architecture can beimplemented with the optoelectronic processor.The temporal performance of such a hierarchicalimplementation is reduced by a factor proportional tothe number of layers. A modification of the SI arraydesign is necessary to optically cascade these modules.In addition to the electronic serial loading of the inputvector, an optical parallel load, or write-with-light,mode is required. At each pixel in this linear array, aphotodetector is added that can be clocked to read thepattern of light on the array and to load this informa-tion into the input buffers. Additional optics arenecessary to map the square subimage onto a linearvector and to arrange the outputs of the first layer forinput to the second layer. This modular approachovercomes the main size limitation of these devices.Macrocircuits for practical size applications can beconstructed from these building blocks.

Conclusion

Adaptive resonance theory neural networks are be-coming an important component of neural networkindustrial applications. Electronic implementationof these network models has proven difficult to scaleto large-input dimensionality. Here a new implemen-tation of ART1 has been proposed that efficientlycombines optical and electronic devices. All parallelcomputations were performed by the optics, whileserial operations were performed in electronics.One possible physical implementation of this architec-ture has been proposed. Macrocircuits can be con-structed from these modules to perform complexlogical functions. Based on the write-with-light mod-ification, this system can also be modified to permitreadout and readin of the learned templates on theSSLM, which permits mass production of trainedsystems.

The author acknowledges the helpful suggestionsof Karel Zikan, John Bell, Donald Wunsch II, DavidCapps, Kris Johnson, Kelvin Wagner, Janusz Kowa-lik, and the journal reviewers.

References1. G. A. Carpenter and S. Grossberg, "A massively parallel

architecture for a self-organizing neural pattern recognitionmachine," in Computer Vision, Graphics, and Image Process-ing, L. Shapiro, N. Badles, H. Freeman, T. Huang, and A.Rosenfield, eds. (Academic, New York, 1987), Vol. 37, pp.54-115.

2. B. Moore, "ARTI and pattern clustering," in Proceedings ofthe 1988 Connectionist Summer School, D. Touretzky and G.Hinton, eds. (Carnegie U. Press, Pittsburgh, Pa., 1989), pp.174-185.

3. G. A. Carpenter, S. Grossberg, and D. B. Rosen, "Fuzzy ART:fast stable learning and categorization of analog patterns by anadaptive resonance system," Neural Networks 4, 759-772(1991).

4. M. Georgiopoulos, G. L. Heileman, and J. Huang, "Properties

of learning related to pattern diversity in ART1," NeuralNetworks 4, 751-758 (1991).

5. T. P. Caudell, S. Smith, R. Escobedo, and C. Johnson, "Aneural database system for reusable design," in Proceedings ofthe International Joint Conference on Neural Networks, C. L.Teo, ed. (Institute of Electrical and Electronics Engineers,New York, 1991), Vol. 1, pp. 254-261.

6. A. A. Baloch and A. M. Waxman, "A neural system forbehavioral conditioning of mobile robots," in Proceedings ofthe International Joint Conference on Neural Networks, W.Freeman and B. Kosko, eds. (Institute of Electrical andElectronics Engineers, New York, 1990), Vol. II, pp. 723-728.

7. V. G. Kaburlasos, D. D. Egbert, and M. Rao, "Hardwareimplementation of the adaptive resonance theory neural net-work," in Proceedings of the First Golden West InternationalConference on Intelligent Systems, C. G. Looney, ed. (Univer-sity of Nevada, Reno, Nev., 1991), Vol. 1, pp. 21-28.

8. S. W. Tsay and R. W. Newcomb, "VLSI implementation ofARTi memories," IEEE Trans. Neural Networks 2, 214-221(1991).

9. D. Psaltis, N. H. Farhat, A. Prata, and E. Peak, "Opticalimplementation of the Hopfield model," Appl. Opt. 24, 1469-1475 (1985).

10. H. J. Caulfield and D. Armitage, "Adaptive resonance theory ofoptical pattern recognition," Appl. Opt. 28, 4060-4061 (1989).

11. D. Wunsch, T. P. Caudell, D. Capps, and A. Falk, "Optoelec-tronic learning machine," in OSA Annual Meeting, Vol. 15 of1990 OSA Technical Digest Series (Optical Society of America,Washington, D.C., 1990), paper MJJ5.

12. A. B. VanderLugt, "Signal detection by complex spatial filter-ing," IEEE Trans. Inf. Theory IT-10, 139-145 (1964).

13. D. 0. Hebb, The Organization of Behavior (Wiley, New York,1949), Introd. and Chap. 4.

14. N. H. Farhat and D. Psaltis, "Optical implementation ofassociative memory based on models of neural networks," inOptical Signal Processing, J. Horner, ed. (Academic, NewYork, 1987), pp. 129-161.

15. G. Amdahl, "The validity of the single processor approach toachieving large scale computing capabilities," AFIPS Nat.Comput. Conf. Expo. Conf. Proc. 30, 483-485 (1967).

16. M. G. Robinson, L. Zhang, K. M. Johnson, and D. A. Jared,"Custom electro-optical devices for optically implementedneuromorphic computing systems," in OSA Annual Meeting,Vol. 15 of 1990 OSA Technical Digest Series (Optical Society ofAmerica, Washington, D.C., 1990), paper MVV9.

17. K. M. Johnson, M. A. Handschy, and L. A. Pagano-Stauffer,"Optical computing and image processing with ferroelectricliquid crystals," Opt. Eng. 25, 385-391 (1987).

18. L. K. Cotter, T. J. Drabik, R. J. Dillion, and M. A. Handschy,"Ferroelectric-liquid-crystal/silicon-integrated-circuit spatiallight modulator," Opt. Lett. 15, 291-293 (1990).

19. M. A. Handschy, T. J. Drabik, L. K. Cotter, and S. D. Gaalema,"Fast ferroelectric-liquid-crystal spatial light modulator withsilicon-integrated-circuit active backplane," in Optical andDigital Gallium Arsenide Technologies for Signal ProcessingApplications, M. P. Bendett, D. H. Butler, A. Prabhokar, andA. Yang, eds., Proc. Soc. Photo-Opt. Instrum. Eng. 1291,158-164 (1990).

20. M. J. Healy and T. P. Caudell, "Application of ART1 hierar-chies to recognition of complex objects," in Cohference onNeural Networks for Automatic Target Recognition, S. Gross-berg, ed. (Wang Institute, Boston University, Boston, Mass.,1991), p. 29.

21. M. J. Healy, "A logical architecture for supervised learning,"in Proceedings of the International Joint Conference on NeuralNetworks, C. L. Teo, ed. (Institute of Electrical and ElectronicsEngineers, New York, 1991), Vol. 1, pp. 190-195.


Hybrid optoelectronic adaptive resonance theory neural...

Documents

Transcript of Hybrid optoelectronic adaptive resonance theory neural...