BAM: Bidirectional Associative Memory

Introduction

The Bidirectional Associative Memory (BAM) model has a neural network of two layers and is fully connected from each layer to the other. That is, there are feedback connections from the output layer to the input layer. However, the weights on the connections between any two given neurons from different layers are the same. You may even consider it to be a single bidirectional connection with a single weight. The matrix of weights for the connections from the output layer to the input layer is simply the transpose of the matrix of weights for the connections between the input and output layer. If we denote the matrix for forward connection weights by W, then WT is the matrix of weights for the output layer to input layer connections. As you recall, the transpose of a matrix is obtained simply by interchanging the rows and the columns of the matrix.

There are two layers of neurons, an input layer and an output layer. There are no lateral connections, that is, no two neurons within the same layer are connected. Recurrent connections, which are feedback connections to a neuron from itself, may or may not be present. The architecture is quite simple. Figure Layout of a BAM network shows the layout for this neural network model, using only three input neurons and two output neurons. There are feedback connections from Field A to Field B and vice-versa. This figure also indicates the presence of inputs and outputs at each of the two fields for the bidirectional associative memory network. Connection weights are also shown as labels on only a few connections in this figure, to avoid cluttering. The general case is analogous.


 Layout of a BAM network

Inputs and Outputs

The input to a BAM network is a vector of real numbers, usually in the set { –1, +1 }. The output is also a vector of real numbers, usually in the set { –1, +1 }, with the same or different dimension from that of the input. These vectors can be considered patterns, and the network makes heteroassociation of patterns. If the output is required to be the same as input, then you are asking the network to make autoassociation, which it does, and it becomes a special case of the general activity of this type of neural network.

For inputs and outputs that are not outside the set containing just –1 and +1, try following this next procedure. You can first make a mapping into binary numbers, and then a mapping of each binary digit into a bipolar digit. For example, if your inputs are first names of people, each character in the name can be replaced by its ASCII code, which in turn can be changed to a binary number, and then each binary digit 0 can be replaced by –1. For example, the ASCII code for the letter R is 82, which is 1010010, as a binary number. This is mapped onto the bipolar string 1 –1 1 –1 –1 1 –1. If a name consists of three characters, their ASCII codes in binary can be concatenated or juxtaposed and the corresponding bipolar string obtained. This bipolar string can also be looked upon as a vector of bipolar characters.

Weights and Training

BAM does not modify weights during its operation, and as mentioned in Chapter 6, like the Hopfield network, uses one-shot training. The adaptive variety of BAM, called the Adaptive Bidirectional Associative Memory, (ABAM) undergoes supervised iterative training. BAM needs some exemplar pairs of vectors. The pairs used as exemplars are those that require heteroassociation. The weight matrix, there are two, but one is just the transpose of the other as already mentioned, is constructed in terms of the exemplar vector pairs.

The use of exemplar vectors is a one-shot learning—to determine what the weights should be. Once weights are so determined, and an input vector is presented, a potentially associated vector is output. It is taken as input in the opposite direction, and its potentially associated vector is obtained back at the input layer. If the last vector found is the same as what is originally input, then there is resonance. Suppose the vector B is obtained at one end, as a result of C being input at the other end. If B in turn is input during the next cycle of operation at the end where it was obtained, and produces C at the opposite end, then you have a pair of heteroassociated vectors. This is what is basically happening in a BAM neural network.


NOTE:  The BAM and Hopfield memories are closely related. You can think of the Hopfield memory as a special case of the BAM.


What follow are the equations for the determination of the weight matrix, when the k pairs of exemplar vectors are denoted by ( Xi, Yi), i ranging from 1 to k. Note that T in the superscript of a matrix stands for the transpose of the matrix. While you interchange the rows and columns to get the transpose of a matrix, you write a column vector as a row vector, and vice versa to get the transpose of a vector. The following equations refer to the vector pairs after their components are changed to bipolar values, only for obtaining the weight matrix W. Once W is obtained, further use of these exemplar vectors is made in their original form.

 
     W =  X1T Y1 + ... + XkT Yk and  WT =  Y1TX1 + ... + YkT Xk

Example

Suppose you choose two pairs of vectors as possible exemplars. Let them be:      X1 = (1, 0, 0, 1), Y1= (0, 1, 1) and  X2 = (0, 1, 1, 0), Y2 = (1, 0, 1)

These you change into bipolar components and get, respectively, (1, –1, –1, 1), (–1, 1, 1), (–1, 1, 1, –1), and (1, –1, 1).

      1 [-1 1 1]   -1 [1 -1 1]   -1  1  1     -1  1 -1    -2  2 0
 W = -1           + 1          =  1 -1 -1  +   1 -1  1  =  2 -2 0
     -1             1             1 -1 -1      1 -1  1     2 -2 0
      1            -1            -1  1  1     -1  1 -1    -2  2 0

and

       -2    2    2    -2
WT =    2   -2   -2     2
        0    0    0     0

You may think that the last column of W being all zeros presents a problem in that when the input is X1 and whatever the output, when this output is presented back in the backward direction, it does not produce X1, which does not have a zero in the last component. There is a thresholding function needed here. The thresholding function is transparent when the activations are all either +1 or –1. It then just looks like you do an inverse mapping from bipolar values to binary, which is done by replacing each –1 by a 0. When the activation is zero, you simply leave the output of that neuron as it was in the previous cycle or iteration. We now present the thresholding function for BAM outputs.

 
     1  if  yj > 0   1   if  xi > 0
bj|t+1 =  bj|t    if  yj = 0   and   ai|t+1 =   ai|t  if  xi = 0
    0   if  yj < 0                                     0   if  xi < 0

where xi and yj are the activations of neurons i and j in the input layer and output layer, respectively, and bj|t refers to the output of the jth neuron in the output layer in the cycle t, while ai|t refers to the output of the ith neuron in the input layer in the cycle t. Note that at the start, the ai and bj values are the same as the corresponding components in the exemplar pair being used.

If X1 = (1, 0, 0, 1) is presented to the input neurons, their activations are given by the vector (–4, 4, 0). The output vector, after using the threshold function just described is (0, 1, 1). The last component here is supposed to be the same as the output in the previous cycle, since the corresponding activation value is 0. Since X1 and Y1 are one exemplar pair, the third component of Y1 is what we need as the third component of the output in the current cycle of operation; therefore, we fed X1 and received Y1. If we feed Y1 at the other end (B field) , the activations in the A field will be (2, –2, –2, 2), and the output vector will be (1, 0, 0, 1), which is X1.

With A field input X2 you get B field activations (4, -4, 0), giving the output vector as (1, 0, 1), which is Y2. Thus X2 and Y2 are heteroassociated with each other.

Let us modify our X1 to be (1, 0, 1, 1). Then the weight matrix W becomes

 
        1  [-1  1  1]       -1  [1  -1  1]       -2    2    0
 W =   -1                  + 1               =    2   -2    0
        1                    1                    0    0    2
        1                   -1                   -2    2    0

and

 
      -2     2     0    -2
WT =    2    -2     0     2
       0     0     2     0

Now this is a different set of two exemplar vector pairs. The pairs are X1 = (1, 0, 1, 1), 1 = (0, 1, 1), and X2 = (0, 1, 1, 0), Y2 = (1, 0, 1). Naturally, the weight matrix is different, as is its transpose, correspondingly. As stated before, the weights do not change during the operation of the network with whatever inputs presented to it. The results are as shown in table below.

Results for the Example

Input vector

activation

output vector

X1 = (1, 0, 1, 1)

(-4, 4, 2)

(0, 1, 1) = Y1

X2 = (0, 1, 1, 0)

(2, -2, 2)

(1, 0, 1) = Y2

Y1 = (0, 1, 1)

(2, -2, 2, 2)

(1, 0, 1, 1) = X1

Y2 = (1, 0, 1)

(-2, 2, 2, -2)

(0, 1, 1,0) = X2

You may think that you will encounter a problem when you input a new vector and one of the neurons has activation 0. In the original example, you did find this situation when you got the third output neuron’s activation as 0. The thresholding function asked you to use the same output for this neuron as existed in the earlier time cycle. So you took it to be 1, the third component in (0, 1, 1). But if your input vector is a new X vector for which you are trying to find an associated Y vector, then you do not have a Y component to fall back on when the activation turns out to be 0. How then can you use the thresholding function as stated? What guidance do you have in this situation? If you keep track of the inputs used and outputs received thus far, you realize that the Field B (where you get your Y vector) neurons are in some state, meaning that they had some outputs perhaps with some training vector. If you use that output component as the one existing in the previous cycle, you have no problem in using the thresholding function.

As an example, consider the input vector X3 = ( 1, 0, 0, 0), with which the activations of neurons in Field B would be (-2, 2, 0). The first component of the output vector is clearly 0, and the second clearly 1. The third component is what is in doubt. Considering the last row of the table where Y2 gives the state of the neurons in Field B, you can accept 1, the last component of Y2, as the value you get from the thresholding function corresponding to the activation value 0. So the output would be the vector (0, 1, 1), which is Y1. But Y1 is heteroassociated with X1. Well, it means that X3 = ( 1, 0, 0, 0) is not heteroassociated with any X vector.

Recall of Vectors

When X1 is presented at the input layer, the activation at the output layer will give ( –4, 4, 2) to which we apply the thresholding function, which replaces a positive value by 1, and a negative value by 0.

This then gives us the vector (0, 1, 1) as the output, which is the same as our Y1. Now Y1 is passed back to the input layer through the feedback connections, and the activation of the input layer becomes the vector (2, –2, 2, 2), which after thresholding gives the output vector (1, 0, 1, 1), same as X1. When X2 is presented at the input layer, the activation at the output layer will give (2, –2, 2) to which the thresholding function, which replaces a positive value by 1 and a negative value by 0, is applied. This then gives the vector (1, 0, 1) as the output, which is the same as Y2. Now Y2 is passed back to the input layer through the feedback connections to get the activation of the input layer as (–2, 2, 2, –2), which after thresholding gives the output vector (0, 1, 1, 0), which is X2.


The two vector pairs chosen here for encoding worked out fine, and the BAM network with four neurons in Field A and three neurons in Field B is all set for finding a vector under heteroassociation with a given input vector.


Continuation of Example

Let us now use the vector X3 = (1, 1, 1, 1). The vector Y3 = (0, 1, 1) is obtained at the output layer. But the next step in which we present Y3 in the backward direction does not produce X3, instead it gives an X1 = (1, 0, 1, 1). We already have X1 associated with Y1. This means that X3 is not associated with any vector in the output space. On the other hand, if instead of getting X1 we obtained a different X4 vector, and if this in the feed forward operation produced a different Y vector, then we repeat the operation of the network until no changes occur at either end. Then we will have possibly a new pair of vectors under the heteroassociation established by this BAM network.

Special Case—Complements

If a pair of (distinct) patterns X and Y are found to be heteroassociated by BAM, and if you input the complement of X, complement being obtained by interchanging the 0’s and 1’s in X, BAM will show that the complement of Y is the pattern associated with the complement of X. An example will be seen in the illustrative run of the program for C++ implementation of BAM, which follows.

C++ Implementation

In our C++ implementation of a discrete bidirectional associative memory network, we create classes for neuron and network. Other classes created are called exemplar, assocpair, potlpair, for the exemplar pair of vectors, associated pair of vectors, and potential pairs of vectors, respectively, for finding heteroassociation between them. We could have made one class of pairvect for a pair of vectors and derived the exemplar and so on from it. The network class is declared as a friend class in these other classes. Now we present the header and source files, called bamntwrk.h and bamntwrk.cpp. Since we reused our previous code from the Hopfield network of Chapter 4, there are a few data members of classes that we did not put to explicit use in the program. We call the neuron class bmneuron to remind us of BAM.

Program Details and Flow

A neuron in the first layer is referred to as anrn, and the number of neurons in this layer is referred to as anmbr. We give the namebnrn to the array of neurons in the second layer, and bnmbr denotes the size of that array. The sequence of operations in the program is as follows:

  We ask the user to input the exemplar vectors, and we transform them into their bipolar versions. The trnsfrm ( ) function in the exemplar class is for this purpose.

  We give the network the X vector, in its bipolar version, in one exemplar pair. We find the activations of the elements of bnrn array and get corresponding output vector as a binary pattern. If this is the Y in the exemplar pair, the network has made a desired association in one direction, and we go on to the next.step. Otherwise we have a potential associated pair, one of which is X and the other is what we just got as the output vector in the opposite layer. We say potential associated pair because we have the next step to confirm the association.

  We run the bnrn array through the transpose of the weight matrix and calculate the outputs of the anrn array elements. If , as a result, we get the vector X as the anrn array, we found an associated pair, (X, Y). Otherwise, we repeat the two steps just described until we find an associated pair.

  We now work with the next pair of exemplar vectors in the same manner as above, to find an associated pair.

  We assign serial numbers, denoted by the variable idn, to the associated pairs so we can print them all together at the end of the program. The pair is called (X, Y) where X produces Y through the weight matrix W, and Y produces X through the weight matrix which is the transpose of W.

  A flag is used to have value 0 until confirmation of association is obtained, when the value of the flag changes to 1.

  Functions compr1 and compr2 in the network class verify if the potential pair is indeed an associated pair and set the proper value of the flag mentioned above.

  Functions comput1 and comput2 in the network class carry out the calculations to get the activations and then find the output vector, in the proper directions of the bidirectional associative memory network.

Program Example for BAM

For our illustration run, we provided for six neurons in the input layer and five in the output layer. We used three pairs of exemplars for encoding. We used two additional input vectors, one of which is the complement of the X of an exemplar pair, after the encoding is done, to see what association will be established in these cases, or what recall will be made by the BAM network.

Header File

As expected, the complement of the Y of the exemplar is found to be associated with the complement of the X of that pair. When the second input vector is presented, however, a new pair of associated vectors is found. After the code is presented, we list the computer output also.

bamntwrk.h

//bamntwrk.h   V. Rao,  H. Rao
 
//Header file for BAM network program
 
#include <iostream.h>
#include <math.h>
#include <stdlib.h>
#define MXSIZ 10  // determines the maximum size of the network
 
class bmneuron{
 protected:
       int nnbr;
       int inn,outn;
       int output;
       int activation;
       int outwt[MXSIZ];
       char *name;
       friend class network;
 
public:
    bmneuron() { };
    void getnrn(int,int,int,char *);
};
 
class exemplar {
protected:
       int xdim,ydim;
       int v1[MXSIZ],v2[MXSIZ];
       int u1[MXSIZ],u2[MXSIZ];
       friend class network;
       friend class mtrx;
 
public:
       exemplar() { };
       void getexmplr(int,int,int *,int *);
       void prexmplr();
       void trnsfrm();
       void prtrnsfrm();
};
 
class asscpair {
protected:
       int xdim,ydim,idn;
       int v1[MXSIZ],v2[MXSIZ];
       friend class network;
 
public:
       asscpair() { };
       void getasscpair(int,int,int);
       void prasscpair();
};
 
class potlpair
{
protected:
       int xdim,ydim;
       int v1[MXSIZ],v2[MXSIZ];
       friend class network;
 
public:
       potlpair() { };
       void getpotlpair(int,int);
       void prpotlpair();
};
 
class network {
public:
       int  anmbr,bnmbr,flag,nexmplr,nasspr,ninpt;
       bmneuron (anrn)[MXSIZ],(bnrn)[MXSIZ];
       exemplar (e)[MXSIZ];
       asscpair (as)[MXSIZ];
       potlpair (pp)[MXSIZ];
       int outs1[MXSIZ],outs2[MXSIZ];
       int mtrx1[MXSIZ][MXSIZ],mtrx2[MXSIZ][MXSIZ];
 
       network() { };
       void getnwk(int,int,int,int [][6],int [][5]);
       void compr1(int,int);
       void compr2(int,int);
       void prwts();
       void iterate();
       void findassc(int *);
       void asgninpt(int *);
       void asgnvect(int,int *,int *);
       void comput1();
       void comput2();
       void prstatus();
};

Source File

The program source file is presented as follows. bamntwrk.cpp

//bamntwrk.cpp   V. Rao, H. Rao
 
//Source file for BAM network program
 
#include “bamntwrk.h”
 
void bmneuron::getnrn(int m1,int m2,int m3,char *y) {
int i;
name = y;
nnbr = m1;
outn = m2;
inn  = m3;
 
for(i=0;i<outn;++i){
outwt[i] = 0 ;
}
 
output = 0;
activation = 0;
}
 
void exemplar::getexmplr(int k,int l,int *b1,int *b2)
{
int i2;
xdim = k;
ydim = l;
 
for(i2=0;i2<xdim;++i2){
v1[i2] = b1[i2]; }
 
for(i2=0;i2<ydim;++i2){
v2[i2] = b2[i2]; }
}
 
void exemplar::prexmplr(){
int i;
cout<<”\nX vector you gave is:\n”;
for(i=0;i<xdim;++i){
       cout<<v1[i]<<”  “;}
 
cout<<”\nY vector you gave is:\n”;
 
for(i=0;i<ydim;++i){
       cout<<v2[i]<<”  “;}
 
cout<<”\n”;
}
 
void exemplar::trnsfrm(){
int i;
 
for(i=0;i<xdim;++i){
       u1[i] = 2*v1[i] -1;}
 
for(i=0;i<ydim;++i){
       u2[i] = 2*v2[i] - 1;}
 
}
 
void exemplar::prtrnsfrm(){
int i;
cout<<”\nbipolar version of X vector you gave is:\n”;
 
for(i=0;i<xdim;++i){
 
       cout<<u1[i]<<”  “;}
 
cout<<”\nbipolar version of Y vector you gave is:\n”;
 
for(i=0;i<ydim;++i){
       cout<<u2[i]<<”  “;}
 
cout<<”\n”;
}
void asscpair::getasscpair(int i,int j,int k)
{
idn = i;
xdim = j;
ydim = k;
}
 
void asscpair::prasscpair(){
int i;
cout<<”\nX vector in the associated pair no. “<<idn<<”   is:\n”;
 
for(i=0;i<xdim;++i){
       cout<<v1[i]<<”  “;}
 
cout<<”\nY vector in the associated pair no. “<<idn<<”   is:\n”;
 
for(i=0;i<ydim;++i){
       cout<<v2[i]<<”  “;}
 
cout<<”\n”;
}
 
void potlpair::getpotlpair(int k,int j){
xdim = k;
ydim = j;
}
 
void potlpair::prpotlpair(){
int i;
cout<<”\nX vector in possible associated pair is:\n”;
 
for(i=0;i<xdim;++i){
       cout<<v1[i]<<”  “;}
cout<<”\nY vector in possible associated pair is:\n”;
 
for(i=0;i<ydim;++i){
       cout<<v2[i]<<”  “;}
cout<<”\n”;
}
 
void network::getnwk(int k,int l,int k1,int b1[][6],int b2[][5]){
anmbr = k;
bnmbr = l;
nexmplr = k1;
nasspr = 0;
ninpt = 0;
int i,j,i2;
flag =0;
char *y1=”ANEURON”, *y2=”BNEURON” ;
 
for(i=0;i<nexmplr;++i){
       e[i].getexmplr(anmbr,bnmbr,b1[i],b2[i]);
       e[i].prexmplr();
       e[i].trnsfrm();
       e[i].prtrnsfrm();
       }
 
for(i=0;i<anmbr;++i){
       anrn[i].bmneuron::getnrn(i,bnmbr,0,y1);}
 
for(i=0;i<bnmbr;++i){
       bnrn[i].bmneuron::getnrn(i,0,anmbr,y2);}
 
for(i=0;i<anmbr;++i){
 
       for(j=0;j<bnmbr;++j){
              mtrx1[i][j]  = 0;
       for(i2=0;i2<nexmplr;++i2){
              mtrx1[i][j]  += e[i2].u1[i]*e[i2].u2[j];}
 
       mtrx2[j][i] = mtrx1[i][j];
       anrn[i].outwt[j] = mtrx1[i][j];
       bnrn[j].outwt[i] = mtrx2[j][i];
    }
}
 
prwts();
cout<<”\n”;
}
 
void network::asgninpt(int *b)
{
int i;
cout<<”\n”;
 
for(i=0;i<anmbr;++i){
       anrn[i].output = b[i];
       outs1[i] = b[i];
       }
 
}
 
void network::compr1(int j,int k){
int i;
for(i=0;i<anmbr;++i){
       if(pp[j].v1[i] != pp[k].v1[i]) flag = 1;
       break;
       }
 
}
 
void network::compr2(int j,int k){
int i;
for(i=0;i<anmbr;++i){
       if(pp[j].v2[i] != pp[k].v2[i]) flag = 1;
       break;}
 
}
 
void network::comput1(){
int j;
for(j=0;j<bnmbr;++j){
       int ii1;
       int c1 =0,d1;
       cout<<”\n”;
       for(ii1=0;ii1<anmbr;++ii1){
              d1 = outs1[ii1] * mtrx1[ii1][j];
              c1 += d1;
              }
 
       bnrn[j].activation = c1;
       cout<<”\n output layer neuron “<<j<<” activation is”<<c1<<”\n”;
 
if(bnrn[j].activation <0) {
       bnrn[j].output = 0;
       outs2[j] = 0;}
 
else
 
       if(bnrn[j].activation>0) {
 
               bnrn[j].output = 1;
               outs2[j] = 1;}
               else
               {cout<<”\n A 0 is obtained, use previous outputvalue \n”;
 
               if(ninpt<=nexmplr){
 
                         bnrn[j].output = e[ninpt-1].v2[j];}
 
               else
 
                         { bnrn[j].output = pp[0].v2[j];}
                         outs2[j] = bnrn[j].output; }
 
       cout<<”\n output layer neuron         “<<j<<” output is”
               <<bnrn[j].output<<”\n”;
       }
}
 
void network::comput2(){
int i;
for(i=0;i<anmbr;++i){
       int ii1;
       int c1=0;
       for(ii1=0;ii1<bnmbr;++ii1){
              c1 += outs2[ii1] * mtrx2[ii1][i];  }
 
       anrn[i].activation = c1;
       cout<<”\ninput layer neuron        “<<i<<”activation is “         <<c1<<”\n”;
 
       if(anrn[i].activation <0 ){
 
              anrn[i].output = 0;
              outs1[i] = 0;}
       else
 
              if(anrn[i].activation >0 ) {
 
                     anrn[i].output = 1;
                     outs1[i] = 1;
                     }
 
              else
 
              { cout<<”\n A 0 is obtained, use previous value if available\n”;
 
              if(ninpt<=nexmplr){
 
                        anrn[i].output = e[ninpt-1].v1[i];}
 
              else
 
                        {anrn[i].output = pp[0].v1[i];}
 
              outs1[i] = anrn[i].output;}
              cout<<”\n input layer neuron
              “<<i<<” output is “
                  <<anrn[i].output<<”\n”;
              }
}
 
void network::asgnvect(int j1,int *b1,int *b2)
{
int  j2;
 
for(j2=0;j2<j1;++j2){
       b2[j2] = b1[j2];}
 
}
 
void network::prwts()
{
int i3,i4;
cout<<”\n  weights—  input layer to output layer: \n\n”;
 
for(i3=0;i3<anmbr;++i3){
 
        for(i4=0;i4<bnmbr;++i4){
                cout<<anrn[i3].outwt[i4]<<”                  “;}
 
        cout<<”\n”; }
 
cout<<”\n”;
 
cout<<”\nweights—  output layer to input layer: \n\n”;
 
for(i3=0;i3<bnmbr;++i3){
 
        for(i4=0;i4<anmbr;++i4){
                cout<<bnrn[i3].outwt[i4]<<”                  “;}
 
        cout<<”\n”;  }
 
cout<<”\n”;
}
 
void network::iterate()
{
int i1;
 
for(i1=0;i1<nexmplr;++i1){
        findassc(e[i1].v1);
        }
 
}
 
void network::findassc(int *b)
{
int j;
flag = 0;
        asgninpt(b);
ninpt ++;
cout<<”\nInput vector is:\n” ;
 
for(j=0;j<6;++j){
       cout<<b[j]<<” “;}
 
cout<<”\n”;
pp[0].getpotlpair(anmbr,bnmbr);
asgnvect(anmbr,outs1,pp[0].v1);
 
comput1();
 
if(flag>=0){
           asgnvect(bnmbr,outs2,pp[0].v2);
 
           cout<<”\n”;
           pp[0].prpotlpair();
           cout<<”\n”;
 
           comput2(); }
 
for(j=1;j<MXSIZ;++j){
       pp[j].getpotlpair(anmbr,bnmbr);
       asgnvect(anmbr,outs1,pp[j].v1);
 
       comput1();
 
       asgnvect(bnmbr,outs2,pp[j].v2);
 
       pp[j].prpotlpair();
       cout<<”\n”;
 
       compr1(j,j-1);
       compr2(j,j-1);
 
       if(flag == 0) {
 
               int j2;
               nasspr += 1;
               j2 = nasspr;
 
               as[j2].getasscpair(j2,anmbr,bnmbr);
               asgnvect(anmbr,pp[j].v1,as[j2].v1);
               asgnvect(bnmbr,pp[j].v2,as[j2].v2);
 
               cout<<”\nPATTERNS ASSOCIATED:\n”;
               as[j2].prasscpair();
               j = MXSIZ ;
      }
 
      else
 
               if(flag == 1)
 
                       {
                       flag = 0;
                       comput1();
                       }
 
     }
}
 
void network::prstatus()
{
int j;
cout<<”\nTHE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY BAM\n\n”;
 
for(j=1;j<=nasspr;++j){
       as[j].prasscpair();
       cout<<”\n”;}
 
}
 
void main(){
int ar = 6, br = 5, nex = 3;
int inptv[][6]={1,0,1,0,1,0,1,1,1,0,0,0,0,1,1,0,0,0,0,1,0,1,0,1,
   1,1,1,1,1,1};
int outv[][5]={1,1,0,0,1,0,1,0,1,1,1,0,0,1,0};
cout<<”\n\nTHIS PROGRAM IS FOR A BIDIRECTIONAL ASSOCIATIVE MEMORY NETWORK.\n”;
cout<<” THE NETWORK ISSET UP FOR ILLUSTRATION WITH “<<ar<<“ INPUT NEURONS, AND “<<br;
cout<<” OUTPUT NEURONS.\n”<<nex <<” exemplars are used to encode \n”;
static network bamn;
bamn.getnwk(ar,br,nex,inptv,outv) ;
bamn.iterate();
bamn.findassc(inptv[3]);
bamn.findassc(inptv[4]);
bamn.prstatus();
}

Program Output

The output from an illustrative run of the program is listed next. You will notice that we provided for a lot of information to be output as the program is executed. The fourth vector we input is not from an exemplar, but it is the complement of the X of the first exemplar pair. The network found the complement of the Y of this exemplar to be associated with this input. The fifth vector we input is (1, 1, 1, 1, 1, 1). But BAM recalled in this case the complement pair for the third exemplar pair, which is X = (1, 0, 0, 1, 1, 1) and Y = (0, 1, 1, 0, 1). You notice that the Hamming distance of this input pattern from the X’s of the three exemplars are 3, 3, and 4, respectively. The hamming distance from the complement of the X of the first exemplar pair is also 3. But the Hamming distance of (1, 1, 1, 1) from the X of the complement of the X of the third exemplar pair is only 2. It would be instructive if you would use the input pattern (1, 1, 1, 1, 1, 0) to see to what associated pair it leads. Now the output:

THIS PROGRAM IS FOR A BIDIRECTIONAL ASSOCIATIVE
    MEMORY NETWORK.
THE NETWORK IS SET UP FOR ILLUSTRATION WITH SIX INPUT
    NEURONS AND FIVE OUTPUT NEURONS.
Three exemplars are used to encode
 
X vector you gave is:
1  0  1  0  1  0
Y vector you gave is:
1  1  0  0  1
 
bipolar version of X vector you gave is:
1  -1  1  -1  1  -1
bipolar version of Y vector you gave is:
1  1  -1  -1  1
 
X vector you gave is:
1  1  1  0  0  0
Y vector you gave is:
0  1  0  1  1
 
bipolar version of X vector you gave is:
1  1  1  -1  -1  -1
bipolar version of Y vector you gave is:
-1  1  -1  1  1
 
X vector you gave is:
0  1  1  0  0  0
Y vector you gave is:
1  0  0  1  0
 
bipolar version of X vector you gave is:
-1  1  1  -1  -1  -1
bipolar version of Y vector you gave is:
1  -1  -1  1  -1
 
  weights—  input layer to output layer:
 
           -1        3      -1     -1     3
           -1        1      -1      3    -1
            1        1      -3      1     1
           -1       -1       3     -1    -1
            1        1       1     -3     1
           -1       -1       3     -1    -1
 
weights—  output layer to input layer:
 
           -1       -1      1      -1       1    -1
            3       -1      1      -1       1    -1
           -1       -1     -3       3       1     3
           -1        3      1      -1      -3    -1
            3       -1      1      -1       1    -1
 
Input vector is:
1 0 1 0 1 0
 
 output layer neuron  0 activation is 1
 output layer neuron  0 output is 1
 output layer neuron  1 activation is 5
 output layer neuron  1 output is 1
 output layer neuron  2 activation is -3
 output layer neuron  2 output is 0
 output layer neuron  3 activation is -3
 output layer neuron  3 output is 0
 output layer neuron  4 activation is 5
 output layer neuron  4 output is 1
 X vector in possible associated pair is: 1  0  1  0  1  0
 Y vector in possible associated pair is: 1  1  0  0  1
input layer neuron 0 activation is 5
 input layer neuron  0 output is 1
input layer neuron 1 activation is -3
 input layer neuron  1 output is 0
input layer neuron 2 activation is 3
 input layer neuron  2 output is 1
input layer neuron 3 activation is -3
 input layer neuron  3 output is 0
input layer neuron 4 activation is 3
 input layer neuron  4 output is 1
input layer neuron 5 activation is -3
 input layer neuron  5 output is 0
 output layer neuron  0 activation is 1
 output layer neuron  0 output is 1
 output layer neuron  1 activation is 5
 output layer neuron  1 output is 1
 output layer neuron  2 activation is -3
 output layer neuron  2 output is 0
 output layer neuron  3 activation is -3
 output layer neuron  3 output is 0
 output layer neuron  4 activation is 5
 output layer neuron  4 output is 1
X vector in possible associated pair is: 1  0  1  0  1  0
Y vector in possible associated pair is: 1  1  0  0  1
PATTERNS ASSOCIATED:
X vector in the associated pair no. 1 is: 1  0  1  0  1  0
Y vector in the associated pair no. 1 is: 1  1  0  0  1
Input vector is: 1 1 1 0 0 0
// We get here more of the detailed output as in the previous case. We will simply not present it here.
PATTERNS ASSOCIATED:
X vector in the associated pair no. 1 is: 1  1  1  0  0  0
Y vector in the associated pair no. 1 is: 0  1  0  1  1
Input vector is: 0 1 1 0 0 0
output layer neuron  0 activation is 0
A 0 is obtained, use previous output value
output layer neuron  0 output is 1
output layer neuron  1 activation is 0
A 0 is obtained, use previous output value
output layer neuron  1 output is 0
output layer neuron  2 activation is -4
output layer neuron  2 output is 0
output layer neuron  3 activation is 4
output layer neuron  3 output is 1
output layer neuron  4 activation is 0
A 0 is obtained, use previous output value
output layer neuron  4 output is 0
X vector in possible associated pair is: 0  1  1  0  0  0
Y vector in possible associated pair is: 1  0  0  1  0
// We get here more of the detailed output as in the previous case. We will simply not present it here.
PATTERNS ASSOCIATED:
X vector in the associated pair no. 1 is: 0  1  1  0  0  0
Y vector in the associated pair no. 1 is: 1  0  0  1  0
Input vector is:
0 1 0 1 0 1
// We get here more of the detailed output as in the previous case. We will simply not present it here.
X vector in possible associated pair is: 0  1  0  1  0  1
Y vector in possible associated pair is: 0  0  1  1  0
// We get here more of the detailed output as in the previous case. We will simply not present it here.
X vector in possible associated pair is: 0  1  0  1  0  1
Y vector in possible associated pair is: 0  0  1  1  0
PATTERNS ASSOCIATED:
X vector in the associated pair no. 1 is: 0  1  0  1  0  1
Y vector in the associated pair no. 1 is: 0  0  1  1  0
Input vector is: 1 1 1 1 1 1
output layer neuron  0 activation is -2
output layer neuron  0 output is 0
output layer neuron  1 activation is 2
output layer neuron  1 output is 1
output layer neuron  2 activation is 2
output layer neuron  2 output is 1
output layer neuron  3 activation is -2
output layer neuron  3 output is 0
output layer neuron  4 activation is 2
output layer neuron  4 output is 1
X vector in possible associated pair is: 1  1  1  1  1  1
Y vector in possible associated pair is: 0  1  1  0  1
input layer neuron 0 activation is 5
input layer neuron  0 output is 1
input layer neuron 1 activation is -3
input layer neuron  1 output is 0
input layer neuron 2 activation is -1
input layer neuron  2 output is 0
input layer neuron 3 activation is 1
input layer neuron  3 output is 1
input layer neuron 4 activation is 3
input layer neuron  4 output is 1
input layer neuron 5 activation is 1
input layer neuron  5 output is 1
output layer neuron  0 activation is -2
output layer neuron  0 output is 0
output layer neuron  1 activation is 2
output layer neuron  1 output is 1
output layer neuron  2 activation is 6
output layer neuron  2 output is 1
output layer neuron  3 activation is -6
output layer neuron  3 output is 0
output layer neuron  4 activation is 2
output layer neuron  4 output is 1
X vector in possible associated pair is: 1  0  0  1  1  1
Y vector in possible associated pair is: 0  1  1  0  1
PATTERNS ASSOCIATED:
X vector in the associated pair no. 1 is: 1  0  0  1  1  1
Y vector in the associated pair no. 1 is: 0  1  1  0  1
THE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY BAM
X vector in the associated pair no. 1 is:   //first exemplar pair
1  0  1  0  1  0
Y vector in the associated pair no. 1 is: 1  1  0  0  1
X vector in the associated pair no. 2 is:   //second exemplar pair
1  1  1  0  0  0
Y vector in the associated pair no. 2 is: 0  1  0  1  1
X vector in the associated pair no. 3 is:   //third exemplar pair
0  1  1  0  0  0
Y vector in the associated pair no. 3 is: 1  0  0  1  0
X vector in the associated pair no. 4 is:   //complement of X of the
0  1  0  1  0  1               first exemplar pair
Y vector in the associated pair no. 4 is:    //complement of Y of the
0  0  1  1  0                    first exemplar pair
X vector in the associated pair no. 5 is:  //input was X = (1, 1, 1,
1  0  0  1  1  1                            1, 1) but result was complement of third exemplar pair
Y vector in the associated pair no. 5 is:  with X of which Hamming
0  1  1  0  1                 distance is the  least.

Additional Issues

If you desire to use vectors with real number components, instead of binary numbers, you can do so. Your model is then called a Continuous Bidirectional Associative Memory. A matrix W and its transpose are used for the connection weights. However, the matrix W is not formulated as we described so far. The matrix is arbitrarily chosen, and kept constant. The thresholding function is also chosen as a continuous function, unlike the one used before. The changes in activations of neurons during training are according to extensions the Cohen-Grossberg paradigm.

Michael A. Cohen and Stephen Grossberg developed their model for autoassociation, with a symmetric matrix of weights. Stability is ensured by the Cohen-Grossberg theorem; there is no learning.

If, however, you desire the network to learn new pattern pairs, by modifying the weights so as to find association between new pairs, you are designing what is called an Adaptive Bidirectional Associative Memory (ABAM).

The law that governs the weight changes determines the type of ABAM you have, namely, the Competitive ABAM, or the Differential Hebbian ABAM, or the Differential Competitive ABAM. Unlike in the ABAM model, which is additive type, some products of outputs from the two layers, or the derivatives of the threshold functions are used in the other models.

Here we present a brief description of a model, which is a variation of the BAM. It is called UBBAM (Unipolar Binary Bidirectional Associative Memory).

Unipolar Binary Bidirectional Associative Memory

T. C. B. Yu and R. J. Mears describe a design of unipolar binary bidirectional associative memory, and its implementation with a Smart Advanced Spatial Light Modulator (SASLM). The SASLM device is a ferroelectric liquid crystal spatial light modulator. It is driven by a silicon CMOS backplane. We use their paper to present some features of a unipolar binary bidirectional associative memory, and ignore the hardware implementation of it.

Recall the procedure by which you determine the weight matrix W for a BAM network, as described in the previous pages. As a first step, you convert each vector in the exemplar pairs into its bipolar version. If X and Y are an exemplar pair (in bipolar versions), you take the product XT Y and add it to similar products from other exemplar pairs, to get a weight matrix W. Some of the elements of the matrix W may be negative numbers. In the unipolar context you do not have negative values and positive values at the same time. Only one of them is allowed. Suppose you do not want any negative numbers; then one way of remedying the situation is by adding a large enough constant to each element of the matrix. You cannot choose to add to only the negative numbers that show up in the matrix. Let us look at an example.

Suppose you choose two pairs of vectors as possible exemplars. Let them be,

X1 = (1, 0, 0, 1), Y1= (0, 1, 1) and X2 = (0, 1, 1, 0), Y2 = (1, 0, 1)

These you change into bipolar components and get, respectively, (1, –1, –1, 1), (–1, 1, 1), (–1, 1, 1, –1), and (1, –1, 1). The calculation of W for the BAM was done as follows.

     1 [-1 1 1]   -1 [1 -1 1]  -1  1  1    -1  1 -1   -2  2 0
W = -1           + 1         =  1 -1 -1    +1 -1  1 =  2 -2 0
    -1             1            1 -1 -1     1 -1  1    2 -2 0
     1            -1           -1  1  1    -1  1 -1   -2  2 0

and

      -2    2    2   -2
WT =    2   -2   -2    2
       0    0    0    0

You see some negative numbers in the matrix W. If you add the constant m, m = 2, to all the elements in the matrix, you get the following matrix, denoted by W~.

       0    4    2
W~ =   4    0    2
       4    0    2
       0    4    2

You need a modification to the thresholding function as well. Earlier you had the following function.

      1  if yj > 0   1   if  xi > 0
      bj|t+1 = bj|t    if  yj = 0   and      ai|t+1 = ai|t     if  xi = 0
      0  if  yj < 0  0   if  xi < 0

Now you need to change the right-hand sides of the conditions from 0 to the product of m and sum of the inputs of the neurons, that is to m times the sum of the inputs. For brevity, let us use Si for the sum of the inputs. This is not a constant, and its value depends on what values you get for neuron inputs in any one direction. Then the thresholding function can be given as follows:

      1   if yj > m Si         1  if  xi > m Si
bj|t+1 = bj|t      if  yj = m Si       and      ai|t+1 = ai|t     if  xi = m Si
       0  if  yj < m Si                         0  if  xi < m Si

For example, the vector X1 = (1, 0, 0, 1) has the activity vector (0, 8, 4) and the corresponding output vector is (0, 1, 1), which is, Y1 = (0, 1, 1). The value of Si is 2 = 1 + 0 + 0 + 1. Therefore, mSi = 4. The first component of the activation vector is 0, which is smaller than 4, and so the first component of the output vector is 0. The second component of the activation vector is 8, which is larger than 4, and so the second component of the output vector is 1. The last component of the activity vector is 4, which is equal to mSi, and so you use the third component of the vector Y1 = (0, 1, 1), namely 1, as the third component of the output.

The example of Yu and Mears, consists of eight 8-component vectors in both directions of the Bidirectional Associative Memory model. You can take these vectors, as they do, to define the pixel values in a 8x8 grid of pixels. What is being accomplished is the association of two spatial patterns. In terms of the binary numbers that show the pixel values. Call them Pattern I and Pattern II.

Two patterns, Pattern I and Pattern II, given by pixel values: 1 for black, 0 for white.

Instead of Pattern I, they used a corrupted form of it as given in figure:

Pattern I and corrupted Pattern I.

There was no problem in the network finding the associated pair (Pattern I, Pattern II).

The corrupted version of Pattern I differs from Pattern I, in 10 of the 64 places, a corruption of 15.6%. This corruption percentage is cited as the limit below which, Yu and Mears state, heteroassociative recall is obtained. Thus noise elimination to a certain extent is possible with this model. As we have seen with the Hopfield memory, an application of associative memory is pattern completion, where you are presented a corrupted version of a pattern and you recall the true pattern. This is autoassociation. In the case of BAM, you have heteroassociative re call with a corrupted input.

Summary

In this chapter, bidirectional associative memories are presented. The development of these memories is largely due to Kosko. They share with Adaptive Resonance Theory the feature of resonance between the two layers in the network. The bidirectional associative memories (BAM) network finds heteroassociation between binary patterns, and these are converted to bipolar values to determine the connection weight matrix. Even though there are connections in both directions between neurons in the two layers, essentially only one weight matrix is involved. You use the transpose of this weight matrix for the connections in the opposite direction. When one input at one end leads to some output at the other end, which in turn leads to output that is the same as the previous input, resonance is reached and an associated pair is found.

The continuous bidirectional associative memory extends the binary model to the continuous case. Adaptive bidirectional memories of different flavors are the result of incorporating different learning paradigms. A unipolar binary version of BAM is also presented as an application of the BAM for pattern completion.