attrasoft

Attrasoft

Attrasoft PolyNet 
for Windows 95/98
Version 4.0 (7/1999)
About

New

Image Search Engine

Products

Services

Order

Shareware/Demos

Future Products

Links

Search This Site

All Attrasoft products are Y2K Compliant!
 

3. User's Guide

3.1 Step 1: Training and Recognition File Format
3.1.1 Examples
3.1.2 To Make A Long Story short
3.1.3 The Details
3.1.4 Part 1 of the Training File: Comment Section
3.1.5 Part 2 of the Training File: Number of Neurons
3.1.6 Part 3 of the Training File: Data Section
3.1.7 Part 1 of the Recognition File: Comment Section
3.1.8 Part 2 of the Recognition File: Data Section
3.2 Step 2: Link Input Files To PolyNet
3.3 Step 3: Set up PolyNet
3.3.1 Position of a Digit in a String
3.3.2 Output File Format
3.3.3 Symmetry
3.3.4 Setup Symmetry Parameters
3.3.5 Symmetry Selections
3.4 Step 4: Run
3.5 Test Input File Format
3.6 1-Neuron-1-Class Classifications
3.7 Training, Retraining and Recognition
3.8 Binary Boltzmann Machine
3.9 Common Errors
 
  3.   User's Guide


Chapter 2 is the quick version of the manual, which listed the minimum requirements. After chapter 2, you should be able to operate the software. However, a few details were left out of chapter 2, such as, what is symmetry and how to set up symmetry.

The operation has 4 steps:

Step 1. Put the training data into the training file; Put your recognition data into the recognition file; together you prepare two files.

Step 2. Link the two files to the PolyNet by clicking "Data/Link".

Step 3. Once the files are in the PolyNet, set up symmetry and output format for PolyNet if necessary;

Step 4. Click 'D' button (Decimal/Distribution command) for a distribution or 'C' button for a classification (Decimal/Classification command).

The command, "Decimal/Distribution", presents you with a distribution, which is all possible classifications and how valuable the PolyNet thinks each prediction is via a confidence number. This number is similar to the number in an Internet Search Engine: the higher that number, the more confidence the PolyNet has in that distribution.

The command, "Decimal/Classification", presents you with a classification, and a confidence number; the higher this number, the more confidence the PolyNet has in that classification.
 
  3.1   Step 1: Training and Recognition File Format
  3.1.1   Examples


The best way to learn the two file formats is to look at the examples in the PolyNet, for example, click "Example/Character5x7". The format is very easy to learn.

An example of a training file is:

* 
5x7 Character Recognition: Training File 
Add your data at the end 
*

46

10000000000
01000
10100
10100
01000
00000
00000
00000

10000000000
01000
10100
10100
10100
01000
00000
00000


An example of a recognition file is:

*
5x7 Character Recognition : Recognition File
*

xxxxxxxxxxx
00100
01010
01010
01010
01010
00100
00000


  3.1.2   To Make A Long Story short


To make a long story short, in the training file, you will:

  • Put a '*' in the first line;
  • Put a '*' in the second line;
  • Write the number of neurons in the third line;
  • Enter the training data to the fourth line and downward.
In the recognition file, you will:
  • Put a '*' in the first line;
  • Put a '*' in the second line;
  • Start from the third line, enter the recognition data with the unknown neurons marked as 'x'.
3.1.3   The Details

The training file has 3 sections and the recognition file has 2 sections:

Training Files                          Recognition Files
----------------------------- ----------------------
Comment Section                     Comment Section
Neuron Number Section
Data Section                             Data Section

The two files must be prepared in the text file format. The PolyNet has its own text editor. Or if you wish, you can use any word processor, like Microsoft Word, or WordPerfect; just make sure you save the data file in the text format. 

The training file has three sections:

  • Comment section;
  • Number of variables;
  • Data section.
The recognition file has two sections:
  • Comment section;
  • Data section.
3.1.4   Part 1 of the Training File: Comment Section

The comment section starts with a "*" and ends with another "*". The purpose of this section is to document your data:

  • Give a description of the data;
  • Specify the data set used;
  • . . .
Example:  *
Title: 5 x7 character recognition
Data: 0, 1, ..., 9
* Note: the PolyNet will ignore this section. This section is for your remarks only.
 
  3.1.5   Part 2 of the Training File: Number of Neurons


This section has a single integer, which will tell the PolyNet how many neurons you will use. 

In the 5x7 character recognition example, each 5x7 character takes 35 neurons, and the number of classes is 11, giving a total of 35 + 11 = 46 neurons.
 
  3.1.6   Part 3 of the Training File: Data Section 


This section contains the data. 

Example 

* 
Character Recognition: Training File
Add your data at the end 
*

46

10000000000
01000
10100
10100
01000
00000
00000
00000
...

3.1.7   Part 1 of the Recognition File: Comment Section 


This section is the same as the comment section of the training file.
 
  3.1.8   Part 2 of the Recognition File: Data Section 


This section is the same as the data section of the training file, however, each pattern contains unknown bits, xxx...xxx. PolyNet will convert the unknown bits to a string of 0's, 1's, ... and 9's.

The rules for the recognition files are:

(1) The unknown digits in a recognition file must be marked as 'x';
(2) The unknown digits in a recognition file must stay together; 
(3) The unknown digits in a recognition file must be the same position.

The following test pattern is wrong because x's are separated:

100 xx11 xx10, 

The following test pattern is wrong because y's are illegal characters:

100 yyyy 0110,

The following test pattern is wrong because x's are not in the same position: 

100 xxxx 0110, 

100 00xx xx10.

If you want PolyNet to fix:

100 xxxx 0110, 
100 00xx xx10,

you have to run twice, the first time with the recognition pattern 

100 xxxx 0110, 

and the second time with the pattern

100 00xx xx10.

Example

    *

    Character Recognition : Recognition File

    *

    xxxxxxxxxxx
    00100
    01010
    01010
    01010
    01010
    00100
    00000

    ...
     
     

3.2   Step 2: Link Input Files To PolyNet

There are two input files to be linked:

  • Training file; and 
  • Recognition file.
They will be used to train and to test PolyNet, respectively. Assume we want to link the two files:
  • Training file: "a.txt"; and 
  • Recognition file "b.txt";
to the PolyNet. There are 4 methods to link the two files:

Method 1: 

  • Save the two files, "a.txt" and "b.txt", to C:\Program Files\Attrasoft\PolyNet 4.0\ (the default directory for the PolyNet);
  • Click command "Data\link", then enter "a.txt" to the first textbox; and enter "b.txt" to the second box.
Method 2: 
  • Save the two files, "a.txt" and "b.txt", anywhere, say C:\abc\def\a.txt, and C:\abc\def\b.txt. 
  • Click command "File/Open" to open one of the two files, this will lead the PolyNet to the directory;
  • Click command "Data\link" (Figure 4, 5), then enter "a.txt" to the first textbox; and enter "b.txt" to the second box.
The default file names are:
C:\Program Files\Attrasoft\PolyNet 4.0\Example1a.txt
C:\Program Files\Attrasoft\PolyNet 4.0\Example1b.txt
If you are using two files for a problem over and over again, you can use the following method to link:

Method 3: 

  • Save your data to: 
C:\Program Files\Attrasoft\PolyNet 4.0\Example1a.txt
C:\Program Files\Attrasoft\PolyNet 4.0\Example1b.txt
  • Your files are linked automatically. 
Method 4: 
  • Save your data to "Example1a.txt" and "Example1b.txt", and leave it anywhere, say C:\abc\def\Example1a.txt, and C:\abc\def\Example1b.txt. 
  • Click command "File/Open" to open one of the two files, this will lead the PolyNet to the directory and your files are linked automatically. 
There is a button on the toolbar for this command "Data\Link. 
 

3.3   Step 3: Set up PolyNet
  3.3.1   Position of a Digit in a String


Let a string be 101, then the position of the first digit is 0; the position of the second digit is 1; and the position of the third digit is 2, ...

position: 012,
pattern: 101.

Similarly, the position of another pattern 1010 1100 is:

position: 0123 4567,
pattern:  1011 1100.
 
  3.1.2   Output File Format


If no output format is given, the default will print one pattern per line. 

If your recognition pattern is long, or they are 2-dimensional, you might want to change this default format. Inserting line breakers will do just that. 

PolyNet allows you to insert up to 4 line breakers. To insert the line breakers, you must specify the positions of the line breakers. Suppose you want the output look like this:

111
0000
11111
000000
1111111

then click: "Data/Line Breaker" and specify (See Figure 7):

Line 2 starts at 3
Line 3 starts at 7
Line 4 starts at 12
Line 5 starts at 18.
 
 


 
 

Figure 7. Set up Line Breakers.
 
 

The position of the first digit is 0; the position of the second digit is 1; and the position of the third digit is 2. Note that after "111", the next position is 3. "Line 2 starts at 3" means a new line starts after "111". The default values are all -1, meaning no line breakers. 

PolyNet allows the output neurons to be printed on five lines. However, if line 3 and line 4 have an equal length, all the rest of the lines will also have the same length. For example, to change the output-vector to:

1111
000
111
000
111
000
111
000
1,
click: "Data/Line Breaker" and specify:
Line 2 starts at 4
Line 3 starts at 7
Line 4 starts at 10
Line 5 starts at 13.
3.3.3    Symmetry


Symmetry means that after you transform the pattern, both the original pattern and the transformed pattern belong to the same class. For example, a picture of a person in the middle of a page and the same person near the edge of the paper represent the same person. Almost all images have some sort of symmetries.

PolyNet supports one-dimensional and two-dimensional patterns. One-dimensional patterns may have the following symmetries: 

  • translation; 
  • scaling; 
Two-dimensional patterns may have the following symmetries: 
  • x- translation;
  • y- translation;
  • x- scaling;
  • y- scaling;
  • z-rotation 
The scaling symmetry automatically implies the translation symmetry. 2 bits define a one-dimensional symmetry:
00 means no symmetry, 
10 means translation symmetry alone, 
01 means scaling and translation symmetry, 
11 means scaling and translation symmetry. 
A two-dimensional symmetry is defined by 5 bits:
 
00000 means no symmetry, 
10000 means x-translation alone, 
11000 means both x- and y- translation,
... ...
What is Symmetry 


We will not formally define these symmetries. We will let you learn from the examples (like neural networks do). Symmetries apply only to pattern-neurons, not to class-neurons. Examples of symmetry are:

x-Translation: After the translation, a '1' is still a '1'.

Before    After

01000    00100
01000    00100
01000    00100
01000    00100
01000    00100
11100    01110
00000    00000

y-Translation: After the translation, a '1' is still a '1'.

Before After

01000     00000
01000     01000
01000     01000
01000     01000
01000     01000
11100     01000
00000     11100

x- and y-Translation: After the translation, a '1' is still a '1'.

Before     After

01000     00000
01000     00100
01000     00100
01000     00100
01000     00100
11100     00100
00000     01110

Linear x-Scaling: After the scaling, a '1' is still a '1'.

Before     After

01000     11100
01000     11100
01000     11100
01000     11100
01000     11100
11100     11111
00000     00000

Linear y-Scaling: After the scaling, a '1' is still a '1'.

Before     After

01000     00000
01000     01000
01000     01000
01000     01000
01000     11100
11100     00000
00000     00000

Rotation: After the rotation, a '1' is still a '1' or it is not a '1' anymore?

Before After

00000     00000
00100     00001
00100     00010
00100     00100
00100     01000
00100     10000
00000     00000
 
  Translation Symmetry


PolyNet fully supports the translation symmetry; it is one of the most useful symmetries. Introducing the translation symmetry significantly reduces the training data, therefore, apart from the data preparation itself, setting up the symmetry is the most helpful step, provided the data does have the chosen symmetry. If your data does not have any symmetry, then you can not use it. In the character recognition example, the 2-D translation symmetry (see last chapter) is used. Besides 2-D translation symmetry, the characters also have partial rotation symmetry and nonlinear scaling symmetry.

Many examples in PolyNet have translation symmetries:

5x7 character recognition
8x8 character recognition
19x19 character recognition, 1023 characters
19x19 character recognition, 4095 characters
32x32 character recognition, 127 characters
50x50 image recognition
100x100 image recognition
Approximate Symmetry
Symmetry settings are so helpful that sometimes even if the symmetry is not exact, it is still used. 

The shifter problem is just such an example. We will study the shifter problem later. Basically, you pick a string and shift it to the left or to the right or no-shift. Then you ask PolyNet to recognize the shift: is it a left shift, a right shift, or a no-shift ?

Shifters satisfy the translation symmetry most of the time and violate it only occasionally. The 8-digit shifter below shows the approximate symmetry:

                                   a string             shifted             Symmetry
                ----         -------------- -------------- ------------
Original:     100         0000 0011     0000 0110 
Translated: 100         0000 0110     0000 1100         Y
                 100         0000 1100     0001 1000         Y
                 100         0001 1000     0011 0000         Y
                 100         0011 0000     0110 0000         Y
                 100         0110 0000     1100 0000         Y
                 100         1100 0001     1000 0000         N

Until the last step, the translation symmetry is O.K:

In line 2, the string "0000 1100" is indeed the left shift of "0000 0110"; 
In line 3, the string "0001 1000" is indeed the left shift of "0000 1100"; 
. . .  In the very last step, the translation symmetry is violated:  The string "1000 0000" is not the left-shift of "1100 0001".  Approximate symmetry might cause some recognition errors. Retraining can compensate for these errors (When you make such a retraining, set the symmetry to "No symmetry"). Without the approximate symmetry, the shifter problem in PolyNet will take more training data to achieve the same recognition success. Try to figure this out for yourself.

Many examples in PolyNet have approximate symmetries:

  • Single shifters
  • Double shifters
  • Triple shifters
  • Quadruple shifters
3.3.4   Setup Symmetry Parameters

Symmetries only apply to the pattern-neurons. In general, the class-neurons do not have any symmetry.
 
 

Figure 8. Set up Symmetry.
 
 

To specify the symmetry, you have to specify:

  • Where is the symmetry ?
  • What symmetry ?
Where (See Figure 8) The position of a pattern and the dimension tell "where":

Position: where does it start, and where does it end:
Dimension: 1-dimension or 2-dimension. 

Select: "Data/Symmetry" to specify:

where symmetry starts
where symmetry ends
the x-dimension of the symmetry
the y-dimension of the symmetry If you do not specify the symmetry, the default values are -2, meaning no symmetry. Please refer to the last section for the position of a pattern.  What (See Figure 8)
To specify a symmetry, check the necessary boxes: x-translation symmetry
y-translation symmetry
x-scaling symmetry
y-scaling symmetry
z- rotation symmetry The rotation symmetry automatically implies the translation symmetry, so is the scaling symmetry. 

Example The 4-Digit Shifter Problem (See chapter 6) has 1-dimensional x-translation symmetry. The class-neurons take digits from 0 to 2, and the pattern-neurons take digits from 3 to 10. The x-dimension is 8 and there is no y-dimension. The position is shown as follows:

Position 012 3456 789 10
Vector  100 0011 011 0

Click: 'Data/Symmetry' to open a dialogue box and specify:

  Symmetry starts at 3
Symmetry ends at 10
x-dimension is 8
y-dimension is -1 check  x-translation symmetry
Leave "y-dimension = -1" unchanged to indicate it is a one-dimensional problem (No y-dimension).  Note the relation:
x-dimension = Symmetry end - Symmetry start + 1 , or 
8 = 10 - 3 + 1.
Example The '5 by 7' Character Recognition Problem (See Chapter 2) has x- and y-translation symmetry. Let a pattern be: 

01000 00000 0
00100
01100
00100
00100
00100
00100
01110

The class-neurons take digits from 0 to 10 and the pattern-neurons takes digits from 11 to 45. x-dimension is 5 and the y-dimension is 7. Click: 'Data/Symmetry' to open a dialogue box and to specify:

 
Symmetry starts at 11
Symmetry ends at 45
x-dimension is 5 
y-dimension is 7
check x-translation symmetry
check y-translation symmetry. Note the relation:
 
x-dimension * y-dimension = Symmetry end - Symmetry start + 1, 
or 5 * 7 = 45 - 11 + 1.


When you specify the translation symmetry, the numbers must satisfy the following conditions:

1-D: x-dimension = Symmetry end - Symmetry start + 1;
2-D: x-dimension * y-dimension = Symmetry end - Symmetry start + 1.

When there are errors in the specification, an error message will be printed.
 

3.3.5   Symmetry Selections

A symmetry is defined by five check boxes:

  • x-translation, 
  • y-translation, 
  • x-scaling, 
  • y-scaling, 
  • rotation.
Let 1 mean the box being checked and 0 mean the box being unchecked, See Figure 8. The possible settings are: 

Setting         Meaning
--------- -------------
00000          No symmetry
10000          x-translation
01000          y-translation
11000          x- and y-translation
00100          x-scaling and x-translation
10100          x-scaling and x-translation
01100          Not supported
11100          Not supported
00010          y-scaling and y-translation
10010          Not supported
01010          y-scaling and y-translation
11010          Not supported
00110          x- and y-scaling , x- and y- translation
10110          x- and y-scaling , x- and y- translation
01110          x- and y-scaling , x- and y- translation
11110          x- and y-scaling , x- and y- translation
00001          z-rotation, x- and y-translation
10001          z-rotation, x- and y-translation
01001          z-rotation, x- and y-translation
11001          z-rotation, x- and y-translation
00101          z-rotation, x- and y-scaling, x- and y-translation
10101          z-rotation, x- and y-scaling, x- and y-translation
01101          not supported
11101          not supported
00011          z-rotation, x- and y-scaling, x- and y-translation
10011          not supported
01011          z-rotation, x- and y-scaling, x- and y-translation
11011          not supported
00111          z-rotation, x- and y-scaling, x- and y-translation
10111          z-rotation, x- and y-scaling, x- and y-translation
01111          z-rotation, x- and y-scaling, x- and y-translation
11111          z-rotation, x- and y-scaling, x- and y-translation

Basically, PolyNet supports:

2D:

x-translation
y-translation
x- and y-translation
x- and y-scaling and x- and y-translation
rotation and x- and y-translation
rotation, x- and y-scaling and x- and y-translation 1D: x-translation
y-translation
x-translation and x-scaling
y-translation and y-scaling More information on scaling and rotation symmetries can be found in chapter 5.

3.4   Step 4: Run

All neural computation starts with training. Training means PolyNet learns the patterns from the training file. 

After the training comes the recognition. Recognition means PolyNet classifies the patterns in the recognition files.

There are two basic commands:

Decimal/Classification -------- the Decimal Hopfield Model;
Decimal/Distribution --------- the Decimal Boltzmann Machine.

The output of the computation goes to the output file. After clicking one of the above two commands, the output file will be opened automatically.

The default output file name is "example1c.txt". You can change the output file name by clicking: "Data/Link", or its button on the toolbar, then type in your new data file name. 

The command, "Decimal/Distribution" presents you with a distribution, which is all possible classifications and how valuable the PolyNet thinks each prediction is via a confidence number. This number is similar to the number in an Internet Search Engine: the higher that number, the more confidence the PolyNet has in that distribution.

The command, "Decimal/Classification" presents you with a classification, and a confidence number. This number is similar to the number in an Internet Search Engine: the higher that number, the more confidence the PolyNet has in that classification.

Each of the above commands has two phases:

  • Training; and 
  • Recognition (or Recognition).
In the training phase, the training file is imposed upon a neural network to force the network to remember the pattern of training data. A neural network can remember the training pattern by adjusting its internal synaptic connections. 

In the recognition phase, a part of the input data in the recognition file is not known. The neural network, based on its internal synaptic connections, will determine the unknown part. 

A typical problem for a neural network to solve is the classification problem. The data is a set of doublets: (pattern, class). In the training phase, the network is taught which pattern belongs to which class. In the recognition phase, only the patterns are given to the network, and the network decides the classification of the patterns. Alternatively, when a part of a pattern and a classification are given, or just a part of a pattern is given, the network is asked to complete the pattern.

Click commands "Decimal/Classification" or "Decimal/Distribution", you can expect three possibilities for each recognition pattern:

Correct classification: this is usually the case when the Boltzmann Machine assigns one predominate probability to one of the output possibilities. 

Incorrect classification: this is usually the case when: the Boltzmann Machine assigns a predominate probability to more than one configuration; or the relative probability is "small". It is hard to define "small" here because it depends on the problem, but for a given problem, an experienced user will know what is "small".

No classification: this is usually the case when the Boltzmann Machine can find little correlation in the test pattern, based on the current training. In such a case, "No classification" is printed in the output data file.

 3.5   Test Input File Format

If you are not sure of the correctness of the training and recognition files, link the files and click: "Data/Test" to test the two files. If the formats of the files are not right, then the "Data/Test" command will tell you.

       
3.6   1-Neuron-1-Class Classifications

There is no restriction on how to represent a class using class neurons. The most common method to represent a class is to use one neuron for each class. Among class neurons, one of the neurons is '1'; all others are '0':

100 ... 00
010 ... 00
001 ... 00
...
000 ... 10
000 ... 01 Here each "1" represents a class classification. Of course, you can choose any way to represent a class.

We call the above representation as "1 neuron for 1 class" or 1N1C representation. If you do choose 1N1C representations, we have two more commands:

Decimal/1N1C Classification -------- similar to Decimal/Classification;

Decimal/1N1C Distribution --------- similar to Decimal/Distribution.

The reason to offer these two commands is that they are 50% to 100% faster than "Decimal/Classification" and "Decimal/Distribution" commands, especially for large problems.

If your problem is reasonably large, meaning:

  • Thousands of training patterns; or
  • Thousands of neurons; or
  • Combined symmetries;
then you should encode your problem in 1N1C representation. 1N1C representation is the preferred mode for character recognition and image recognition.
 

3.7   Training, Retraining and Recognition

Command "Decimal/Classification" is made up of two commands:

  • Decimal/Train.
  • Decimal/Recognition|Classification.
Clicking these two commands in order will produce the command, Decimal/Classification.

Command "Decimal/Distribution" is made up of two commands:

  • Decimal/Train;
  • Decimal/Recognition|Distribution. 
Command "Decimal/Retrain" is used to retrain the network. Retraining can not take place until training has been done first. You can have as many retraining as you wish. Every retraining looks for data from the training file. Make sure you offer fresh data to the PolyNet for each retraining.
 
  3.8   Binary Boltzmann Machine


PolyNet also supports the binary Boltzmann machine and the binary Hopfield model. These commands are exactly the same as ABM 2.7. 

The binary Boltzmann machine will treat '0' as the ground state and all the other digits (1, 2, ..., 9) as the excited state.
 
  3.9   Common Errors
Common errors are:

  • The number of neurons is out of range. To correct the problem, make the number larger than 5 and less than 10001.
  • The training and recognition files are not linked. To check, click "Data/Input". To correct the problem, click "Data/Link".
  • The formats of the training and recognition files are not right. To check, click "Data/Test".
  • PolyNet produces too much "garbage" (occupied memory no longer in use). To solve this problem, exit PolyNet to force "garbage collection".
When you run a large problem, it will be helpful if you close all programs and restart your computer, this will give the maximum recourses to the PolyNet.
 
 

Online Order or if you prefer, Online Fax order: 
PolyNet for Windows 95/98 
Version 4.0 ($124.99 + $5 US Shipping and Handling)

Mail Order: $129.99 (S&H included)

PolyNet 4.0
Attrasoft, Attn.: Gina
P. O. Box 13051
Savannah, GA. 31406, USA
Products New Lottery Stock jpg/gif Order Order Help Demos
Please join Attrasoft Discussion Group.
Send questions or comments to:
webmaster@attrasoft.com
or contact us at: 
          Attrasoft, P. O. Box 13051, Savannah, GA. 31406, USA
Copyright © 1998 Attrasoft, Inc. All rights reserved.