1.   Introduction

1.1   Neural Networks
1.2   Hopfield Model and Boltzmann Machine
1.3   History of PolyApplet

1.3.1   ABM 2.7
1.3.2   PolyNet 4.0
1.3.3   ImageFinder
1.3.4   DecisionMaker 2.5
1.3.5   Predictor 2.6

1.4   Training and Recognition
1.5   ABM2

---

1.   Introduction

Biological neurons are believed to be the structural constituents of the brain.  A neural network can:

• Learn by adapting its synaptic weights to changes in the surrounding environments;
• Handle imprecise, fuzzy, noisy, and probabilistic information; and
• Generalize from known tasks or examples to unknown ones.

Artificial neural networks (ANN) are an attempt to mimic some, or all, of these characteristics.

A neural network can learn. One of the neural net paradigms is supervised learning. In supervised learning, a neural net directly compares the network output with a known correct or desired answer in the training process.

1.1   Neural Networks

Neurons are living nerve cells that come together to form complicated biological neural networks. A typical human brain has 100 billion neurons. Each neuron is connected to 1,000 to 10,000 neurons. The human brain contains approximately 100 to 1,000 trillion interconnections.

An artificial neural network (ANN) is either software or hardware that can simulate biological neural networks. Artificial neural networks can perform computations and have learning abilities. These features distinguish neural network software from other software.

 Attrasoft PolyApplet is a .Net Class Library, which simulates two types of neural networks:

• Polytomous Hopfield Model (multi-valued Hopfield Model)
• Polytomous Boltzmann Machine (multi-valued Boltzmann Machine)

Attrasoft PolyApplet is a programming tool. You can design your neural network using Attrasoft PolyApplet and Microsoft Visual Studio .Net.  In addition, the software also provides the examples of using the Attrasoft PolyApplet to produce the following Attrasoft software:

• Predictor
• DecisionMaker
• PolyNet
• ABM (Attrasoft Boltzmann Machine)

1.2   Hopfield Model and Boltzmann Machine

One of the most popular networks is the Hopfield network. This recurrent net is completely connected. The Boltzmann Machine is closely related to the Hopfield model. The Boltzmann Machine is a special type of neural network, in which each neuron configuration has a certain probability to appear. (The name comes from the following fact: the Boltzmann Machine is a probabilistic neural network, which forms a Markov chain; the invariant distribution of the Markov chain is similar to the "Boltzmann Distribution" in statistical physics).

Most software is "programmed" to perform certain tasks. These tasks are fixed. For example, chess game software will not play solitary. Neural network software is not programmed; it is trained. Neural networks do what they are trained to do. What you train is what you get.

1.3   History of PolyApplet

 The Attrasoft PolyApplet is the class library used to build the following Attrasoft software:

• Predictor
• DecisionMaker
• PolyNet
• ABM (Attrasoft Boltzmann Machine)

The software also provides the examples of using the Attrasoft PolyApplet to produce the above software. In this section, we will briefly introduce the software.

1.3.1   ABM 2.7

To give the machine the ability to solve complex perceptual problems, Attrasoft developed its first generation neural software: ABM (Attrasoft Boltzmann Machine) for Windows 95/98. The first software, ABM 1.0, was released in 1995.

ABM is a software simulation of the Boltzmann Machine. ABM is not "programmed" for a particular task. ABM has to be "trained" to perform certain tasks; therefore, before you can use the software, you have to train it. It is a medium size neural network, designed to operate between 1,000 to 1,000,000 external neurons (external neurons mean input and output neurons; a hidden neuron is not an external neuron).

ABM:

• Nominated by PC Magazine for the 1996 Ziff-Davis Shareware Award;
• Simulates two powerful types of neural networks, the Boltzmann Machine and the Hopfield Model;
• If it fails to recognize a pattern, one retraining will fix it;
• If you train with the wrong information by accident or by design, retraining will correct it;
• Can handle conflicting training data;
• Supports 1D and 2D translation, scaling, and rotation symmetries;
• Works for any pattern recognition problem;
• Supports up to 10,000 external neurons;
• Includes more than ten examples to walk you through, including one with more than 4,000 classes;
• Learns more than 4,000 characters in 20 seconds and recognizes one of the 4,000 characters in 0.5 seconds, a speed unachievable by humans;
• The larger the ABM is, the more accurate the classification rates are.

The latest version of the software is ABM 2.7.

1.3.2   PolyNet 4.0

PolyNet 4.0 is similar to ABM 2.7, except it uses multi-valued neurons instead of binary neurons. With PolyApplet, you can create a simple version of the PolyNet 4.0, or ABM 2.7 in matter of minutes. PolyApplet supports 10,000 neurons. A larger neural net can also be ordered.

The PolyApplet has two key components, one for ABM 2.7 and the other for PolyNet 4.0.

1.3.3   ImageFinder

By far the most important applications are the Attrasoft ImageFinder, which is universal image recognition software. The Attrasoft ImageFinder has been used in many real world projects, including:

• Palm print recognition with a biometrics company;
• Stamp recognition for Process Automation with a SP 500 company;
• Document recognition with a document company;
• Quality Control (i.e., product labels) with a food company;
• Microscopic Image recognition with a medical equipment startup company;
• Face recognition with PriceWaterHouseCoopers;
• Content-based image recognition with a cell phone company;
• Automobile Wheel recognition with an auto part company;
• Automobile Chair recognition with an auto company.

Attrasoft has two class libraries:

PolyApplet 6.0; and
TransApplet 6.0.

The Attrasoft PolyApplet is responsible for:

• Predictor
• DecisionMaker
• PolyNet
• ABM (Attrasoft Boltzmann Machine)

The Attrasoft TransApplet is responsible:

• ImageFinder.

We will refer user’s to the TransApplet User’s Guide on how to build a software like the ImageFinder.

1.3.4   DecisionMaker 2.5

The Attrasoft DecisionMaker inferences a function, y = f(x), from data { (x1,y1), (x2,y2), …}, where x and y can represent many variables. It attempts to solve problems like this,

• “now all of your blood tests are back, how likely will you have a heart attack problem next year? During the next 5 years? …”.

The DecisionMaker is also called a vertical predictor; the Predictor is called a horizontal predictor.

1.3.5   Predictor 2.6

The Attrasoft Predictor will predict the next row of numbers in a sequence. It attempts to solve problems like this,

• “based on the history of the Dow Jones 30 index, what will be the index Tomorrow?  …”.

The DecisionMaker is also called a vertical predictor; the Predictor is called a horizontal predictor.

1.4   Training and Recognition

The ability to learn is a fundamental trait of intelligence. Neural nets can learn from examples. There are two basic phases in using PolyApplet:

• Training; and
• Recognition.

In the training phase, data is imposed upon a neural network to force the network to remember the pattern of training data. A neural network can remember the training data pattern by adjusting its internal synaptic connections.

In the recognition phase, a part of the input data is not known. The neural network, based on its internal synaptic connections, will determine the unknown part.

A typical problem for a neural network to solve is the classification problem. The data is a set of doublets: (pattern, classification). In the training phase, the network is taught which pattern belongs to which classification. In the recognition phase, only the patterns are given to the network, and the network decides the classification of the patterns. Alternatively, when a part of a pattern and a classification are given, or just a part of a pattern is given, the network is asked to complete the pattern.

1.5   ABM2

The neural network attempts to simulate the human brain. The human brain has front-end subsystems and can only handle preprocessed information. The front-end subsystems for humans are eyes, ears, ...  The same is also true for neural networks.

There are two types of data used by neural network systems: user data (or application data), and neural data. Neural networks use neural data. User data depends on the application. Neural nets speak neural language; users speak user language.

The information processed by a neural network has to be prepared by a front-end subsystem. This is called Data Encoding. A neural network cannot usually handle the user-application data directly. Similarly, after neural computation, the result usually does not make sense to humans directly; the front-end system is responsible for converting the neural output data back into user-application data.  This is called Data Decoding.

 A neural network system consists of the front-end subsystem and a neural network subsystem:

Front-End Subsystem

• Data Encoding: to convert application data to neural input data;
• Data Decoding: to convert neural output data back to application data.

Neural Network Subsystem

• Classification: patterns are given to the network, and the network decides the classification of the patterns;
• Pattern Fix: parts of a pattern and a classification are given, or just a part of a pattern is given; the network is asked to complete the pattern.

 Examples of Front-End Subsystems:

• Images to neural data;
• Video to neural data;
• Fax data;
• Various databases;
• Handwritten words;
• Scanned documents;
• Sound data;
• Stock Market;
• …

To summarize, the neural computation process has three stages: Data Encoding, Neural Computation, and Data Decoding. Data Encoding and Decoding are application-dependent, while the neural network is not application-dependent.

PolyApplet is a class library, which has several components:

• PolyApplet63 (“abm” + “polynet”)
• Abm2
• Predictor63, DecisionMaker63, PredictorPoly63, DecisionMakerPoly63

PolyApplet63 component is a neural network subsystem, i.e. a neural network simulator that is not application-dependent. It can be interfaced with any front-end systems to solve any problem.

Attrasoft PolyApplet is a brain to train for your specific problem.

Abm2 component is responsible for Data Encoding, and Data Decoding.

A typical neural net application will have three layers:

• GUI (Graphical User Interface);
• Translation Layer;
• Neural Network Layer.

The Predictor63, DecisionMaker63, PredictorPoly63, DecisionMakerPoly63 components are the GUI layer libraries.
 

Return