Attrasoft Boltzmann machine (ABM)/chap. 1

1. Introduction

1.1 Neural Networks
1.2 Hopfield Model and Boltzmann Machine
1.3 Why ABM ?
1.4 ABM 2.7
1.5 ABM Applications
1.6 Training and Recognition
1.7 Neural Network System
1.8 ABM's Version

1 Introduction

Biological neurons are believed to be the structural constituents of the brain. A neural network can:

learn by adapting its synaptic weights to changes in the surrounding environments;
handle imprecise, fuzzy, noisy, and probabilistic information; and
generalize from known tasks or examples to unknown ones.

Artificial neural networks (ANN) are an attempt to mimic some, or all, of these characteristics.

A neural network can learn. One of the neural net paradigms is supervised learning. In supervised learning, a neural net directly compares the network output with a known correct or desired answer in the training process.

1.1 Neural Networks

Neurons are living nerve cells that come together to form complicated biological neural networks. A typical human brain has 100 billion neurons. Each neuron is connected to 1,000 to 10,000 neurons. The human brain contains approximately 100 to 1,000 trillion interconnections.

An artificial neural network (ANN) is either software or hardware that can simulate biological neural networks. Artificial neural networks can perform computations and have learning abilities. These features distinguish neural network software from other software.

When you bought ABM, you bought software that simulates a neural network. This is a blank neural net and there is nothing stored in the network. Your job is to train the network. After that, the ABM is ready to serve you.

A neural network is characterized by:

network topology,
connection strength between neurons,
node properties,
internal controls, and
the updating rules.

In ABM, the only allowed neuron states are the ground state, and the excited state, mathematically represented by 0 and 1, respectively. To train a network, training data is provided to the network. Training data consists of many patterns like this:

00100

01100

00100

01110.

This is an image of a 5x7 character, '1'. These training patterns will influence the network. Every time the network looks at a training pattern, the network stores the information by modifying the neuron synaptic connections. Modifying the values of the connections represents a learning process: the neural networks learn their environment by changing their internal connections. After a while, these synaptic connections hold certain values. These values represent the neural network's memory and it can be used to perform certain tasks.

1.2 Hopfield Model and Boltzmann Machine

One of the most popular networks is the Hopfield network. This recurrent net is completely connected. The Boltzmann Machine is closely related to the Hopfield model. The Boltzmann Machine is a special type of neural network, in which each neuron configuration has a certain probability to appear. (The name comes from the following fact: the Boltzmann Machine is a probabilistic neural network which forms a Markov chain; the invariant distribution of the Markov chain is similar to the "Boltzmann Distribution" in statistical physics).

Most software is "programmed" to perform certain tasks. These tasks are fixed. For example, chess game software will not play solitary. Neural network software is not programmed; it is trained. Neural networks do what they are trained to do. What you train is what you get.

1.3 Why ABM ?

Figure 1. Attrasoft Boltzmann Machine.

Modern digital computers outperform humans in the domain of numerical computation and related symbol manipulation. However, humans can effortlessly solve complex perceptual problems, like recognizing a person in a crowd from a mere glimpse of his face, so quickly that it would dwarf the world's fastest computer.

ABM shows signs that even in perceptual problems, computers are catching up. ABM can learn more than 4,000 characters in 20 seconds and recognize one of the 4,000 characters in 0.5 second.

There are several reasons you should choose ABM:

ABM is large: ABM supports up to 10,000 neurons. Customized software can increase the number of neurons to 1,000,000.
ABM is fast: ABM can learn more than 4,000 characters in 20 seconds and recognize one of the 4,000 characters in 0.5 seconds.
ABM is accurate: ABM can solve any problem (for example, image recognition, prediction of stock market and Internet image search engine).
ABM is neural-net-size invariant, i.e., you can encode a problem with a smaller net and a larger net; the results for the larger net is equally good as the smaller net.

1.4 ABM 2.7

Attrasoft Boltzmann Machine for Windows 95/98 (ABM for Windows 95/98) is a software simulation of the Boltzmann Machine. ABM is not "programmed" for a particular task. ABM has to be "trained" to perform certain tasks, therefore, before you can use the software, you have to train it. It is a medium size neural network, designed to operate between 1,000 to 1,000,000 external neurons (external neurons mean input and output neurons; a hidden neuron is not an external neuron).

ABM 2.7:

Nominated by PC Magazine for the 1996 Ziff-Davis Shareware Award;
Simulates two powerful types of neural networks, the Boltzmann Machine and the Hopfield Model;
If it fails to recognize a pattern, one retraining will fix it;
If you train with the wrong information by accident or by design, retraining will correct it;
Can handle conflicting training data;
Supports 1D and 2D translation, scaling, and rotation- symmetries;
Works for any pattern recognition problem;
Supports up to 10,000 external neurons;
Includes more than ten examples to walk you through, including one with more than 4,000 classes;
Learns more than 4,000 characters in 20 seconds and recognizes one of the 4,000 characters in 0.5 seconds, a speed unachievable by humans;
ABM does not lose focus or clarity as the net gets larger.

ABM 2.7 will allow a maximum number of 10,000 external neurons to be used. There are two basic types of tasks we expect the software to perform:

Classification: Given a pattern, find its class;
Determine a Pattern: Given a classification and part of a pattern, complete the pattern.

1.5 ABM Applications

ABM is the brain for all Attrasoft software. The applications are:

Content-based Image Retrieval for local drive

Attrasoft ImageFinder for Window 95/98, v3.4, will do this. Application examples are:

Finger print recognition;
Signature recognition;
Handwriting zip code recognition.

Content-based Image Retrieval for the Internet Attrasoft Internet ImageFinder, v3.3, will do this. Application examples are:

Internet Image Search Engine.

Image Recognition Attrasoft ImageClassifier 3.1 will do this. Application examples are:

Finger print recognition;
Signature recognition;
Handwriting zip code recognition.

Time Sequence Prediction: Attrasoft Predictor 2.6 will do this. Application examples are:

Predict Stock Market
Predict earnings and revenue of a company
Predict short term and long term interest rates
Predict commodity (gold, oil, corn,...) prices
Predict regional/worldwide price fluctuations for a particular merchandise item
Predict various indicators/indexes of the economy for strategic thinking and policy issues
Predict Lottery Numbers
Predict Dynamic Systems
Predict Markov Chains
Predict other complex systems

Function Inference Attrasoft DecisionMaker 2.5 will do this. Application examples are:

Decide whether to grant a loan to a customer, student *, ...
Decide whether to hire a person or not
Decide whether a patient has cancer *, heart disease *, thyroid disease *, ...., or not
Decide how likely a patient is going to get cancer, heart disease *...
Decide the classification of a virus/bacteria within a group of virus/bacteria *, ...
Decide how much a house *, a car *, a boat, ... , is worth
Decide whether your supervisors evaluate your employees fairly
Decide whether your system (electronics systems, computer network, retail store, sexual harassment situation) is healthy
Decide regional supermarket price predictability based on accumulated database, ...
Decide which career a child should choose
Decide the outcome of a random event, like a horse race, ...
Identify an item in Forensic Science *, ...
ETC....

*Examples based on real data are in DecisionMaker 2.5. Sound Recognition

Will be available.

Internet Image Search Engine

Will be available soon.

1.6 Training and Recognition

The ability to learn is a fundamental trait of intelligence. ANN can learn from examples. As an ANN simulator, ABM can also learn from examples. ABM is a blank artificial neural network. ABM has to be trained before it can perform even one task for you. There are two basic phases in using ABM:

Training; and
Testing (or Recognition).

In the training phase, data is imposed upon a neural network to force the network to remember the pattern of training data. A neural network can remember the training data pattern by adjusting its internal synaptic connections.

In the recognition or testing phase, a part of the input data is not known. The neural network, based on its internal synaptic connections, will determine the unknown part.

A typical problem for a neural network to solve is the classification problem. The data is a set of doublets: (pattern, class). In the training phase, the network is taught which pattern belongs to which class. In the testing phase, only the patterns are given to the network, and the network decides the classification of the patterns. Alternatively, when a part of a pattern and a classification are given, or just a part of a pattern is given, the network is asked to complete the pattern.

1.7 Neural Network System

The neural network attempts to simulate the human brain. The human brain has front-end subsystems and can only handle preprocessed information. The front-end subsystems for humans are eyes, ears, ... The same is also true for neural networks.

There are two types of data used by neural network systems: user data (or application data), and neural data. Neural networks use neural data. User data depends on the applications. Put it in another way: neural nets speak neural language, users speak user language.

The information processed by a neural network has to be prepared by a front-end subsystem. This is called data encoding. A neural network can not usually handle the user-application data directly. Similarly, after neural computation, the result usually does not make sense to humans directly; the front-end system is responsible for converting the neural output data back into user-application data. This is called data decoding.

Front-end systems are basically a language translator. The ABM has several interfaced packages:

Predictor 2.6
DecisionMaker 2.5
Internet ImageFinder 3.3
ImageFinder 3.4
ImageClassifier 3.1

A neural network system consists of the front-end subsystem and a neural network subsystem:

Front-end subsystem

Data encoding: to convert application data to neural input data;
Data decoding: to convert neural output data back to application data.

Neural network subsystem

Classification: patterns are given to the network, and the network decides the classification of the patterns;
Pattern Fix: parts of a pattern and a classification are given, or just a part of a pattern is given, the network is asked to complete the pattern.

Examples of front-end subsystems:

JPEG Images to neural data;
GIF images to neural data;
Fax data;
Various databases;
Handwritten words;
Scanned documents;
Sound data;
Stock Market.

ABM is a neural network subsystem. It takes the text files as inputs and produces the text files as neural output. Several interfaced software can be ordered from Attrasoft. Customized software can be ordered for any application.

To summarize, the neural computation process has three stages: Data encoding, Neural Computation, and Data Decoding. Data encoding and decoding are application-dependent, while the neural network is not application-dependent. ABM is a neural network simulator that is not application-dependent. It can be interfaced with any front-end systems to solve any problem.

Attrasoft ABM is a brain to train for your specific problem.

1.8 ABM's Version

The ABM has several versions:

Version Neurons prices

_____________________________________________

Standard version         10,000                             $99.99
100K version             100,000                             $499
250K version             250,000                             $999
1M version                 1,000,000                         $9999
Other customized versions                                 $999 and up

Consulting fee:

The free customer support covers how to operate your ABM.

If you need help on how to encode your problem, the consulting fees are:

$250 per problem if the problem has 1000 rows or less;
$500 per problem if the problem has 1,001-10,000 rows or less;
$1000 per problem if the problem has more than 10,000 rows.