4.1 Linear Scaling Symmetry
We will study the linear scaling symmetry in this section. An example of linear scaling is the example in ABM, which is given as follows:
0000000000000000000000000000000000000000
0000000000000000000000000010000000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000111111111000000000
0000000000000000001111111111111111100000
0000000000000000000000000111100000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000000111000000000000
0000000000000000000000011111111000000000
0000000000000000000000000111100000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000001100000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000111111111111000000000
0000000000000001111111111111111111100000
0000000000000001111111111111111111100000
0000000000000000000000011111000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000000011110000000000000
0000000000000000000011111111111000000000
0000000000000000000011111111111000000000
0000000000000000000000011111000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000100000000000000000000
0000000000000000001110000000000000000000
0000000000000000011111000000000000000000
0000000000000000111111100000000000000000
0000000000000000111111110000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000011111111111111111111110000000000
0000000111111111111111111111111000000000
1111111111111111111111111111111111111100
1111111111111111111111111111111111111100
1111111111111111111111111111111111111100
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000000001111111111000000000000000
0000000000011111111111111111110000000000
0000000000011111111111111111110000000000
0000000000011111111111111111110000000000
0000000000000011111111111100000000000000
0000000000000001111111111000000000000000
0000000000000000000000000000000000000000
In the above pictures, 3 airplanes have 3 different scales.
Character recognition is another example, which has linear scaling symmetry. For example, a character of 'A' is :
00100
01010
01110
10001
10001
A linear scaling of the above 'A' is:
0000110000
0000110000
0011001100
0011001100
0011111100
0011111100
1100000011
1100000011
1100000011
1100000011
The non-linear scaling is:
0000100000
0000100000
0010001000
0010001000
0011111000
0010000100
1000000010
1000000010
1000000010
1000000010
ABM supports only linear scaling. The nonlinear
scaling can be ordered as a customized version.
Internal Parameters for Scaling Symmetries
The scaling symmetry depends on a large set of internal parameters, which are situation dependent. We set the parameters to work optimally in the range of 40x40 images to 60x60 images. Of course, it will work in other ranges too.
The scaling symmetries are computationally expensive, i.e., it requires a complicated internal structure. For N external neurons, the scaling requires O(N*N) internal neurons to get the maximum support. As a result, it significantly slows down the training and testing speed.
The scaling symmetries are sensitive to the internal parameters. For example, the parameters for 10x10 images are different from those for 100x100 images.
The scaling symmetry is very powerful and reduces the training data drastically.
There are several approaches to the scaling symmetry:
Partial support of the scaling symmetry: this approach
significantly speeds up the training and testing time, at some cost of
the performance.
One-dimension Scaling Symmetry Example
The scaling symmetry automatically implies the translation symmetry. The example for testing 1-dimensional scaling symmetry is called "1-D Segment Counter", and this example is under "Example/1D Segment Counter". The operation of this problem is very similar to the Shifter example. The nature of the problem is also very similar to the shifter problem: they are generated randomly. The problem can also be scaled to any neuron size, like the shifter problem.
The "1D Segment counter" is a very difficult problem for neural networks. This example counts the number of segments. For example, a string, 1111001111, has two segments: 1111 and 1111. Obviously, this problem has the scaling symmetry. If we scale down this pattern by 50%, the new pattern is: 1101100000. This new pattern still has two segments.
The class-neurons have 40 bits. This problem counts from 0 to 38 segments. For the class-neurons, 1000... represents 0 segment; 0100... represents 1 segment, ...
10000000... : 0 segment
01000000... : 1 segment
00100000... : 2 segments
00010000... : . . .
If the data has more than 38 segments, it is class 00 . . . 001.
The pattern-neurons are set to be 120 by default, but it can be changed by the user. The training patterns have 160 (40 + 120 = 160) neurons. The data is generated as follows: the probability of '1' is 34% and that of '0' is 66%. The segment length, which is the number of bits in a segment, is 4 or higher.
An example of the training pattern, which has 160 neurons, is:
0000010000000000000000000000000000000000
0000011111000000000111111110000000000000
0000011111111111000000000000000011111111
1111111111100000000000000000011111111100.
The first line is class-neurons, 40 bits total, and it
indicates the whole pattern has 5 segments. The pattern-neurons take the
next 3 lines and they have five segments. The size of the pattern-neurons
can be set to any number. This problem has 1D scaling symmetry.
The default values are:
120 pattern-neurons and 40 class-neurons (160 total),
250 training-patterns, and
50 testing-patterns.
This is an extremely difficult problem because of its large sample data space. To get better results, generate 5000 training-patterns and run again. The training and testing files are generated by clicking:
Using a Pentium 400 MHz, under the default setting, the run time for "Run/Classification" is about 5 seconds. The testing rate for 250 training patterns is not very high. The results can be improvement by increasing the training patterns or retraining. For example:
Click "Example/1D Segment Counter/Parameter" to set:
120 bits segment counter (160 neurons),
5000 training-patterns, and
50 testing-patterns.
2D scaling symmetry is computational expensive; to speed up the computation, please use 1N1C commands for this example.
The "2D Scaling Symmetry" has 40x40 images of 16 airplanes in the training file. To generate the example, click "Example/2D Scaling symmetry". In the testing file, 10 images of the airplane are scaled up and down. Use the command "Run/1N1C Classification" to classify the planes. The run time is 15 seconds (Pentium 400 MHz).
The rotation symmetry is similar to scaling symmetry:
they are both sensitive to the internal parameters. Again, in designing
the rotation symmetry, we seek the balance between the quality of the performance
and the speed.
ABM supports z-direction rotation symmetry in the xy-plane. The rotation symmetry naturally implies the x- and y-translation symmetries. There are two types of rotation symmetries:
360° (Complete) Rotation Symmetry
Partial Rotation Symmetry (For example, -10° to 10° )
The complete rotation is computationally more expensive than the partial rotation symmetry. ABM supports the complete rotation symmetry. ABM currently supports two different settings of the rotation symmetry:
Rotation + translation
Rotation + scaling + translation
An example of rotation symmetry is in ABM, which can be opened by clicking:
"Example/2D Rotation + Translation".
Two images are given as follows (one is the rotated image of the other):
0000000000000000010000000000000000000000
0000000000000000010000000000000000000000
0000000000000000111000000000000000000000
0000000000000000111000000000000000000000
0000000000000000111000000000000000000000
0000000000000001111100000000000000000000
0000000000000001111100000000000000000000
0000000000000001111100000000000000000000
0000000000000011111110000000000000000000
0000000000000111111111000000000000000000
0000000000001111111111100000000000000000
0000000000001111111111100000000000000000
0000000000011111111111110000000000000000
0000000000111111111111111000000000000000
0000000001111111111111111100000000000000
0000000011111111111111111110000000000000
0000000011111111111111111110000000000000
0000000111111111111111111111000000000000
0000001111111111111111111111100000000000
0000011111111111111111111111110000000000
0000111111100011111110001111111000000000
0001111110000011111110000011111100000000
0011111000000011111110000000111110000000
0011100000000011111110000000001110000000
0010000000000011111110000000000010000000
0000000000000011111110000000000000000000
0000000000000001111100000000000000000000
0000000000000011111110000000000000000000
0000000000000111111111000000000000000000
0000000000001111111111100000000000000000
0000000000011111111111110000000000000000
0000000000111111111111111000000000000000
0000000000111001111100111000000000000000
0000000000000001111100000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000110000000000000000000000000
0000000000000110000000000000000000000000
0000000000000111000000000000000000000000
0000000000000111100000000000000000000000
0000000000000111110000000000000000000000
0000000000000111110000000000000000000000
0000000000000111111100000000000000000000
0000000000000111111111000000000000000000
0000000000000111111111110000000000000000
0000000000000111111111111000000000000000
0000000000000111111111111110000000000000
0000000000001111111111111111100000000000
0000000000001111111111111111110000000000
0000000000011111111111111111111100000000
0000000000011111111111111111111111000000
0000000000011111111111111111111111100000
0000000000011111111111111111111111110000
0000000000111111111111110000011111111000
0000000000111111111111110000000000000000
0000000001111111111111111000000000000000
0000000001111111001111111000000000000000
0000000001111110001111111100000000000000
0000000011111000001111111100000000000000
0000000011111000000111111100000000000000
0000000011010000000111111110000000000000
0000000011100000000011111111000000000000
0000000111000000000011111111110000000000
0000000110000000000011111111111100000000
0000000110000000000111111111111100000000
0000000000000000000111111111000000000000
0000000000000000001111111111100000000000
0000000000000000001110011111000000000000
0000000000000000001110011000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
These pictures look odd because '1' and '0' are longer in the y-direction than in the x-direction.
Rotation and Scaling Symmetries
Rotation symmetry naturally implies x- and y-translation symmetries. Rotation symmetry can be combined with scaling symmetry. In the above example, after rotation and scaling of the above images, an image is:
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000001100000000000000000
0000000000000000001111100000000000000000
0000000000000001111111000000000000000000
0000000000001111111111000000000000000000
0000000111111111111111000000000000000000
0000111111111111111111000000000000000000
0001111111111111111111000000000000000000
0000000000111111111111000000000000000000
0000000000111111111111000000000000000000
0000000000111111111110000000000000000000
0000000000111111111110000000000000000000
0000000000111111111110000000000000000000
0000000011111111011110000000000000000000
0000111111111110011110000000000000000000
0001111111111000011110000000000000000000
0000111111111000011110000000000000000000
0000001111110000001110000000000000000000
0000001111110000001110000000000000000000
0000000110111000001111000000000000000000
0000000000011000000011000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
Internal Parameters for Rotation Symmetries
The rotation symmetry depends on a large set of internal parameters, which are situation dependent. We set the parameters to work optimally in the range of 40x40 images to 60x60 images. Of course, it will work in other ranges too.
The rotation symmetries are computationally expensive, i.e., it requires a complicated internal structure. For N external neurons, the rotation symmetry requires O(N) internal neurons to get the maximum support. As a result, it significantly slows down the training and testing speed.
The combined rotation and scaling symmetries are computationally expensive, i.e., it requires a complicated internal structure. For N external neurons, the combined symmetry requires O(N*N*N) internal neurons to get the maximum support. As a result, it significantly slows down the training and testing speed.
The rotation symmetries are sensitive to the internal parameters. For example, the parameters for 10x10 images are different from those for 100x100 images.
The rotation symmetry is very powerful and reduces the
training data drastically.
There are two examples, which test the rotation symmetry:
2D Rotation + Translation; and
2D Rotation + Scaling
The "Example/2D Rotation + Translation" example has 40x40 images of 16 airplanes as training data. In the testing file, 11 images of the airplane are pointed at various directions. The run time for "Run/Classification" is 30 seconds and the run time "Run/1N1C Classification" is less than 20 seconds, using a Pentium 400 MHz and 90 MB RAM. You must have 60 MB hard disk space to run this example, otherwise, the software will quit.
The "Example/2D Rotation + Scaling" example has 40x40 images of 3 airplanes as training data. In the testing file, 10 images of the airplane are pointed at various directions and scaled up and down. 2D rotation and scaling symmetry is computational expensive; to speed up the computation, please use 1N1C commands for this example.
This is a long example; the training time is about 2 minutes, and the testing time is 1.5 minutes per image. If you use the default testing file which has 10 images, it will take 15 minutes. You must have 200 MB hard disk space to run this example, otherwise, the software will quit. You might want to delete 9 images from the testing file and leave only 1 image in the testing file to see how long it will take. The command "Run/1N1C Classification" for 1 testing image will take 3.5 minutes (2 minutes for training and 1.5 minutes for recognition).
Note, these two examples take significantly longer time than all other examples. This is an indication that the combined symmetries are computationally expensive. It is important that you have enough hard disk storage space for the execution of ABM software, otherwise, the program will quit.
The rotation and scaling symmetries are not optimized.
If you want to have these symmetries optimized, a customized version has
to be ordered.