About Attrasoft TransApplet
TransApplet is a .Net Class Library that enables System Integrators, Solution Developers, and Individuals to quickly add Image Recognition capability to their products and services.
Software Requirements
The software requirements are:
- Microsoft Visual Studio .Net, or
- C#.Net, or
- VB.Net
Installing the Software
Copying the “CD:\transapplet80” to the C-driver will complete the installation.
Information & Support
Attrasoft TransApplet
Attrasoft
P. O. Box 13051
Savannah, GA, 31406
USA
imagefinder@attrasoft.com
(Email Subject: Attrasoft)
Phone: (912) 484-1717
© Attrasoft 1998 - 2011
License Agreement
THIS LICENSE AGREEMENT ("AGREEMENT") IS BETWEEN YOU, THE END USER, AND Attrasoft. IT GOVERNS THE USE OF THE SOFTWARE PROGRAM AND DOCUMENTATION KNOWN AS Attrasoft TransApplet (THE "PRODUCT"). IF YOU USE THE PRODUCT, THEN YOU AGREE TO THE TERMS OF THIS AGREEMENT. IF YOU ARE NOT WILLING TO BE BOUND BY THE TERMS OF THIS AGREEMENT, PROMPTLY RETURN THIS PACKAGE TO THE PLACE OF PURCHASE WITH A COPY OF THE RECEIPT, AND YOUR LICENSE FEE WILL BE REFUNDED.
Attrasoft licenses use of the PRODUCT, which may be manufactured and distributed by Attrasoft or by a third party (either, the "Manufacturer"). You, the end-user, assume responsibility for the selection of the PRODUCT to achieve your intended results, and for its installation and subsequent use.
GRANT OF LICENSE
Attrasoft hereby grants you
a non-exclusive license to use the PRODUCT in object code form only, upon the
terms and conditions contained in this Agreement.
You may:
1. Use the PRODUCT
on the number of workstations for which you have purchased PRODUCT licenses. The
workstations must be owned, leased or otherwise controlled by you, whether in a
network or other configuration.
2. Create a
quantity of backup copies of the PRODUCT, in any machine-readable or printed
form, equal to the number of PRODUCT licenses you have purchased.
3. Transfer the
PRODUCT and your rights under this Agreement to another party if the other
party agrees to accept the terms and conditions of this Agreement. If you
transfer the PRODUCT, you must, at the same time, either transfer all copies of
PRODUCT to the same party, or destroy any copies not transferred. You must
immediately notify Attrasoft of the transfer.
4. Print out one
copy of the Product documentation from the Attrasoft program, TransApplet,
for each license purchased. If you print out any part of the Product
documentation from the Attrasoft program, TransApplet, you must
reproduce and include all the copyright notices that appear in the
documentation on any such copy of the documentation.
You May Not:
1. Use or copy the
PRODUCT, in whole or in part, except as expressly provided in this Agreement.
2. Use the PRODUCT
concurrently on more than the number of workstations for which you have
purchased licenses.
3. Copy, rent,
distribute, sell, license or sub-license, or otherwise transfer the PRODUCT or
this license, in whole or in part, to another party, except as specifically set
forth above.
4. Incorporate the
PRODUCT or any portion of the PRODUCT into, or use the PRODUCT, or any portion
of the PRODUCT to develop, other software without a license from Attrasoft, or
otherwise modify or create a derivative work from the PRODUCT without a license
from Attrasoft.
5. Reverse
engineer, decompile, or disassemble the PRODUCT.
To use the PRODUCT
as described in Sections 2 or 4 above, or for any other use not specifically
set forth above, additional licensing from Attrasoft is required. For further
information, please contact Attrasoft at:
Attrasoft,
Inc.
Phone: (912)
484-1717
gina@attrasoft.com
PROPRIETARY RIGHTS
This Agreement
gives you ownership only of the physical Attrasoft program, TransApplet,
on which the PRODUCT is stored, but not of the PRODUCT itself. You acknowledge
that Attrasoft owns all rights, title, and interest in the PRODUCT, and that
you will acquire no rights in the PRODUCT through your use of it. You agree
that you will take no action that interferes with Attrasoft's rights in the
PRODUCT.
TERMS
This Agreement is
effective until terminated. You may terminate it at any time by destroying the
PRODUCT together with all copies and documentation in any form. This Agreement
will also terminate automatically and without notice from Attrasoft if you fail
to comply with any term or condition of this Agreement. You agree upon such
termination to destroy the PRODUCT and all copies of the PRODUCT.
DISCLAIMER; LIMITED WARRANTY
EXCEPT AS PROVIDED
BELOW, THE PRODUCT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND,
EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE
RISK AS TO THE QUALITY AND PERFORMANCE OF THE PRODUCT IS WITH YOU. SOME
JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE
EXCLUSION MAY NOT APPLY TO YOU. NEITHER ATTRASOFT NOR MANUFACTURER WARRANTS THAT
THE FUNCTIONS CONTAINED IN THE PRODUCT WILL MEET YOUR REQUIREMENTS OR THAT THE
OPERATION OF THE PRODUCT WILL BE UNINTERRUPTED OR ERROR-FREE. However, where
Attrasoft is the Manufacturer, Attrasoft warrants that the Attrasoft program, TransApplet,
on which the software is furnished will be free from defects in materials and
workmanship under normal use for a period of ninety (90) days from the date of
delivery as evidenced by a copy of your receipt.
LIMITATION OF REMEDIES
Where Attrasoft is
the Manufacturer, Manufacturer’s entire liability and your exclusive remedy
shall be:
1. The replacement
of the Attrasoft program, TransApplet, not meeting the Limited Warranty,
which is returned to Manufacturer with a copy of your receipt.
2. If Manufacturer
is unable to deliver replacement Attrasoft program, TransApplet, which
is free of defects in materials or workmanship, you may terminate this
Agreement by returning the PRODUCT and a copy of your receipt to the place of
purchase, and your money will be refunded. Where Attrasoft is not the
Manufacturer, Attrasoft shall have no liability to replace or refund, and you
agree to look to Manufacturer to meet the obligations described above.
LIMITATION OF LIABILITY
IN NO EVENT WILL
ATTRASOFT OR MANUFACTURER BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT
LIMITED TO, ANY LOST PROFITS, LOST SAVINGS, OR OTHER INDIRECT, SPECIAL,
EXEMPLARY, INCIDENTAL OR CONSEQUENTIAL DAMAGES, ARISING OUT OF THE USE OR
INABILITY TO USE THIS PRODUCT, EVEN IF ATTRASOFT OR MANUFACTURER HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. FURTHER, IN NO EVENT WILL ATTRASOFT
OR MANUFACTURER
BE LIABLE FOR ANY CLAIM BY ANY OTHER PARTY ARISING OUT OF
YOUR USE OF THE PRODUCT. SOME JURISDICTIONS DO NOT ALLOW THE LIMITATION OR
EXCLUSION OF LIABILITY FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THE ABOVE
LIMITATION OR EXCLUSION MAY NOT APPLY TO
YOU.
TRADEMARKS
Attrasoft is a trademark of Attrasoft, Inc. Microsoft,
C# logo are registered trademarks of Microsoft Corporation. No rights, license,
or interest in such trademarks is granted hereunder.
U.S. GOVERNMENT RESTRICTED RIGHTS IN DATA
This computer
software product and documentation are provided with Restricted Rights. Use,
duplication, or disclosure by the U.S. Government is subject to restrictions as
set forth in the Rights in Technical Data and Computer Software clause at DFARS
252.227-7013, or subparagraphs (c)(1) and (2) of the
Commercial Computer Software - Restricted Rights at 48 CFR 52.227-19, as
applicable. Contractor/Manufacturer is Attrasoft, Inc.
EXPORT CONTROLS
You agree not to
export or re-export the PRODUCT, directly or indirectly, to any countries,
end-users or for any end uses that are restricted by U.S. export laws and
regulations, without first obtaining permission to do so as required by the
U.S. Department of Commerce's Bureau of Industry and Security, or other
appropriate government agency. These restrictions change from time to time. If
you have any questions regarding your obligations under U.S. export
regulations, you should contact the Bureau of Industry and Security, U.S.
Department of Commerce, Exporter
Counseling Division,
Washington D.C. (202) 482-4811, http://www.bis.doc.gov.
TABLE OF CONTENTS
1.3 Installing
the TransApplet
1.4 Attrasoft
TransApplet Class Library
2.1 Attrasoft
Image Recognition Basics
2.2 Image
Recognition Internal Structures
6.3 Set Up the
Chapter Project
7.2
Implementing H, >, and < Buttons
8.4 Codes for
Each of the Steps
9.6 Signature
File Implementation
10.1 Whole
Image Matching Menu
10.3 N:N
Matching Implementation
10.4 1:N Matching Implementation
12.3 Creating
A Master Library
14.1
Sub-Images and Variations
16.3 Set Up
the Chapter Project
20.2 Image
Preprocessing & Processing
20.4 Neural
Filter and Neural Net Filter
21.1 Image
Preprocessing Filter
21.7 Skip the
Empty Border by Content Percent
22.3 Set Image
Processing Filters
24.1 Pushing
Images Through Filters
24.3 Grouping
Parameters Together
25.5
Unsupervised Filterand BioFilter
25.7.8
Internal/External Weight Cut
25.7.11 Use
BioFilter & Use Neural Filter
26. ImageFinder Support Service Packages
26.1 Basic
Annual Maintenance & Support Package
26.3 What is a
Feasibility Study?
1. Introduction
Attrasoft TransApplet for Windows is a .Net Class Library that enables the addition of Image Recognition capability to products & services. It can be used for:
- Image Verification (1:1 Matching);
- Image Identification (1:N Matching);
- Image Search or Retrieval (1:N Matching);
- Multiple Verification or Identification (N:N and N:M Matching); and
- Video Matching
1.1 What is TransApplet?
Attrasoft TransApplet for Windows is a .Net Class Library that enables the addition of Image Recognition capability to products & services.
Before you continue, you should be familiar with the Attrasoft ImageFinder for two reasons:
- Learn the structure of the Attrasoft Image Recognition Approach;
- Learn the ImageFinder software.
The TransApplet will introduce the class library, “Attrasoft.TransApplet80”. This library is used in the ImageFinder. As a teaching tool, the TransApplet will show you how to build an ImageFinder via C# .Net. Therefore, if you have not learned the ImageFinder, you should start with the ImageFinder now.
The entire ImageFinder C# project is in the software. It is located in the ImageFinder subfolder. You can run the ImageFinder from there.
1.2 Software Requirements
The software requirements are:
- Microsoft Visual Studio .Net, or
- C#.Net, or
- VB.Net
In addition, we assume users understand C# or VB programming.
1.3 Installing the TransApplet
Copying the “CD:\transapplet80” to the C-driver will complete the installation. After copying, you should have both the Library Files and the Chapter Example.
1.4 Attrasoft TransApplet Class Library
The TransApplet library has many *.dll files:
C:\transapplet80\xyz80.dll
…
Each Library file has one
main library, which has the same name as the file name:
Attrasoft.TransApplet80.xyz80
Each Library has one main
class, which has the same name as the file name:
Attrasoft.TransApplet80.xyz80.xyz80
The advantage of making
class-name = library-name = file-name
is that you will know what to use. If you go to a file xyz80.dll, you will use Library, “Attrasoft.TransApplet80.xyz80”, and you will use class, xyz80. There is no guessing which class you should choose if the Library has more than one class.
The disadvantage is
that the compiler will not be able to distinguish between the class-name and
the Library-name unless you use the full name.
Xyz80 alone is not
understandable by the C# compiler. To declare an object, you must use the full
path:
Attrasoft.TransApplet80.xyz80.xyz80 myObject80;
Each chapter has a project. Many chapters may share the same project. These projects provide an integration source code so you can cut and paste to produce your own application.
1.5 Linking the TransApplet
To link to these Libraries:
- Start C# project;
- Right click References in Solution Explorer and select Add Reference;
- Click Browse Tab;
- Select “c:\transapplet80\*.dll” files;
- Click the “OK” button.
2. Image Recognition Overview
2.1 Attrasoft Image Recognition Basics
The Attrasoft Image Recognition Approach has two basic phases:
- Whole Images
- Sub-Images
In matching whole images, a
unique image signature is computed for each image. Different images will have
different signatures. The Signature Matching is based on these image
signatures. This matching can easily reach the speed of 100,000 to 1,000,000
whole images per second and is very fast.
In the Image Segment matching, image signatures may or may not be used. If no image signatures are used, the recognition speed is much slower.
Image Matching is further divided into filters. It is these filters that will perform the image matching tasks.
2.2 Image Recognition Internal Structures
An image recognition application is roughly divided into:
Level 5: User Interface;
Level 4: Data Management;
Level 3: Image-Matching Layer;
Level 2: Scanner, Camera, and their Drivers;
Level 1: PC with Windows.
The TransApplet Library will be used to implement the Image-Matching layer. Stand-alone software, like the Attrasoft ImageFinder, is roughly divided into 3 layers:
Level 3: User Interface;
Level 2: Image-Matching Layer;
Level 1: PC with Windows.
3. TransApplet Overview
ImageFinder for Windows is an off-the-shelf Application Software that enables System Integrators, Solution Developers, and Individuals to quickly test their own Image Recognition ideas.
VideoFinder for Windows is an off-the-shelf Application Software that enables System Integrators, Solution Developers, and Individuals to quickly test their own video Recognition ideas.
TransApplet is .Net Class Library that enables System Integrators, Solution Developers, and Individuals to quickly add Image Recognition capability to their products and services.
This User’s Guide will show you how to build software like the ImageFinder using C# .Net. If you have not learned how use to the ImageFinder, you should start with the ImageFinder now. The entire ImageFinder project is located in the ImageFinder sub-folder.
The TransApplet consists of a set of class libraries. Each chapter will introduce one of these libraries as you continue to build the “ImageFinder”. Some minor graphical user interface issues will be ignored here because it is already complicated enough.
3.1 TransApplet API
Chapter 4, API,
describes the API (Application Programming Interface) of the
“Attrasoft.TransApplet80” Class Library.
3.2 User Interface
Chapter 5, User Interface, briefly describes how to set up the forms and the menu bar for the ImageFinder, and also how to link to class libraries.
3.3 Input & Display
Chapter 6, Input, briefly describes how to enter data to the ImageFinder. The input images are further divided into:
- Key image – the sample image the ImageFinder will match against;
- Search Source – a list of previously stored images to be searched.
An image path specifies the training image. A search-folder specifies the Search Source.
Chapter 7 will also briefly describe how to display the images. The Image Display will play the following role:
- Display Key Segment;
- Display the first image in the search source;
- Display the next image and the previous image in the search source.
- Display the first and next image in the matching results.
- Display processed images (how an image is viewed by the software).
3.4 ImageFinderLite
Chapter 8, ImageFinderLite, briefly describes how to build the software, ImageFinderLite.
3.5 Signature Recognition
Chapter 9, Image Signatures, will describe how to generate image signatures.
3.6 Whole Image Matching
Chapter 10, Whole Image Matching, will describe the steps for whole image matching.
Chapter 11, Binary Search, will describe the steps for binary whole image matching.
3.7 Dynamic Library
Chapter 12, Dynamic Library, introduces the dynamic library used in the ImageFinder. Dynamic Library allows the master library to be updated via insertion, deletion, and update.
3.8 Image Segment Matching
Chapter 13, Image
Variation, will describe how to generate image signatures for some simple
variations, such as rotation and flips.
Chapter 14, Sub-Image Matching,
introduces the sub-image matching used in the ImageFinder.
Chapter 15, Fast Sub-Image Matching, gives
a more options for matching sub-images.
3.9 Videos
Chapter 16, Video Input,
briefly describes how to enter video data to the ImageFinder.
Chapter 17, Whole Video Matching, introduces matching with a whole video as a unit.
Chapter 18, Video Frame Matching, introduces matching with a video frame as a unit.
Chapter 19, Video Object Matching, introduces matching with an object inside a video frame as a unit.
3.10 Image Processing
Chapter 20, Image Preprocessing Concept, chapter 21, Image Preprocessing and chapter 22, Image Processing, introduces the image processing process required for the ImageFinder.
Chapter 23, Normalization, introduces image representation required for the ImageFinder.
Chapter 24 and Chapter 25 discuss the internal parameters in the ImageFinder.
3.11 Customized Software
Chapter 26, Development Process, introduces the Image Recognition Implementation Process using the TransApplet.
4. API
This chapter will introduce the interfaces of the important classes and important functions. The interface for other classes will be introduced in the later chapters as they are used.
4.1 Image Signature
Whole Image Matching is done through Image Signature. An image has a set of computed values called features. A collection of features is grouped into a signature.
This section introduces the Image Signature interface. An Image Signature consists of the following attributes:
ID
Name (For example, xyz.jpg)
Path (c:\abc\)
Attribute 1
Attribute 2
…
The interface for Image Signature is:
public interface I_ImageSignature
{
int getStatus ();
string getID();
string getImageName ();
string getImagePath();
string getAbsolutePath ();
int getNumberOfAttributes ();
int [] getSignatureAttributes();
int getSignatureAttributes (int index);
string toString ();
}
The following table lists the functions.
|
Function |
Descriptions |
Comments |
|
int getStatus () |
Returns the status associated with the signature. Output: 1: signature ready. 0: signature not ready. -1: no image -2: image segmentation specification error -3: other error. |
|
|
string getID() |
Returns the ID associated with the signature. Output: image ID. |
|
|
string getImageName () |
Returns the image name associated with the signature. Output: image name. |
|
|
string getImagePath() |
Returns the image path associated with the signature. Output: image path. |
|
|
string getAbsolutePath () |
Returns the absolute path associated with the signature. Output: image absolute path. |
|
|
int getNumberOfAttributes () |
Returns the number of attributes associated with the signature. Output: number of attributes. |
|
|
int [] getSignatureAttributes() |
Returns the attribute array associated with the signature. Output: attribute array. |
|
|
int getSignatureAttributes() (int index) |
Returns the attribute associated with the input index. Input: index. Output: attribute associated with the input index. |
|
|
string toString () |
Returns the entire image signature as a string with fields separated by Tab. |
|
4.2 Signature Filter
Signature filter computes signatures in the TransApplet; the input is an image and the output is a signature.
The filter has two classes: Signature Generator and Signature Filter. The interface for Signature Generator is:
public interface I_SignatureGenerator
{
bool setSignatureFilter (int x);
int getSignatureFilter ();
ImageSignature getSignature (Bitmap b);
int getSignatureFilterSize ();
string [] getSignatureFilterNames();
}//class
The following table lists the functions.
|
Function |
Descriptions |
Comments |
|
ImageSignature getSignature (Bitmap b) |
Gets the signature of the input image. Bitmap b Output: Signature. |
|
|
int getSignatureFilterSize () |
Gets the number of attributes the library of signatures. |
|
|
bool setSignatureFilter (int x) int getSignatureFilter () |
Selects a Signature Generator. |
|
The interface for Signature Filter is:
public interface I_SignatureFilter
{
bool setSignatureFilter (int x);
int getSignatureFilter ();
string [] getSignatureFilterNames();
ImageSignature getSignature (string imagePath, string ID);
ImageSignature getSignature (string imagePath);
ImageSignature getSignature (Bitmap b, string ID);
ImageSignature getSignature (Bitmap b);
ImageSignature getSignature
( string imagePath, string ID,int x, int y, int w, int h);
ImageSignature getSignature
( Bitmap bImg, string path, string name, string ID,
int x, int y, int w, int h);
bool getLibrary ( string [] imageAbsolutePath, string fileName );
bool getLibrary
( string [] imageAbsolutePath, string [] IDs, string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath,
string [] IDs,
string [] xs,
string [] ys,
string [] ws,
string [] hs,
string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath,
string [] IDs,
int [] xs,
int [] ys,
int [] ws,
int [] hs,
string fileName );
}//class
The following table lists the functions.
|
Function |
Descriptions |
Comments |
|
bool setSignatureFilter (int x) int getSignatureFilter () |
Selects a Signature filter. |
|
|
string [] getSignatureFilterNames(); |
Gets a list of Signature filter names. |
|
|
ImageSignature getSignature (string imagePath, string ID);
ImageSignature getSignature (string imagePath); ImageSignature getSignature (Bitmap b, string ID); ImageSignature getSignature (Bitmap b); |
Gets the Signature of the input image. |
|
|
ImageSignature getSignature ( string imagePath, string ID,int x, int y, int w, int h); ImageSignature getSignature ( Bitmap bImg, string path, string name, string ID, int x, int y, int w, int h); |
Gets the Signature of the input image segment.
string imagePath, or Bitmap bImg string ID int x, int y, int w, int h. Output: Signature. |
|
|
bool getLibrary ( string [] imageAbsolutePath, string fileName ); bool getLibrary ( string [] imageAbsolutePath, string [] IDs, string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath, string [] IDs, string [] xs, string [] ys, string [] ws, string [] hs, string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath, string [] IDs, int [] xs, int [] ys, int [] ws, int [] hs, string fileName ); |
Generates a Signature library from all images in string [] imageAbsolutePath and produces a file that contains all signatures. Input: string [] imageAbsolutePath Output: A text file that contains the library of signatures. |
|
4.3 Matching Results
Attrasoft ImageFinder matches whole images or image segments. The ImageFinder can be used for:
- Image
Verification (1:1
Matching);
- Image
Identification (1:N
Matching);
- Image
Search or Retrieval (1:N
Matching);
- Multiple Matching (N:N and N:M Matching); and
- Video Matching (1:1, 1:N, M:N).
The results for N:N Matching always goes to a file. The results for 1:1 and 1:N Matching go to a data structure called Results_1N. The interface for class, Results_1N, is:
public interface I_Results_1N
{
bool getStatus ();
int getNumberOfMatches();
string getImageID (int i) ;
string [] getImageID () ;
string [] getImageID_N ( int N );
string getScore(int i);
string [] getScore();
string [] getScore_N ( int N );
string getImageName (int i) ;
string [] getImageName () ;
string [] getImageName_N ( int N ) ;
string getImagePath (int i) ;
string []getImagePath () ;
string [] getImagePath_N ( int N ) ;
string getX(int i);
string [] getX();
string [] getX_N ( int N );
string getY(int i);
string [] getY();
string [] getY_N ( int N );
string getW(int i);
string [] getW();
string [] getW_N ( int N );
string getH(int i);
string [] getH();
string [] getH_N ( int N );
Results_1N sort ();
string toString ();
}
The following table lists the functions.
|
Functions |
Descriptions |
Comments |
|
int getStatus () |
Returns the status of the current signature comparison: > 0: OK; < 0: Error. |
|
|
int getNumberOfMatches() |
Returns the number of matches of the current signature comparison. |
|
|
string getImageID (int i) string [] getImageID () string [] getImageID_N ( int N ) |
Returns the matching IDs of the current signature. |
|
|
string getScore(int i) string [] getScore() string [] getScore_N ( int N ) |
Returns the matching scores of the current signature. |
|
|
string getImageName (int i) string [] getImageName () string [] getImageName_N ( int N ) |
Returns the matching names of the current signature. |
|
|
string getImagePath (int i) string []getImagePath () string [] getImagePath_N ( int N ) |
Returns the matching paths of the current signature. |
|
|
string getX(int i) string [] getX() string [] getX_N ( int N ) string getY(int i) string [] getY() string [] getY_N ( int N ) string getW(int i) string [] getW() string [] getW_N ( int N ) string getH(int i) string [] getH() string [] getH_N ( int N ) |
Returns the matching (x, y, w, h) of the current signature. |
|
|
string toString () |
Returns the entire results as a string with fields separated by Tab. |
|
4.4 Whole Image Matching
NeuralFilter also matches two whole images. The interface for NeuralFilter is:
public interface I_NeuralFilter
{
bool training ( string a1_txt, string match_txt);
Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11
( Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig1,
Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig2);
Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11
(string path1, string path2);
Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11
(Bitmap left, Bitmap right);
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N
(Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig,
string a1File, string b1File);
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N
(string keyuPath, string a1File, string b1File);
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N
(Bitmap keyImage, string a1File, string b1File);
bool findMatchNN (string a1File, string b1File);
bool findMatchNM (string a1File, string a2File, string b1File);
}
The following table lists the functions.
|
Functions |
Descriptions |
|
bool training ( string a1_txt, string match_txt) |
Trains the BioFilter. |
|
Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 ( Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig1, Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig2); Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 (string path1, string path2); Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 (Bitmap left, Bitmap right); |
Makes a 1:1 matching |
|
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig, string a1File, string b1File); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (string keyuPath, string a1File, string b1File) Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (Bitmap keyImage, string a1File, string b1File) |
Makes a 1:N matching |
|
bool findMatchNN (string a1File, string b1File); bool findMatchNM (string a1File, string a2File, string b1File); |
Matches all image signatures in file a1File against all image signatures in a1File or a2File and saves the results to b1File. |
4.5 ImageLibrary
Dynamic Library allows the library (N images in a 1:N Matching) to be updated via insertion, deletion, and update.
The interface for Image Library filter is:
public interface I_ImageLibrary
{
string getLibraryID ();
bool setLibraryID (string x);
bool load ();
bool load ( string fileName);
bool load ( string fileName1, string fileName2);
bool print ( );
bool clear();
bool backup ();
bool backup (string fileName);
bool addSignature (ImageSignature sig);
bool deleteSignature (ImageSignature sig);
bool deleteSignature (string ID);
bool replaceSignature (ImageSignature sig);
bool mergeLibrary (string libFile1, string libFile2, string outputLib);
}
The following table lists the functions.
|
Functions |
Descriptions |
Comments |
|
int getLibraryID () |
Gets the Library ID (optional). |
|
|
void setLibraryID (string x) |
Sets the Library ID (optional). |
|
|
bool load () |
Loads the default master library, lib1.txt. |
|
|
bool load ( string fileName) |
Loads master library specified by fileName. |
|
|
bool load ( string fileName1, string fileName2) |
Loads two libraries specified by fileName1 and fileName2. |
|
|
bool clear() |
Clears the current library from RAM only. |
|
|
bool backup () |
Saves current library to the default file, lib1_bk.txt. |
|
|
bool backup (string fileName) |
Saves current library to a back file. |
|
|
bool addSignature (ImageSignature sig) |
Adds a signature to a loaded image library in RAM. |
|
|
bool deleteSignature (ImageSignature sig) |
Deletes a signature from a loaded image library in RAM. |
|
|
bool deleteSignature (string ID) |
Deletes a signature from a loaded image library in RAM. |
|
|
bool replaceSignature (ImageSignature sig) |
Replaces a signature from a loaded image library in RAM. |
|
|
bool mergeLibrary (string libFile1, string libFile2, string outputLib) |
Merges two signature libraries into a single library. Input: string libFile1 string libFile2 string outputLib Output: A text file that contains the library of signatures from both input libraries. |
|
4.6 Matching Engine
The interface for the Neural Filter Matching Engine is:
public interface I_MatchingEngine
{
int getLowCut();
void setLowCut(int x);
int getHighCut();
void setHighCut(int x);
void setNeuralOutputType(int x);
int getNeuralOutputType();
void setThreshold(int x);
int getThreshold();
void setSensitivity(int x);
int getSensitivity();
void setBlurring(int x);
int getBlurring();
void setUseRelativeScore(int x);
int getUseRelativeScore();
bool setLibrary
(Attrasoft.TransApplet80.ImageLibrary80.ImageLibrary lib);
bool setSignature
(Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig);
bool setNFTraining
(Attrasoft.TransApplet80.MatchingEngineTraining80.MatchingEngineTraining80 a_1);
bool setBFTraining
(Attrasoft.TransApplet80.MatchingEngineTrainingBF80.MatchingEngineTrainingBF80 bf80_1);
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch11
( Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig1,
Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig2);
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch ();
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch
(Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig);
}
The following table lists the functions.
|
Functions |
Descriptions |
Comments |
|
int getLowCut() void setLowCut(int x)
int getHighCut() void setHighCut(int x)
void setNeuralOutputType (int x) int getNeuralOutputType() void setThreshold(int x) int getThreshold() void setSensitivity (int x) int getSensitivity() void setBlurring (int x) int getBlurring()
void setUseRelativeScore (int x) int getUseRelativeScore() |
Gets and Sets matching engine parameters. |
|
|
bool setLibrary (ImageLibrary lib) |
Sets the library to be used in matching signatures. |
|
|
bool setSignature (ImageSignature sig) |
Sets the signature for matching. |
|
|
bool setNFTraining (MatchingEngineTraining80 a_1) bool setBFTraining (MatchingEngineTrainingBF80 bf80_1) |
Sets the training objects for matching. |
|
|
Results_1N findMatch11 (ImageSignature sig1, ImageSignature sig2); |
Gets 1:1 match results for sig1 and sig2. |
|
|
Results_1N findMatch () |
Gets the match results for the signature, sig, specified by function, setSignature (ImageSignature sig) Output: Results_1N. |
|
|
Results_1N findMatch (ImageSignature sig) |
Gets the match results for the signature, sig. Input: ImageSignature sig; Output: Results_1N. |
|
4.7 NeuralNet Filter
The NeuralNet Filter matches a segment of an image(s). The interface for the NeuralNet Filter Matching Engine is:
public interface I_NeuralNetFilter
{
bool train (Bitmap img);
bool train (string sPath);
bool train (Bitmap img, int x, int y, int w, int h);
bool train (string sPath, int x, int y, int w, int h);
bool retrain (Bitmap img);
bool retrain (string sPath);
bool retrain (Bitmap img, int x, int y, int w, int h);
bool retrain (string sPath, int x, int y, int w, int h);
Attrasoft.TransApplet80.Results_1N.Results_1N
findMatch11 ( Bitmap img1);
Attrasoft.TransApplet80.Results_1N.Results_1N
findMatch11 ( string path1);
Attrasoft.TransApplet80.Results_1N.Results_1N
findMatch1N ( string [] fileList);
bool findMatchNN (string [] fileList, string c1File);
bool findMatchNM (string [] keyList, string [] libraryList, string c1File);
}
The following table lists the functions.
|
Functions |
Descriptions |
|
bool train (Bitmap img) bool train (string sPath) bool train (Bitmap img, int x, int y, int w, int h) bool train (string sPath, int x, int y, int w, int h) |
Trains the neural net. |
|
bool retrain (Bitmap img) bool retrain (string sPath) bool retrain (Bitmap img, int x, int y, int w, int h) bool retrain (string sPath, int x, int y, int w, int h) |
Retrains the neural net. |
|
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch11 ( Bitmap img1); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch11 ( string path1); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N ( string [] fileList); |
Makes a 1:N Matching. |
|
bool findMatchNN (string [] fileList, string c1File) bool findMatchNM (string [] keyList, string [] libraryList, string c1File); |
Makes a N:N Matching and N:M Matching. |
4.8 Other API
We will introduce other API’s in later chapters as we encounter them.
5. User Interface
This User’s Guide is written as if you will create your own version of the ImageFinder. We will do a step-by-step instruction for implementing an “ImageFinder”. Eventually, you will have your own “ImageFinder”.
This chapter briefly describes how to set up the form and menu bar for the ImageFinder. We will use C# .Net as the implementation language. We will build the user interface form with a Menu Bar and Input buttons.
5.1 How to Create a Project
To create a new C# project:
- Start the Visual Studio .Net, (see Figure 5.1).
- Click File è New Project command. The New Project dialog box is displayed (Figure 5.2).
- Language and Templates: Highlight the Visual C# project folder in the Project Type list to display the templates that are available for C#. Then, highlight the Windows Application template (Figure 5.2).
- Location and Name: Enter a name for the project and select the location for the project. A folder with the same name as the project is automatically added to the location you specify. We will use chap4 as the project name.
- Click the ‘OK’ button in Figure 5.2 to start the new project (Figure 5.3).
Now go to the property window and set (Figure 5.4):
- Form1.text to “ImageFinder”;
- Resize Form1.
Figure 5.1 Create C# Project.
Figure 5.2 Select Project Templates.
Figure 5.3 Form1.
Figure 5.4 Set Form1.Size.
5.2 How to Create Menus
To add a menu bar to a form:
- Add a “MenuStrip” or “MainMenu” control to a form from the toolbox. The control will appear in the component Design Tray at the bottom of the Form Design window. The menu of a form is determined by the Menu property.
- To add items to the menu, click the “MenuStrip” or “MainMenu” control and click wherever it says “Type Here” in the menu designer. Then, type the text of the menu item and press the Enter key. Additional entry areas appear below and to the right of the entry.
- To edit the property for each menu item, right-click the menu and select the Property command.
5.3 Link to Class Library
Figure 5.5 References in
the Solution Explorer.
In this section, XYZ
denotes any class located in c:\transapplet80\.
To use the class library,
“Attrasoft.TransApplet80.XYZ”:
(1)
Add a “using” statement:
using
Attrasoft.TransApplet80.XYZ;
(2)
Right click References and select Add reference in the Solution Explorer
(Figure 5.5);
(3)
Use the Browse button to find “XYZ.dll”; highlight the library under “Selected
Components” and click the “OK” button.
Now the class library,
“Attrasoft.TransApple80.XYZ”, is ready to be used. To make sure the link is successful, you should see XYZ under References in the
Solution Explorer.
5.4 Declare Objects
Each Library has one main class, which has the same name as the Library-name:
Attrasoft.TransApplet80.xyz80.xyz80.
…
The advantage of naming the class-name as the library-name is that you will know what
to use, i.e.
Attrasoft.TransApplet80.xyz80.xyz80.
There is no guessing which
class you should choose if the Library has more than one class.
The disadvantage is the compiler will not be able to distinguish between the class-name
and the Library-name unless you use the full name.
To declare all TransApplet
objects using the full class-name like this:
public Attrasoft.TransApplet80.xyz80.xyz80 myObj80
= new Attrasoft.TransApplet80. xyz80.xyz80 ();
6. Input
This chapter describes how to enter data to the ImageFinder. The input images are further divided into:
- Key
- Search Source.
An Image Path specifies the training image. A Search-Folder specifies the Search Source.
This chapter will introduce several class libraries,
Attrasoft.TransApplet80.InputModule80,
Attrasoft.TransApplet80.InputList80,
Attrasoft.TransApplet80.GUI.
The chapter project is in “.\ transapplet80.chap6”.
6.1 Class Library Name
The class library is:
Attrasoft.TransApplet80.InputModule80.
The main class in this library will be:
Attrasoft.TransApplet80.InputModule80. InputModule.
The interface is:
public interface I_ InputModule80
{
//Input:
bool
setDirList(string sInput);
bool setSubDirList(string sInput);
bool setFileList(string sInput);
bool setFileSegmentList(string sInput);
bool setFileSegmentKeyList(string
sInput);
bool setAccessList(string sInput,
string sSQL);
bool setAccessSegmentList(string
sInput, string sSQL);
//Output:
Attrasoft .TransApplet80 .InputList80 .InputList getList();
}
This class has two parts:
- Input;
- Output.
The input can be:
- Folder path
- File path containing images
- File path containing image segments
- Access file containing images
- Access file containing image segments
- …
Output is an object of the following class:
public
class InputList
{
public int status = 0;
public string[] imageAbsoultePath =
null;
public string[] imageID = null;
public string[] imageName = null;
public string[] imagePath = null;
public string[] imageX = null;
public string[] imageY = null;
public string[] imageW = null;
public string[] imageH = null;
public string[] key0 = null;
public string[] key1 = null;
…
}
6.2 Class Library Overview
The functions in classes are listed in the following table.
|
Function |
Description |
|
string [] setDirList ( string sInput) |
To get a string list of the absolute paths of all images in directory, set the input directory, sInput. |
|
string [] setSubDirList ( string sInput) |
To get a string list of the absolute paths of all images in all sub-directories, set the input directory, sInput. |
|
string [] setFileList ( string sInput) |
To get a string list of the absolute paths of all images in file, set the input file, sInput. |
|
string [] getFileSegmentList ( string sInput) |
To get a string list of the absolute paths of all image segments in file, set the input file, sInput. |
|
string [] getAccessList ( string sInput, string sSQL) |
To get a string list of the absolute paths of all images in access file, sInput, specified by a SQL statement, sSQL, set the two input variables. |
|
string [] getAccessSegmentList ( string sInput, string sSQL) |
To get a string list of the absolute paths of all image segments in the access file, sInput, specified by a SQL statement, sSQL, set the two variables. |
|
Attrasoft .TransApplet80 .InputList80 .InputList getList() |
After the input is specified, get the output. |
6.3 Set Up the Chapter Project
Figure 6.1 Chapter 6 Project.
We will add the following controls to our form:
Key Button, Key Text Box
Use the “Key” button to display the selected key image and print the image path in the key text box. For example, click the button, go to the directory, "C:\transapplet80\ex_label", and then select an image. The selected image will be shown in the ImageFinder to indicate the specification is successful and the key-image is ready for training or retraining.
Source Button, Source Text Box
Use the “Source” button to select a search source and display the source in the text box.
Clear Button
Use the “Clear” button to clear the rich text box.
Rich Text Box
Use the Rich Text Box to display messages.
Key Picture Box
Use the Key Picture Box to display key images.
Source Picture Box
Use the Source Picture Box to display images in the search source.
OpenFileDialog
Use the OpenFileDialog to select a key image.
FolderBrowserDialog
Use the FolderBrowserDialog to select a source path.
After adding these controls, the form will look like Figure 6.1.
6.4 Link to Class Library
Figure 6.2 Link to Class library.
To include the class library to the project,
- Right click References and select Add Reference in the Solution Explorer;
- Browse to find “InputModule80.dll” in “c:\transapplet80”;
- Highlight it and click the “OK” button (Figure 6.2).
To declare an object, please use the full path for class:
internal
Attrasoft.TransApplet80.InputModule80.InputModule
inputModuleSource;
inputModuleSource
= new Attrasoft.TransApplet80.InputModule80.InputModule(this.richTextBox1);
Now the object is ready to use. In addition, we will add several class libraries:
Helper80;
GUI;
InputList80.
6.5 Key Segment
We will add a class, “Input”, in the project. We will implement the input buttons in this class. In this way, the main class is simpler. We will declare an object, input, as follows:
Input input;
private void
Form1_Load(object sender, System.EventArgs e)
{
h80 = new
Attrasoft.TransApplet80.Helper80.Helper80(this.richTextBox1);
inputModuleSource
=
new Attrasoft.TransApplet80.InputModule80.InputModule(this.richTextBox1);
guiSource = new
Attrasoft.TransApplet80.GUI.GUI_Key(this.pictureBox2, h80);
guiResults = new
Attrasoft.TransApplet80.GUI.GUI_Results(this.pictureBox3, h80);
input = new Input(this);
}
The helper80 object is used to print a message:
void
appendText (string s)
{
f.h80 .appendText ( s);
}
void appendLine (string s)
{
appendText ( s +"\n");
}
We can double click the Key Segment buttons to complete the programming:
private void button1_Click(object sender, System.EventArgs e)
{
input.keySegment();
}
which in turn, calls a function:
public bool keySegment()
{
if ( f.openFileDialog1.ShowDialog () != DialogResult.OK )
return false;
string fileName = f.openFileDialog1.FileName;
try
{
bTrain =new Bitmap (fileName);
f.pictureBox1.Image = bTrain;
f.textBox1.Text = fileName ;
f.richTextBox1.AppendText
( "Display Key Segment:\n " + fileName +"\n");
}
catch
{
appendText ("Invalid key image !\n");
return false;
}
return true;
}//keySegment()
The first section of code opens a file dialog so the user can select a file:
if ( f.openFileDialog1.ShowDialog () != DialogResult.OK )
return false;
The next section of code gets the selected key:
string fileName = f.openFileDialog1.FileName;
The last section of code creates an image and displays the image:
bTrain =new Bitmap (fileName);
f.pictureBox1.Image = bTrain;
f.textBox1.Text = fileName ;
f.richTextBox1.AppendText ( "Display Key Segment:\n " + fileName +"\n");
6.6 Search Source Button
Now, we implement the “Source” button:
private void button2_Click(object sender, System.EventArgs e)
{
input.source( );
}
which in turn, calls the following function in the Input class:
internal bool source()
{
try
{
if
(f.folderBrowserDialog1.ShowDialog() != DialogResult.OK)
return false;
string cDir = f.folderBrowserDialog1.SelectedPath;
f.textBox2.Text = cDir;
f.inputModuleSource.setSubDirList(cDir);
f.guiSource.setInput(f.inputModuleSource.getList());
if (!f.guiSource.getStatus())
{
f.appendText("Data
Source fails!\n");
return false;
}
f.guiSource.setLists();
}
catch (Exception e)
{
f.appendText(e.ToString() +
"\n");
return false;
}
return
this.f.guiSource.Button_First(false);
}
The first section of code gets the current search directory via the folder browser dialog:
if (f.folderBrowserDialog1.ShowDialog()
!= DialogResult.OK)
return false;
string cDir =
f.folderBrowserDialog1.SelectedPath;
f.textBox2.Text = cDir;
The next section enters the selected folder to the InputModule object:
f.inputModuleSource.setSubDirList(cDir);
The third section gets the string list of the image paths:
f.inputModuleSource.getList().
And passes this list to the display section:
f.guiSource.setInput(f.inputModuleSource.getList());
The last section displays the first images:
f.guiSource.setLists();
f.guiSource.Button_First(false);
To test, run the software and click the first image in folder, “c:\transapplet80\ex_label”.
7. Image Display
This chapter describes how to display images in a search source. To display these search images, three buttons will do:
First Button
Use the “First” button to display the first image in the search source.
> (Next) Button
Use the “Next” button to display the Next image in the search source.
< (Previous) Button
Use the “Previous” button to display the Previous image in the search source.
After adding the three buttons, the form looks like Figure 7.1.
Figure 7.1 First, Next (>), and Previous (<) Buttons.
7.1 Class Library Name
The class library is:
Attrasoft.TransApplet80.GUI.
It has three libraries:
new Attrasoft.TransApplet80.GUI.GUI_Key
new Attrasoft.TransApplet80.GUI.GUI_Source
Attrasoft.TransApplet80.GUI.GUI_Results
They are designed for:
Key;
Source;
Results.
GUI_Key is the base class of GUI_Source.
GUI_Source is the base class of GUI_Results.
At this point, GUI_Key and GUI_Source are the same.
All three class libraries function in a similar way.
The GUI objects are:
guiSource = new
Attrasoft.TransApplet80.GUI.GUI_Key(this.pictureBox2, h80);
guiResults = new
Attrasoft.TransApplet80.GUI.GUI_Results(this.pictureBox3, h80);
The functions in this class are:
|
Function |
Description |
|
Bool setInput ( ImageList80) |
Sets an input list of image paths, image ID, …. |
|
void setList ( ) |
Sets the class ready for the first button, next button, and previous button. |
|
bool getStatus () |
Gets the status. |
|
Bool Button_First(bool); |
Displays the first image. |
|
Bool Button_Next(bool); |
Displays the next image. |
|
Bool Button_Previous(bool); |
Displays the previous image. |
The sample codes are:
Input
f.guiSource.setInput(f.inputModuleSource.getList());
Computation
f.guiSource.setLists();
Display
f.guiSource.Button_First(false);
f.guiSource.Button_Next(false);
f.guiSource.Button_Previous(false);
7.2 Implementing H, >, and < Buttons
In this section, we will implement the First, Next, and Previous buttons. To see images in a search source, start with the “First” button and then, keep clicking the “Next” button to display the next image; eventually, all of the images in the search source will be shown.
The code for the three buttons consists of three parts:
- Populating GUI Object;
- Computation to get the object ready;
- First, Next, Previous buttons.
First of all, the object has to be populated. This is accomplished by the source button in the last chapter:
f.guiSource.setInput(f.inputModuleSource.getList());
if (!f.guiSource.getStatus())
{
f.appendText("Data
Source fails!\n");
return false;
}
Then we will set up the object:
f.guiSource.setLists();
Finally, the code for the First, Next, and Previous buttons are:
private void button3_Click(object
sender, EventArgs e)
{
bool b =
this.guiSource.Button_First(true);
if (!b)
setText(guiSource.getMessage());
}
private void
button5_Click(object sender, EventArgs e)
{
bool b =
guiSource.Button_Next();
if (!b)
setText(guiSource.getMessage());
}
private void
button6_Click(object sender, EventArgs e)
{
bool b =
guiSource.Button_Previous();
if (!b)
setText(guiSource.getMessage());
}
7.3 Test
This section tests the project:
(1) Run the project;
(2) Click the “Key” button, go to “c:\transapplet80\ex_wheel\”, and select a key image;
(3) Click the “Source” button, go to “c:\transapplet80\ex_wheel\”, and select an image.
(4) Click the “First” button;
(5) Click the “Next” button several times;
(6) Click the “Previous” button several times.
7.4 Output Images
In the last a few sections, we demonstrated how to display Input Source images. After a matching, the output images will be displayed in a similar fashion.
At the end of a 1:1 or 1:N matching, a Result object will be produced. This object will be entered into GUI_Result object:
guiResults.outputResults(results_1N,
softwareDir);
To display result images:
guiResults.Button_First(false);
guiResults.Button_Next(false);
guiResults.Button_Previous(false);
8. ImageFinderLite
In this chapter, we will briefly describe how to build a quick and dirty image recognition application. In particular, we will discuss how to identify a full image.
8.1 ImageFinderLite
Attrasoft.TransApplet80.ImageFinderLite
is a component Attrasoft TransApplet.
To use the ImageFinderLite object, you will need 5 steps:
Step 1. Set the Folder that contains the images you
want to search through.
Step 2. Create a Image Signature Library from the
folder.
Step 3. Load the Signature Library into the
ImageFinderLite object.
Step 4. Get a key image file name, which will be
matched against the library.
Step
5. Make a 1:N search.
8.2 Interface
The ImageFinderLite interface is given below:
public interface
I_ImageFinderLite
{
//variables:
//1.
Foldername (string)
//2.
Library Name (string)
//3.
KeyImage (string)
//4.
Signature (Attrasoft object)
//5.
Result (Attrasoft object)
//step 1
Set Library Folder
bool
setFolder(string folderName);
string
getFolder();
//step 2 Create Image Signature Library
bool
computeSignatureLibrary();
bool
setLibraryName(string a1File);
string
getLibraryName();
//setp 3
Load Signature Library
bool
loadLibrary();
//step 4
Get key file name
bool
setKeyImage (string key);
string
getKeyImage();
//step 5
search
bool search ( );
bool
setOutputFileName (string b1File);
string
getOutputFileName ( );
}
Step 1. Set the Folder that contains the images you want to search
through.
|
bool setFolder(string folderName) string getFolder() |
Use setFolder (string folderName) to set the folder that contains the
images you want to search through; use getFolder() to see the current
setting. |
Step 2. Create Image Signature Library from the folder.
|
bool computeSignatureLibrary() |
Use computeSignatureLibrary() to get a file that contains the image
signatures from the specified folder. |
|
bool setLibraryName(string a1File) string getLibraryName() |
Use setFolder setLibraryName(string a1File) to set the image signature file
name that contains the images you want to search through. The default file is
“.\data\a1.txt”, where “.” is the application folder. Use getLibraryName() to
see the current setting. |
Step 3. Load the Signature Library into the ImageFinderLite object.
|
bool loadLibrary(); |
Use loadLibrary() to load the Signature Library into the ImageFinderLite
object. |
Step 4. Get a key image file name, which will be matched against the
library.
|
bool setKeyImage (string key) string getKeyImage() |
Use setKeyImage (string key) to set the unknown image that you want to
search; use getKeyImage() to see the current setting. |
Step 5. Make a 1:N search.
|
bool search ( ) |
Use search ( ) to make an 1:N search. |
|
bool setOutputFileName (string b1File) string getOutputFileName ( ) |
Use setOutputFileName (string b1File) to set the output file name; the
default name is “.\data\b1.txt”, where “.” is the application folder. Use getOutputFileName ( ) to see the
current setting. |
8.3 Programming Example
Now we will show a programming example.
1. Create a User Interface in Figure 8.1.
Figure
8.1 A User Interface, which contains
the five steps.
2. Link to the Library
Figure
8.2 Add the class library; see the
highlighted areas in the Solution Explorer.
3. Initial Files
Apart from
linking the class libraries, there are several data files required by the class
libraries. You must provide these data files in order for the class libraries
to function properly. You will need to copy “.\data\*.*” from the CD into your
project folder. These files must be placed in the “.\data\” sub-folder of your
project
4. ImageFinderLite Object
Add an ImageFinderLite object:
Attrasoft.TransApplet80.ImageFinderLite.ImageFinderLite
ifl;
Create the object in the form constructor:
public Form1()
{
InitializeComponent();
ifl = new
Attrasoft.TransApplet80.ImageFinderLite.ImageFinderLite
( this.richTextBox1 );
}
Figure
8.3 Add the ImageFinderLite object; see
the highlighted areas in the Solution Explorer.
5. Following the 5 steps:
Step 1. Set the Folder that contains the images you
want to search through.
Step 2. Create the Image Signature Library from the
folder.
Step 3. Load the Signature Library into the
ImageFinderLite object.
Step 4. Get a key image file name, which will be
matched against the library.
Step
5. Make a 1:N search.
The details are given in the
next section.
8.4 Codes for Each of the Steps
Step 1. Set the Folder that contains the images you want to search
through.
private void button1_Click(object sender,
EventArgs e)
{
if
(folderBrowserDialog1.ShowDialog() != DialogResult.OK)
return ;
string cDir = folderBrowserDialog1.SelectedPath;
textBox1.Text = cDir;
ifl
.setFolder ( cDir );
}
Step 2. Create the Image Signature Library from the folder.
private void button2_Click(object sender,
EventArgs e)
{
bool
b = ifl.computeSignatureLibrary();
}
Step 3. Load the Signature Library into the ImageFinderLite object.
private void button3_Click(object sender,
EventArgs e)
{
if (openFileDialog1.ShowDialog() != DialogResult.OK)
return;
string fileName =
openFileDialog1.FileName;
textBox2.Text = fileName;
ifl.setKeyImage(fileName);
}
Step 4. Get a key image file name, which will be matched against the
library.
private void
button4_Click(object sender, EventArgs e)
{
bool
b = ifl.loadLibrary ();
}
Step 5. Make a 1:N search.
private void
button5_Click(object sender, EventArgs e)
{
ifl.search () ;
}
8.5 Test Run
Run the software to get Figure 8.1; then
·
Click Button 1 to set the Folder that contains the images you
want to search through.
·
Click Button 2 to create the Image Signature Library from the
folder.
·
Click Button 3 to load the Signature Library into the
ImageFinderLite object.
·
Click Button 4 to get a key image file name, which will be
matched against the library.
·
Click Button 5 to Make
a 1:N search.
The results are given in the text window in Figure 8.1 and in a text file, “.\data\b1.txt”.
9. Image Signatures
Image Matching is implemented through something called Image Signature. An image has a set of computed values called features. A collection of features is grouped into a signature.
In this project, we will use Signature Menu Items to compute. The input is an image and the output is a signature. We will show how to use the Image Signatures in image matching in the next chapter. This chapter introduces how to compute signatures only, which will take an input image and produce a signature.
The chapter project is located at:
C:\transapplet80\imagefinder\.
The executable file is located at:
c:\transapplet80\imagefinder\ imagefinder
Lite\bin\Release\.
We
also call this folder “.\”. Most of the
functions will be implemented in two classes:
·
MainMenuToAPI
· GUIRelated
Class, MainMenuToAPI, will implement all TransApplet related functions. This class will declare an object of “Attrasoft.TransApplet80.EntryPoint80.EntryPoint80”, which contains all of the required objects.
Class, GUIRelated, will implement all functions related to GUI (Graphical User Interface), such as input/output, display images, messaging, …
The basic object is:
Attrasoft.TransApplet80.SignaturFilter80.SigantureFilter80.
Starting from this chapter, we will use a new object
from this class:
Attrasoft.TransApplet80.EntryPoint80.EntryPoint80
This class has a list of objects; one of the objects is
the SignatureFilter80.
9.1 Signature Menu
First of all, we will create the menu shown in Figure 9.1 and create the menu items under the Signature Menu.
Since the project is already implemented, you can simply open the project in:
.\imagefinder\,
and see the implementation of the menu items. We will explain the main functions in each menu item.
Figure 9.1 Signature Menu.
9.2 API
The basic idea is:
Image Signature = SignatureFilter. getSiganture (image);
As we can see, there are two objects here, Signature and Signature Filter. The
Signature functions are:
|
Function |
Descriptions |
Comments |
|
int getStatus () |
Returns the status associated with the signature. Output: 1: signature ready. 0: signature not ready. -1: no image -2: image segmentation specification error -3: other error. |
|
|
string getID() |
Returns the ID associated with the signature. Output: image ID. |
|
|
string getImageName () |
Returns the image name associated with the signature. Output: image name. |
|
|
string getImagePath() |
Returns the image path associated with the signature. Output: image path. |
|
|
string getAbsolutePath () |
Returns the absolute path associated with the signature. Output: image absolute path. |
|
|
int getNumberOfAttributes () |
Returns the number of attributes associated with the signature. Output: number of attributes. |
|
|
int [] getSignatureAttributes() |
Returns the attribute array associated with the signature. Output: attribute array. |
|
|
int int [] getSignatureAttributes() (int index) |
Returns the attribute associated with the input index. Input: index. Output: attribute associated with the input index. |
|
|
string toString () |
Returns the entire image signature as a string with fields separated by Tab. |
|
The Signature Filter functions are:
|
Function |
Descriptions |
Comments |
|
bool setSignatureFilter (int x) int getSignatureFilter () |
Selects a Signature filter. |
|
|
string [] getSignatureFilterNames(); |
Gets a list of Signature filter names. |
|
|
ImageSignature getSignature (string imagePath, string ID);
ImageSignature getSignature (string imagePath); ImageSignature getSignature (Bitmap b, string ID); ImageSignature getSignature (Bitmap b); |
Gets the Signature of the input image. |
|
|
ImageSignature getSignature ( string imagePath, string ID,int x, int y, int w, int h); ImageSignature getSignature ( Bitmap bImg, string path, string name, string ID, int x, int y, int w, int h); |
Gets the Signature of the input image segment.
string imagePath, or Bitmap bImg string ID int x, int y, int w, int h. Output: Signature. |
|
|
bool getLibrary ( string [] imageAbsolutePath, string fileName ); bool getLibrary ( string [] imageAbsolutePath, string [] IDs, string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath, string [] IDs, string [] xs, string [] ys, string [] ws, string [] hs, string fileName );
bool getSegmentLibrary ( string [] imageAbsolutePath, string [] IDs, int [] xs, int [] ys, int [] ws, int [] hs, string fileName ); bool getKeywordLibrary( string [] imageAbsolutePath, string [] IDs, int [] xs, int [] ys, int [] ws, int [] hs, string filename, string [] key0, string [] key1, … ); |
Generates a Signature library from all images in string [] imageAbsolutePath and produces a file that contains all signatures. Input: string [] imageAbsolutePath Output: A text file that contains the library of signatures. |
|
9.3 TransApplet Objects
The
first thing we will do is to create all of the objects required for the ImageFinder. As we discussed earlier in this chapter, all of
the objects required for the ImageFinder project is grouped into a
single class. All we have to do is to create an object of this class and we
will call it script.
We
will create a new class, MainMenuToAPI. We will do all of the menu item implementation
in this class.
internal
Attrasoft.TransApplet80.EntryPoint80.EntryPoint80 script;
MainMenu f;
public
MainMenuToAPI(MainMenu f1)
{
try
{
f = f1;
script =
new
Attrasoft.TransApplet80.EntryPoint80.EntryPoint80 (f.richTextBox1);
}
catch (Exception e )
{
f.richTextBox1.Text = e.ToString ()
+"\n";
}
}
9.4 Key Signature
To compute the signature of a key image:
- Click the “Key” button to select a key image or sample image;
- Click the “Signature/1:1 First Signature” to compute the signature.
Example. Key Signature.
- Click the “Key” button to select “.\ex_wheel\2067(1).jpg”;
- Click the “Signature/1:1 First Signature” to compute the signature.
Figure 9.2 Click “Signature/Left Signature (Key)”.
The Signature looks like:
3055 29 33 30355 255 18711 195 1 1 …
In 1:1 Matching or 1:N Matching, the signatures are computed behind the screen; you do not need to use the signature directly. The menu item, “Signature/1:1 First Signature”, shows you the mechanism behind the screen so you can see what a signature looks like.
Now, we will implement these two menu items. Double click these menu items and you will see:
private void menuItem2_Click(object sender, System.EventArgs e)
{
mainMenuToAPI.signature_Key_Signature ( textBox1.Text );
}
Here, mainMenuToAPI is an object, which will implement all functions. The main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public
bool signature_Key_Signature ( string key)
{
if ( ! System.IO .File .Exists
(key) )
{
appendText (
"Please enter a valid Key!\n");
return false;
}
int x =0, y = 0, w =0, h = 0;
script.imageSignature
=
script.signatureFilter.getSignature (key, "1of1N",x, y, w, h );
if ( script.imageSignature !=
null )
{
setLine ("The
Signature for the left picture box:");
appendLine
(script.imageSignature.toString () );
return true;
}
else
return false;
}
The following section of code simply makes sure the key image exists:
if ( ! System.IO .File
.Exists (key) )
{
appendText ( "Please enter a valid Key!\n");
return false;
}
The following section of code computes the signature of the key segment:
script.ImageSignature
= script.signatureFilter.getSignature (key, "1 of 1:N", x, y, w, h );
In this statement, the script object contains all of the objects required for the ImageFinder project. In particular, “script.ImageSignature” is the image signature object; and script.signatureFilter is the signature filter object. The parameter, "1 of 1:N", is the ID, which we arbitrarily assigned to the key image. The following section of code prints the signature of the key segment in the text window:
if
( script.imageSignature != null )
{
setLine ("The Signature for the left
picture box:");
appendLine (script.imageSignature.toString ()
);
return true;
}
else
return false;
9.5 Signature File Concepts
In 1:N Matching, the Key image will be matched against all images in a directory. The key signature is computed first, as we demonstrated in the last section. Then this signature will be matched against all signatures in the N-signature file. The N-signature file has a fixed name, “.\data\a1.txt”.
In N:M Matching, the N-signature file is a1.txt, and the M-signature file is a2.txt. The two files, a1.txt and a2.txt, are computed first. Then all of the signatures in a1.txt will be matched against all of the signatures in a2.txt. The M-signature file has a fixed name, “.\data\a2.txt”.
There are 2 menu items:
- Signature/1:N Signature (.\data\a1.txt)
- Signature/N:M Signature (.\data\a2.txt)
These commands compute the signatures for all images in a folder, including all of its subfolders. The only difference between the two commands is where to save the signatures.
Signature/1:N Signature (.\data\a1.txt)
Menu item, “Signature/1:N Signature (.\data\a1.txt)”, computes the signatures for all images in a directory and saves the signatures to a1.txt, the N-Signature file.
Signature/N:M Signature (.\data\a2.txt)
Menu item, Signature/N:M Signature (.\data\a2.txt), computes the signatures for all images in a directory and saves the signatures to a2.txt, the M-Signature file.
9.6 Signature File Implementation
Now, we will implement menu item, Signature/1:N Signature (.\data\a1.txt). Double click this menu item and enter:
private
void button1_Click(object sender, EventArgs e)
{
this.mainMenuToAPI.signature_Search_source
(dataDir + "a1.txt",
SIGNATURE_LIMIT);
}
Again, mainMenuToAPI is an object, which will implement all functions. The main form simply links menu items to functions in the mainMenuToAPI object. The implementation of function, signature_Search_source, is:
f.mainMenuToAPI.script.signatureFilter.getKeywordLibrary
( f.guiSource.libraryList.imageAbsoultePath,
f.guiSource.libraryList.imageID,
f.guiSource.libraryList.imageX, f.guiSource.libraryList.imageY,
f.guiSource.libraryList.imageW, f.guiSource.libraryList.imageH,
outputFile,
f.guiSource.libraryList.key0, f.guiSource.libraryList.key1,
f.guiSource.libraryList.key2, f.guiSource.libraryList.key3,
f.guiSource.libraryList.key4,
f.guiSource.libraryList.key5,
f.guiSource.libraryList.key6, f.guiSource.libraryList.key7,
f.guiSource.libraryList.key8, f.guiSource.libraryList.key9
);
In this statement, the script object contains all of the objects required for the ImageFinder project. In particular, “script.signatureFilter” is the image signature filter object. The parameter, “f.gui.imageAbsoultePath”, is a string list that has the absolute paths of all source images to be matched. After the completion of signature computation, the results will be stored in a file specified by the parameter.
9.7 Examples
Example. Compute N-Signatures:
- Click the “Source” button;
- Select folder, ./ex_wheel/, here “./” means the folder where the ImageFinder is located;
- Click “Signature/N Signature (.\data\a1.txt)” to compute image signatures;
- Click “Signature/Open a1.txt” to see the results.
10. Whole Image Matching
The NeuralFilter matches whole images. The chapter project is located at:
C:\transapplet80\imagefinder\.
The executable file is located at:
c:\transapplet80\imagefinder\ imagefinder Lite\bin\Release\.
We also call this folder “.\”.
10.1 Whole Image Matching Menu
The NeuralFilter is responsible for whole image matching. The “Whole Image” menu items are given in Figure 10.1.
Figure 10.1 NeuralFilter
Menu.
10.2 NeuralFilter API
The following table lists the NeuralFilter functions.
|
Functions |
Descriptions |
|
bool training ( string a1_txt, string match_txt) |
Trains the NeuralFilter. |
|
Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 ( Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig1, Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig2); Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 (string path1, string path2); Attrasoft.TransApplet80.Results_1N .Results_1N findMatch11 (Bitmap left, Bitmap right); |
Makes a 1:1 matching. |
|
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (Attrasoft.TransApplet80.ImageSignature80.ImageSignature sig, string a1File, string b1File); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (string keyuPath, string a1File, string b1File) Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N (Bitmap keyImage, string a1File, string b1File) |
Makes a 1:N matching. |
|
bool findMatchNN (string a1File, string b1File); bool findMatchNM (string a1File, string a2File, string b1File); |
Matches all image signatures in file a1File against all image signatures in a1File or a2File and saves the results to b1File. |
10.3 N:N Matching Implementation
Double click menu item “Whole Imager/N:N Matching (a1.txt vs. a1.txt)” and you will see:
private void toolStripMenuItem21_Click (object sender, System.EventArgs e)
{
mainMenuToAPI.neuralFilter_Matching_NToM
( dataDir + "a1.txt", dataDir + "a1.txt", dataDir + "b1.txt" );
}
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
Public bool neuralFilter_Matching_NToM
( string a1File, string a2File, string b1File)
{
try{
script.neuralFilter.findMatchNM ( a1File, a2File, b1File);
}
catch (Exception e)
{
appendText ( e.ToString () + "\n");
return false;
}
return true;
}
In this statement, the script object contains all of the objects required for the ImageFinder project. In particular, script.neuralFilter is the NeuralFilter object. The parameters, (a1File, a2File, b1File), indicate all signatures in file, a1File, will be matched against all signatures in file, a2File. After completion of the matching, the results will be stored in a file specified by the last parameter, b1File.
10.4 1:N Matching Implementation
Double click menu item “Whole Image/1:N Matching (Key vs. a1.tx)” and you will see:
mainMenuToAPI .neuralFilter_Matching_1ToN
( textBox1.Text , dataDir + "a1.txt", dataDir + "b1.txt" );
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public bool neuralFilter_Matching_1ToN
( string key, string a1File, string b1File)
{
bool b = false;
if ( ! System.IO .File .Exists (key) )
{
appendText ( "Please enter a valid Key!\n");
return false;
}
b = this.signature_Key_Signature (key);
if ( ! b )
{
appendText ( "Key Signature computation fails!\n");
return false;
}
try
{
script.results_1N = script.neuralFilter.findMatch1N
( script.imageSignature, a1File, b1File);
}
catch (Exception e)
{
appendText ( "NeuralFilter 1:N Matching fails:\n"
+ e.ToString () + "\n");
return false;
}
if ( script.results_1N == null )
{
appendText ( "NeuralFilter 1:N Matching fails!\n" );
return false;
}
if ( script.results_1N.getStatus () )
{
setText ( script.results_1N.toString () + "\n");
appendText ("" + script.results_1N.getNumberOfMatches ()
+" matches!\n");
}
else
{
appendText ( "No Match!\n");
return false;
}
return createFile (key, b1File);
}
The following code simply makes sure the key image exists:
if ( ! System.IO .File .Exists (key) )
{
appendText ( "Please enter a valid Key!\n");
return false;
}
The next section of code computes the key signature:
b = this.signature_Key_Signature (key);
The next section code makes a 1:N Match, the results are in script.results_1N:
try
{
script.results_1N = script.neuralFilter.findMatch1N
( script.imageSignature, a1File, b1File);
}
catch (Exception e)
{
appendText ( "NeuralFilter 1:N Matching fails:\n"
+ e.ToString () + "\n");
return false;
}
The final section of code prints the 1:N Matching results:
if ( script.results_1N.getStatus () )
{
setText ( script.results_1N.toString () + "\n");
appendText ("" + script.results_1N.getNumberOfMatches ()
+" matches!\n");
}
else
{
appendText ( "No Match!\n");
return false;
}
In section 7.4, we have already discussed how to display the matching results from the result object, script.results_1N. At the end of a 1:1 or 1:N matching, a Result object will be produced. This object will be entered into GUI_Result object
guiResults.outputResults(script.results_1N, softwareDir);
To disply the first, next,
and previous result images, call:
guiResults.Button_First(false);
guiResults.Button_Next(false);
guiResults.Button_Previous(false);
11. Binary Search
For a large amount of images, say above 1 million images, binary search is much faster. However, binary search requires additional steps, signature sorting and signature management. The “Binary Search” menu is given in Figure 11.1.
The chapter project is located at:
C:\transapplet80\imagefinder\.
The executable file is located at:
c:\transapplet80\imagefinder\ imagefinder
Lite\bin\Release\.
We also call this folder “.\”. Run the project to get the figure below.
Figure 11.1 Binary Search Menu Items.
The search steps are:
1. Select a folder (Click the “1” Button in
the Toolbar, see Figure 11.1)
2. Get Library Signatures (Click the “2” button
in the Toolbar, see Figure 11.1)
3. Sort Signature (see Figure 11.1).
4. Select a Key image (Click the “3” Button
in the Toolbar, see Figure 11.1).
5. Search (There are several options in
Figure 11.1).
11.1 Signatures
For binary search, the signature file is sort1.txt. Unlike sequential search, which allows many signature files to be loaded via a library, the sorted signatures will use one single file, so you have to manage this file.
To get a signature file sort1.txt:
1. Select a folder (Click the “1” Button in
the Toolbar, see Figure 11.1).
2. Get Library Signatures (Click the “2”
button in the Toolbar, see Figure 11.1).
3. Click menu item, “Binary Search/Create
sort1.txt from a1.txt”.
If you have several signature files, you have to merge them:
- Create sort1.txt;
- Copy sort1.txt to sort2.txt;
- Create sort1.txt with new data;
- Merge sort1.txt and sort2.txt to
sort3.txt.
The merger will be done by clicking “Binary Search/Merge sort1.txt, sort2.txt to sort3.txt”.
Sorting a signature file takes two steps:
- Create an index file from a signature file;
- Sort signatures from the signature file and index file.
Click the menu item, “Binary
Search/Create sort1.txt from a1.txt”, you will see:
private void createSort1txtFromA1txtToolStripMenuItem_Click
(object sender, EventArgs e)
{
mainMenuToAPI.Sort_CreateSort1Froma1
(dataDir + "a1.txt",
dataDir +
"index1.txt",
dataDir +
"sort1.txt");
}
Which
will call the following function.
internal bool Sort_CreateSort1Froma1
(string a1_txt, string index1_txt, string sort1_txt)
{
bool bTmp = Sort_indexing(
a1_txt, index1_txt);
if ( ! bTmp )
{
appendText("Indexing
Fails!\n");
return false;
}
else
{
appendText("Indexing
Completed!\n");
}
bTmp = Sort_sortitngFile(a1_txt,
index1_txt, sort1_txt);
if (!bTmp)
{
appendText("sort1.txt failed!\n");
return false;
}
else
{
appendText("sort1.txt
created!\n");
}
return true;
}
The first step is to create an index file:
private bool Sort_indexing(string
a1_txt, string index1_txt)
{
bool b;
try
{
b =
script.se.setInputFile(a1_txt );
if (!b)
{
appendText("Input File
fails: " + a1_txt +"\n");
return false;
}
b =
script.se.setOutputIndexFile(index1_txt);
if (!b)
{
appendText("Cannot
create Index File" + index1_txt + "\n");
return false ;
}
b = script.se.indexing();
if (!b)
{
appendText("Indexing
fails\n");
return false;
}
}
catch (Exception ex)
{
appendText(ex.ToString());
return false;
}
return true;
}
This
function use an SortEngine object, script.se. It first assign the input image
signature file, and assign output index file, then it creates an sorted index
file by:
b = script.se.indexing();
The
next step is to sort the signature file,
private bool
Sort_sortitngFile(string a1_txt, string index1_txt, string
sort1_txt)
{
bool
b;
try
{
b =
script.se.setInputFile(a1_txt);
if
(!b)
{
appendText("Input File fails: " + a1_txt + "\n");
return false;
}
b =
script.se.setOutputIndexFile(index1_txt);
if
(!b)
{
appendText("Cannot create Index File" + index1_txt
+ "\n");
return
false;
}
b =
script.se.setOutputLibraryFile(sort1_txt);
if
(!b)
{
appendText("Cannot create library File" + sort1_txt
+ "\n");
return
false;
}
b = script.se.sorting();
if
(!b)
{
appendText("Sorting fails\n");
return
false;
}
appendText("Output file : " + sort1_txt + "\n");
}
catch
(Exception ex)
{
appendText(ex.ToString());
return
false;
}
return
true;
}
This
function will first specify:
input signature
file – file containing image signautres;
index file –
file containing sorted index; and
output signature file – file to be produced with all
signatures sorted.
The
main statement in this function is:
b = script.se.sorting();
At
this point, a sorted signature file is created and saved in the output
signature file.
11.2 Search
To make a search:
- Select a Key image (Click the “3” Button
in the Toolbar, see Figure 11.1).
- Search.
There are several search options:
- Binary Search/1:N Search (key vs. sort1.txt)
- Binary Search/1:N Search (10%)
- Binary Search/1:N Search (5%)
- Binary Search/1:N Search (3%)
- Binary Search/1:N Search (1%)
Assume you have 1 million images, the first command above will search 1 million images; the second command will search 100 thousand images; the third command will search 50 thousand images; the fourth command will search 30 thousand images; and the last command will search 10 thousand images. By definition, the Binary Search will only ignore the parts that are not likely to yield a match.
Assume you use Sequential Search (Chapter 4) for 1 million images; you will search through 1 million images. Then you use the “Binary Search/1:N Search (1%)” for the same 1 million images, provided the image signatures are sorted, you will search through only 10 thousand images. The difference is a factor of 100.
Binary Search is more restrictive than 1:N whole image matching; so the two image search results do not always match. In general, Binary Search generates less results than 1:N while image matching.
All of these menu items use the same functions, they only differ in one parameter. Click menu item, “Binary Search/1:N Search (10%)”,
private void nSearch10ToolStripMenuItem_Click(object sender, EventArgs
e)
{
int
percent = 10;
bool
bTmp = this.mainMenuToAPI.Sort_Match_1N_KeyVsSort1
(textBox1.Text,
dataDir + "sort1.txt",
dataDir + "b1.txt", percent);
if
(!bTmp)
return;
guiResults.outputResults(mainMenuToAPI.script.results_1N,
softwareDir);
guiResults.Button_First(false);
}
This function first callS a
function for 1:N matching, giving the results in mainMenuToAPI.script.results_1N.
Then it passes this result to an image display object, guiResults. The last line simply
displays the first image in the results:
guiResults.Button_First (false);
The function below does the 1:N matching:
internal bool Sort_Match_1N_KeyVsSort1
(string key, string sort1File,
string b1File, int percent)
{
bool b = false;
if (!System.IO.File.Exists(key))
{
appendText("Please enter
a valid Key!\n");
return false;
}
b = this.signature_Key_Signature(key);
if (!b)
{
appendText("Key Signature
computation fails!\n");
return false;
}
try
{
script.results_1N =
script.neuralFilterSorted .findMatch1N
(script.imageSignature, sort1File, b1File, percent);
}
catch (Exception e)
{
appendText("Quick 1:N
Matching fails:\n" + e.ToString() + "\n");
return false;
}
if (script.results_1N == null)
{
appendText("NeuralFilter
1:N Matching fails!\n");
return false;
}
if (script.results_1N.getStatus())
{
appendText(script.results_1N.toString() + "\n");
appendText("" +
script.results_1N.getNumberOfMatches() + " matches!\n");
}
else
{
appendText("No
Match!\n");
return false;
}
return createFile(key, b1File);
}
The code here is similar to the sequential 1:N matching in the last chapter. The matching is done by:
script.results_1N = script.neuralFilterSorted
.findMatch1N
(script.imageSignature, sort1File, b1File, percent);
The parameters are:
script.imageSignature –
signature of a new image;
sort1File – signature file;
b1File – output file;
percent – percent used in the search.
12. Dynamic Library
In the last a few chapters, we introduced 1:N Match and N:N Match, where N is fixed. This chapter will introduce the Dynamic Library where N can be updated via:
- insertion,
- deletion, and
- replacement.
The chapter project is located at:
c:\transapplet\80\imagefinder\.
12.1 Dynamic Library Menu
Figure 12.1 and Figure 12.2 shows the dynamic library menu.
Figure 12.1 Dynamic Library Menu.
Figure 12.2 Dynamic Library/Binary Search Menu.
12.2 Dynamic Library API
The following table lists the dynamic library functions.
|
Functions |
Descriptions |
Comments |
|
int getLibraryID () |
Gets the Library ID (optional). |
|
|
void setLibraryID (string x) |
Sets the Library ID (optional). |
|
|
bool load () |
Loads the default master library, lib1.txt. |
|
|
bool load ( string fileName) |
Loads master library specified by fileName. |
|
|
bool load ( string fileName1, string fileName2) |
Loads two libraries specified by fileName1 and fileName2. |
|
|
bool clear() |
Clears the current library from RAM only. |
|
|
bool backup () |
Saves current library to the default file, lib1_bk.txt. |
|
|
bool backup (string fileName) |
Saves current library to a back file. |
|
|
bool addSignature (ImageSignature sig) |
Adds a signature to a loaded image library in RAM. |
|
|
bool deleteSignature (ImageSignature sig) |
Deletes a signature from a loaded image library in RAM. |
|
|
bool deleteSignature (string ID) |
Deletes a signature from a loaded image library in RAM. |
|
|
bool replaceSignature (ImageSignature sig) |
Replaces a signature from a loaded image library in RAM. |
|
|
bool mergeLibrary (string libFile1, string libFile2, string outputLib) |
Merges two signature libraries into a single library. Input: string libFile1 string libFile2 string outputLib Output: A text file that contains the library of signatures from both input libraries. |
|
12.3 Creating A Master Library
Up to this point when we have done 1:N or N:N Matching, the N-images have been fixed. The Dynamic Library allows you to update the N-images, including inserting, deleting, and replacing signatures. We now create the master library. The master library file is lib1.txt.
Assuming we have created the signature file, a1.txt. To use the Dynamic Library, you have to create a library file lib1.txt. To create a library file, open a1.txt and save it to lib1.txt, or click:
“Library/Maintenance/Create lib1.txt from a1.txt”.
12.4 Load Dynamic Library
The dynamic library uses file, lib1.txt, as a master library file. Before we can use the dynamic library, this file must be loaded into the dynamic library. To load the library file, click “Library/Maintenance/Load lib1.txt”.
Double click menu item “Library/Maintenance/Load lib1.txt” and you will see:
mainMenuToAPI.library_loadLib (dataDir +"lib1.txt" );
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public bool library_loadLib ( string lib1_txt)
{
if (script.imageLibrary == null )
{
appendText ( "Unable to find a library !\n");
return false;
}
try
{
bool b = script.imageLibrary.load ( lib1_txt);
if ( b )
{
appendText ("Library 1 loaded!\n");
}
}
catch (Exception ee )
{
appendText ( ee.ToString () + "\n" );
return false;
}
return true;
}
The following statement loads the library:
bool b = script.imageLibrary.load ( lib1_txt);
12.5 Library M:N Matching
In an N:M
Matching, a1.txt will be matched against the loaded library. Click menu item
“Library/ N:M Matching (a1.txt vs. lib1.txt)” menu
item to complete a N:M Match.
Double click menu item, Library/ N:M Matching (a1.txt vs. lib1.txt), and enter:
bool b = mainMenuToAPI.library_Matching_N_M
( dataDir + "a1.txt", dataDir + "b1.txt" );
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public bool library_Matching_N_M ( string a1_txt, string b1_txt)
{
if ( script.imageLibrary == null )
{
appendText ("Dynamic Library not available!\n");
return false;
}
bool b = script.neuralFilter .setLibrary ( script.imageLibrary );
if (!b)
{
appendText ("Dynamic Library Assignment Fails!\n");
return false;
}
try
{
script.neuralFilter.findMatchNN ( a1_txt, b1_txt);
}
catch (Exception e)
{
appendText ( e.ToString () + "\n");
return false;
}
return true;
}
In this statement, the script object contains all of the objects required for the ImageFinder project. In particular, “script.imageLibrary” is the dynamic library object.
The following statement assigns the dynamic library to the NeuralFilter object, “script.NeuralFilter”:
bool b = script.neuralFilter .setLibrary ( script.imageLibrary );
if (!b)
{
appendText ("Dynamic Library Assignment Fails!\n");
return false;
}
Once the NeuralFilter object obtains this library, it will match images against this master library by the following statement:
script.neuralFilter.findMatchNN ( a1_txt, b1_txt);
12.6 Library 1:N Matching
Double click menu item “Library/ 1:N Matching (Key vs. lib1.txt)” and enter:
bool b = this.mainMenuToAPI.library_Matching_1_N
(textBox1.Text , dataDir + "b1.txt" );
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public bool library_Matching_1_N( string key, string b1File)
{
if ( script.imageLibrary == null )
{
appendText ("Dynamic Library not available!\n");
return false;
}
bool b = script.neuralFilter .setLibrary ( script.imageLibrary );
if (!b)
{
appendText ("Dynamic Library Assignment Fails!\n");
return false;
}
b = false;
if ( ! System.IO .File .Exists (key) )
{
appendText ( "Please enter a valid Key!\n");
return false;
}
b = this.signature_Key_Signature (key);
if ( ! b )
{
appendText ( "Key Signature computation fails!\n");
return false;
}
try
{
script.results_1N = script.neuralFilter.findMatch1N
( script.imageSignature);
}
catch (Exception e)
{
appendText ( "NeuralFilter 1:N Matching fails:\n"
+ e.ToString () + "\n");
return false;
}
if ( script.results_1N == null )
{
appendText ( "NeuralFilter 1:N Matching fails!\n" );
return false;
}
if ( script.results_1N.getStatus () )
{
setText ( script.results_1N.toString () + "\n");
appendText ("" + script.results_1N.getNumberOfMatches ()
+" matches!\n");
}
else
{
appendText ( "No Match!\n");
return false;
}
}
The only difference between this 1:N Matching and the NeuralFilter 1:N Matching introduced earlier is the following statement:
bool b = script.neuralFilter .setLibrary ( script.imageLibrary );
In a normal 1:N Matching, the neural filter gets the library from a1.txt. This statement assigns the dynamic library, “script.imageLibrary”, to the NeuralFilter object, “script.NeuralFilter”.
12.7 Update Implementation
Double click menu item “Library/Maintenance/Add (Key)” and enter:
if ( ! System.IO .File .Exists (textBox1.Text ) )
{
this.mainMenuToAPI.appendText
("File does not exist: \n" + textBox1.Text + "\n");
return;
}
this.mainMenuToAPI.library_add (textBox1.Text );
Here, mainMenuToAPI is an object, which will implement all functions. As we discussed earlier, the main form simply links menu items to functions in the mainMenuToAPI object. The implementation is:
public bool library_add ( string sImage)
{
script.imageSignature
= script.signatureFilter.getSignature (sImage);
if (script.imageSignature == null )
{
this.setLine ("Image Signature computation fails!");
return false;
}
if (script.imageSignature.getStatus () <= 0 )
{
this.setLine ("Image Signature computation fails!");
return false;
}
bool b = script.imageLibrary.addSignature (script.imageSignature);
return b;
}
The script object contains all of the objects required for the ImageFinder project. In particular, “script.imageLibrary” is the dynamic library object. The first section of the code computes the image signature. The next section of code checks the validity of the signature. The following statement inserts the signature to the dynamic library:
bool b = script.imageLibrary.addSignature (script.imageSignature);
The menu item Delete and the menu item Replace can be implemented in a similar fashion and the codes are provided in the project.
13. Image Variations
The chapter project is located at:
c:\transapplet\80\imagefinder\.
13.1 Variation Signatures
Figure 13.1 A change beyond the tolerance of a single image signature.
Figure 13.2 Signature/Variation Signatures Menu.
Image Matching is done through something called Image Signature. An
image has a set of computed values called features. A collection of features is
grouped into a signature. A single image signature will take care of a certain
amount of variation. To increase the variations, additional signatures will be
added, which is the purpose of the Variation Signatures menu.
An image can be changed, for example, a rotation of 90 degrees to the left. As the image is changed, the signature will change accordingly. In general, this change will not be maintained inside a signature, i.e. a small variation allowed inside a single signature will not take care of large variations like 90 degree rotations.
The “Signature/Variation Signatures” menu item (Figure 13.2) provides a list of menu items to compute signatures. The net result is that an image will have multiple image signatures.
The “Signature/Variation Image” menu item (Figure 13.3) will compute various variations of images from the key image and will deposit these images into “.\tmp\” folder. To use this menu item, you must first select a key image. This allows a user to see what the variations are.
Figure 13.3 Signature/Variation Images Menu.
This menu supports the following variations:
Signatures/Variation Signatures/Scale (100, 90, 80, …)
This menu
item generates 7 image signatures:
100%, 90%, 80%, 70%, 60%, 50%, 40%.
Here 90% is
an image that has 5% border region removed.
Signatures/ Variation Signatures/Rotation (90, 180, 270)
This menu
item generates 4 image signatures:
0, 90, 180, 270.
Here 90 is
an image that has a 90 degree rotation.
Signatures/ Variation Signatures/Flip (X, Y, XY)
This menu
item generates 4 image signatures:
0, X, Y,
XY.
Here X is
an image that has flipped around x axis.
Signatures/ Variation Signatures/Flip Rotation (16
variations)
This menu
item generates 16 image signatures:
{0, 90, 180,
270} ^ {0, X, Y, XY}.
Here (90,
X) is an image that has 90 degree rotation and flipped around x axis.
Signatures/ Variation Signatures/Stretch (4 variations)
This menu
item generates 5 image signatures:
(0, 0), (120%, 100%), (80%, 100%), (100%, 120%), (100%, 80%).
Here (120%,
100%) are stretch percentages along the x and y directions.
Signatures/ Variation Signatures/Resize
This menu
item generates 5 image signatures:
100, 150, 200, 300, 400.
Here 150
increases the image size by 50% in terms of pixel counts in each direction.
The implementation for menu item, Signatures/Variation Signatures/Scale (100, 90, 80, …), is:
script.signatureFilterSym
.getKeywordLibrary_Scale
(f.guiSource.libraryList.imageAbsolutePath,
f.guiSource.libraryList.imageID,
f.guiSource.libraryList.imageX, f.guiSource.libraryList.imageY,
f.guiSource.libraryList.imageW, f.guiSource.libraryList.imageH,
outputFile,
f.guiSource.libraryList.key0, f.guiSource.libraryList.key1,
f.guiSource.libraryList.key2, f.guiSource.libraryList.key3,
f.guiSource.libraryList.key4, f.guiSource.libraryList.key5,
f.guiSource.libraryList.key6, f.guiSource.libraryList.key7,
f.guiSource.libraryList.key8, f.guiSource.libraryList.key9,
f.mainMenuToAPI.script.tranAppletPara .signatureFilter_LibraryScaleType
);
13.2 Segments
A collection of
features is grouped into a signature. One image has one signature. The signature matches whole images against whole images,
i.e. it does not look inside the image.
An image can be many logical segments. As the focus changes from a whole image to an image segment inside the image (Figure 13.3), the signature will change accordingly. In general, this change will not be maintained inside a basic signature.
The “Signature/Segment
Signatures” menu item attempts to partially address this
image segmentation problem. The menu item will look at some segments inside an
image. On average, it will look at six segments.
The “Signature/Segment Images” menu item (Figure 13.2) computes signatures for image segments. The net result is that an image will have multiple image signatures, one for each segment.
The “Signature/Segment Images” menu item (Figure 13.3) will compute segments from the key image and will deposit them into “.\tmp\” folder. To use this menu item, you must first select a key image. This allows a user to see what the segments are.
Figure 13.4 Image Segmentations.
The object, imgSigFilterSeg, used for computing segment
signatures, is an instance of the following class:
Attrasoft.TransApplet80.ImageSignatureFilterSegment80.ImageSignatureFilterSegment80 imgSigFilterSeg;
The implementation of the menu item, Signature/Segment Signatures, is:
imgSigFilterSeg.getSegmentLibrary_Modelbased(sourceFolder,
outputFile);
This function will convert images in parameter, sourceFolder, into image signatures and save them to a file, outputFile.
14. Sub-Image Search
The chapter project is located at:
c:\transapplet80\imagefinder\.
The sub-image matching requires a different set of parameters. There are two different settings:
Setting/Search Sub-Images
Setting/Search Whole Images
To match sub-images, click menu item, “Setting/Search Sub-Images”. To switch back to the original setting for whole images, click: “Setting/Search Whole Images”.
Figure 14.1 Setting Change.
14.1 Sub-Images and Variations
Starting from this chapter, we will search inside an image. We will deal with the problems of:
- Sub-image
- Variations.
The number of sub-images is very large, measured by exponential functions of the number of pixels. The number of variations is also very large:
- Translated segments;
- Rotated segments;
- Scaled segments;
- Rotated and Scaled segments;
- Brighter or Darker segments;
· ...
- All of the above simultaneously.
We will not use image signatures because of the large amount of variations. This will significantly slow down the search speed.
Because the variations for searching inside an image are huge, we might impose some limits on where to search. There are several approaches:
- Automatic Segmentation;
- Enumerative.
Automatic segmentation deploys certain algorithms to look at a small number of segment locations inside the image. There are two menu items to address this approach (Chapter 13):
The “Signature/Segment Signatures” menu item (Figure 13.2) computes signatures for a selected number of image segments. The net result is that an image will have multiple image signatures, one for each segment.
The “Signature/Segment Image” menu item (Figure 13.3) will compute segment images from the key image and will deposit them into “.\tmp\” folder. To use this menu item, you must first select a key image. This allows a user to see what the segments are.
The advantages of automatic segmentation are that it will dramatically reduce the number of search locations. The disadvantages of segmentation algorithms are that it will not be able to address some human cropped images which cut right through the middle of a segment.
The enumerative approaches are not restricted by the automatic segmentation algorithm; however, the variations for enumerative sub-image searching are huge; some sort of limits on where to search will have to be imposed.
Figure 14.2 Two parameters.
Figure 14.3 Sub-Image Menu.
14.2 Search Process
In this chapter, we will use Enumerative search by imposing these two conditions:
- Translation invariance only;
- No scale, rotation, …, invariance.
In each search, there are two parameters N x M (Figure 14.2), which
specify an array of sub-images to search. The software will search via an “N by
M” grid for each unknown image for the specified key image. The key will not be
rotated or scaled.
Initially, you will need three steps for an image search:
1. Select a Key;
2. Select a folder;
3. Search.
To see the results:
1. Click the Results Home button (Figure 14.3);
2. Click the Results “=>” button (Figure 14.3).
Starting from the second search, you do not have to select the folder
again because it is already selected; each search has only two steps:
1. Select a Key;
3. Search;
1. Select a Key;
3. Search;
…
Note:
Step 1, selecting a Key can be completed by
dragging & dropping an image to the first text box;
Step 2, select a
folder, which can be completed by dragging & dropping a folder to the
second text box.
14.3 Learn by Example
Figure 14.4 1:1
Matching.
The first example (see Figure 14.4) is to match the key,
.\ ex_object1\key.jpg,
against the image,
.\ ex_object1\CIMG4643_mid_shelf1.JPG.
The example will take three clicks:
1.
Click the “Key” button in the Toolbar,
and select .\ ex_object1\ key.jpg. You can also drag & drop .\ ex_object1\key.jpg to the first text box.
Figure 14.5 Step 1, selecting a key. After selecting a key image, the image will be displayed in the Key Panel.
2.
Click the “Source” button in the
Toolbar and select the folder, .\ ex_object1\.
You
can also drag & drop the folder, .\ ex_object1\, to the second text box; the use “=>’ button
to select image, .\
ex_object1\CIMG4643_mid_shelf1.JP, see
Figure 14.6.
Figure 14.6 Step 2, selecting a folder. After selecting a folder, the first
image will be displayed in the Source Panel (middle picture box). The list of
image files will be displayed in the text window. The Home button goes to the
first image; the “=>” button goes to the next image; and the “<=” button goes to the previous image. All sub-folders
will be included.
Figure 14.7 Search the N x M grid of sub-images. For example, setting them to
30 x 5 will result in searching 150 sub-images.
3.
Before we make a search, you will
specify how to search via an “N by M” grid, i.e. how many sub-images we will
look at for the specified key image. The default value is a 10 by 10 grid.
To
make a 1:1 search,
·
Enter 30 and 5 in Figure 14.7.
·
Click the “Sub-Image/1:1 Matching”
menu item in Figure 14.3.
You
will get:
ID Name Path Score X Y W H
CIMG4643_mid_shelf1.JPG_146 CIMG4643_mid_shelf1.JPG .\ex_object1\ 84 366 34 73 184
CIMG4643_mid_shelf1.JPG_134 CIMG4643_mid_shelf1.JPG .\ex_object1\ 84 197 34 73 184
CIMG4643_mid_shelf1.JPG_123 CIMG4643_mid_shelf1.JPG .\ex_object1\ 83 42 34 73 184
CIMG4643_mid_shelf1.JPG_110 CIMG4643_mid_shelf1.JPG .\ex_object1\ 83 282 25 73 184
CIMG4643_mid_shelf1.JPG_117 CIMG4643_mid_shelf1.JPG .\ex_object1\ 81 380 25 73 184
CIMG4643_mid_shelf1.JPG_100 CIMG4643_mid_shelf1.JPG .\ex_object1\ 80 141 25 73 184
Note, if you do not get this, click menu
item, “Setting/Search Sub-Images” and then try again.
Figure 14.8
Search the selected folder. Images
that matched the “key” will be displayed in the Results Panel. Use the Home,
=>, and <=buttons to see matched images. The Home button goes to the
first image; the “=>” button goes to the next image; and the “<=” button goes to the previous image.
Figure 14.9
The matched score and coordinates,
(x, y, w, h), are given in a text file: “.\data\c.txt”.
To see the first matched image, click the Home button in the Results Panel (see Figure 14.8). To see the next matched image, click the “=>” button in the Results Panel. The key and the matched images are displayed side-by-side.
14.4 Implementation
The basic idea is to cut a segment of a source image, which has the
same size as the key image. These two images are matched via image signatures.
This process is repeated for various locations.
The basic function is:
r80
= script.smallSegment.findmatch11_NeuralFilter_LargeImage
(b1,
b2, theID, thePath, theName, f.pictureBox4);
Where:
- findmatch11_NeuralFilter_LargeImage (…) is the function;
- B1 is the key bitmap;
- B2 is the source bitmap;
- theID is the image ID used in the output file;
- thePath is the image Path used in the output file;
- f.pictureBox4 displays each sub-image of B2 that is matched against Bitmap, B1.
15. Fast Sub-Image Matching
The sub-image matching requires a different set of parameters. There are two different settings:
Setting/Search Sub-Images
Setting/Search Whole Images
To match sub-images, click menu item, “Setting/Search Sub Images”. To switch back to the original setting for whole images, click: “Setting/Search Whole Images”.
The objective for the last chapter menu is to find a cropped segment at
all cost, provided there are no scaling factors. This can take a long time if
the N x M search grid is set to be very large.
The first objective of this chapter is to find a cropped segment much
faster.
The second objective is to add scale invariance and rotation
invariance.
Figure 15.1 “Fast” menu items.
15.1 Fast Menu
The objective of the “Fast” menu items is to find a cropped segment
from an image within 1 second.
The “Fast” menu items provide a much faster search than the menu items
of the last chapter, but these menu items are less accurate than those in the
last chapter. It is a trade-off between accuracy and computation time. Figure
15.1 shows the menu items under this menu.
Example. We will match (see Figure 15.4 from last
chapter) the key,
.\ ex_object1\key.jpg,
against the image,
.\ ex_object1\CIMG4643_mid_shelf1.JPG.
Figure 15.2 Fast match results.
Before starting this example, click menu item, “Setting/Search Sub Images”.
The example will take three clicks:
1.
Click the “Key” button, and select .\ ex_object1\key.jpg. You can also drag & drop
.\ex_object1\key.jpg to the first text box.
2.
Click the “Source” button and select
the folder, .\ ex_object1\. You can also drag & drop the folder, .\ ex_object1\, to the second text box.
3.
Fast Search.
Click Fast/1:1 Matching and see the result in Figure 15.2.
The menu items are given below:
Sub Image/Fast 1:1
Use “Sub
Image/Fast 1:1” to make a 1:1 Matching.
Sub Image/Fast 1:N (Same size)
Use “Sub Image/Fast 1:N (Same size)” to
make a 1:N Matching; it will search through all images in a folder for the
specified cropped image. This command is faster than “Sub Image/1:N (Different
size)”, but it requires all images in the search folder to have the same pixel
size.
Sub Image/Fast 1:N (Different size)
Use “Sub Image/Fast 1:N (Different size)” to make a 1:N Matching; it will search through all
images in a folder for the specified cropped image. This command is slower than
“Sub Image/1:N (Same size)”, but it does not requires all images in the search folder
to have the same pixel size.
To implement these menu items, the object is:
f.mainMenuToAPI.script.neuralNetFilter
This object will need to learn the sample bitmap, b:
try
{
f.mainMenuToAPI.script.neuralNetFilter.setSymmetry(3);
f.mainMenuToAPI.script.neuralNetFilter.train(b, right.Width,
right.Height);
return
true;
}
catch
(Exception ex)
{
f.mainMenuToAPI.appendText(ex.ToString () + "\n");
return
false;
}
After that, it will make a fast 1:N matching:
f.mainMenuToAPI.script.results_1N
=
f.mainMenuToAPI.script.neuralnet2.findMatch1N
(f.guiSource .libraryList .imageAbsolutePath, matchingCode);
15.2 NeuralNet Filter API
The following table lists the NeuralNet Filter functions.
|
Functions |
Descriptions |
|
bool train (Bitmap img) bool train (string sPath) bool train (Bitmap img, int x, int y, int w, int h) bool train (string sPath, int x, int y, int w, int h) |
Trains the neural net, either use whole image or image segement (x, y, w, h). |
|
bool train (Bitmap b, int w, int h) |
Place an image b into a white image w pixels by h pixels, the train the neural net. |
|
bool retrain (Bitmap img) bool retrain (string sPath) bool retrain (Bitmap img, int x, int y, int w, int h) bool retrain (string sPath, int x, int y, int w, int h) |
Retrains the neural net. |
|
Attrasoft.TransApplet80.Results_1N.Results_1N findMatch11 ( Bitmap img1); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch11 ( string path1); Attrasoft.TransApplet80.Results_1N.Results_1N findMatch1N ( string [] fileList); |
Makes a 1:1 and 1:N Matching. |
|
bool findMatchNN (string [] fileList, string c1File) bool findMatchNM (string [] keyList, string [] libraryList, string c1File); |
Makes a N:N Matching and N:M Matching. |
16. Video Input
Video recognition largely mirrors the image recognition. Identify a whole video is similar to identify a whole image. This chapter describes how to enter video data to the VideoFinderLite. The input videos are further divided into:
- Key
- Search Source
A Key-folder specifies the sample videos. A Search-Folder specifies the Search Source. The chapter project is in “.\videofinderlite”. The video input programming mirrors the image input.
16.1 Class Library Name
The class library is:
Attrasoft.TransApplet80.InputModuleVideo.
The main class in this library will be:
Attrasoft.TransApplet80.InputModuleVideo .InputModuleVideo.
The objects for key and source are:
internal
Attrasoft.TransApplet80.InputModuleVideo .InputModuleVideo inputModuleKey;
internal Attrasoft.TransApplet80.InputModuleVideo.InputModuleVideo inputModuleSource;
The interface is:
public interface I_ InputModule80
{
Attrasoft .TransApplet80 .InputList80 .InputList getList( string);
}
The input is a folder path. Output is an object of the following class:
public
class VideoInputList
{
public int status = 0;
public string[] imageAbsoultePath =
null;
public string[] VideoID = null;
public string[] VideoName = null;
public string[] VideoPath = null;
public string[] key0 = null;
public string[] key1 = null;
public string[] key2 = null;
public string[] key3 = null;
public string[] key4 = null;
}
16.2 Class Library Overview
The functions in classes are listed in the following table.
|
Function |
Description |
|
Attrasoft .TransApplet80 . VideoInputList. VideoInputList getList( string folder) |
After input folder is specified, get the output object. |
16.3 Set Up the Chapter Project
Figure 16.1 Chapter Project.
The project has the following controls:
Key Button, Key Text Box
Use the “Key” button to display the selected key video folder.
Source Button, Source Text Box
Use the “Source” button to select a search video folder.
Clear Button
Use the “Clear” button to clear the rich text box.
Rich Text Box
Use Rich Text Box to display messages.
Key Picture Box
Use the Key Picture Box to display key images.
Source Picture Box
Use the Source Picture Box to display images in the search source.
OpenFileDialog
Use the OpenFileDialog to select a key image.
FolderBrowserDialog
Use the FolderBrowserDialog to select a source path.
After adding these controls, the form will look like Figure 16.1.
16.5 Key Segment
We can double click the Key Segment buttons to complete the programming:
public bool Button_Keys()
{
try
{
if
(f.folderBrowserDialog1.ShowDialog() != DialogResult.OK)
return false;
string cDir =
f.folderBrowserDialog1.SelectedPath;
f.textBox1.Text = cDir;
f.guiKey.setInput(f.inputModuleKey.getList(cDir));
if (!f.guiKey.getStatus())
{
f.appendText("Data
Source fails!\n");
return false;
}
f.guiKey.setLists();
}
catch (Exception e)
{
f.appendText(e.ToString() +
"\n");
return false;
}
return
this.f.guiKey.Button_First(false);
}
The first section of code opens a file dialog so the user can select a file:
if
(f.folderBrowserDialog1.ShowDialog() != DialogResult.OK)
return false;
string cDir =
f.folderBrowserDialog1.SelectedPath;
The next section enters the selected folder to the InputModuleKey object and gets the string list of the image paths:
f.inputModuleKey.getList(cDir);
Then it passes this list to display object:
f.guiKey.setInput(f.inputModuleKey.getList(cDir));
The last section displays the first images:
f.guiKey.setLists();
f.guiKey.Button_First(false);
The video input process is basically the same as image input process.