Attrasoft VideoFinder
About Attrasoft VideoFinder
Attrasoft VideoFinder consists of two components, VideoFinder and FrameFinder. The software will perform three independent tasks:
- Attrasoft VideoFinder matches a newly obtained video clip against a pre-established library of video clips.
- Attrasoft FrameFinder searches video clips for an object specified by a key image.
- Attrasoft FrameFinder searches live video for an object specified by a key image.
Software Requirements
Software
Requirements:
(1) Microsoft
Windows;
(2) Windows
.Net Framework.
To get the latest version of .Net Framework, use Internet Explorer, then click
“Tools\Windows Update".
Installing the Software
1. If you have not done so, go to Internet Explorer, then click “Tools\Windows Update" to download Windows .Net Framework .
2. Click “CD:\setup.exe” to install the VideoFinder.
3. The serial number is the word “attrasoft”.
Information and Support
Attrasoft ImageFinder
Attrasoft
P. O. Box 13051
Savannah, GA. 31406
USA
gina@attrasoft.com (Email Subject: Attrasoft)
Phone: (912)
484-1717
© Attrasoft 1998 - 2008
Statement of Copyright Restriction
Attrasoft has copyrighted the Attrasoft program you have purchased, and your rights of ownership are subject to the limitations and restrictions imposed by the copyright laws outlined below.
It is against the law to copy, reproduce or transmit (including, without limitation, electronic transmission over any network) any part of the program except as permitted by the copyright act of the United States (title 17, United States code). However, you are permitted by law to write the contents of the program into the machine memory of your computer so the program may be executed. You are also permitted by law to make a back-up copy of the program subject to the following restrictions:
· Each back-up copy must be treated in the same way as the original copy purchased from Attrasoft;
·
No copy
(original, or back-up) may be used while another copy, (original, or back-up)
is in use;
· If you ever sell or give away the original copy of the program, all back-up copies must also be sold or given to the same person, or destroyed.
© Attrasoft 2004 - 2008
Statement of Copyright Restriction
1. Introduction To VideoFinder
1.5 Supported Video Formats and
Camera
4. VideoFinder Reference Manual
6. Introduction to FrameFinder
10.5 Unsupervised Filter &
BioFilter
10.7.5 Area of Interest (AOI) 78
10.7.8 Internal/External Weight Cut
10.7.11 Use BioFilter & Use Neural Filter
11. FrameFinder Reference Manual
12. Video FrameFinder Examples
14.1 About Attrasoft VideoFinder
14.6 Supported Video Formats and
Camera
1. Introduction To VideoFinder
1.1 VideoFinder Functions
Attrasoft VideoFinder consists of two components, VideoFinder and FrameFinder. The software will perform three independent tasks:
- Attrasoft VideoFinder matches a newly obtained video clip against a pre-established library of video clips.
- Attrasoft FrameFinder searches video clips for an object specified by a key image.
- Attrasoft FrameFinder searches live video for an object specified by a key image.
Figure 1.1 Attrasoft
VideoFinder verifies the copyright information of a newly obtained video clip by
matching it against a pre-established library of video clips.
After installing the software, the folder structure is:
Videofinder\
Framefinder\
The VideoFinder software is stored in “videofinder\” folder;
the FrameFinder is stored in “framefinder\” folder; and
the examples are stored in “examples\” folder.
1.2 Beginner Menu
Both VideoFinder and FrameFinder have a “Beginner” menu. You can operate the software from the “Beginner” menu alone.
Figure 1.2 The “Beginner” Menu
and its menu items.
1.3 Signature Concepts
The unique attributes of a video clip – its “video signature” or “video fingerprint” - are used.
The matching between a new video clip and previously stored video clips is implemented via “Signature” matching, i.e. all videos must be converted into signatures first. A library is a collection of previously stored signatures. The matching is done via a new key signature against a particular library.
You can have as many libraries as you want. Before matching, you will need to select a library and load it to the Attrasoft VideoFinder; then, you are ready to verify the copyright information.
VideoFinder uses signatures; FrameFinder does not use signatures.
1.4 User Interface
Both VideoFinder and FrameFinder have a similar design. The VideoFinder has three video panels: ‘Key’, ‘Library’, and ‘Results’.
Figure 1.3 Three Video Panels:
‘Key’, ‘Library’, and ‘Results’. The Key video panel is labeled by the ‘Key’
label.
Key:
A key is a newly captured/obtained video clip. A key video is selected via the “Key” button. Use the “Play” and “Stop” button below the label ‘Key’ to play this video clip.
Library:
A library is a collection of previously stored video clips. Library videos are selected via the “Library” button. Use the “Play” and the “Stop” button below the label ‘Library’ to play a video clip. Use the “F”, “>”, and “<” button to select a video clip in the library, here “F” stands for First; “>” stands for Next; and “<” stands for previous.
Results:
Use the “Play” and the “Stop” button below the label ‘Results’ to play this video clip. Use the “F”, “>”, and “<” button to select a video clip in the matched collections.
Figure 1.4 Three Video Panels:
‘Key’, ‘Library’, and ‘Results’. The Library video panel is labeled by the
‘Library’ label.
1.5 Supported Video Formats and Camera
The VideoFinder uses Windows .Net Framework for video decoding and camera. It supports *.avi files, *.wmv files, and limited *.mpg files. It supports Logitech and several other cameras.
2. How to Use VideoFinder
Initially, you will need the following steps for video matching:
0. Initialization (Only Once after starting the software)
1. Select Library (Source button)
2. Get Library Signatures (source.txt)
3. Select Key (Key button)
4. Get Key Signatures (key.txt)
5. Create Library File (lib1.txt)
6. Load Library (lib1.txt)
7. 1:N Matching (Match/1:N Match menu item)
8. First Results (2nd F button)
9. Next Results (2nd > button)
Later, you do not have to create a library each time:
0. Initialization (Only Once After Starting the Software)
6. Load Library (lib1.txt)
3. Select Key (Key button)
4. Get Key Signatures (key.txt)
7. 1:N Matching (Match/1:N Match menu item)
3. Select Key (Key button)
4. Get Key Signatures (key.txt)
7. 1:N Matching (Match/1:N Match menu item)
…
2.1 How to Use VideoFinder
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking:
· “Beginner/0. Initialization (Only Once)” menu item, OR
· “Init” button.
1. Select
Library (Source button)
A video clip is converted into a video clip signature. A set of video clip signatures forms a library. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the VideoFinder.
This step selects a video clip directory. This step is implemented by clicking:
· “Beginner/1. Select Library (Source button)” menu item, OR
· “Source” button.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
Figure 2.1 Step 1, Select
Library (Source button)
2. Get Library Signatures (source.txt)
This step converts all video clips in the selected directory into signatures and saves them to the file, “source.txt”. This step is implemented by clicking:
· “Beginner/2. Get Library Signatures (source.txt)” menu item, OR
· “Signature/Library Signatures” menu item.
3. Select Key (Key button)
A key is a newly obtained video clip. The video clip is converted into a signature. The key (for example, “c:\abc\def\ghi.avi”) will be entered into the VideoFinder.
This step selects a key. This step is implemented by clicking:
· “Beginner/3. Select Key (Key button)” menu item, OR
· “Key” button.
The selected key will be displayed in the first picture box. Use the “Play” and “Stop” buttons to play the video clip.
Figure 2.2 Step 3, Select Key
(Key button)
4. Get Key Signatures
(key.txt)
This step converts a key video clip into a signature and saves them to the file, “key.txt”. This step is implemented by clicking:
· “Beginner/4. Get Key Signatures (key.txt)” menu item, OR
· “Signature/Key Signature” menu item.
5. Create Library File
(lib1.txt)
Before matching, you will need to select a library and load it to the VideoFinder. The library to be loaded is fixed to the file, “lib1.txt”. We have created a library file, “source.txt”; now we will simply copy it to the loading-library file, “lib1.txt”.
This step is implemented by clicking:
· “Beginner/5. Create Library File (lib1.txt)” menu item, OR
· “Library/Copy source.txt to lib1.txt” menu item.
6. Load Library (lib1.txt)
Before matching, you will need to select a library and load it to the VideoFinder. This step is implemented by clicking:
· “Beginner/6. Load Library (lib1.txt)” menu item, OR
· “Library/Load (lib1.txt)” menu item.
7. 1:N Matching (Match/1:N Match menu item)
“1:N Matching” (read as 1 to N matching) compares “1” (i.e., the new video clip) with “N” (i.e., the previously stored video clips). This step is implemented by clicking:
· “Beginner/7. 1:N Matching” menu item, OR
· “Match/1:N Matching” menu item.
Figure 2.3 Step 7, 1:N
Matching
Figure 2.4 Step 7, 1:N
Matching Text File Output
8. First Results (2nd
F Button)
The 1:N matching results are also displayed in the third picture box (labeled ‘Results’).
To select the first retrieved video clip, use:
· “Beginner/8. First Results (2nd F Button)” menu item, OR
· Second “F” button.
Use the “Play” and “Stop” buttons to play the video clip.
9. Next Results (2nd
> Button)
The 1:N matching results are also displayed in the third picture box (labeled ‘Results’).
To select the next retrieved video clip, use:
· “Beginner/9. Next Results (2nd > Button)” menu item, OR
· Second “>” button.
Use the “Play” and “Stop” buttons to play the video clip.
2.2 Example
In this section, we will use the “Friends” Video Clips. After installing the software, the folder structure is:
.\Videofinder\
.\Framefinder\
.\Examples\
where “.\” is the folder where the software is installed. The data stored used in this section is in folder, “.\examples\mwv_friends\”.
0. Initialization (Only Once
After Starting the Software)
After starting the software, click:
· “Beginner/0. Initialization (Only Once)” menu item, OR
· “Init” button.
1. Select Library (Source
button)
Click:
· “Beginner/1. Select Library (Source button)” menu item, OR
· “Source” button.
Go to folder, “.\examples\mwv_friends\”, and select any file, see Figure 2.1. The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
2. Get Library Signatures (source.txt)
This step converts all video clips in the selected directory into signatures and saves them to the file, “source.txt”. This step is implemented by clicking:
· “Beginner/2. Get Library Signatures (source.txt)” menu item, OR
· “Signature/Library Signatures” menu item.
3. Select Key (Key button)
Click:
· “Beginner/3. Select Key (Key button)” menu item, OR
· “Key” button.
And select “.\examples\mwv_friends\FRIENDS_SEASON_2_0001.mwv”, see Figure 2.3. The selected key will be displayed in the first picture box. Use the “Play” and “Stop” buttons to play the video clip.
4. Get Key Signatures
(key.txt)
Click:
· “Beginner/4. Get Key Signatures (key.txt)” menu item, OR
· “Signature/Key Signature” menu item.
5. Create Library File
(lib1.txt)
Click:
· “Beginner/5. Create Library File (lib1.txt)” menu item, OR
· “Library/Copy source.txt to lib1.txt” menu item.
6. Load Library (lib1.txt)
Click:
· “Beginner/6. Load Library (lib1.txt)” menu item, OR
· “Library/Load (lib1.txt)” menu item.
7. 1:N Matching (Match/1:N Match menu item)
Click (Figure 2.3 and 2.4):
· “Beginner/7. 1:N Matching” menu item, OR
· “Match/1:N Matching” menu item.
8. First Results (2nd
F Button)
Click:
· “Beginner/8. First Results (2nd “F” Button)” menu item, OR
· Second “F” button.
Use the “Play” and “Stop” buttons to play the video clip.
9. Next Results (2nd
> Button)
Click:
· “Beginner/9. Next Results (2nd > Button)” menu item, OR
· Second “>” button.
Use the “Play” and “Stop” buttons to play the video clip.
2.3 Beginner Tool Bar
Figure 2.5 The Beginner Tool Bar.
The beginner toolbar duplicate the Beginner menu.
3. Advanced Users
3.1 Signatures
Figure 3.1 Signature Menu.
The matching between a new video clip and previously stored video clips is implemented via “Signature” matching, i.e. all video must be converted into signatures first. A library is a collection of previously stored signatures. The matching is done via a new key signature against a particular library. You can have as many libraries as you want.
Before matching, you will need to select a library and load it to the VideoFinder; then, you are ready to verify the copyright information. The Signature menu provides you the menu items required for converting a video into signature files.
3.1.1 Library Signatures
The default library signatures are computed at the rate of 1 signature every 1.0 second. The menu items are:
Signature/Library Signature
Use "Signature/Library Signature" menu item to collect signatures from a source specified by the "Library" button and displayed in the ‘Library’ text box.
Signature/Get Current Source Signature
Use "Signature/Get Current Source Signature" menu item to collect signatures from a single video currently selected via "F"-button, ">"-button, or "<"-button and save it to file, “source_current.txt”.
This rate of 1 signature per second can be reset via the “Parameter” button, which will specify the starting frame, ending frame, and interval between signatures. The following menu item will compute the signatures with user-specified starting frame, ending frame, and interval between signatures.
Signature/Library Signature (User Setting)
Use "Signature/Library Signature" menu item to collect signatures from a source specified by the "Library" button and displayed in the ‘Library’ text box. Users via the “Parameter” button determine the start frame, ending frame, and interval.
Example:
Let a video clip be 60 seconds, the default setting will be:
Starting Frame = 0
Ending Frame = 60
Interval = 1
You set it to:
Starting Frame = 10
Ending Frame = 30
Interval = 2
Now you collect 10 frames.
3.1.2 Key Signatures
The default key signatures are computing 40 – 80 signatures, depending on the length of the video. The menu item is:
Signature/Key Signature (Default)
Use "Signature/Key Signature (Default)" to convert a key video clip into a signature and save them to the file, “key.txt”.
The Key Signature Rules are:
Let N be the video Length In Seconds,
If N < 40 seconds, then the number of signatures is N, one per second;
Else if N < 80 seconds, then the number of signatures is 40, one per second;
Else if N < 120 seconds, then the number of signatures is 40, one every 2 seconds;
Else if N < 160 seconds, then the number of signatures is 40, one every 3 seconds;
Else if N < 200 seconds, then the number of signatures is 40, one every 4 seconds;
Else if N < 240 seconds, then the number of signatures is 40, one every 5 seconds;
Else if N < 280 seconds, then the number of signatures is 40, one every 6 seconds;
Else if N < 320 seconds, then the number of signatures is 40, one every 7 seconds;
Else if N < 360 seconds, then the number of signatures is 40, one every 8 seconds;
Else if N < 400 seconds, then the number of signatures is 40, one every 9 seconds;
Else if N < 440 seconds, then the number of signatures is 40, one every 10 seconds;
Else if N < 500 seconds, then the number of signatures is 40, one every 10 seconds;
Else if N < 550 seconds, then the number of signatures is 50, one every 10 seconds;
Else if N < 600 seconds, then the number of signatures is 50, one every 11 seconds;
Else if N < 650 seconds, then the number of signatures is 50, one every 12 seconds;
…
Besides the default setting, there are several options:
Signature/Key Signature (20)
Use "Signature/Key Signature (20)" menu item to collect 20 signatures from a video, regardless of the video length.
Signature/Key Signature (40)
Use "Signature/Key Signature (40)" menu item to collect 40 signatures from a video, regardless of the video length.
Signature/Key Signature (80)
Use "Signature/Key Signature (80)" menu item to collect 80 signatures from a video, regardless of the video length.
Signature/Key Signature (Every 2 Sec)
Use "Signature/Key Signature (Every 2 Sec)" to convert a key video clip into a signature and save them to the file, “key.txt”. The number of signatures is the video length in seconds divided by 2.
Again, the “Parameter” button will specify the starting frame, ending frame, and interval between signatures. The following menu item will compute the signatures with user-specified starting frame, ending frame, and interval between signatures.
Signature/Key Signature (User Setting)
Use "Signature/Key Signature (User Setting)" to convert a key video clip into a signature and save them to the file, “key.txt”. Users via the “Parameter” button determine the start frame, ending frame, and interval.
3.2 Detailed Matching Results
You can get detailed information on “which frame in the key has matched with which frame of a video in a library”.
Please note that the frame
with the same ID in the key will not necessary mean the same frame in the
library, because the default library signatures are computed at the rate of 1
signature every 1.0 second, while the key signatures are not computed at the
rate of 1 signature every 1.0 second.
To see the detailed matching, go to the Text tab. (See Figure 3.2)
Figure 3.2 Detailed Matching.
In this example,
FRIENDS_SEASON_2_0001_000001
Matched with the following frame:
FRIENDS_SEASON_2_0001_000001 760057
3.3 Libraries
You can have as many libraries as you want. Before matching, you will need to select a library and load it to the VideoFinder.
The library to be loaded can be a single file, or several files in a folder.
The folders used for the Library Tab are fixed:
.\data\lib1.txt
.\data\library
.\data\libraries
where ".\" is the folder where the VideoFinder is located.
To select and load ONE library, go to the Library tab and use the “Open Single Library” and the “Load Library” buttons.
To select and load All files in a folder, “.\Release\data\libraries”, go to the Library tab and use the “Open Library List” and the “Load Library” buttons. The buttons are:
Open Single Library
Use the "Open Single Library" button to list all library files in ".\data\library".
To select a library, simply highlight a file from the list.
Load Library
Use the "Load Library" button to load the highlighted file in ".\data\library".
Open Library List
Use the"Open Library List" button to list all library files in ".\data\libraries".
Load Library
Use the "Load Library" button to load all of the files in ".\data\libraries".
Figure 3.3 Select A Signature
Library.
Additional functions can be found in the Library menu.
Figure 3.4 Library Menu.
3.4 Additional Buttons
Figure 3.5 Extra buttons.
Clear Button
Use the “Clear” button to clear the text area.
Library: F, >, <
Use these buttons to select library video clips.
Result: F, >, <
Use these buttons to select retrieved video clips.
3.5 Parameters
Figure 3.6 Setting parameters.
Parameter Button
Use the “Parameter” button to open the Parameter form and set the parameters. The parameters apply to both Key signatures and Library signatures.
“Play Interval” Text Box
The default play speed is 1 second per frame. This speed can be changed. One signature is obtained in each Interval, so this parameter also determines how many signatures you will collect from each video clip.
“Set Parameter” Button
Use the “Set Parameter” button to set the newly selected parameters.
“Video Output Folder” Text Box
Use the “Video Output Folder” Text Box to set the folder used for converting the video clip to images.
“Video to Images” Button, “Starting Frame” Text Box, “Ending Frame” Text Box
Use the “Video to Images” button to convert the key video clip to images from the “Starting Frame” to the “Ending Frame”, one image every “Play Interval”.
3.6 Play Menu
Figure 3.7 Play Menu.
Play Menu plays a video in a separate window with better quality.
To play a video, you must open and play:
Open opens a video file.
Play plays the video.
Pause pauses the video.
Stop stops the video.
Open is further divided into 3 menu items:
Open Key Key must be selected first.
Open Current Source Video Source must be specified first.
Open Current Result Video Results must be available first.
For example, to play a Key video:
(1) Click the “Key” button to select a video;
(2) Click "Video/Open Key" to open the video;
(3) Click “Video/Play”.
To play a Source video:
(1) Click the “Library” button to select a source folder;
(2) Click "Video/Open Current Source Video" to open the video;
(3) Click “Video/Play”.
4. VideoFinder Reference Manual
4.1 Beginner Menu
Beginner/0. Initialization (Only Once)
Use “Beginner/0. Initialization (Only Once)” menu item to initialize the software. This step is done only once after starting the software.
Beginner/1. Select Library (Source button)
Use “Beginner/1. Select Library (Source button)” menu item to select a folder. A video clip is converted into a video clip signature. A set of video clip signatures forms a library. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the VideoFinder.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory.
Use the “>” button to select the next video clip.
Use the “<” button to select the previous video clip.
Beginner/2. Get Library Signatures (source.txt)
Use “Beginner/2. Get Library Signatures (source.txt)” menu item to convert all video clips in the selected directory into signatures and save them to the file, “source.txt”.
Beginner/3. Select Key (Key button)
Use “Beginner/3. Select Key (Key button)” menu item to select a key. A key is a newly obtained video clip. The video clip is converted into a signature. The selected key will be displayed in the first picture box (labeled ‘Key’). Use the “Play” and “Stop” buttons to play the video clip.
Beginner/4. Get Key Signatures (key.txt)
Use “Beginner/4. Get Key Signatures (key.txt)” to convert a key video clip into a signature and save them to the file, “key.txt”.
Library/Copy source.txt to lib1.txt
Use “Library/Copy source.txt to lib1.txt” menu item to create “lib1.txt”. Before matching, you will need to select a library and load it to the VideoFinder. The library to be loaded is fixed to the file, “lib1.txt”. We have created a library file, “source.txt”; now we will simply copy it to the loading-library file, “lib1.txt”, via this menu item.
Beginner/6. Load Library (lib1.txt)
Use “Beginner/6. Load Library (lib1.txt)” menu item to load library, “lib1.txt”. Before matching, you will need to select a library and load it to the VideoFinder.
Beginner/7. 1:N Matching
Use “Beginner/7. 1:N Matching” menu item to make a 1:N matching. “1:N Matching” (read as 1 to N matching) compares “1” (i.e., the new video clip) with “N” (i.e., the previously stored video clips).
Beginner/8. First Results (2nd “F” button)
Use “Beginner/8. First Results (2nd F Button)” to select the first matched video clip. The 1:N matching results are also displayed in the second picture box (labeled ‘Library’). Use the “Play” and “Stop” buttons to play the video clip.
Beginner/9. Next Results (2nd “>” button)
Use “Beginner/9. Next Results (2nd > Button)” to select the next matched video clip. The 1:N matching results are also displayed in the third picture box (labeled ‘Results’). Use the “Play” and “Stop” buttons to play the video clip.
4.2 Signature
Signature/Key Signature (Default)
Use "Signature/Key Signature (Default)" to convert a key video clip into a signature and save them to the file, “key.txt”.
Signature/Key Signature (20)
Use "Signature/Key Signature (20)" menu item to collect 20 signatures from a video, regardless of the video length.
Signature/Key Signature (40)
Use "Signature/Key Signature (40)" menu item to collect 40 signatures from a video, regardless of the video length.
Signature/Key Signature (80)
Use "Signature/Key Signature (80)" menu item to collect 80 signatures from a video, regardless of the video length.
Signature/Library Signature
Use "Signature/Library Signature" menu item to collect signatures from a source specified by the "Library" button and displayed in the ‘Library’ text box.
Signature/Get Current Source Signature
Use "Signature/Get Current Source Signature" menu item to collect signatures from a single video currently selected via "F"-button, ">"-button, or "<"-button and save it to file, “source_current.txt”.
Signature/Key Signature (Every 2 Sec)
Use "Signature/Key Signature (Every 2 Sec)" to convert a key video clip into a signature and save them to the file, “key.txt”. The number of signatures is the video length in seconds divided by 2.
Signature/Key Signature (User Setting)
Use "Signature/Key Signature (User Setting)" to convert a key video clip into a signature and save them to the file, “key.txt”. Users via the “Parameter” button determine the start frame, ending frame, and interval.
Signature/Library Signature (User Setting)
Use "Signature/Library Signature" menu item to collect signatures from a source specified by the "Library" button and displayed in the ‘Library’ text box. Users via the “Parameter” button determine the start frame, ending frame, and interval.
4.3 Library Menu
Before matching, you will need to select a library and load it to the VideoFinder.
The library to be loaded is fixed to the file, “lib1.txt”, “lib2.txt”, .... You can select arbitrary library file names from the library tab, but not under this menu.
The “Signature/Library Signatures” menu item will create a library file, “source.txt”. Now we will simply copy it to the loading-library file, “lib1.txt”.
Library/Copy source.txt to lib1.txt
Use “Library/Copy source.txt to lib1.txt” menu item to copy from “source.txt” to “lib1.txt”.
Library/Load (lib1.txt)
Use “Library/Load (lib1.txt)” menu item to load the library file, “lib1.txt”, to the Attrasoft Matching Engine. Before matching, you will need to select a library and load it to the VideoFinder.
Library/Load (lib2.txt)
Use “Library/Load (lib1.txt)” menu item to load the library file, “lib2.txt”, to the Attrasoft Matching Engine. Before matching, you will need to select a library and load it to the VideoFinder.
Library/Add (key)
Use “Library/Add (key)” menu item to add the current key signature to the loaded library file. The requirements are:
(1) You have selected a key;
(2) You have obtained the key signatures.
Library/Print Source Library
Use “Library/Print Source Library” menu item to print the loaded library to the text window. The requirement is that a library has been loaded.
Library/Print Key Signature
Use “Library/Print Key Library” menu item to print the key signature in file, “key.txt”, to the text window. The requirement is that a key signature has been computed.
Library/Save (lib1.txt)
Use the “Library/Save (lib1.txt)” menu item to save a copy of the currently loaded library to “lib1.txt”.
“Library/Backup (lib1_bk.txt)” Menu Item
Use the “Library/Backup (lib1_bk.txt)” menu item to create a backup copy of the currently loaded library to “lib1_bk.txt”.
4.4 Match Menu
Figure 4.1 Match Menu.
Match Menu makes 1:N matching.
Match/1:N Match
Use “Match/1:N Match" menu item to make a “1:N Matching” (read as 1 to N matching). “1:N Matching” compares “1” (i.e., the new video clip) with “N” (i.e., the previously stored video clips).
This menu item requires:
(1) Signature Library(s) is already loaded;
(2) Newly captured video is already converted into signatures.
4.5 Play Menu
Play/Open Key
Use "Video/Open Key" to open a video specified by the “Key” button.
Play/Open Current Source Video
Use "Video/Open Current Source Video" to open a video specified by the Source “F”, “>”, “<” button.
Play/Open Current Result Video
Use "Video/Open Current Result Video" to open a video specified by the Result “F”, “>”, “<” button.
Video/Play
Use “Video/Play” to play the opened video.
Video/Pause
Use “Video/Pause” to pause a video.
Video/Stop
Use “Video/Stop” to stop a video.
4.6 Library Tab
Open Single Library
Use the "Open Single Library" button to list all library files in ".\data\library".
To select a library, simply highlight a file from the list.
Load Library
Use the "Load Library" button to load the highlighted file in ".\data\library".
Open Library List
Use the "Open Library List" button to list all library files in ".\data\libraries".
Load Library
Use the "Load Library" button to load all of the files in ".\data\libraries".
5. Examples: ‘News’ search
5.1 Data
In this example, data are divided into two sections, reference and unknown.
The reference video file names are:
a_reference.wmv,
b_reference.wmv,
…
Each video is one hour long. This project has two videos: c, and j, two hours long.
The unknown video clips are cut from the news video. They are:
Unknown-length-X.wmv.
For example:
Unknown-5s-a.wmv
Unknown-30s-a.wmv
Unknown-5m-a.wmv
They stand for video clips from reference, X, and they are 5 seconds, 30 seconds, and 5 minutes long respectively. The unknown Video clips will also combine together to form a clip from several references. For example,
Unknown-5s-abcd.wmv
Which is a 20 seconds video clip with 5 seconds from references, a, b, c, and d. This project has 8 unknown video clips, representing 14 individual references. Seven out of 14 references are c and j; the rest are from other reference videos. The goal of this project is to run through each unknown video to find matches.
The data is stored at:
.\examples\news\reference_c\
.\examples\news\reference_j\
.\examples\news\unknown\
5.2 Procedure
0. Initialization (Only Once
After Starting the Software)
After starting the software, click the “Init” button.
1. Select Library
(Source button)
Click the “Source” button. Go to folder, “.\examples\news\reference_c\”, and select a file. The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
2. Get Library Signatures (source.txt)
Click “Beginner/2. Get Library Signatures (source.txt)” menu item, wait until the signature computation is completed. Now we will save the result, “source.txt”, to a new file, “reference_c.txt”.
Now, we get another signature library so we will repeat step 1 and 2 again.
1a. Select Library
(Source button)
Click “Source” button. Go to folder, “.\examples\news\reference_j\”, and select a file. The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
2a. Get Library Signatures (source.txt)
Click “Beginner/2. Get Library Signatures (source.txt)” menu item, wait until the signature computation is completed. Now we will save the result, “source.txt”, to a new file, “reference_j.txt”.
3. Select Key (Key button)
Click the“Key” button, go to “.\examples\news\unknown\”, and select a file. The selected key will be displayed in the first picture box (labeled ‘Key’). Use the “Play” and “Stop” buttons to play the video clip.
4. Get Key Signatures
(key.txt)
· Click “Beginner/4. Get Key Signatures (key.txt)” menu item.
5. Create Library File
You can load multiple libraries with a single click; but all libraries must be stored in a fixed folder: ".\data\libraries". Copy two files, “reference_cj.txt” and “reference_j.txt”, into this folder. Use the "Open Library List" button to list all library files in ".\data\libraries".
6. Load Library (lib1.txt)
Click the "Load Library" button to load all of the files in ".\data\libraries".
7. 1:N Matching (Match/1:N Match menu item)
Click:
· “Beginner/7. 1:N Matching” menu item, OR
· “Match/1:N Matching” menu item.
5.3 Results
unknown-5m-F-WhiteNoise100.wmv
No match
unknown-5m-J-WhiteNoise100.wmv
j_reference.wmv 3175527
unknown-5s-GCJD.wmv
c_reference.wmv 68276
j_reference.wmv 32169
unknown-10s-DCJG.wmv
c_reference.wmv 1749793
j_reference.wmv 1416628
unknown-30s-C-30s-Volume50.wmv
c_reference.wmv 2528799
unknown-30s-D-WhiteNoise50.wmv
No Match
unknown-30s-J-WhiteNoise50.wmv
j_reference.wmv 273225
unknown-30s-G-WhiteNoise50.wmv
No Match
Each of the seven video clips in the two hours of the original reference video are identified correctly.
6. Introduction to FrameFinder
6.1 FrameFinder Functions
Attrasoft VideoFinder consists of two components, VideoFinder and FrameFinder. The software will perform three independent tasks:
- Attrasoft VideoFinder matches a newly obtained video clip against a pre-established library of video clips.
- Attrasoft FrameFinder searches video clips for an object specified by a key image.
- Attrasoft FrameFinder searches live video for an object specified by a key image.
Figure 6.1 Attrasoft
FrameFinder searches video clips for an object specified by a key image.
6.2 Beginner Menu
Both VideoFinder and FrameFinder have a “Beginner” menu. You can operate the software from the “Beginner” menu alone.
Figure 6.2 The “Beginner” Menu
and its menu items.
The matching between a sample image, called key, and previously stored video clips is implemented directly without signature.
You can have as many keys as you want. Before matching, you will need to select a key by the “Key” button and select a folder containing video clips by the “Source” button; then, you are ready to search.
6.3 User Interface
Both VideoFinder and FrameFinder have a similar design. The VideoFinder has three video panels: ‘Key’, ‘Library’, and ‘Results’.
The FrameFinder has three video panels: ‘Key’, ‘Source’, and ‘Results’.
Key:
A key is an image, which specifies an object. A key is selected via the “Key” button.
Source:
A Source is a directory of video clips. Source videos are selected via the “Source” button. Use the “Play” and the “Stop” button below the label ‘Source’ to play a video clip. Use the “F”, “>”, and “<” button to select a video clip in the library: here “F” stands for First; “>” stands for Next; and “<” stands for previous.
Results:
Results consist of two sections: matched videos and matched frames. Use the “Play” and the “Stop” button below the label ‘Results’ to play the matched video clip. Use the “VF”, “V>”, and “V<” button (Video First, Video Next, Video Previous) to select a video clip in the matched collections. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
Figure 6.3 Three Video Panels:
‘Key’, ‘Source’, and ‘Results’. The Key image is labeled by the ‘Key’ label.
Figure 6.4 Three Video Panels:
‘Key’, ‘Source’, and ‘Results’. The Source video panel is labeled by the
‘Source’ label.
7. How to Use FrameFinder
Initially, you will need the following steps for video searching:
0. Initialization (Only Once after starting the software)
1. Select Key (Key button)
2. Learn Key
3. Select Source (Source button)
4. Search
7.1 How to Use FrameFinder
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking:
· “Beginner/0. Initialization (Once)” menu item, OR
· “Init” button.
Figure 7.1 Step 1. Select Key
(Key button)
1. Select Key (Key button)
A key is an image, which specifies an object. A key is selected via the “Key” button. This step selects a key. This step is implemented by clicking:
· “Beginner/1. Select Key” menu item, OR
· “Key” button.
The selected key will be displayed in the first picture box.
2. Learn Key
In this step, the FrameFinder will learn the object specified by the key image. This step is implemented by clicking:
· “Beginner/2. Learn Key” menu item, OR
·
“V(Video)Match/Learn Key” menu item.
Figure 7.2 Step 3, Select
Source
3. Select Source
A source is a folder consisting a set of video clips. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the FrameFinder.
This step selects a video clip directory. This step is implemented by clicking:
· “Beginner/3. Select Source” menu item, OR
· “Source” button.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
4. Searching
“Searching” compares the key image with all video frames in all of the video clips. This step is implemented by clicking:
· “Beginner/4. Search” menu item, OR
· “V(Video) Match/Search” menu item.
The results are also displayed in the third picture box (labeled ‘Results’). Results consist of two sections: matched videos and matched frames. Use the “Play” and the “Stop” button below the label ‘Results’ to play the matched video clip. Use the “VF”, “V>”, and “V<” button (Video First, Video Next, Video Previous) to select a video clip in the matched collections. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
Figure 7.3 Step 4, Searching.
Figure 7.4 Step 4, 1:N Matching Text File Output.
7.2 Example 1
In this section, we will use the “Friends” Video Clips. After installing the software, the folder structure is:
.\Videofinder\
.\Framefinder\
.\Examples\
where “.\” is the folder where the software is installed. The data stored used in this section is in folder, “.\examples\mwv_friends\”.
0. Initialization (Only Once
After Starting the Software)
After starting the software, click:
· “Beginner/0. Initialization (Once)” menu item, OR
· “Init” button.
1. Select Key (Key button)
Click (Figure 7.1):
· “Beginner/1. Select Key” menu item, OR
· “Key” button.
And select “.\examples\mwv_friends\FRIENDS_SEASON_2_0001_0006.jpg”, see Figure 7.1 The selected key will be displayed in the first picture box (labeled ‘Key’).
2. Learn Key
Click:
· “Beginner/2. Learn Key” menu item, OR
·
“V(Video)Match/Learn Key” menu item.
3. Select Source
Click:
· “Beginner/3. Select Source” menu item, OR
· “Source” button.
Go to folder, “.\examples\mwv_friends\”, and select any file, see Figure 7.2. The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
4. Searching
Click:
· “Beginner/4. Search” menu item, OR
· “V(Video) Match/Search” menu item.
The results are also displayed in the third picture box (labeled ‘Results’), see Figure 7.3 and 7.4. Results consist of two sections: matched videos and matched frames. Use the “Play” and the “Stop” button below the label “Results” to play the matched video clip. Use the “VF”, “V>”, and “V<” button (Video First, Video Next, Video Previous) to select a video clip in the matched collections. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
7.3 Example 2
After installing the software, the folder structure is:
.\Videofinder\
.\Framefinder\
.\Examples\
where “.\” is the folder where the software is installed. The data stored used in this section is in folder, “.\examples\avi1\”.
The question of this section is this: I want to search for frames that look like that tiger in that video clip, but how do I locate that tiger?
In this section, we will take frames out of a video clip and then, search for that frame.
0. Initialization (Only Once
After Starting the Software)
After starting the software, click the “Init” button.
1. Select Source for video to
image conversion
Click the ‘Source’ button.
Go to folder, “.\examples\avi1\”, and select any file. The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’). Use the “Play” and “Stop” buttons to play the video clip. Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
Now click the “>” button until you find a video file “tiger-eating.avi”, see Figure 7.5
Figure 7.5 Find a video.
Figure 7.6 Open Video Parameter Form.
2. Open Video Parameter Form
Click “VMatch/Parameter” menu item to open Video Parameter form, see Figure 7.6.
3. Video To Image
Conversion
Click button “Video to Image” and wait until “Conversion Completed”, see Figure 7.6.
4. Results
The results are stored in the folder, “./framefinder/temp/”, see Figure 7.7.
Figure 7.7 Video To Image
Conversion Results.
Now that we have obtained the images we want to search, we can start to search for tigers.
1. Select Key (Key button)
Click the “Key” button and select “./framefinder/temp/tiger-eating_00001.jpg”. The selected key will be displayed in the first picture box.
2. Learn Key
Click “Beginner/2. Learn Key” menu item.
3. Select Source
Click the ‘Source’ button and go to folder, “.\examples\avi1\”, and select any file.
4. Searching
Click “Beginner/4. Search” menu item and the results are also displayed in the third picture box (labeled ‘Results’).
8. Advanced Users
Figure 8.1 Video Matching Menu.
8.1 Searching Multiple Keys
You can search more than one key at a time. To select two or more keys:
1. Select Key (Key button)
This step selects a key. This step is implemented by clicking the “Key” button.
2. Learn Key
In this step, the FrameFinder will learn the object specified by the key image. This step is implemented by clicking “V(Video)Match/Learn Key” menu item.
3. Select Another Key (Key
button)
A second key is selected again via the “Key” button. This step is implemented by clicking the “Key” button.
4. Learn Another Key
This step is implemented by clicking “V(Video)Match/Learn Another Key” menu item.
5. Select Another Key (Key
button)
A third key is selected again via the “Key” button. This step is implemented by clicking the “Key” button.
6. Learn Another Key
This step is implemented by clicking “V(Video)Match/Learn Another Key” menu item.
Note:
- “VMatch/Learn
Key” menu item will first delete the old keys and start learning a new key
from the beginning.
- “VMatch/Learn Another Key” menu item will keep old keys.
8.2 1-to-1 Matching
Before you search a long video or multiple long videos, you can match a few frames to test your settings.
To make 1-to-1 matching:
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking:
· “Beginner/0. Initialization (Once)” menu item, OR
· “Init” button.
1. Select Key (Key button)
A key is an image, which specifies an object. A key is selected via the “Key” button. This step selects a key. This step is implemented by clicking:
· “Beginner/1. Select Key” menu item, OR
· “Key” button.
The selected key will be displayed in the first picture box.
2. Learn Key
In this step, the FrameFinder will learn the object specified by the key image. This step is implemented by clicking:
· “Beginner/2. Learn Key” menu item, OR
·
“V(Video)Match/Learn Key” menu item.
3. Select Source
A source is a folder consisting of a set of video clips. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the FrameFinder.
This step selects a video clip directory. This step is implemented by clicking:
· “Beginner/3. Select Source” menu item, OR
· “Source” button.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’).
Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
Use the “Play” and “Stop” buttons to play the video clip. When you stop at a frame, you can make a 1-to-1 matching.
4. 1-to-1 Matching
1-to-1 Matching compares the key image with the frame currently displayed in the second picture box. This step is implemented by clicking “V(Video) Match/1:1 Match” menu item.
The results are also displayed in the textbox. You might want to click the “Clear” button to clear the textbox first.
8.3 Detailed Matching Results
You can get detailed information on which frame has matched with key(s). To see the detailed matching, go to the Text tab.
Figure 8.2 Detailed Matching.
This tab will log all matching events. It starts with video 0. It lists all the frames that matched with the keys. Those frames that matched with keys will be saved to a file. The filename is:
.\temp\VideoFileName_MatchingFrameNumber.jpg.
Here “.\” is the folder where the FrameFinder is located. “temp\” folder will be used to store the matched frames as images. You should constantly clear the folder “.\temp\”.
The contents in the text tab look like this:
Video 0. C:\attrasoft_videofinder70\wmv\FRIENDS_SEASON_2_0001.wmv
ID Name Path Score X Y W H R
Right Right Right 102912 24 44 140 132
Save to :
C:\attrasoft_videofinder70\framefinder\temp\FRIENDS_SEASON_2_0001_000025.jpg
Video 1. C:\attrasoft_videofinder70\wmv\FRIENDS_SEASON_2_0002.wmv
ID Name Path Score X Y W H R
Right Right Right 37152 34 0 138 162
Save to :
C:\attrasoft_videofinder70\framefinder\temp\FRIENDS_SEASON_2_0002_000038.jpg
ID Name Path Score X Y W H R
Right Right Right 125376 0 0 138 162
Save to :
C:\attrasoft_videofinder70\framefinder\temp\FRIENDS_SEASON_2_0002_000055.jpg
Video 2. C:\attrasoft_videofinder70\wmv\FRIENDS_SEASON_2_0003.wmv
ID Name Path Score X Y W H R
Right Right Right 45278 26 10 138 162
Save to :
C:\attrasoft_videofinder70\framefinder\temp\FRIENDS_SEASON_2_0003_000059.jpg
ID Name Path Score X Y W H R
Right Right Right 45278 26 10 138 162
At the end, the text window contains two summary sections:
- The first section contains a list of all videos that matched with the keys and their scores;
- The first section contains a list of all video frames that matched with the keys and their scores.
It looks like this:
===========================================================
ID Name Path Score X Y W H
FRIENDS_SEASON_2_0002_000038 FRIENDS_SEASON_2_0002.wmv C:\attrasoft_videofinder70\wmv\ 162528 24 44 140 132
FRIENDS_SEASON_2_0001_000025 FRIENDS_SEASON_2_0001.wmv C:\attrasoft_videofinder70\wmv\ 102912 34 0 138 162
FRIENDS_SEASON_2_0003_000059 FRIENDS_SEASON_2_0003.wmv C:\attrasoft_videofinder70\wmv\ 90556 26 10 138 162
===========================================================
ID Name Path Score X Y W H
FRIENDS_SEASON_2_0002_000055 FRIENDS_SEASON_2_0002_000055.jpg C:\attrasoft_videofinder70\framefinder\temp\ 125376 24 44 140 132
FRIENDS_SEASON_2_0001_000025 FRIENDS_SEASON_2_0001_000025.jpg C:\attrasoft_videofinder70\framefinder\temp\ 102912 34 0 138 162
FRIENDS_SEASON_2_0003_000060 FRIENDS_SEASON_2_0003_000060.jpg C:\attrasoft_videofinder70\framefinder\temp\ 45278 0 0 138 162
FRIENDS_SEASON_2_0003_000059 FRIENDS_SEASON_2_0003_000059.jpg C:\attrasoft_videofinder70\framefinder\temp\ 45278 26 10 138 162
FRIENDS_SEASON_2_0002_000038 FRIENDS_SEASON_2_0002_000038.jpg C:\attrasoft_videofinder70\framefinder\temp\ 37152 26 10 138 162
In addition, a text file, “b1.txt”, will also be opened at this point which contains the two summary sections. This output file can be opened again later by clicking menu item, “VMatch/Results”.
Figure 8.3 Text File Output
8.4 Video Parameters
Video Search has a set of parameters. The default setting is as follows:
Starting frame: 0
Ending frame: Length of the video in seconds
Interval: 1 second
This setting can be changed via menu item “VMatch/Parameters”.
Figure 8.4 Setting parameters.
“VMatch/Parameters”
Use the “VMatch/Parameters” menu item to open the Video Parameters form and set the parameters.
“Play Interval” Text Box
The default play speed is 1 second per frame. This speed can be changed. One frame is obtained in each Interval, so this parameter also determines how many frames you will search from each video clip.
“Set Parameter” Button
Use the “Set Parameter” button to set the newly selected parameters.
“Video Output Folder” Text Box
Use the “Video Output Folder” Text Box to set the folder used for converting the video clip to images.
“Starting Frame” Text Box, “Ending Frame” Text Box
Use the “Starting Frame” and “Ending Frame” to specify the range inside a video; only frames inside this range will be searched. The unit for “Starting Frame” and “Ending Frame” is seconds.
8.5 Additional Buttons
Figure 8.5 Extra buttons.
Clear Button
Use the “Clear” button to clear the text area.
Source: F, >, <
Use these buttons (First, Next, Previous) to select library video clips.
Result: VF, V>, V<
Use these buttons (First, Next, Previous) to select retrieved video clips.
Result: IF, I>, I<
Use these buttons (First, Next, Previous) to select retrieved images.
8.6 Play Menu
Figure 8.6 Play Menu.
Play Menu plays a video in a separate window with better quality.
To play a video, you must open and play:
Open opens a source video file.
Play plays the video.
Pause pauses the video.
Stop stops the video.
To play a Source video:
(1) Click the “Source” button to select a source folder;
(2) Click "Video/Open" Key to open the video;
(3) Click “Video/Play”.
9. Live Video
Figure 9.1 The “Mode” button. In the Camera mode, Attrasoft FrameFinder searches live video for an object specified by a key image.
Figure 9.2 Clicking the “Source” button and the “Live” button.
9.1 Camera Mode
The FrameFinder has two modes:
- File Mode for searching video files;
- Camera Mode for searching live video.
You do need to have a camera, which is not included in the VideoFinder. Any camera that is compatible with Microsoft should work. We recommend Logitech cameras.
The default setting is live file. To change the mode, simply click the “Mode” button (Figure 9.1). The mode will switch between the “Video File” mode and the “Live Camera” mode.
After setting the “Mode” to Camera, your will need to:
- Click the “Source” button to bring camera live in the small window;
- Click the “Live” button to bring live picture to the second picture box.
9.2 Keys, Source, and Results
The matching between a sample image, called the key, and live video clips is implemented via the “LMatch” menu.
You can have as many keys as you want. Before matching, you will need to select a key by the “Key” button and set the live video by clicking three buttons: “Mode”, “Source”, and “Live” button; then, you are ready to search.
The FrameFinder has three video panels, ‘Key’, ‘Source’, and ‘Results’.
Key:
A key is an image, which specifies an object. A key is selected via the “Key” button.
Source:
A Source is live video. Set live video by clicking three buttons: “Mode”, “Source”, and “Live” buttons.
Results:
Results consist of matched frames. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
Figure 9.3 Three Video Panels:
‘Key’, ‘Source’, and ‘Results’. The Key image is labeled by the ‘Key’ label.
Figure 9.4 Three Video Panels:
‘Key’, ‘Source’, and ‘Results’. The Source video panel is labeled by the
‘Source’ label.
Figure 9.5 The “LMatch” (Live
Match) Menu and its menu items.
9.3 Live Search Menu
Figure 9.5 shows the live search menu. Initially, you will need the following steps for video searching:
0. Initialization (Only Once after starting the software)
1. Select Key (“Key” button)
2. Learn Key
3. Set Live video (Click “Mode”, “Source”, and “Live” buttons)
4. Search
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking the “Init” button.
1. Select Key (Key button)
A key is an image, which specifies an object. A key is selected via the “Key” button. This step selects a key. This step is implemented by clicking the “Key” button. The selected key will be displayed in the first picture box.
2. Learn Key
In this step, the FrameFinder will learn the
object specified by the key image. This step is implemented by clicking
“L(Live)Match/Learn Key” menu item.
3. Set Live Video Source
Set live video by clicking three buttons: “Mode”, “Source”, and “Live” buttons.
4. Searching
“Searching” compares key image with all video frames in all of the video clips. This step is implemented by clicking “L(Live) Match/Search” menu item.
The results are also displayed in the third picture box (labeled ‘Results’). Results consist of matched frames. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
9.4 Searching Multiple Keys
You can search more than one key at a time. To select two or more keys:
1. Select Key (Key button)
This step selects a key. This step is implemented by clicking the “Key” button.
2. Learn Key
In this step, the FrameFinder will learn the object specified by the key image. This step is implemented by clicking “LMatch/Learn Key” menu item.
3. Select Another Key (Key
button)
A second key is selected again via the “Key” button. This step is implemented by clicking the “Key” button.
4. Learn Another Key
This step is implemented by clicking “LMatch/Learn Another Key” menu item.
5. Select Another Key (Key
button)
A third key is selected again via the “Key” button. This step is implemented by clicking the “Key” button.
6. Learn Another Key
This step is implemented by clicking “LMatch/Learn Another Key” menu item.
…
Note:
- “LMatch/Learn
Key” menu item will first delete the old keys and start learning a new key
from the beginning.
- “LMatch/Learn Key” menu item will keep old keys.
9.5 Example
In this example, we will first use the camera to capture several images; then we will search for the image from live video.
0. Initialization (Only Once
After Starting The Software)
After start the software, click the “Init” button.
1. Set Live Video Source
Set live video by clicking three buttons: “Mode”, “Source”, and “Live” buttons.
Figure 9.6 Live Camera.
Figure 9.7 Open Video Parameter Form.
2. Open Video Parameter Form
Click “LMatch/Parameter” menu item to open Video Parameter form, see Figure 9.7.
3. Capture Live Image
Click button “Live to Image” and wait until “Capture Completed”, see Figure 7.7. This form will also specify the number of seconds, and interval between frames in seconds, if the default setting is too large for you.
4. Results
The results are stored in the folder, “./framefinder/temp/”, see Figure 9.8.
Figure 9.8 Video To Image
Conversion Results.
Now that we have gotten several live images, we can use it to search.
0. Initialization (Only Once
After Starting the Software)
This step is already done.
1. Select Key (Key button)
Click the “Key” button and select a newly captured image.
2. Learn Key
Click “L(Live)Match/Learn Key” menu item.
3. Set Live Video Source
This step has already been done earlier.
4. Searching
Click “L(Live) Match/Search” menu item.
You will see a text file output in Figure 9.9.
Figure 9.9 Text File Output.
The results are also displayed in the third picture box (labeled ‘Results’). Results consist of matched frames. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
10. Matching Parameters
The matching engine under the VideoFinder and the FrameFinder is the ImageFinder. You can tweak the VideoFinder and FrameFinder via their parameters in a file, “.\data\abm70.txt”, provided you know how to set these parameters and what these parameters are for. The best way to learn the parameters is to learn the Attrasoft ImageFinder. After you have learned the ImageFinder, you will know how to modify “abm70.txt”. This chapter assumes you already know the ImageFinder; otherwise, please skip this chapter.
This chapter will describe the parameters in the ImageFinder, which is deployed in both VideoFinder and the FrameFinder. The VideoFinder uses the Neural Filter; and the FrameFinder uses the NeuralNet Filter. For the VideoFinder, the parameters can be changed via a file “.\data\abm70.txt”. For the FrameFinder, the parameters can be changed from the software directly via clicking the “Parameter” button.
10.1 Overview
Attrasoft ImageFinder can:
· Match whole images;
· Match a portion of an image.
When matching a portion of an image, similar images are defined as images containing the sample segments, or:
- Translated segments;
- Rotated segments;
- Scaled segments;
- Rotated & Scaled segments;
- Brighter or Darker segments.
To match an image, the ImageFinder pushes the image through many filters. For example, a set of filters could be:
Preprocessing Filters
Edge Filters
Threshold Filters
Clean-Up Filters
Reduction Filters
Unsupervised Filters
BioFilters
NeuralFilters
NeuralNet Filters
Many parameters and options of the ImageFinder are hidden. The users have only limited control of the parameters. Still, the ImageFinder has many parameters, which can be adjusted by users.
The ImageFinder for Windows has 70 open parameters for the user to adjust for their particular image type. You should get Identification Rates ranging from 60% to 89%; this is because the off-the-shelf ImageFinder only has 70 open parameters for users to adjust. The best rate, one of our customers (without any customization) was able to obtain, was an 89% Identification Rate.
However, the ImageFinder itself has 3000+ internal parameters, which the users have no access to at all. Fine-tuning these 3000+ internal parameters is called customization, which is Attrasoft’s area of expertise. If you need increased accuracy beyond what you are able to achieve when using the ImageFinder for Windows, then customization will provide you with Your desired level of accuracy (ranging from 95% to 99.9%). If you need a customized version of the ImageFinder, please contact imagefinder@attrasoft.com .
In a typical search, you
will set these parameters and leave the other parameters with default values.
10.2 Image Preprocessing
Use the “Stick Shift” parameter to speed up
the computation. Set “Stick Shift” in
Figure 10.2 between 0 and 5, with 0 being the slowest and 5 being the fastest.
0 No skip;
1 Skip the white empty border space;
2 Skip the black empty border space;
3 Skip x percent of the contents on the white background space;
4 Skip x percent of the contents on the black background space;
5 Skip empty border space on user defined Threshold Filter;
6 Skip x percent of the contents on user defined Threshold/Edge Filters.
10.3 Image Processing
10.3.1 Edge Filters
Edge
Filters extract and enhance edges & contours in an image by expressing
intensity differences (gradients) between neighboring pixels as an intensity
value. The basic variables are the differences between the top and bottom rows,
the differences between the left and right columns, and the differences between
the center point and its neighbors.
Edge Filters have the following selections:
Code Meaning
0 No Edge Filter
1 Sobel 1 (Prewitt)
2 Sobel 2 (Sobel)
3 Sobel 3
4 Sobel 4
5 Gradient
6 Gradient, 45°
7 Sobel 1, 45°
8 Sobel 1, - 45°
9 Laplacian 4
10 CD 11
11 FD 11
12 FD 9
13 FD 7
14 FD 13
15 Laplacian 5
16 Laplacian 8
17 Laplacian 9
18 Laplacian 16
19 Laplacian 17
All other filters have to be ordered in a Customized Version.
These names really do not make any sense to common users; the best way to figure out what these filters are, is to select a training image and try each of the filters. In general, these filters require the “Dark Background 128” Threshold Filter.
If you do not want to know the details, please skip the rest of this section.
The details will be given below so you will know how to order a customized filter:
Sobel 1:
-1 0 1 -1 -1 -1
-1 0 1 0 0 0
-1 0 1 1 1 1
Sobel 2:
-1 0 1 -1 -2 -1
-2 0 2 0 0 0
-1 0 1 1 2 1
Sobel 3:
-1 0 1 -1 -3 -1
-3 0 3 0 0 0
-1 0 1 1 3 1
Sobel 4:
-1 0 1 -1 -4 -1
-4 0 4 0 0 0
-1 0 1 1 4 1
Gradient:
0 0 0 0 -1 0
-1 0 1 0 0 0
0 0 0 0 1 0
Gradient, 45°
0 0 1 -1 0 0
0 0 0 0 0 0
-1 0 0 0 0 1
Sobel 1, 45°
0 1 1 1 1 0
-1 0 1 1 0 -1
-1 -1 0 0 -1 -1
Sobel 2, 45°
0 1 2 2 1 0
-1 0 1 1 0 -1
-2 -1 0 0 -1 -2
Laplacian 4
0 -1 0
-1 4 -1
0 -1 0
Laplacian 5
0 -1 0
-1 5 -1
0 -1 0
Laplacian 8
-1 -1 -1
-1 8 -1
-1 -1 -1
Laplacian 9
-1 -1 -1
-1 9 -1
-1 -1 -1
Laplacian 16
0 0 -1 0 0
0 -1 -2 -1 0
-1 -2 16 -2 -1
0 -1 -2 -1 0
0 0 -1 0 0
Laplacian 17
0 0 -1 0 0
0 -1 -2 -1 0
-1 -2 17 -2 -1
0 -1 -2 -1 0
0 0 -1 0 0
10.3.2 Threshold Filters
After Edge Filters, the Threshold Filter will be applied to images. Choose these two filters where the sample objects stand out, otherwise change the filters.
If no filter in this version fits your problem, a Customized Filter has to be built. DO NOT make too many things stand out, i.e. as long as the area of interest stands out, the rest should show as little as possible.
Once you make a selection, the objects in the images are black and the background is white (like a book: white paper, black print). You should make the black area as small as possible, as long as it covers the key-segment(s). Otherwise, switch to a different background.
Figure
10.4 Threshold Filter Parameters.
There are 30 Threshold filters in the ImageFinder.
A few filters, including the average-filter and the customized-filter, allow you to specify any color range. Color is specified by three separate colors: Color = (red, green, blue). Each of the colors ranges from 0 to 255. (0, 0, 0) is black; (255, 255, 255) is white.
You should choose a filter where the sample object(s) stand out. You may want to know the meaning of the filters; example, "Light Background 128" means:
· “RGB Average in 0 – 127 “ č objects; and
· “RGB Average in 128 - 255“ č background.
To
Summarize:
· Choose an Edge Filter and a Threshold Filter where the sample object(s) stand out;
· Choose an Edge Filter and a Threshold Filter where the black area is as small as possible, as long as it covers the key-segment(s).
10.3.3 Clean-Up Filters
Clean-Up Filters will clear noise off the image, but it will take more computation time.
10.4 Normalization Filter
The Normalization Filter connects the image to the underlying neural nets.
Let the underlying neural net be 100x100: if an image is larger than 100x100, say 350x230, then this image will be reduced to 100x100 or smaller.
When reducing images, a scaling factor can be introduced easily. Although scaling symmetry can compensate for this scaling factor, scaling symmetry is computationally expensive.
It is important to know that the Reduction Filter will match the selected underlying neural net, therefore, the behavior of the Reduction Filter not only depends on the selection of this filter itself, but also depends on the NeuralNet Filter chosen.
Figure 10.5 Selecting Reduction Filter.
There are several ways to reduce images:
- Integer,
- Real, or
- All images are reduced by the same amount.
Integer Reduction
Images are reduced by an integer factor to maximally fit 100x100 without distortion. For example, a 350x230 image will be reduced to 87x57.
Real Reduction
Images are reduced by a real number to maximally fit 100x100 without distortion. For example, a 350x230 image will be reduced to 100x65.
Within each type of reduction, there are 3 more settings. Assume a 3x3 pixel array is reduced to 1 pixel,
- Avg: Assign the average of the 3x3 pixel array to the new pixel;
- Max: Assign the maximum of the 3x3 pixel array to the new pixel; or
- Min: Assign the minimum of the 3x3 pixel array to the new pixel.
To select the Reduction Filter, use the fourth drop down list. The Reduction Filter has seven parameters.
Segment Cut
This parameter deals with the edges of the segments in the images. The Segment Cut parameter ranges from 0 to 12. The larger this parameter is, the smaller the segment the ImageFinder will use. The possible settings in the user interface are: 0, 1, 2, .., and 12.
Size Cut
In some applications, the users only want to
search images of certain dimensions and ignore other images. An example is
given below:
In this example, the two stamps belong to two different classes based on the image dimension alone.
The Size Cut parameter ranges from 0 to 9. If the setting is 0, this parameter will be ignored.
· If the parameter is 1, then the longest edge of the image to be considered must be at least 100, but less than 199.
· If the parameter is 2, then the longest edge of the image to be considered must be at least 200, but less than 299; …
The Border Cut parameter ranges from 0 (no cut) to 9 (18% border cut). For some images (see the picture below), you might want to get rid of the sections of images close to the borders. To get rid of the border section, use the Border Cut.
The possible settings in the user interface are: 0, 1, 2, .., and 9.
Assume an image is (0,0; 1,1),
· setting Border Cut to 1 means the ImageFinder will look at the section (0.02, 0.02; 0.98, 0.98);
· setting Border Cut to 2 means the ImageFinder will look at the section (0.04, 0.04; 0.96, 0.96); … .
The Look-At Area is the area the ImageFinder will use in a matching operation. A 100 x 100 window specifies a whole image. If an integer Reduction Filter is used, the actual area can be less than 100x100.
Four numbers specify the Look-At Area:
(x, y, w, h)
(x, y) are the coordinates of the upper-left corner and (w, h) are the width and height of the Look-At window.
To use this Look-At window, enter (x, y, w, h) to the 4 text boxes.
10.5 Unsupervised Filter & BioFilter
The Unsupervised Filter and BioFilter have similar parameters; so we have combine these two filters together.
Figure 10.6 Unsupervised Filter or BioFilter Parameter.
Fault Tolerance Scale
Use this parameter to control the amount of output. This parameter ranges
from 0 to 100. The larger this number is, the more matches you will get. To set
this parameter, enter a number between 0 and 100 to the text box.
Blurring
Use this parameter to control the amount of output. This parameter
ranges from 0 to 100. The larger this number is, the more matches you will get.
To set this parameter, enter a number between 0 and 100 to the text box.
Sensitivity
Use this parameter to control the amount of output. This parameter
ranges from 0 to 100. The larger this number is, the more matches you will get.
To set this parameter, enter a number between 0 and 100 to the text box.
Threshold
The result of image comparison is a "score", indicating the
degree to which a match exists. This score is then compared to a pre-set Threshold
to determine whether or not to declare a match. This parameter sets the
threshold. To decide what threshold to use, you should make a test run first
and look at the scores. Matching images have higher scores; unmatched images
have lower scores. Select a threshold to separate these two groups. There will
be a few images in the middle, representing both groups. Under these
circumstances, the threshold selection depends on your application.
Relative Score
Use the relative score to set the range of matching score between 0 and
100.
Show File
This parameter is set to 1 by default, which will show the output file.
If this parameter is set to 0, then output file will not be shown.
10.6 Neural Filters
Figure 10.7 NeuralFilter Parameter.
Fault Tolerance Scale
Use this parameter to control the amount of output. This parameter
ranges from 0 to 100. The larger this number is, the more matches you will get.
To set this parameter, enter a number between 0 and 100 to the text box.
Blurring
Use this parameter to control the amount of output. This parameter
ranges from 0 to 100. The larger this number is, the more matches you will get.
To set this parameter, enter a number between 0 and 100 to the text box.
Sensitivity
Use this parameter to control the amount of output. This parameter
ranges from 0 to 100. The larger this number is, the more matches you will get.
To set this parameter, enter a number between 0 and 100 to the text box.
Threshold
The result of image comparison is a "score", indicating the
degree to which a match exists. This score is then compared to a pre-set
Threshold to determine whether or not to declare a match. This parameter sets
the threshold. To decide what threshold to use, you should make a test run
first and look at the scores. Matching images have higher scores; unmatched
images have lower scores. Select a threshold to separate these two groups.
There will be a few images in the middle, representing both groups. Under these
circumstances, the threshold selection depends on your application.
Relative Score
Use the relative score to set the range of matching scores between 0
and 100.
Neural Filter Opening
This parameter controls
the amount of output. This parameter has 5 settings:
Very Large
Large
Normal
Small
Very Small
Large openings will allow more output than small openings. To set the parameter, keep clicking the button; the setting will switch from one to the next each time you click the Blurring button.
Show File
This parameter is set to 1 by default, which
will show the output file. If this parameter is set to 0, then output file will
not be shown.
10.7 NeuralNet Filter
Figure 10.8 Neural Net Parameter.
The available NeuralNet filters are:
- 100x100 (Most Accurate)
- 90x90
- 80x80
- 70x70
- 60x60
- 50x50 (Least Accurate)
Let the speed of 100x100 filter be a base, then the overall speed for:
- 90x90 filter is 1 times faster;
- 80x80 filter is 1.6 times faster;
- 70x70 filter is 2.7 times faster;
- 60x60 filter is 5 times faster; and
- 50x50 filter is 10 times faster.
The NeuralNet Filter has many parameters. The following sections will explain these parameters.
10.7.1 Symmetry
Symmetry or Invariance means similarity under certain types of changes. For example, considering two images, one with a face in the middle and the other with the face moved to the edge; we say these two images are similar because of the face.
The symmetry defines "similar images". The Attrasoft ImageFinder supports five symmetry settings:
· No symmetry (0);
· Translation symmetry (3);
· Scaling symmetry (4);
· Rotation symmetry (5); and
· Rotation & Scaling symmetries (6).
The numbers are the codes in the batch file. Currently, Scaling symmetry and Oblique symmetry are the same.
Other symmetries, or combination of symmetries, can be built for Customized Orders.
A customized Attrasoft ImageFinder can implement any symmetry (or combination of symmetries), which can be described by mathematics.
However, symmetries are computationally expensive.
Every symmetry setting has the Translation symmetry, except "No Symmetry". In addition, each of the above settings support:
· Intensity symmetry.
Symmetries are computationally expensive, meaning it will take a longer time to do the job. You should use them only when they are required.
To set the Symmetry, keep clicking the Symmetry button; the setting will switch from one to the next each time you click the button. The default setting in this version is Translation Symmetry.
For example, it seems that Stamp Recognition requires Translation and Rotation symmetries. But because the edges of a stamp can be detected easily, the stamp can be rotated and shifted to a fixed position where the horizontal side is longer than the vertical side. All you need to do is recognize a stamp or an upside-down stamp. Therefore, Stamp Recognition does not really require Translation and Rotation symmetries.
10.7.2 Translation Type
The Translation Type defines the accuracy of the Translation symmetry.
The Translation Type settings (and their codes) are:
· Most Accurate (0);
· Accurate (1); and
· Least Accurate (2).
To set the Translation Type, keep clicking the “T Type” button; the setting will switch from one to the next each time you click the button. The default setting is 0, the most accurate setting.
10.7.3 Scaling Type
The Scaling Type defines the accuracy of the Scaling symmetry.
The Scaling Type settings (and their codes) are:
· Least Accurate (0);
· Accurate (1);
· Accurate (2); and
· Most Accurate (3).
To set the Scaling Type, keep clicking the “T Type” button; the setting will switch from one to the next each time you click the button. The default setting is 0, the least accurate setting.
10.7.4 Rotation Type
The Rotation Type defines the accuracy of the Rotation symmetry.
The Rotation Type settings (and their codes) are:
· 360° rotation, least accurate (0);
· -5° to 5° rotation (1);
· -10° to 10° rotation (2);
· 360° rotation, accurate (3);
· 360° rotation, more accurate (4);
· 360° rotation, most accurate (5).
To set the Rotation Type, keep clicking the “Rotation Type” button; the setting will switch from one to the next each time you click the button. The default setting is 360° rotation, the least accurate setting (0).
10.7.5 Area of Interest (AOI)
Selecting an image segment is very important for training.
- Use image segments for searching similar
images.
- Only use the whole image for exact matches.
Training requires an "Area of Interest" (AOI) or "Set Focus", which selects a key-segment. If an AOI is not chosen, the whole image is the AOI. Four numbers specify AOI: the upper-left corner coordinates, and the length & width. Once the segment specification is successful, a box will cover the selected area. When you look at the training image, if the selected area is not what you want, just re-select the area again and click the “Segment” button.
The default setting is the whole image; the code is (x y w h) = (0 0 0 0). (0000) means ignore the segment. The units are pixels.
There are two situations where you should create a new sample image out of a sample segment:
· You repeatedly use an image segment;
· The image segment is not a rectangle; say a polygon.
The Windows Paint program will help you to create an image from a segment. When you create an image segment, please do not change the original image size. For example, if your image is 512x512 and you want create a segment of 400x200, please paste the 400x200 segment into a 512x512 empty image.
10.7.6 Blurring
This is one of the most important search parameters and the first
parameter you should adjust.
Blurring compensates for minor image changes, which are not visible to human eyes. For example, if you use software to compress an image, to change the intensity of an image, or to translate, scale, or rotate an image, the image will be distorted a bit at the pixel level. You have to set “Blurring” to compensate for this.
The Blurring setting ranges from 0 to 50. The default setting is 10. You should set the parameters in the following order:
Blurring,
Internal Weight Cut, Sensitivity, External Weight Cut.
To Summarize:
- When a search yields no results, increase
Blurring;
- When a search yields too many results, decrease
Blurring.
10.7.7 Sensitivity
The Sensitivity parameter ranges from 0 (least sensitive) to 100 (most sensitive).
- To search small segment(s), use high sensitivity search.
- To search large segment(s), use low sensitivity search.
- The higher this parameter is, the more results you will get.
The Sensitivity parameter ranges
from 0 to 100. The default is 50. To Summarize:
- When a search yields no results, increase
sensitivity;
- When a search yields too much result, decrease sensitivity.
10.7.8 Internal/External Weight Cut
You can set the "Internal Weight Cut" (Internal Cut) or "External Weight Cut" (External Cut) to list only those retrieved images with scores or weights greater than a certain value (called Threshold).
It is better to give no answer than a wrong answer.
Assume you are searching images and all similar images have weights ranging from 1,000 to 10,000. It is possible that some other images pop up with weights ranging from 10 to 100. To eliminate these images, you can set the “External Weight Cut” to 1,000.
The Internal Cut plays a similar role as the External Cut. There are two differences between these two cuts:
· The Internal Cut ranges from 0 to 99; the External Cut can be any number;
· The Internal Cut stops the images from coming out, whereas the External Cut can bring the eliminated images back if you set the External Cut to 0. You might need to see the eliminated images sometimes for the purpose of adjusting the parameters.
To Summarize:
·
Set the “Internal Cut” or “External Cut” to
eliminate errors.
10.7.9 Segment Size
The ImageFinder is currently tuned to search for large image segments (size of the whole image). It can look for small segments via the "Small Segment" setting; however, only Translation symmetry is supported for small segments.
A Customized Version can be ordered for other symmetries.
To search large segments, use setting 0.
To search small segments, use setting 1.
For example:
- If a sample segment is one quarter of the sample image, it is a large segment.
- If the segment is 1/20 of the sample image, it is a small segment.
Currently, "S
Segment" only supports Translation symmetry. If you need Rotation or/and
Scaling symmetry, please use "L Segment".
Other symmetries can be
added in a Customized Version.
10.7.10 Image Type
There are BW and Color images. For each of them, there are “sum-search”, “maximum-search”, and “average-search”. This generates 6 image types:
- BW Sum
- BW Max
- BW Avg
- Color Sum
- Color Max
- Color Avg
"BW Sum” is like an integration of function f (x).
"BW Max” is like a maximum value of f (x); and
"BW Avg” is the average of the above two.
"Color Sum” is like an integration of function f (x).
"Color Max” is like a maximum value of f (x); and
"Color Avg” is the average of the above two.
To set the image type, keep clicking the “Image Type” button; the setting will switch from one to the next each time you click the “Image Type” button.
10.7.11 Use BioFilter & Use Neural Filter
These two parameters, “Use BioFilter” and “Use NeuralFilter” will decide whether the BioFilter and NeuralFilter will be used before the NeuralNet Filter is used to eliminate some images.
10.7.12 Auto Segment
The training segment can be specified in two ways:
Manual Specification
Automatic Specification
The default is Manual Specification. In this
setting the segment will be specified by the four text boxes (x, y, w, h), as
we discussed earlier.
If you do not want to pick up a training segment, then let the ImageFinder pick up the segment for you by using the Automatic Specification. This parameter has several settings:
NO Auto Segment
Very Large Segment
Very Large Segment
Large Segment
Large Segment
Medium Segment
Medium Segment
10.7.13 Summary
The NeuralNet filter is hard to use because it has so many parameters. Not all parameters are equal. We divide the parameters into two groups. The beginners should use only parameters in the first group. Note that:
- The most important parameters for Training
are Image Processing, AOI, Symmetry, and Segment Cut (in the Reduction
Filter).
- The most important parameters for Matching are Blurring, and Sensitivity.
In a typical search, you will
set these parameters and leave other parameters with default values. These
are the 7 parameters you should focus on first:
Training (3 parameters):
- Segment: selecting “Large AutoSeg 3” so the ImageFinder will select a training segment for you at the beginning.
- Segment Cut (in the Reduction Filter)
Matching (4 parameters):
- Sensitivity
- Blurring
- External Weight Cut
- Internal Weight Cut
Ignore the rest of the parameters at the beginning.
11. FrameFinder Reference Manual
11.1 Beginner Menu
Beginner/0. Initialization (Once)
Use “Beginner/0. Initialization (Once)” menu item to initialize the software. This step is done only once after starting the software.
Beginner/1. Select Key
Use “Beginner/1. Select Key” menu item to select a key. A key is an image, which specifies an object.
Beginner/2. Learn Key
Use “Beginner/2. Learn Key” menu item to learn the object specified by the key image.
Beginner/3. Select Source
Use “Beginner/3. Select Source” menu item to select a folder. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the VideoFinder.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’). Use the “Play” and “Stop” buttons to play the video clip. Use the “F” button to select the First video in the directory; use the “>” button to select the next video clip; and use the “<” button to select the previous video clip.
Beginner/4. Search
Use “Beginner/4. Search” menu item to compares key image with all video frames in all of the video clips.
11.2 Video Match
VMatch/Learn Key
Same as “Beginner/2. Learn Key”.
VMatch/Learn Another Key
Use “VMatch/Learn Another Key”menu item to learn second, third, fourth, …, key image.
VMatch/Search
Same As “Beginner/4. Search”
VMatch/1:1 Match
Use the “VMatch/1:1 Match” menu item to make a 1:1
Matching. 1:1 Matching compares the key in the first picture box with the image
displayed in the second picture box. The “Key” button selects the key image.
The second image can be selected by the “Play” and “Stop” buttons below the
second image. You must learn a key first before 1:1 Matching. Note that, next
to each image, a score is printed. The larger the number is, the more
similarity between the training image(s) and the retrieved images.
VMatch/Results
Use the “VMatch/Results” menu item to open “b1.txt”, which is the file containing the last matching result in text format.
VMatch/Parameters
Use the “VMatch/Parameters” menu item to open the Parameter form and set the parameters.
“Play Interval” Text Box
The default play speed is 1 frame per second. This speed can be changed.
“Video to Images” Button, “Starting Frame” Text Box, “Ending Frame” Text Box
Use the “Video to Images” button to convert the current source video clip to images from the “Starting Frame” to the “Ending Frame”, one image every “Play Interval”.
“Set Parameter” Button
Use the “Set Parameter” button to set the newly selected parameters.
“Video Output Folder” Text Box
Use the “Video Output Folder” Text Box to set the folder used for converting the video clip to images.
“Live to Images” Button
Use the “Live to Images” button to convert the live clip to images from 0 seconds to the “Ending Frame”, one image every “Play Interval”.
11.3 Live Match
LMatch/Learn Key
Same as “Beginner/2. Learn Key”.
LMatch/Learn Another Key
Use “LMatch/Learn Another Key”menu item to learn second, third, fourth, …, key image.
LMatch/Search
Same As “Beginner/4. Search”
LMatch/1:1 Match
Use the “LMatch/1:1 Match” menu item to make a 1:1
Matching. 1:1 Matching compares the key in the first picture box with the image
displayed in the second picture box. The “Key” button selects the key image.
The second image come from a camera. You must learn a key first before 1:1
Matching. Note that, next to each image, a score is printed. The larger the
number is, the more similarity between the training image(s) and the retrieved
images.
LMatch/Results
Use the “LMatch/Results” menu item to open b1.txt, which is the file containing the last matching result in text format.
LMatch/Parameters
Similar to “VMatch?Parameter” menu item.
11.4 Play Menu
Play/Open
Use "Video/Open" to open a source video specified by Source “F”, “>”, “<” button.
Video/Play
Use “Video/Play” to play the opened video.
Video/Pause
Use “Video/Pause” to pause a video.
Video/Stop
Use “Video/Stop” to stop a video.
12. Video FrameFinder Examples
In this chapter, we present an example to:
- Recognize an object in video clips;
- Track an object in video clips;
- Report its coordinates; and
- Count the number of frames in which the object appears.
12.1 Data
After installing the software, the folder structure is:
.\Videofinder\
.\Framefinder\
.\Examples\
where “.\” is the folder where the software is installed. The data stored used in this section is in folder, “.\examples\mpeg1\”.
We are looking for the image of “Peter Pan” and want to:
- Recognize an object in video clips;
- Track an object in video clips;
- Report its coordinates; and
- Count the number of frames in which the object appears.
The coordinate system is (0, 0) to (200, 200). The key image is in Figure 12.1.
Figure 12.1 Key Image.
12.2 Procedure
1. Select Key (Key button)
Click the “Key” button; go to “.\examples\mpeg1\”, and select the only jpeg image there. The selected key will be displayed in the first picture box.
2. Learn Key
Click “Beginner/2. Learn Key” menu item.
3. Select Source
Click the “Source” button and go to folder, “.\examples\mpeg1\, and select any file.
4. Searching
Click “Beginner/4. Search” menu item and the results are also displayed in the third picture box (labeled ‘Results’).
12.3 Results
Figure 12.2 Video clip at 27, 28, and 29 seconds.
Figure 12.3 Video clip at 76 and 77 seconds.
The output is:
ID Name Path Score X Y W H
carsintro_000037 carsintro.mpg C:\attrasoft_videofinder70\examples\mpeg1\ 502272 54 54 100 52
ID Name Path Score X Y W H
carsintro_000037 carsintro_000037.jpg C:\attrasoft_videofinder70\framefinder\temp\ 89920 54 54 100 52
carsintro_000038 carsintro_000038.jpg C:\attrasoft_videofinder70\framefinder\temp\ 125824 52 58 100 52
carsintro_000039 carsintro_000039.jpg C:\attrasoft_videofinder70\framefinder\temp\ 105600 52 62 100 52
carsintro_000076 carsintro_000076.jpg C:\attrasoft_videofinder70\framefinder\temp\ 74624 54 52 100 52
carsintro_000077 carsintro_000077.jpg C:\attrasoft_videofinder70\framefinder\temp\ 106304 52 58 100 52
The first section lists the video clips and the second section lists all frames.
Click “IF” button to see the first output, “carsintro_000037.jpg’. Keep clicking “I>” button to see carsintro_000038.jpg, …, carsintro_000077.jpg.
Now we go back to the original the questions:
Recognize an object in video clips:
Yes, objects are found.
Track an object in video clips:
The object appears at 37 seconds and lasts 3 seconds. The object also appears at 76 seconds and last 2 seconds.
Report its coordinates:
Their coordinates of the first match is: upper-left (x, y) = (52, 58) and (w, h) = (100, 52), in the scale of (0, 0, 200, 200). The rest are more or less in the same area.
Count the number of frames in which the object appears:
There are a total of 5 seconds in one video clip.
13. Live FrameFinder Examples
In this chapter, we will use a few toy cars to run through a table. We will use a live camera to detect whether there is a car on the road or not.
We will first use the camera to capture several images; then we will search the image from live video.
13.1 Key Images
0. Initialization (Only Once
After Starting The Software)
After start the software, click the “Init” button.
1. Set Live Video Source
Set live video by clicking three buttons: “Mode”, “Source”, and “Live” buttons.
Figure 13.1 Live Camera.
Figure 13.2 Open Video Parameter Form.
2. Open Video Parameter Form
Click “VMatch/Parameter” menu item to open Video Parameter form, see Figure 13.2.
3. Capture Live Image
Click button “Live to Image” and wait until “Capture Completed”, see Figure 13.2. This form will also specify the number of seconds, and interval between frames in seconds, if the default setting is too large for you.
4. Results
The results are stored in the folder, “./framefinder/temp/”, see Figure 13.3.
Figure 13.3 Live To Image
Conversion Results.
Repeat the above process several times until you collect enough data.
13.2 Procedure
Now that we have gotten several live images, we can use it to search.
0. Initialization (Only Once
After Starting the Software)
This step is already done.
1. Select Key (Key button)
Click the “Key” button and select a newly captured image.
2. Learn Key
Click “L(Live)Match/Learn Key” menu item.
We will learn each of the four keys in Figure 13.3. To learn more than one key, select another key via the “Key” button and click “L(Live)Match/Learn Another Key” menu item. The data is stored in “.\examples\live\keys\”. There are seven images, image 1, 3, 5, and 7; these are used to teach the FrameFinder what to look for.
3. Set Live Video Source
This step has already been done earlier.
4. Searching
Set the fourth text box in Figure 13.2 (Ending Frame) to 300 (300 seconds). Click “L(Live) Match/Search” menu item.
You will see the results in Figure 13.4.
Figure 13.4 Live search Results.
The results are also displayed in the third picture box (labeled ‘Results’). Results consist of matched frames. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
14. Readme.Txt
14.1 About Attrasoft VideoFinder
Attrasoft VideoFinder consists of two components, VideoFinder and FrameFinder. The software will perform three independent tasks:
- Attrasoft VideoFinder matches a newly obtained video clip against a pre-established library of video clips.
- Attrasoft FrameFinder searches video clips for an object specified by a key image.
- Attrasoft FrameFinder searches live video for an object specified by a key image.
14.2 How to Use VideoFinder
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking:
· “Beginner/0. Initialization (Only Once)” menu item, OR
· “Init” button.
2. Select
Library (Source button)
A video clip is converted into a video clip signature. A set of video clip signatures forms a library. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the VideoFinder.
This step selects a video clip directory. This step is implemented by clicking:
· “Beginner/1. Select Library (Source button)” menu item, OR
· “Source” button.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Library’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory;
Use the “>” button to select the next video clip; and
Use the “<” button to select the previous video clip.
2. Get Library Signatures (source.txt)
This step converts all video clips in the selected directory into signatures and saves them to the file, “source.txt”. This step is implemented by clicking:
· “Beginner/2. Get Library Signatures (source.txt)” menu item, OR
· “Signature/Library Signatures” menu item.
3. Select Key (Key button)
A key is a newly obtained video clip. The video clip is converted into a signature. The key (for example, “c:\abc\def\ghi.avi”) will be entered into the VideoFinder.
This step selects a key. This step is implemented by clicking:
· “Beginner/3. Select Key (Key button)” menu item, OR
· “Key” button.
The selected key will be displayed in the first picture box. Use the “Play” and “Stop” buttons to play the video clip.
4. Get Key Signatures
(key.txt)
This step converts a key video clip into a signature and saves them to the file, “key.txt”. This step is implemented by clicking:
· “Beginner/4. Get Key Signatures (key.txt)” menu item, OR
· “Signature/Key Signature” menu item.
5. Create Library File
(lib1.txt)
Before matching, you will need to select a library and load it to the VideoFinder. The library to be loaded is fixed to the file, “lib1.txt”. We have created a library file, “source.txt”; now we will simply copy it to the loading-library file, “lib1.txt”.
This step is implemented by clicking:
· “Beginner/5. Create Library File (lib1.txt)” menu item, OR
· “Library/Copy source.txt to lib1.txt” menu item.
6. Load Library (lib1.txt)
Before matching, you will need to select a library and load it to the VideoFinder. This step is implemented by clicking:
· “Beginner/6. Load Library (lib1.txt)” menu item, OR
· “Library/Load (lib1.txt)” menu item.
7. 1:N Matching (Match/1:N Match menu item)
“1:N Matching” (read as 1 to N matching) compares “1” (i.e., the new video clip) with “N” (i.e., the previously stored video clips). This step is implemented by clicking:
· “Beginner/7. 1:N Matching” menu item, OR
· “Match/1:N Matching” menu item.
8. First Results (2nd
F Button)
The 1:N matching results are also displayed in the third picture box (labeled ‘Results’).
To select the first retrieved video clip, use:
· “Beginner/8. First Results (2nd F Button)” menu item, OR
· Second “F” button.
Use the “Play” and “Stop” buttons to play the video clip.
9. Next Results (2nd
> Button)
The 1:N matching results are also displayed in the third picture box (labeled ‘Results’).
To select the next retrieved video clip, use:
· “Beginner/9. Next Results (2nd > Button)” menu item, OR
· Second “>” button.
Use the “Play” and “Stop” buttons to play the video clip.
14.3 How to Use FrameFinder
0. Initialization (Only Once
After Starting the Software)
This step initializes the software. This step is implemented by clicking:
· “Beginner/0. Initialization (Once)” menu item, OR
· “Init” button.
1. Select Key (Key button)
A key is an image, which specifies an object. A key is selected via the “Key” button. This step selects a key. This step is implemented by clicking:
· “Beginner/1. Select Key” menu item, OR
· “Key” button.
The selected key will be displayed in the first picture box.
2. Learn Key
In this step, the FrameFinder will learn the object specified by the key image. This step is implemented by clicking:
· “Beginner/2. Learn Key” menu item, OR
·
“V(Video)Match/Learn Key” menu item.
3. Select Source
A source is a folder consisting a set of video clips. A set of video clips will be placed in a directory and the directory path (for example, “c:\abc\def\”) will be entered into the FrameFinder.
This step selects a video clip directory. This step is implemented by clicking:
· “Beginner/3. Select Source” menu item, OR
· “Source” button.
The video clips in the selected directory will be displayed in the second picture box (labeled ‘Source’).
Use the “Play” and “Stop” buttons to play the video clip.
Use the “F” button to select the First video in the directory;
Use the “>” button to select the next video clip; and
Use the “<” button to select the previous video clip.
4. Searching
“Searching” compares the key image with all video frames in all of the video clips. This step is implemented by clicking:
· “Beginner/4. Search” menu item, OR
· “V(Video) Match/Search” menu item.
The results are also displayed in the third picture box (labeled ‘Results’). Results consist of two sections: matched videos and matched frames. Use the “Play” and the “Stop” button below the label ‘Results’ to play the matched video clip. Use the “VF”, “V>”, and “V<” button (Video First, Video Next, Video Previous) to select a video clip in the matched collections. Use the “IF”, “I>”, and “I<” button (Image First, Image Next, Image Previous) to display matched image frames.
14.4 Software Requirements
Software Requirements:
- Microsoft Windows;
- Windows .Net Framework.
To get the latest version of .Net Framework, use Internet Explorer; then click “Tools\Windows Update".
14.5 Installing the Software
If you have not done so, go to Internet Explorer, then click “Tools\Windows Update" to download Windows .Net Framework .
Click “CD:\setup.exe” to install the VideoFinder.
The serial number is the word “attrasoft”.
14.6 Supported Video Formats and Camera
The VideoFinder uses Windows .Net Framework for video decoding and camera. It supports *.avi files, *.wmv files, and limited *.mpg files. It supports Logitech and several other cameras.
14.7 Information and Support
Attrasoft ImageFinder
Attrasoft
P. O. Box 13051
Savannah, GA. 31406
USA
©
Attrasoft 1998 – 2008
gina@attrasoft.com (Email Subject: Attrasoft)
Phone: (912) 484-1717