(This image
magnified x2.)
Original 8 bit grey image captured from a video camera pointed at a whiteboard.
This example shows successive stages of processing of a hand-drawn sketch.
The example sketch depicts a command to a diagrammatic user interface
controlling a whiteboard scanner. The command is drawn directly on the
whiteboard, which is monitored by a video camera. In the working system,
it takes anywhere between 5 and 30 seconds for the system to respond to a drawn
diagrammatic command, depending mainly on the speed of the computer. The
diagrammatic command language supports a "GO" button (nested boxes), plus
specification of a region of interest on the whiteboard, and character
annotations (the C and X hanging off the bottom of the button).
(This image
magnified x2.)
Original 8 bit grey image captured from a video camera pointed at a whiteboard.
(This image
magnified x2.)
After image processing, thinning, and tracing, chain-coded curves are extracted
representing the hand-drawn strokes.
(This image
magnified x2.)
Rectangles portray line segment approximations to curve segments. Symolic
descriptors such as line segments are represented as "tokens" placed in a
spatial and scale-indexed data structure (the Scale-Space Blackboard). Circles
show pairs of line segment tokens forming an "upper left corner" configuration,
found by token-grouping operations. These are used as candidate locations to
look for nested box buttons.
(This image magnified x2.)
Magnified view of the line segment tokens associated with the nested box
button. Note the fragmentation of the strokes as they were originally drawn by
the user.
(This image magnified x2.)
Combined result of two kinds of perceptual grouping by smooth continuation.
Rectangles show all simply aligning structures. Red lines show groups of line
segments forming circular arcs.
(This image magnified x2.)
Pairs of curve tokens forming corners, nested, corners, and parallels. An
appropriate configuration of these intermediate level structures constitutes
sufficient evidence to assert the presence of a nested box button.
(This image magnified x2.)
Once the nested box button is found, routines are executed to identify the
enclosing region and character annotations associated with the nested box
button. A tracing routine follows the path specifying the enclosing region,
then models it as a polygon. The path may be either continuous or broken.
Circles and dots show steps of a "flashlight search" routine used in path
tracing. Tracing is also used to follow leader lines starting at the bottom
center of the button, to the character annotations. Two handwritten text
regions are found as shown by their bounding rectangles; these image regions
are to be passed to a handwritten character recognition module.
