It has always been important for people to know where their fellow humans are looking. From business deals to courting, people rely heavily on eye contact.

We make important inferences about someone's level of interest and state of mind based on observations of what they are doing with their eyes.

Edge Analysis Systems collect information about what people are doing with their eyes.
 

The Edge Analysis System is a tool for measuring, recording, playing back, and analyzing what a person is doing with his eyes.

It is an instrument for both developing and running custom eyetracking applications.
It includes all the basic video equipment, computer hardware, and eyegaze software and C/C++ development libraries necessary to build and run eyetracking application programs.

• Gazepoint tracking measurements are made unobtrusively via one or more a remote video cameras mounted below the computer monitor.

• Nothing is attached to the subject.

• The Edge Analysis System tracks the subject's gazepoint on the screen automatically and in real time, with gazepoint measurements made at a 60 HZ or 120 Hz rate.

• Gaze direction is determined using the pupil-center-corneal reflection (PCCR) method and the image processing and gazepoint calculations are performed in software on a Windows computer.

• At each camera image sample, the Edge Analysis System generates:

  • an eye-found flag indicating whether or not the eye is visible to the camera and thus whether or not a valid gazepoint is calculated,
  • x-y coordinates of the subject's gazepoint on the computer screen,
  • pupil diameter,
  • 3-dimensional location of the eyeball center within the camera field-of-view, an indicator of head location and movement, and
  • fixation and saccade analysis.

• The Edge Analysis System is robust and extremely easy to calibrate. RMS tracking errors are typically 0.25 inch (0.63 cm) or less and the advanced image processing algorithms in the System explicitly accommodate several common sources of gazepoint tracking error:

  • Head Range Variation
  • Pupil Diameter Variation
  • Corneal Reflection Straddling Pupil Edge

Detailed Specifications:

• The Edge Analysis System uses the Pupil-Center/Corneal-Reflection method to determine the eye's gaze direction.
• A video camera located below the computer screen remotely and unobtrusively observes the subject's eye.
• No attachments to the head are required.
• A small, low power, infrared light emitting diode (LED) located at the center of the camera lens illuminates the eye. The LED generates the corneal reflection and causes the bright pupil effect, which enhances the camera's image of the pupil.
   

• Specialized image-processing software in the Edge Analysis System computer identifies and locates the centers of both the pupil and corneal reflection.
• Trigonometric calculations project the person's gazepoint based on the positions of the pupil center and the corneal reflection within the video image.
• The Edge Analysis System generates raw gazepoint location data at the camera field rate of 60 Hz or 120 Hz.

Fixation Analysis
Development Tool Kit
Run-Time Operation
Calibration Procedure
Eyetracking Application Programs
Eyetracker Outputs
Edge Analysis System Performance Specifications
Operational Requirements
Single vs Double Computer Configurations

Fixation Analysis

A function called DetectFixation analyzes the eye's motions to identify its saccades and fixations. This function takes the raw gazepoint data as uniformly sampled at the camera frame rate, detects the saccadic motions between fixations, and reports the positions and durations of fixations as they occur. The raw gazepoint data may be passed to the DetectFixation function either in real time or in post analysis.

LC Technologies' fixation function source code is available here. This code may be reused as desired as long as proper credit is given.

The Trace program (discussed below) illustrates the use of the DetectFixation function and plots fixation histories on the computer screen.

return to top

Development Tool Kit

The Edge Analysis System includes a C/C++ Language software tool kit to support custom eyetracking applications programs in a Windows 2000/XP/Vista environment. The tool kit contains a library of user-linkable functions that track the eye, analyze fixations, and make the eyetracking data readily accessible to the application program, either on a real-time or after-the-fact basis. Source code is provided for a variety of functions to analyze, store and display eye-motion data. Complete source code is also provided for several example eyegaze applications programs to illustrate typical usage of the eyegaze functions. The Edge Analysis System software supports applications programs that run either on the Edge Analysis System computer itself or on a client computer which treats the eyegaze computer as a peripheral device (see Single vs Double Computer Configurations below).

return to top

Run-Time Operation

The image processing functions in the Edge Analysis System are all implemented in software. The eyetracking functions compute the raw gazepoint coordinates each 60^th of a second in synchronization with the field rate of the video camera. At each sampling iteration, the eyetracking software places the measured eyegaze data into data structures where it is readily available to the application program.

return to top

Calibration Procedure

Prior to tracking a person's eye, the Edge Analysis System must learn several physiological properties of the person's eye in order to be able to project his gazepoint accurately. It must know the radius of curvature of the eye's cornea and the angular offset between the eye's optic and focal axes. The system learns these parameters by performing a calibration procedure where the subject sequentially looks at a series of calibration dots displayed at different locations on the screen. The procedure takes about 15 seconds and need not be repeated for a given person on different work sessions. The calibration results are passed to the eyetracking functions for purposes of future gazepoint projection, and are also placed in disk files for access by the applications programs if desired.

return to top

Eyetracking Application Programs

Eyegaze applications programs access the raw eyetracking data generated by the eyetracking functions and perform the display, recording, analysis, and replay functions desired by the developer. The Edge Analysis System includes the source code for several complete applications programs that may be used as is, modified to meet custom needs, or used as references for preparing other eyegaze applications programs. The following are brief descriptions of the applications programs.
 

Gaze Demonstration Program

The simplest application program is the GazeDemonstration program. It positions the mouse cursor on the desktop display at the person's projected gazepoint, i.e., the cursor simply follows the user's gazepoint.
 

Trace Program

The Trace program displays a user-prepared image on the computer monitor and passively collects the eyegaze activity as a subject observes the screen. The eyegaze history is stored in a disk file. After the data collection phase, the eyegaze history is played back both as a time history and as a trace superimposed on the original screen image. The trace may be paused, reversed and replayed at different speeds. Different eyegaze variables, such as the pupil diameter or the x and y coordinates of the gazepoint, may be plotted out as a function of time.

The following figures show examples of a graphical gaze trace replay. A link below the figures points to the corresponding gaze history file produced by the trace program.

Raw Gaze Trace (Individual points are 60th of a second samples)

Fixation Gaze Trace (circle sizes indicate fixation duration)
Gazetrack Data File for above gaze trace

return to top

Eyetracker Outputs

The Edge Analysis System generates the following data from each camera field image:
 

Eyegaze System Performance Specifications

Accuracy
 

* Bias errors result from inaccuracies in the measurement of head range, asymmetries of the pupil opening about the eye's optic axis, and astigmatism. They are constant from frame to frame and cannot be reduced by averaging or smoothing.

+ Frame-to-frame variations result from image brightness noise and pixel position quantization in the camera image and may be reduced by averaging or smoothing.

Speed
 

Angular Gazetrack Range
 

As the eye's gaze axis rotates away from the camera, the corneal reflection moves away from the center of the cornea. Accurate gaze angle calculation ceases when the corneal reflection "falls off" the edge of the cornea. The eye's gaze axis may range up to 40 degrees away from the camera, depending on the arc of the person's cornea. The lower 15 degrees of the gaze cone, however, is generally clipped due to the upper eyelid blocking the corneal reflection when the eye is looking down below the camera.

 

Tolerance To Head Motion (Fixed-camera Eyegaze Systems)
 

In fixed-camera Edge Analysis Systems, the eye must remain within the field of view of the camera. However, if the subject moves away from the camera's field of view, eyetracking will resume once he returns to a position where his eye is again visible to the camera.

 

Computer Usage
 

Light Emitting Diode

return to top

Operational Requirements
 

Low Ambient Infrared Light

There must be low levels of ambient infrared light falling on the subject's eye. Stray IR sources obscure the lighting from the Edge Analysis System's light emitting diode and degrade the image of the eye. The sun and incandescent lamps contain high levels of infrared light. The environment may be brightly illuminated with lights such as fluorescent or mercury-vapor which do not emit in the infrared region of the spectrum. The Edge Analysis System also works well in the dark.

Eye Visibility

The camera(s) must have a clear view of the subject's eye(s). If either his pupil or the corneal reflection are occluded, there may be insufficient image information to make an accurate gaze measurement. The camera's view of the eye can be obstructed by, for example, 1) an object between the camera and the eye, 2) the person's nose or cheek if his head is rotated too much with respect to the camera, or 3) by excessive squinting. Alternative Edge Analysis software is included to accommodate for an obstructed image of the top of the pupil, usually caused by a droopy eyelid or an unusually large pupil. The software returns a "false" condition for the EyeFound flag whenever an adequate image of the eye is not present.
 

Glasses and Contact Lenses

In most cases, eyetracking works with glasses and contact lenses. The calibration procedure accommodates for the refractive properties of the lenses. When wearing glasses, the glasses may not be tilted significantly downward, or the reflection of the LED off the surface of the glass is reflected back into the camera and obscures the image of the eye. The lens boundary in hard-line bifocal or trifocal glasses often splits the camera's image of the eye, and the discontinuity in the image invalidates the image measurements.

Soft contact lenses that cover all or most of the cornea generally work well with the Edge Analysis System. The corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can cause problems, however, if the lenses move around considerably on the cornea, and the corneal reflection moves across the discontinuity between the contact lens and the cornea.

return to top

Single vs Double Computer Configurations

With the Edge Analysis System, eyetracking application programs may be configured to run directly on the Edge Analysis System computer, or to run on another computer with the Edge Analysis System acting as a peripheral eyetracking instrument.

Single Computer Configuration

If it is desired to run the eyetracking application program directly on the Edge Analysis System computer,

1. the eyegaze camera is mounted below the eyegaze computer's monitor,
2. the eyegaze computer performs the applications functions and drives the application display on its monitor, and
3. the eyegaze computer also performs the eyegaze image processing functions required to track the test subject's gazepoint.

The single computer configuration is preferable if it is desired to minimize the amount of equipment needed to implement the project.

Double Computer Configuration

If it is desired to run the eyetracking application program on a separate (client) computer with the Edge Analysis System acting as a peripheral device,

1. the client computer performs the application functions and drives the applications display on its monitor,
2. the eyegaze camera is mounted below the client computer's monitor,
3. the eyegaze computer performs the eyegaze image processing functions and transfers the measured gazepoint data to the client computer via an ethernet or a serial communications link in real time, and
4. the eyegaze computer's monitor displays the subject's relative gazepoint in real time, allowing a test observer to view the gazepoint activity online during a test.

The double computer configuration is preferable if the application code consumes a large amount of CPU time (i.e. if there is not enough CPU time to execute both the application and eyegaze image processing code in real time), if the application program is not compatible with Windows 2000/XP/Vista, or if an observer station is desired for viewing gaze activity online.

Programming Interface

The programmer's interface to eyegaze is accomplished through a calibration program and three basic functions, illustrated in the following pseudo code:

																												{
 if (new user)
 {
    run Calibrate.exe    //have person perform the calibration procedure
 }
 setup eyegaze data structures
 EgInit();           // create and start the eyegaze thread
 for (loop)
 {
    EgGetData();        // wait for eyegaze to process next camera field
    use eyegaze data
 }
 EgExit();           // terminate the eyegaze thread
}

In the single computer configuration, the eyegaze software operates as a thread within your applications program.

In the double computer configuration, the client computer program uses an almost identical programming approach but image processing runs on the eyegaze computer and data is sent over the ethernet/serial link.

The Edge Analysis System includes source code for the EgClientDemo program which runs on the client computer and demonstrates communication with the Eyegaze System. To act as a peripheral device to a client computer, the EgServer program is run on the Edge Analysis System computer.

return to top