2D Visualisations for Gaze
Scanpath
- Basis of the association between eye fixation and visual context is the point of regard (PoR) (Dodge 1907)
- Scanpaths show the sequence of several point of regards (Yarbus 1967, but russian original earlier in 1965)
- Scanpaths are a qualitative visualization
- Several variations exist: size of PoR scaled to duration, animated scanpaths, etc.
Scanpath on a 2D image
Yarbus, A. L. (1967). Eye Movements and Vision. Plenum Press.
2D Visualisations for Gaze
Regions of Interest
- Aggregate fixations over certain regions
- Regions of Interest (RoI) provide quantitative feedback
- Several variations exists, example: links between regions reveal transition probabilities
Mapping from fixations to Regions of Interest with transition probabilities
Fitts, Jones and Milton (1950). Eye movements of aircraft pilots during instrument-landing approaches. Aeronautical Engineering Review.
2D Visualisations for Gaze
Heatmaps / Fixation Maps / Attentional Landscapes
- Concept introduced by Pomplun, Ritter and Velichkovsky (1996), today often referred to as Heatmaps
- Re-Interpretation of Attentional Landscapes (Elias, Sherwin and Wise, 1984) on images
- Elaborated by Wooding (2002) as Fixation Maps
- Model area of high acuity of gaze as gaussian distribution (SD 1 degree of visual angle)
- Aggregate over several PoR
- Heatmaps provide qualitative feedback
Heatmap of gaze on Boring figure
Pomplun, Ritter and Velichkovsky (1996). Disambiguating complex visual information: Towards communication of personal views of a scene. Perception.
Geometry-based Approaches
2.5D Gaze Tracking
2.5D object geometry acquisition used by Rötting et al. (1999)
Rötting, Göbel and Springer (1999). Automatic object identification and analysis of eye movement recordings. MMI-Interaktiv
2.5D Gaze Tracking
- offline process for the semi-automatic detection of 2.5D PoR (Rötting et al. 1999)
- based on monocular eye tracking, scene-camera and Ascension Flock of Birds 6DoF tracker
- 2 staged process
- object regions were manually labeled in different views to extract 2.5D geometry-model
- object regions were projected onto scene-camera for each frame and fixations clustered on the image plane
Geometry-based Approaches
Duchowski et al. (2001)
- use binocular eye tracking with a HMD
- designed for diagnostic use
- returns visual line of sight for each eye and computes intersection point
- creates virtual line of sight (cyclopean)
- computes geometry intersection based on virtual line of sight
- depth information of 3D PoR is thrown away in favor of geometry-based approach (!)
Setup for binocular eye tracking in HMD-VR
Duchowski et al. (2001): Binocular Eye Tracking in VR for Visual Inspection Training. Virtual Reality Software and Technology.
Holistic Approaches
- determine the 3D PoR based on measurements alone
- require at least two viewing directions (binocular eye tracking or temporal monocular eye tracking)
- mapping to geometries only done for quantitative interpretation
3D PoR triangulation based on vergence
Pfeiffer et al. (2009). Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments. JVRB.
Holistic Approaches
Essig et al. (2006)
- binocular eye tracking on desktop VR with anaglyph stereo presentation of target dots
- machine-learning approach (parameterized self-organizing map) to 3D PoR estimation
- significant reduction of error compared to naive 3D triangulation (45 percent)
Setup with anaglyph stereo glasses and binocular eye tracker (Eye Link I)
Essig et al. (2006). A neural network for 3D gaze recording with binocular eye trackers. The International Journal of Parallel, Emergent and Distributed Systems.
Holistic Approaches
Pfeiffer et al. (2009)
- extended the approach of Essig et al. (2006) to generic 3D environments
- Virtual Reality environments presented with shutter glasses
- Real World environments
- applied this approach to object selection tasks
Binocular eye tracker (Arrington Research), polarized stereo glasses and marker for optical tracking
Visualizations for Geometry-based Approaches
Attentional Maps
- proposed by Stellmach, Nacke and Dachselt (2010)
- come in different flavours
- Projected Attentional Maps (2D projection == common heatmap)
- Object-based Attentional Maps
- Surface-based Attentional Maps
- proposed solution based on geometry-based 3D POR/Desktop VR
Visualizations for Holistic Approaches
3D Scanpath - VP 09
Individual 3D scanpath of VP 09, specific sequence of objects, holistic 3D PoR, static radius
3D Scanpath
- $POR_{sphere}(\vec{x}): (\vec{x}-\vec{p_{POR}})^2 \le r(\vec{y})^2$
- $r(\vec{y}): |\vec{y}-\vec{p_{eye}}|\tan \alpha$
- with
- $POR_{sphere}$: membership function for $\vec{x}$
- $\vec{p}_{POR}$: 3D point of regard
- $\vec{p}_{eye}$: 3D position of the observing eye
- $\alpha$: angle of high visual acuity
- $\vec{y}$: either $\vec{p_{POR}}$ (static for one POR, as in 2D) or $\vec{x}$ (dynamic)
Visualizations for Holistic Approaches
3D Attention Volumes
3D Attention Volume aggregating over 10 participants, holistic 3D PoR
3D Attention Volumes
- Basic idea: compute the share of visual attention for every point $\vec{x}$ in space (instead of a plane)
- $3DAV(\vec{x}): d(t) e^{-\frac{|\vec{x} - \vec{p}_{POR}|^2}{\sigma(\vec{p}_{eye},\vec{x})}}$
- with $d(t)$: amplification factor depending on the duration
- and computing $\sigma$ as a function of the actual distance of $\vec{x}$ from the eye $\vec{p}_{eye}$ (here cyclopean)
- Image to the left shows version with fixed $\sigma$, dynamic $\sigma$ shown in video on the next slide
Visualizations for Holistic Approaches
3D Attention Volumes for Real Objects
Object
Image of the real world 3D object
Attention Volumes from Different Perspectives
