We try our first biomedical project and create the world's first ElectroOculogram-controlled console game. Using the principles of the eye, we can track the movement using some sticky sensors and some custom circuitry. Watch us blink and nod our way through classic Mario. You can create your own Eye Gaming System using LabVIEW, a LEGO MINDSTORMS NXT, and a biomedical sensor.
Eyes + Electrodes + Filter/Amp + sbRIO + NES = Awesome
In this episode we show off our Eye Mario system that allows you to play any NES video game just using your eye movements. In this video we give you a basic overview of how this system works.
Play NES with your Eyes - Episode 04
The Team
Huge thanks to the engineers that made this project possible:
Doug Farrell
Will Schoettler
Stephen Meserve
Hunter Smith
Elizabeth Jameson
"The Compound" Staff
Chris Culver
Barron Stone
System Overview
The goal of this project is to control a Nintendo Entertainment System using solely eye movement. This involves reading in eye position using a technique known as electrooculography (EOG) as well as interfacing with the NES standard protocols to simulate a game controller all using an NI sbRIO and a custom PCB Daughter card.
System Set Up
Electrodes placed on the face around the eyes feed a small electrical signal into isolation and analog input circuitry that was custom built on an NI Singleboard RIO daughter card. All processing is done on the NI Singleboard RIO and information is passed back out digitally to the daughter card which communicates with an original Nintendo Entertainment System.
Isolation Circuitry
The use of wet electrodes, as previously stated, drops the natural electrical resistance of the skin for improved conductivity. Unfortunately this does not only make for improved measurements but also weakens the body’s defenses against external shock, for instance from a power surge in the measurement circuitry. To protect the test subject from a possibly deadly shock isolation circuitry is used.
Electrooculogram (EOG)
The human eye is polarized due to the concentration of electrodes in your retina on the back of the eye. This create a dipole potential. When your eye moves around in your head the electric field aroudn your eye rotates with it. We used electrodes ( Ambu Blue Sensor N wet electrodes) to detect very small changes in voltage that would indicate which way you are looking .
These electrodes are placed around the eye in a diamond pattern to detect its current position using electrooculography. The voltage differences between the horizontal and vertical pairs are taken to determine eye position. Additionally a ground reference electrode is applied to the neck. This reference is used to filter out any base biological electrical activity not related to eye position. By using a wet electrode a conductive material is used to drop the electrical resistance of the skin and allows for better electrical conduction.
The Waterloo Labs Eye Mario Team
photo credit: Jay Janner - Austin Statesman
diagram of the polarized human eye
NI Singleboard RIO
After the information has been digitized by the isolation and analog input circuitry it is passed down to the Singleboard RIO for analysis. It is initially acquired on the FPGA of the RIO and then passed through DMA transfer to the built in Real Time Controller for processing.
Both horizontal and vertical eye position signals are compared to preset thresholds. If the measured signal surpasses the threshold a controller command is triggered and sent back out to the daughter card for communication to the Nintendo. For example, if the test subject looks far enough to the left a digital line corresponding to a left button being pressed is asserted and handed out to the daughter card to be passed off the gaming system.
Nintendo
The Nintendo controller is essentially a shift register that serializes 8 digital lines, each corresponding to a button on the controller. All of this is done based on signals coming in from the Nintendo such as a latch and clock signal. By recreating this on the daughter card and replacing the buttons from controller with digital lines the team could successfully communicate with the NES.
The EyeMario EOG Filter & Amplification Circuit
How to Make a Custom Daughter Card Circuit Board
How to Make a Custom Daughter Card Circuit Board
A 3D render of the daughter card
The layout schmatic for the daughter card
The Daughter Card Display
Software Overview
The EyeMario software has 3 compontnets: The FPGA code, Real-Time code, and the Windows host. The FPGA code is used to read the signal from the daughter card using a Synch 3 filter to extract the data from the Data Sigma ADC. The Real-Time code takes the processed data and compares it to range limits to determine if it is a blink or a look. The Windows host shows the signal on a plot that allows the user to set the calibration limits. The Real-Time code then decides which controller output to send to the NES.
EOG Data