Virtual Gesture Input Device Hack using Wiimote

This projects aims at designing a low cost gesture based input device for computers using a Nintendo Wii Remote(Wiimote). This work is inspired by the virtual whiteboard projects done by Johnny Lee of Carnegie Mellon University. He had used windows based bluetooth stack & wiimote interface library with Visual C#. Here I made use of Cwiid library for python and implemented in Linux environments too.

Components used:

Wii-remote
USB Bluetooth dongle*
Infrared-LED
Python Libraries - Cwiid, Pygame

*If your computer has built-in bluetooth, a dongle is not necessary.

WIIMOTE:

Wiimote is a hand-held primary controller for Nintendo's Wii console. Besides just being a joystick, it has multiple features as follows:

3-axis accelerometer
Infra-Red Camera
Piezo Speaker
Rumble-Vibration feedback
Player LEDs (also indicate battery level)
Buttons
Bluetooth connectivity
External I2C port

IR CAMERA:

The wiimote has a 128x96 resolution Infra-Red camera in front that can track upto 4 IR sources. The resolution of the camera is extrapolated to 1024x768 by the built-in processor with a response rate of 100Hz (100 frames/second). The camera is covered with Infra-Red filter for better sensitivity. The IR camera has a field of view of around 33 degrees horizontally and 23 degrees vertically. The python-cwiid library is used to access the x and y co-ordinates of the IR sources.

INFRA-RED LED PEN:

An infra-red LED is connected in series with 100 Ohm resistor and a switch. This arrangement is placed inside an empty pen with the LED facing outwards and taped together.

WORKING:

Firstly, the pairing between Wiimote and the host PC is established. Then, the input reporter is initiated so that the joystick's buttons,accelerometer or IR camera data can be retrieved. Since we are tracking an Infrared LED, we need to initialize the IR camera.
After initializing the IR camera, in the basic mode,one among the three modes of operation, we get around 5 bytes for 2 IR sources combined. Each position is 20-bit wide;10-bit for X and 10-bit for Y coordinate. The format of data for two IR sources detected by camera is given as follows:

Image Source: wiibrew.org

So, the Byte[0] can be combined with Byte[2]<9:8> to form a 10-bit data which corresponds to the x-position of first IR source. Similarly, the y-position is also determined. So by mapping these coordinates to those of the computer screen, we can make the IR-LED's motion to be sensed and tracked in real time.

RESULTS:

The following image is drawn using the IR pen.

VIDEO DEMO:

Comments

Shell Scripting to Display Bar Graphs in Linux Terminal

This week was quite hectic for me with course assignments and their merciless deadlines. One such assignment in Advanced Computer Architecture was simulation of various benchmarks in alpha architecture in simplescalar , a well-known computer architecture simulator. I was supposed to run a total of 4 benchmarks with different configurations of cache memory, instruction issue widths, commit widths, in-order execution modes, etc and I had to plot the required performance parameters for every benchmark. A conservative estimate would be around 40 plots! Since the simulation platform was Linux, I could breathe a sigh of relief since most things can be automated using a powerful tool called shell. Scripting made my life easier here since I can automate a bunch of simulations without having to keep an eye on each and every simulation which would take anywhere from 20 minutes to 20 hours. The problem arose when it came to plotting the performance results. Becau...

Programming STM32 ARM microcontrollers in Arch Linux

Once upon a time, not so long ago, the 8-bit microcontrollers were ruling the hobbyist embedded world. But today, the 32-bit ARM Cortex Microcontrollers are so inexpensive and power efficient that there is no good reason to ignore them. Here, let us see how to program a STM32 ARM cortex Microcontroller in Linux environment. The specific microcontroller used here is an STM32F103C8 (ARM cortex M3) programmed in Arch linux. Components Generic STM32F103 board (blue pill) STLINK-V2 (STM32 programmer) Female-Female connectors All the above components can be bought from ebay for less than $10 total. The STLINK-V2 is optional since you can use any of USB-SERIAL converters like FT232, CP2102, PL2303, CH340 and the built-in UART bootloader of STM32 chip to program. So if you already have any of the above serial converters, you don't really need STLINK to program the STM32F103 microcontroller. But STLINK helps if you plan to use in circuit debugging functionalities. Software The...

JTAG - Test Access Port (TAP)Controller based Xilinx FPGA configuration using Raspberry Pi

JTAG - Joint Test Action Group is an IEEE 1149.1 standard used in many silicon devices for programming and debugging purposes. Xilinx FPGAs support this JTAG protocol for their configuration. Here I have designed a JTAG FPGA bitstream programmer using Raspberry Pi which programs the bit file into FPGA in fraction of seconds! JTAG physical bus has four lines: TMS (Test Mode Select) TDI (Test Data In) TDO (Test Data Out) TCK (Test Clock) Components Used: Raspberry-Pi Xilinx Spartan 3E FPGA (XC3S250E in Papilio One) Jumper wires BLOCK DIAGRAM: TAP CONTROLLER: The TAP(Test Access Port) controller is a state machine inside the FPGA which changes it's state based on TMS input. For instance, let us assume that the state machine is in " Test-Logic-Reset " state. Now if I drive the TMS pin low and toggle the TCK pin, the state machine will go to " Run-Test/ Idle " state. This is how we move to different states. Note ...

Electronics Adventures

Search This Blog