I built a multitouch pad music player that uses gesture recognition to execute various sounds. This application produces two sound styles and includes eight different instruments to choose from! Mini-project for 6.810 Engineering Interactive Technologies.
To create the multitouch pad, I designed, laser cut, and bent the acryllic into a touchpad. For the electronics, I soldered in the LEDs and USB cable, using an ON/OFF switch to control the power input. For gesture recognition, I used OpenCV to track finger movements inn order to play the appropriate sounds accordingly.
For more information on the implementation and features of this application, check out the images and video below!
Camera for gesture recognition mounted at the bottom of the multitouch pad
Lasercut and bent acryllic
USB port to power the multitouch pad
The LED Candle simulates a real candle by turning off when wind is detected, and turning back on with a button press. The entire system can be toggled with an on/off switch. Mini-project for 6.810 Engineering Interactive Technologies.
What if you could move the game of pong into the real world? Pong is an arcade game that simulates table tennis. Players compete against each other by controlling in-game paddles on the left or right side of the screen to hit a ball back and forth. Points are earned when a player fails to return the ball to the other. In our real-world version of Pong, Air Pong, a drone is the ball and the people are the paddles. Final project for 6.111 Introductory Digital Systems Laboratory.
This team project consisted of a closed feedback loop that was divided into three parts: Visualization, Controls, and Communications. I implemented object detection and visualization. My goal was to decode and process data from the camera, and output the necessary coordinate information for Controls, so that the drone knew where to fly.
A camera that hangs from the ceiling uses computer vision to track the real-time locations of the drone and players within a defined playing field. The physical positions of the players represent the locations of the paddles, and the physical position of the drone represents the location of the ball. We used an FPGA to process camera and sensor data, create game logic, and define controls to send commands to the drone in 3D space. While the game is in motion, a visualization of the real world pong game appears on the VGA monitor.
This project was primarily implemented in Verilog, compiled with the Xilinx ISE software, and programmed into the labkit's FPGA. Labs involved learning about using breadboards, clocks, LEDs, switches, generic i/o signals, and logic analyzer connectors. The labkit's setup came with a dedicated video DAC that drives the VGA monitor, and a 4MB of high-speed synchronous ZBT memory that was soldered onto the labkit PCB. A parrot bebop drone was used as the "ball".
Using movable crosshairs to specify objects to track. Includes pong and paddle overlay.
Chroma keying objects by thresholding HSV values. The colored pixels represent values within threshold.
Example of badly thresholded values. Keyed the green background instead of the pink paper.
NTSC camera mounted to the ceiling.
System Setup: FPGA, VGA Display Monitor, and Xilinx ISE software.
Parrot Bebop Drone
Green screen taped to the floor, with real-life pong (drone) and paddles (colored paper).
LED Color Organ
By listening to an audio file and using a low/high-pass filter to filter out treble, mid, and bass frequencies, the LED color organ emits light via the respective LEDs. Final project for 6.070 Electronics Project Laboratory
Bass Freq: Red
Mid Freq: Green
High Freq: Blue
Debugging our circuit via the oscilloscope and recalculating our math