Overview

What if you could move the game of pong into the real world? Pong is an arcade game that simulates table tennis. Players compete against each other by controlling in-game paddles on the left or right side of the screen to hit a ball back and forth. Points are earned when a player fails to return the ball to the other. In our real-world version of Pong, Air Pong, a drone is the ball and the people are the paddles.

This team project consisted of a closed feedback loop that was divided into three parts: Visualization, Controls, and Communications. I implemented object detection and visualization. My goal was to decode and process data from the camera, and then output the necessary coordinate information for Controls, so that the drone knew where to fly. Ultimately, we were unable to integrate all the separate modules together in order to present a complete product, so I will show just the Visualization part here!

Implementation

A camera that hangs from the ceiling uses computer vision to track the real-time locations of the drone and players within a defined playing field. The physical positions of the players represent the locations of the paddles, and the physical position of the drone represents the location of the ball. We used an FPGA to process camera and sensor data, create game logic, and define controls to send commands to the drone in 3D space. While the game is in motion, a visualization of the real world pong game appears on the VGA monitor.

This project was primarily implemented in Verilog, compiled with the Xilinx ISE software, and programmed into the labkit's FPGA. Labs involved learning about using breadboards, clocks, LEDs, switches, generic i/o signals, and logic analyzer connectors. The labkit's setup came with a dedicated video DAC that drives the VGA monitor, and a 4MB of high-speed synchronous ZBT memory that was soldered onto the labkit PCB. A parrot bebop drone was used as the "ball".

Final project for 6.111 Introductory Digital Systems Laboratory.

Read our Final Project Paper!
View the code here.

Using movable crosshairs to specify objects to track. Includes pong and paddle overlay.

Chroma keying objects by thresholding HSV values. The colored pixels represent values within threshold.

Example of badly thresholded values. What happens when you key the green background instead of the pink paper.

NTSC camera mounted to the ceiling.

Labkit's FPGA

System Setup: FPGA, VGA Display Monitor, and Xilinx ISE software.

Parrot Bebop Drone

Green screen taped to the floor, with real-life pong (drone) and paddles (colored paper).