Line-Following Car That Plays Music

A ROS 2 car on a Jetson Nano that follows a track and plays a note when it sees a colored marker.

A small ROS 2 car that follows a line and plays a note when it sees a colored marker.

For our MAE 148 final project, my team combined autonomous driving with music. The car follows a line track using computer vision, and when its camera sees a colored marker it plays the matching note through an onboard speaker. Red, green, and blue map to the first three notes of a C major scale (C4, D4, E4), enough to play the opening of “Au Clair de la Lune.”

The team was Kim Garbez, Kenneth Ho, Daniel Weng, and me. We built on UCSD’s Robocar ROS 2 framework, running in Docker on Ubuntu 20.04 on a Jetson Nano with an OAK-D camera.

My part

Hardware setup: mounting the camera, electronics, and speaker.
Integration and debugging: getting the modified ROS 2 and OpenCV packages to run together on the Jetson.
Early testing: DonkeyCar simulation before moving to the physical car.

How it works

The Robocar lane detection node handles line following, and we modified it to run alongside a color-detection node. The color node looks for the target marker colors in the camera image and triggers the matching note on the speaker.

What didn’t work

Following the line around curves while also detecting colors was unreliable.
The car sometimes played the same note more than once for one marker.

Our proposed fixes: run lane detection in its own thread so it doesn’t compete with color detection, and control the car’s speed to set the song’s tempo.

Close-up of the car at night — The car at night.

Demo clips

One of our first times playing notes on seeing colors.

An early run on the track.

Playing "Au Clair de la Lune."

The team standing outdoors holding the finished car — The team with the car.