Open Co-Pilot is an idea I started experimenting with around 6 years ago — a hands-free driving assistant that uses real-time computer vision to automatically detect and identify road hazards. No screen taps, no looking away from the road. Just mount your phone on the windshield and drive. It’s also fully open source, built as a free alternative to the manual reporting workflows in apps like Waze.
To date, no one has really fully executed and capitalized on this idea. Some day in the near future I may pick it back up, finish it, and publish it on the App Store for fun.
What It Does
The idea is to combine Waze-style navigation with automated visual awareness. The system runs two core capabilities simultaneously:
Object Detection & Localization — Using YOLOv4, the system identifies and localizes objects on the roadway in real time during actual driving scenarios.
Monocular Depth Perception — Rather than requiring stereo cameras or LiDAR, Open Co-Pilot estimates depth and spatial relationships from a single camera using self-supervised monocular depth prediction. This means any modern smartphone can serve as the sensor.
Hardware
The project is designed to run on consumer Apple devices with an A12 chip or newer (iPhone XR and later). All you need is a stable windshield mount and the app.
Why It Matters
Apps like Waze showed that crowdsourced road data is incredibly valuable, but reporting still requires manual interaction — tapping buttons while driving. Open Co-Pilot automates the detection entirely so drivers never have to take their hands off the wheel or their eyes off the road. And because it’s open source, anyone can run it, improve it, or adapt it — no expensive ADAS package required, just the phone in your pocket.