respirAI: A DIY AI Stethoscope Built in a Weekend

DEV Community

We built a talking stethoscope in a weekend. The hardest bug was the word "Great!"

At UniHack 2026 our team built respirAI: a low-cost digital stethoscope that guides you through recording your own lung sounds, screens them with a neural network on your phone, and tells you whether they sound normal or are worth showing to a doctor. It is a screening aid built by students, not a medical device. Try it: https://denis-furtuna.github.io/respirAI/ (there is a practice mode, no device needed) Code: https://github.com/denis-furtuna/respirAI This post is how we built it, step by step, and the bug that cost us the most sleep. The problem A stethoscope is only useful when a trained ear is on the other end. We wanted the opposite: a device that anyone can hold on their own chest, with the phone doing the guiding and the first round of listening. Step 1: the device No custom board. A regular stethoscope chest piece sits on an INMP441 MEMS microphone, wired to a Seeed XIAO ESP32-S3 with a small LiPo battery. The firmware captures audio at 16 kHz, downsamples it to 8 kHz and streams it over Bluetooth Low Energy in 10 ms packets: a 2-byte sequence number followed by 160 bytes of 16-bit mono PCM. The sequence number matters more than it looks. BLE drops packets, and a lung recording with silent holes in it is worse than no recording, so the app counts what is missing and refuses a recording that has too many gaps. Step 2: the app The app is plain JavaScript, HTML and CSS with no framework. In a browser it talks to the device with Web Bluetooth; the same code is wrapped with Capacitor into an Android app. A check goes through the chest positions one at a time. For each one, a body diagram shows where the chest piece goes, then a breathing circle takes over the whole screen and paces two slow breaths while the device records for 12 seconds. The app runs in five languages. Two decisions we would make again: A practice device. A fake Bluetooth device that plays stored recordings let us build and test the whole flow without hardware in hand, and it is why the browser tests can run on every push. Fixed interfaces first. We wrote down the BLE packet format and the app-to-AI contract on day one, so firmware, app and AI were built in parallel on separate branches and merged at the end. Step 3: the AI The model is a convolutional network trained on the ICBHI 2017 respiratory sound database (920 recordings from 126 participants). It looks at a log-mel spectrogram of each recording and flags crackles and wheezes. Two rules we held to. We trained on sound labels, not disease labels, because the disease labels are dominated by one condition and a model can score well by always guessing it. And we split by patient, never randomly, because a random split puts the same person in training and test and inflates every number. Step 4: the voice guide Here is the thing we missed at first: the person's hands are busy. They are holding a chest piece on their own back. They cannot tap "start". So we added a voice guide on top of the flow, using an ElevenLabs conversational agent. It says where to put the chest piece, hears "ready", starts the recording through a client tool, answers questions like "where exactly?", and reads the result. The timer, the breathing circle and every button stay in our code; the agent only speaks and presses two buttons. Offline, the same switch plays pre-recorded instructions instead. One rule is absolute: while the stethoscope records, nobody talks. The phone microphone is muted and the agent's volume is zero, otherwise the stethoscope records the voices too. The hardest bug: the agent kept getting cut off The flow looked simple: The person says "ready". The agent calls our start_recording tool. The tool answers: "tell the person to breathe slowly and stay silent". We wait until the agent has finished saying that. We mute everything and record. Step 4 was the bug. The agent would start "Now breathe slowly, in and out with the ci" and go silent. Every time. Our first version of "wait until it finished" was the obvi...

Powered by Readybot.io