Assisted Songwriting Machine
A real-time embedded system for interactive music creation.
This project combines hardware signal processing, AI-assisted composition, and a custom-built web interface to help users generate and explore original melodies. Built on the ESP32, the system records input from a button piano, then offers multiple playback and completion options—including a trained RNN, a Markov chain engine, and user editing through a web-based MIDI note editor.
Hardware Design
Developed the user-facing input and playback system on the ESP32 microcontroller, including key and tempo calibration, real-time melody recording via an 8-button piano interface, and audio playback through a buzzer speaker. The state-machine-based flow enabled clean transitions between recording, selection, and playback stages.
Markov Chain Completion
From my background as a musician, I implemented a probabilistic algorithm that extends user-written melodies using a weighted matrix of scale-degree transitions. The system dynamically blends corpus-based music theory data with user-specific input patterns to generate new, musically coherent phrases in both major and minor keys. The design preserves each individual user’s style while keeping the melody within general pleasing musical boundaries.
Real-Time Audio Processing
Since immediate audio feedback is an important part of writing music, I wrote a custom MIDI-to-audio synthesis pipeline that converts user melodies into sinusoidal waveforms with configurable harmonic profiles. The audio is generated and streamed directly from the ESP32’s DAC in real time, optimized for memory-constrained playback without sacrificing musicality.
Technical Skills
ESP32 Firmware: Real-time input handling, audio playback, and RESTful web requests
Digital Signal Processing: Custom pitch detection and MIDI-based audio synthesis on microcontroller
Algorithm Design: Markov Chain melody generation in Python and C with major/minor key transitions
Hardware Prototyping: Button matrix input, DAC audio output, LCD feedback
Toolchain: PlatformIO, LabVIEW, Python MIDI scripting, Google Colab for AI integration