I don’t know how to play an instrument, so obviously I built one as an app.
Literally, everyone in my family can sign or play an instrument, and I’m the odd one out.
And I know what you’re thinking, “Who cares?
With AI, you can build almost anything.” I’m more excited about the technique I chose to build the app with my agent.
Specifically I used context from the agent session that built the app to find and fix the most important gap in its Playwright tests.
Here’s how I did it.
Step 1: Install Entire Entire captures the prompts, transcripts, tool calls, and decisions behind agent-generated code, seamlessly connecting that underlying context to your Git commits through lightweight checkpoints.
On macOS: Check out these instructions to install on your operating system.
Step 2: Create the project I created an empty directory (or you can ask your agent to do this) Step 3: Enable Entire Before handing off any work to the agent, I initialized Entire directly within the repository because I wanted to capture my agent sessions: You can also target a specific agent (I personally use Codex): This sets up the background hooks Entire relies on to capture agent activity, binding that session context directly to the commits generated along the way.
Step 4: Turn the vague idea into a plan Rather than starting with a rigid technical spec, I simply shared my initial idea: Working together, the agent helped refine this into an “Air Jam” concept: a browser app where MediaPipe tracks hand gestures, a custom gesture engine interprets them, and Tone.js handles the audio output.
Wanting it to double as a learning tool, I followed up with: That added a crucial new dimension.
Beyond being a fun novelty, the app could now render notes, highlight active scales and chord tones, and eventually break down the music theory behind what was being played.
To lock this down, I asked the agent to document everything: It generated , detailing the vision, architecture, development stages, MVP targets, and explicit success criteria.
Phase 1 focused on laying the foundation: Bootstrap the browser application.
Request camera access and show mirrored video.
Integrate MediaPipe Hand Landmarker.
Draw hand landmarks and confidence information.
Define eight virtual strings.
Detect a fingertip crossing one string.
Trigger a basic synth note.
Add filtering, hysteresis, and note cooldowns.
Measure tracking rate and gesture-to-sound latency.
Most importantly, it defined a clear exit condition: Deliberate movements reliably produce one corresponding sound with few false triggers.
Step 5: Set up atomic agent commits I prefer keeping commits bite-sized, which is especially helpful when an agent is touching multiple parts of a codebase.
To enforce this, I added a repo rule: Since I didn't want that configuration change committed just yet, I quickly clarified: From then on, the agent created neat, focused commits as features were built.
And thanks to Entire, every single commit remained tied to the exact session context that created it.
Step 6: Build Phase 1 With the roadmap in place, we dove straight into implementation: The agent systematically assembled the foundation: Webcam access MediaPipe hand tracking Eight virtual strings Gesture-to-note conversion Tone.js audio Velocity-sensitive notes Gesture smoothing and cooldowns Landmark and string visualizations Camera-free note buttons Live performance diagnostics And just like that, it worked.
I could wave a finger across a virtual string and play a fully functional air harp.
Step 7: Generate Playwright tests Before expanding the app, I wanted a solid browser test suite to protect what we’d built.
I kept my request broad: The agent generated five passing Playwright tests covering: The initial interface Camera-free note buttons Starting and stopping the synthetic camera Camera permission errors The mobile layout On paper, everything was green.
But a passing test suite doesn't automatically mean you're testing what actually matter