By Dillon Simeone
Long before we understand what sound is, we feel and hear it together. In the womb, as our sensory systems develop, the fluid environment delivers the mechanical thud of our mother’s heart and the physical impact of her steps directly through tissue and bone. That kinetic pulse is the very first rhythm any of us experience.
Thousands of years ago, around campfires, human beings used rhythm to connect with one another. It was not just a quiet sound carried through thin air; it was feet hitting packed earth, hands slapping wood and stone, and deep chest vibrations. Rhythm brought people together into a single tribe.
I was born deaf. Lowercase “deaf” describes my audiological condition, while uppercase “Deaf” describes the linguistic and cultural community I call home. Growing up, I learned very quickly how our school system views sound. When music class started, the hearing kids stayed in the room together to sing, drum, and play bells. I was routinely pulled out of class and sent to the library to read or do worksheets by myself. Don’t get me wrong: I loved the library. Nobody demanded my attention on their flapping lips. I could hoard a stack of “Goosebumps” paperbacks in absolute peace.
Dillon Simeone developed HAXEL (Haptic Pixel Engine), an open-source firmware that treats touch the same way a screen treats light. Credit: Meghan Sinnott
It was only much later that I started working out what that music room trade had really cost me, losing the early, shared wiring that anchors you to other people. Today, I am anchored to hardware, logic analyzers, and my machines. If I had been born hearing, would that still be true? I do not know.
Music, in the emotional sense that hearing culture reveres, has always been an alien concept to me. I often imagine sitting in a packed concert hall while an orchestra plays something meant to move the room to tears. Within minutes, I would be bored, looking around at the crowd. I would turn to a hearing friend next to me, see them crying too, and simply ask if they were okay.
For readers who grew up with music and watched it slip away as their hearing shifted, that silence is a profound, legitimate grief. For me, however, born profoundly deaf, music was never something I lost; it was something I never had. Intriguingly, neuroscientists recognize a condition called specific musical anhedonia, where individuals with perfectly intact hearing simply cannot experience emotional reward from music, finding it as neutral as static.
For those mourning a lost acoustic world, haptics offers a physical way back in. For me, it offers an entirely new language.
The Strength of Touch
Because my brain developed without acoustic input, my auditory cortex underwent cross-modal neuroplasticity, reallocating its neural real estate to visual and tactile processing.
Touch offers a pathway that does not depend on the cochlea. Although human skin cannot follow a cymbal’s shimmer up into the kilohertz, it does not need to. What fingertips excel at is timing, edge detection, and texture. Deep Pacinian corpuscles respond with exquisite sensitivity to vibrations around 200 to 300 hertz (Hz), while shallow Meissner receptors pick up low frequency flutters and rapid taps.
When sound is routed through hardware, haptics work like an acoustic crossover network. In high fidelity speaker design, a crossover splits complex audio, sending low frequencies to a woofer and high frequencies to a tweeter. Haptics do the same thing across sensory modalities.
Instead of forcing damaged hair cells to process smeared high frequency transients, we hand off what touch does best: the exact attack of a drumstick, the sudden pluck of a bass string, how hard a note hits, and how fast it decays. A listener who can no longer hear the acoustic bite of a snare can feel the attack land crisply on a fingertip.
Nature and the Haptic Zoo
By narrowing our cultural definition of music to airborne pressure waves collected by an ear canal, modern society has artificially severed sound from physical touch. In doing so, we have turned music into an acoustic abstraction, ignoring the evolutionary reality that rhythm began as a full-body seismic event.
Animals have always relied on physical vibration to communicate across space. Male alligators bellow with deep, low-frequency rumbles that dip directly into sub-audible infrasound, vibrating the surrounding water so intensely that the surface erupts into dancing droplets. Elephants transmit and detect seismic rumbles through their footpads, communicating across miles of savannah.
This biological connection inspired an interactive installation I am building called the Haptic Petting Zoo. Instead of using acoustic speakers, the Petting Zoo uses custom mechanical vibrators and kinetic actuators. Visitors hold and touch physical animal forms that reproduce distinct tactile patterns from nature. You feel the heavy, low rumble of an alligator in your ribcage, or the rapid flutter of an insect wing directly on your fingertips.
Simeone and Shawn Trail, Ph.D., and team are creating a tactile, Deaf-centered STEAM music curriculum to improve engagement, learning outcomes, and creative expression for K-12 students, especially those who are deaf or hard of hearing. Credit: Eric Buchner
To create these sensations, we use solenoids. These are electromagnetic actuators with an internal plunger that snaps forward and retracts, much like the mechanical flippers and pop-bumpers inside a classic pinball machine. When dialed in with precision, solenoids do not produce a cheap, buzzing vibration like a smartphone on a desk. They deliver sharp, percussive physical impacts with real physical mass.
To coordinate those impacts across different sculptures, I built HAXEL (short for haptic pixel), open-source firmware that treats touch the way a screen treats light. Just as a pixel collapses color into a single point of light, a “haxel” streams addressable waves of physical motion to mechanical actuators from any browser.
When both deaf and hearing visitors try these devices, their expressions shift immediately. But there is a second thing that happens in the room, and it is the part I care about most: the moment raw mechanical vibration transforms into genuine emotion.
On its own, an isolated buzz is merely telemetry. But when a visitor pets a mechanical creature and solenoids tap out a rhythmic heartbeat beneath their palm, an ancient neurological switch flips. We are all primed by that first rhythm in the womb. If a tactile heartbeat accelerates into a rapid, frantic tempo, our nervous systems instinctively tighten with anxiety; if it slows to a steady, calm cadence, our shoulders drop and we are lulled toward calm.
Because the vibration is delivered through a warm, living context, the brain stops analyzing the signal as cold machinery and begins feeling it as empathy. If we can attach emotional meaning to a kinetic pulse in an artificial beast, we can do the exact same thing with a kick drum, a bassline, or an ensemble groove.
Bringing Haptics to the Classroom
Bringing these ideas out of the workshop and into real classrooms led our team at CymaSpace’s Universal Music Design program to an active research grant from the National Endowment for the Arts.
Working alongside research director Shawn Trail, Ph.D., at the University of Arizona and the Cochlear Implant and Hearing Loss Lab run by Lina Reiss, Ph.D., at Oregon Health & Science University, we are asking a direct question: How does a tactile, Deaf-centered STEAM music curriculum improve engagement, learning outcomes, and creative expression for K-12 students, particularly those who are deaf or hard of hearing?
The Universal Music Design Team includes Sean Wolfe, our lead audio engineer, and Misty Jones Simpson, our lead curriculum designer. Both are experienced musicians who lost their hearing in one or both ears, respectively, and now use cochlear implants. We’re collaborating to consider how students with no or partial hearing can equally engage in our music curriculum.
Our starting point was GestoLumina, a modular interface we built at CymaSpace. These are fingertip rings equipped with micro-force sensors and vibration motors, paired with a wrist-mounted LED bracelet that translates audio into color. The first prototypes were laser-cut basswood, hot glue, and an ESP32 microcontroller whose pin layout was, frankly, evil.
In musical performance, timing is everything. While passive listeners can tolerate wider sensory discrepancies, active musicians notice timing delays at roughly 15 to 20 milliseconds (ms), where rhythmic synchronization starts to degrade. That tight timing budget leaves very little margin.
Simeone leading a HAXEL workshop. Credit: Alya/@cwbanana
On our test bench, our firmware and driver circuitry process gestures and fire trigger pulses in under 2.3 ms. Mechanical actuator rise time adds additional latency on top of that depending on the motor design, which is why characterizing end-to-end physical onset with bench accelerometers is part of our ongoing work. Keeping electrical latency tiny ensures the physical pulse has the best possible chance to align with the visual flash and acoustic sound.
The moment I knew the concept worked was during a duet. My musical partner is a cochlear implant user, while I am completely deaf. We did not need to speak, and we did not need to hear each other to stay in sync. I squeezed a rhythm between my thumb and forefinger, and my partner felt that exact kinetic impact land on their finger. It was multitrack monitoring conducted entirely through skin. Then they tapped a syncopated polyrhythm back onto mine.
We are taking these tools into public school classrooms with our regional inclusive education partners. Students explore musical scales, dynamic variations, and complex rhythms both with and without haptic feedback, while we track their engagement through structured surveys and observations. We are not simply handing deaf and hard of hearing (D/HH) kids a novel gadget; we are testing whether tactile physics keeps them anchored in the room alongside their hearing peers.
True Sonic Agency
Sound is physical energy, but music is the deliberate architecture of that energy. Music is not merely a sequence of acoustic pitches; it is the physical relationship of attack, duration, syncopation, tension, and decay over time.
Much of the pleasure human beings derive from music comes from pattern recognition and musical expectancy: Our brains delight in anticipating exactly when a note will land, predicting the pocket of a groove, and feeling the release of a delayed beat.
When a drummer strikes a snare or a synthesist modulates a low frequency oscillator, they are orchestrating physical shockwaves. When properly engineered hardware delivers those impulses directly to the skin with very low latency, a student is not receiving a pale acoustic imitation; they are experiencing music's structural foundation. They can feel the pocket, track the groove, and shape the dynamics themselves.
Why, then, does hearing culture experience music with such overwhelming emotion, while someone like me remains neutral? Part of the answer, I suspect, brings together neurodevelopment and classical conditioning.
In early childhood, our brains rely on critical developmental windows to build the cross-modal scaffolding that binds our senses together. When that early sensory access is in place, life experiences take over through classical conditioning, particularly in its evaluative form: pairing a neutral stimulus with an emotional event until the signal itself evokes the feeling. Think of Pavlov’s dog.
Nobody is born weeping at the sound of a cello. Hearing culture spends a lifetime pairing acoustic frequencies with joy, heartbreak, grief, and camaraderie. Sound is present during all of life's most profound moments, repeated hundreds of times until the acoustic frequency alone can ring the emotional bell.
I had rich, meaningful experiences throughout my life, too, but because I was isolated from sound, the acoustic frequency was never paired with the emotion. I simply never heard the bell.
Rhythm possesses an ancient, hardwired pull, but what was taken from me when I was sent to the library was not the musical scale. It was the room: the shared, synchronized, full-body experience where sound is forged into human belonging. The research we’re working on is meant to tap into this primal experience and replicate it across classrooms around the world.
That is the part we can give back. I do not build these instruments because I have fallen in love with melody, or because I feel a poetic calling to teach deaf children how to sing. I build them because access should never be dictated by the limits of a biological sensor.
Dillon Simeone lives in Oregon. The Haptic Petting Zoo and its accompanying tactile learning frameworks have been shaped by continuous support across Oregon’s creative and technical ecosystems and through multiple collaborations. CymaSpace’s Universal Music Design research was seeded through the Oregon Community Foundation, with ongoing K-12 haptic interface research funded in part by a grant from the National Endowment for the Arts. This research is conducted in collaboration with Oregon Health & Science University and the University of Arizona. Rapid fabrication, community prototyping, and tour development for The Haptic Petting Zoo are rooted at Control+H (PDX Hackerspace) under OCF’s Creative Heights grant program. Lina Reiss, Ph.D., is a 2027 Emerging Research Grants scientist, generously funded by Royal Arch Research Assistance. She also was a 2012–2013 ERG scientist.


For those mourning a lost acoustic world, haptics offers a physical way back in. For me, it offers an entirely new language.