Meet the Researcher
Jenelle Feather, Ph.D., received her doctorate in brain and cognitive sciences from MIT and completed postdoctoral training at the Flatiron Institute’s Center for Computational Neuroscience. She is an assistant professor in the department of psychology and the Neuroscience Institute at Carnegie Mellon University, with a courtesy appointment in the machine learning department. She runs the Laboratory for Computational Perception at CMU, which builds computational models of how we perceive the world, with a particular focus on hearing, and develops methods to test how closely these models match human perception. Feather is a 2027 recipient of an Elizabeth M. Keithley, Ph.D. Early Stage Investigator Award, generously funded by William Randolph Hearst Foundations.
This project grew out of my work on auditory models that perform multiple tasks at once. Humans can understand speech while simultaneously recognizing who is talking and what is happening in the background, but current models struggle to do this, and when we tried to evaluate how well models matched human listeners on those “background” sounds, we realized that no suitable task existed to compare them. That gap led us to develop the dataset and behavioral task at the center of this proposal, and we quickly realized that the same tools could address a largely overlooked question in hearing impairment: how well people and their hearing devices recognize everyday, non-speech sounds.
My path to becoming a scientist was a bit unconventional. I don’t have any scientists in my family; in fact, I am the first in my family to attend college, let alone get a Ph.D. Going into undergrad, I didn’t really know that one could be a scientist as a career! I was lucky to be able to work in a lab starting freshman year (including as a summer job) through MIT's Undergraduate Research Opportunities Program, in part supported by my financial aid package. I got a taste for neuroscience research and ended up working in a lab as a research assistant after undergrad, as I planned to take a year or two before going to medical school. But during this time, I realized that I could become a scientist as a full-time job, and that I would love such a career path.
My primary experience with hearing loss is not my own, but rather from interacting with those around me. Family and friends are often the first to notice that a loved one has hearing loss, and frequently express frustration at not being heard or understood. Yet, current assistive hearing devices do not result in hearing in a “natural” way. Sounds are amplified in ways that do not emphasize the critical features, and multiple sound sources are often overlapping and unable to be decomposed by the listener. Even though some listeners adapt over time, complaints about the quality of sound and the annoyance of the devices often lead to hearing aids sitting in a drawer instead of being in use, leading to even more concerns from family members and isolation of the individual with hearing loss. Much of my interest in hearing is to build systems that restore hearing in a natural way, with a focus on the full auditory environment, and to make devices that feel natural enough that people actually want to wear them rather than seeming like a medical burden.
One of my first Ph.D. projects was an attempt to synthesize auditory textures (sounds like rain, wind, or fire) from various stages of deep neural networks. To our surprise, in every model I tested, including those that performed extremely well on common evaluation tasks, the sounds synthesized from the models were weird and unnatural. These sounded nothing like the rain or fire sounds we were expecting to hear! We quickly realized that this was not a bug in a single model or in the synthesis pipeline, but rather a fundamental problem with neural networks in use at the time in both the visual and auditory domains. By tracking down how to think about these “weird” sounds, I developed the framework of model metamers: multiple stimuli that have different waveforms, but are indistinguishable to a model. This work highlighted a critical failure mode in both speech recognition and image classification models. At the time, the commonly used models in the auditory and visual domains had model metamers that were completely unrecognizable to human observers and did not sound or look anything like natural sounds or images. Much of my work since has involved ways to improve the models and make how they hear more similar to how humans hear, often using tasks involving model metamers or other synthetic stimuli.
I started taking pottery classes during grad school with some lab mates, and it has stuck as a hobby. While bouncing between cities as an academic, I have been able to find community studios to join, learn from new artists, and have a general outlet where I don’t need to look at my computer or phone and just focus on the piece in front of me. At the beginning of making a pot on the wheel, you must “center” the clay and get it perfectly aligned, otherwise the pot will wobble as you pull up the walls and shape it. This often feels like a meditative practice, requiring focus but also relying on built-in muscle memory to let the form take shape. Pottery also teaches you a lot about non-attachment. Often things don’t go as planned. For instance, you might put a piece into the kiln expecting it to look one way when it comes out, but instead the glaze behaves in a surprising way. Sometimes these are good surprises, but sometimes they are not. Because of this, pottery has taught me to let go of things that are out of my control and to appreciate the surprises that sometimes come our way. This is often the case in science—occasionally experiments do not go how you would expect, and it is important to recognize that sometimes this means that you must pivot, but sometimes it means that there is a different interesting finding to explore.
I grew up in a fairly rural area in Pennsylvania. My family lived in a location where, at the time, we could not get high-speed internet at our house, even though it had become the norm for most homes and workplaces, because there were no cable lines or cell towers nearby. We did have dial-up internet, but it was painfully slow. Because of this, I was likely one of the last people to apply to my college with a paper application sent through the mail (the next year, multiple schools stopped taking paper applications and moved everything online). I often think about this when teaching and mentoring, to ensure that all students have the resources they need to succeed, as sometimes things that seem simple for one person can have hidden challenges for another.
In the near term, I hope to establish the behavioral benchmarks and computational tools needed to rapidly build and test new candidate models of hearing, and to extend these tests from normal-hearing listeners to listeners with hearing impairments. Longer term, my goal is a validated “digital twin” of the auditory system that can predict the perceptual errors of an individual listener and be used to design and evaluate hearing aid algorithms before they ever reach a patient. Additionally, a goal of establishing my research group is to train students who can easily translate between the fields of cognitive science, neuroscience, and artificial intelligence. Hearing research could immensely benefit from cross-disciplinary approaches, and I aim to help build a community where theoretical and computational approaches become a standard part of how we study hearing.
The Research
Carnegie Mellon University
Models and mechanisms for auditory event categorization
Our auditory input is often a cacophony of sounds composing an auditory scene. When speaking to a friend in a crowded city park, one can simultaneously hear the music of buskers nearby, the water flowing in a nearby fountain, the chirping of birds in nearby trees, and cars honking at the intersection. However, an individual with hearing impairment may have difficulty recognizing these everyday auditory events. Although there has been immense progress to optimize hearing aids and cochlear implants to improve speech recognition, there has been relatively little work on understanding how these devices, and the listeners using them, process broad categories of non-speech sounds. Unlike deficits in speech understanding, which have immediate social consequences, deficits in audio event perception often go unnoticed and untested clinically, yet these can be detrimental to listener well-being. While much of the field focuses on narrow acoustic domains (like speech or pitch) or the detection of sounds in complex mixtures, this project will systematically evaluate how we recognize broad categories of natural sounds.
In recent years, rapid advancements in artificial intelligence have led to the development of deep neural networks (DNNs) that are top-performing models for both engineering purposes and in capturing properties of human hearing. These computational models can be harnessed to design technology that augments sensory input for devices like hearing aids or cochlear implants, provided that the models process sound the way the brain does. We will develop a sound dataset and novel behavioral testing paradigm to characterize human and model auditory sound categorization, and will use this task to compare artificial neural networks and human observers on both natural stimuli and hearing-aid–processed stimuli. We will additionally develop machine learning models with more perceptual alignment to humans, and test human listeners on synthetic sounds designed to highlight potential differences between models and humans. By taking steps towards a perceptually aligned “digital twin” of the auditory system, this work provides the foundational framework required to optimize hearing devices for real-world listening.
Long-term goal of research: An overarching, long-term goal of my research program is to establish a perceptually aligned “digital twin” of the human auditory system: a computational model that closely approximates human auditory perception in broad domains ranging from speech recognition to auditory event categorization. Such a model serves as an in-silico testbed to rigorously evaluate hypotheses about hearing without requiring exhaustive human testing. Once a highly predictive human-aligned model is defined, it will enable translational advances. For example, by extending models to different types of hearing loss, we can use them to propose stimulus sets that will maximally pull apart different observers to use for identification of different hearing loss mechanisms. Additionally, such a model could be used in the future to guide the development of novel (potentially individualized) hearing aid algorithms. Ultimately, this approach can support the development of hearing technologies that better preserve the perceptual organization of real-world sounds, improving environmental awareness and overall quality of life for individuals with hearing loss.