Audio testing is, arguably, one of the most difficult areas for evaluation. There is no question that the test equipment available today is vastly better than that of 40 years ago. Back then, the leading lights of Brüel & Kjær, HP and Rohde & Schwartz, were on the edge of moving from analogue test equipment to the new digital era. Swept tones and pen chart recorders were the norm, even with physical cabling between analyzer and paper recorder to try to keep everything in sync.
With the move to digital acquisition, and the push in the 1980s from analogue to digital delivery to the consumer, there was a wave of rapid evolution. MLSSA time analysis became possible, with the arrival of waterfall plots for loudspeaker energy dissipation. The equipment just got better, more sensitive, and gave deeper insight. The digital domain for audio allowed for an entirely new era of measurement, including jitter, noise distortion, and digital to analogue convertor linearity – all things that would have seemed almost alien to the earlier analogue testing era.
Today, companies like Audio Precision lead the way with audio testing equipment. Our new APx517 analyzer has everything you need for audio and acoustical testing, including Bluetooth. Connect it to the GRAS Head and Torso simulator, and you can fully evaluate the performance of a headphone or headset in a matter of a few seconds, generating data that can take hours to evaluate and learn. Never before in my 40 years of audio/acoustical testing has things been so good and so fast.
Yet, even the best equipment out there is almost comically poor compared to the evaluation of how people actually hear sound and relate to music. When I started 40 years ago, I had hoped that by now I could have a piece of test equipment which showed me the layout of the orchestra in the sound space. That told me that there were four desks of first violins, and that one of the players was a little more out of tune than the others. Or to help me understand how a band had a real groove, with an almost hypnotic rhythm and drive.
The reality is that the human ear is a massively non-linear transducer coupled to a supercomputer. And we have almost no idea how it all works, let alone be able to model test equipment to evaluate this. We are the result of millions of years of evolution, where we have become trained to ignore ongoing noise sources, such as the sound of wind in the trees, and to focus on specific transient events.
By comparison, taking a steady tone at 1 KHz, and measuring its Total Harmonic Distortion is useful, but hardly answering the question of, “Why do I think Buddy Rich was a great drummer?” Modern test equipment goes further, of course. We can have multi tone generation and analysis. Chirp impulses. Fast sweeps. Pink and white noise. We can look at the performance of a speaker and measure it across multiple test positions and analyze the results.
A transducer like a Bluetooth loudspeaker has multiple problems, from small case volume size, limited frequency response, poor acoustic beaming, distortion and physical buzzes and rattles. And that’s before you start looking at the aggressive DSP put in place to attempt to liven up the sound and stop it shaking itself to pieces.
That’s why subjective evaluation is always critical in the assessment of a product. I can spend hours with our equipment, and still not have a comprehensive view of how a product actually sounds. I will have insights into specific issues, but nothing more. For this, you need an expert panel of assessors who can evaluate the whole product, playing music and speech, and all the other sorts of materials that real users play on these devices, day after day. Getting ISO 17025 for such a subjective evaluation process is not easy, but we did it. I just wish that I had a magic box that could tell me what was going on. Maybe in another 40 years, my dream will be realized.
Jon Honeyball is CEO of Woodleyside Lab, a company that specializes in consumer electronics evaluation and R&D work.