The protocol was developed after an in-depth global customer survey conducted by DXOMARK found that wind noise was “the number one pain point” for consumers recording video outdoors.
While many newer smartphones employ noise-cancelling algorithms intended to reduce the noise interference from wind, it was unclear how well these algorithms worked. In a bid to find out, DXOMARK recently devised a protocol to ascertain which phones are better suited to outdoor recording.
The protocol was designed to test wind interference for the three most common recording use cases — videos, selfie videos, and audio-only memo recordings. The testing setup consists of a wind generator, four high-end speakers, a rotating smartphone holder, and special controlling software, DXOMARK said.
The smartphone is placed in the holder in front of the wind generator, which can produce light (3 m/s), moderate (5 m/s), and strong wind (6.5 m/s) during recording.
DXOMARK's test set-up for evaluating how well smartphones record video in windy environments. Picture: DXOMARK
For each device under test, DXOMARK engineers record the following test signals: (a) white noise under no-wind conditions; (b) white noise under with-wind conditions; and (c) wind noise only.
DXOMARK audio engineers also conduct tests of the same tracks at different wind speeds - light, moderate, and strong – and with the smartphone facing the wind machine and then turned perpendicular to it. They also conduct recordings without wind to use as references.
The recordings are then analyzed to produce three metrics: (1) the wind reduction ratio (WRR), which is the ratio of spectral energies between the signal alone and the wind noise only; (2) wind energy, which is the ratio of white noise recorded in the presence of wind (low, moderate, or strong) to the wind-only recording; and (3) the average correlation between the reference audio track (a recording without wind) and the test audio track (a recording made in the presence of wind).
Proprietary software measures and analyzes the reference signals that the smartphone records against the recordings it makes at the different wind levels, the company said.
Graph shows the performance scores of 33 smartphones under windy conditions. Picture: DXOMARK
To complement the objective tests, trained engineers also use voice tracks to perceptually evaluate how well a device’s wind noise attenuation affects intelligibility, DXOMARK said.
For each use case, a phone that does a good job attenuating wind noise is awarded bonus points or receives a penalty for a poor performance. The graph above shows the results of DXOMARK’s initial round of testing of 33 smartphones released in 2021 and 2022.
DXOMARK said that it plans to expand its wind noise attenuation protocol to include action/sports cameras, microphones, and other professional recording devices.