Although many experiments have been conducted to evaluate perceptual quality of codecs using both subjective and objective techniques, little has been done in the context of end-to-end network transmission systems, for which additional factors such as robustness to network congestion, packet-delay variation, and error concealment are critical. To address this shortcoming, we have developed an objective method for end-to-end Quality of Service (QoS) analysis for audio transmission, which includes accurate measurement of latency and jitter, as well as an evaluation of the number of glitches based only on the delivered audio data. These parameters, which have not previously been evaluated together, provide a meaningful indicator of quality for the entire transmission path, rather than being restricted to audio coding quality alone.
We apply these methods to an empirical evaluation of several uncompressed audio streaming engines, taking into account these issues in addition to the choice of parameter configuration.
These methods have also been applied to seven popular compressed audio streaming systems, and compare these against a reference audio transmission in which no compression was used. Coding formats included G.722, MPEG 1 Layers 2 and 3, and AAC-HE. The results allow for evaluation of the latency introduced by the streaming engine, independent of network delays, and a comparison with algorithmic delays.
In 127th Audio Engineering Society convention, New York, October. 9 pages
Performance metrics for network audio systems: methodology and comparison Bibtex
In New Interfaces for Musical Expression (NIME'09), Pittsburgh. 6 pages
Challenges and Performance of High-Fidelity Audio Streaming for Interactive Performances Bibtex