[PATCH 00/15] reduce audio playback latency

Volker Rümelin posted 15 patches 2 years, 2 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/cfcae86f-59c3-a2c5-76cd-1ab5e23e20f3@t-online.de
There is a newer version of this series
audio/alsaaudio.c   |   1 +
audio/audio.c       | 194 ++++++++++++++++++++++++--------------------
audio/audio_int.h   |   9 +-
audio/coreaudio.c   |  13 +--
audio/dsoundaudio.c |  30 ++++---
audio/jackaudio.c   |   5 +-
audio/noaudio.c     |   1 +
audio/ossaudio.c    |  17 +++-
audio/paaudio.c     |  49 ++++++-----
audio/sdlaudio.c    |  21 +++--
audio/wavaudio.c    |   1 +
11 files changed, 199 insertions(+), 142 deletions(-)
[PATCH 00/15] reduce audio playback latency
Posted by Volker Rümelin 2 years, 2 months ago
This patch series reduces the playback latency for audio backends,
in some cases significantly. For PulseAudio, the audio buffer is
also moved from the QEMU side to the PulseAudio server side. This
improves the drop-out safety for PulseAudio.

I actually measured the latency reduction with the PulseAudio
backend. For the test I used my Linux host configured to play
audio with PulseAudio. The guest was a Linux guest, also
configured to use PulseAudio.

Measuring audio latencies is difficult. I played a sine tone in
the guest with Audacity and measured the time from releasing the
left mouse button until the tone can be heard. A few seconds
before the measurement I started playback of an audio file with
10 minutes of silence to fill the audio buffers. The over-all
latency can't be used to estimate the playback latency, but it
can be used to calculate the playback latency reduction.

The measured over-all latency with PulseAudio is around 200ms
without these patches and around 135ms with these patches. The
difference of 65ms agrees well with the expected value of
46.4ms * 2 + 15ms - 46.4ms = 61.4ms. 46.4ms * 2 is the size of
the mixing-engine buffer ("[PATCH 14/15] paaudio: fix samples vs.
frames mix-up" explains the factor 2), 15ms is the server side
PulseAudio buffer size used before these patches and 46.4ms is
the new server side PulseAudio buffer size.

Volker Rümelin (15):
   audio: replace open-coded buffer arithmetic
   audio: move function audio_pcm_hw_clip_out()
   audio: add function audio_pcm_hw_conv_in()
   audio: inline function audio_pcm_sw_get_rpos_in()
   paaudio: increase default latency to 46ms
   jackaudio: use more jack audio buffers
   audio: copy playback stream in sequential order
   audio: add pcm_ops function table for capture backend
   audio: revert tests for pcm_ops table
   audio: restore mixing-engine playback buffer size
   paaudio: reduce effective playback buffer size
   dsoundaudio: reduce effective playback buffer size
   ossaudio: reduce effective playback buffer size
   paaudio: fix samples vs. frames mix-up
   sdlaudio: fix samples vs. frames mix-up

  audio/alsaaudio.c   |   1 +
  audio/audio.c       | 194 ++++++++++++++++++++++++--------------------
  audio/audio_int.h   |   9 +-
  audio/coreaudio.c   |  13 +--
  audio/dsoundaudio.c |  30 ++++---
  audio/jackaudio.c   |   5 +-
  audio/noaudio.c     |   1 +
  audio/ossaudio.c    |  17 +++-
  audio/paaudio.c     |  49 ++++++-----
  audio/sdlaudio.c    |  21 +++--
  audio/wavaudio.c    |   1 +
  11 files changed, 199 insertions(+), 142 deletions(-)

-- 
2.31.1

Re: [PATCH 00/15] reduce audio playback latency
Posted by Christian Schoenebeck 2 years, 2 months ago
On Donnerstag, 6. Januar 2022 10:21:47 CET Volker Rümelin wrote:
> This patch series reduces the playback latency for audio backends,
> in some cases significantly. For PulseAudio, the audio buffer is
> also moved from the QEMU side to the PulseAudio server side. This
> improves the drop-out safety for PulseAudio.
> 
> I actually measured the latency reduction with the PulseAudio
> backend. For the test I used my Linux host configured to play
> audio with PulseAudio. The guest was a Linux guest, also
> configured to use PulseAudio.

I haven't reviewed all the patches yet, but from what I read so far, does that 
mean the additional 3rd buffer is solely for PulseAudio, so for JACK and other 
backends these changes would overall be a degradation, wouldn't they?

> Measuring audio latencies is difficult. I played a sine tone in
> the guest with Audacity and measured the time from releasing the
> left mouse button until the tone can be heard. A few seconds
> before the measurement I started playback of an audio file with
> 10 minutes of silence to fill the audio buffers. The over-all
> latency can't be used to estimate the playback latency, but it
> can be used to calculate the playback latency reduction.
> 
> The measured over-all latency with PulseAudio is around 200ms
> without these patches and around 135ms with these patches. The
> difference of 65ms agrees well with the expected value of
> 46.4ms * 2 + 15ms - 46.4ms = 61.4ms. 46.4ms * 2 is the size of
> the mixing-engine buffer ("[PATCH 14/15] paaudio: fix samples vs.
> frames mix-up" explains the factor 2), 15ms is the server side
> PulseAudio buffer size used before these patches and 46.4ms is
> the new server side PulseAudio buffer size.
> 
> Volker Rümelin (15):
>    audio: replace open-coded buffer arithmetic
>    audio: move function audio_pcm_hw_clip_out()
>    audio: add function audio_pcm_hw_conv_in()
>    audio: inline function audio_pcm_sw_get_rpos_in()
>    paaudio: increase default latency to 46ms
>    jackaudio: use more jack audio buffers
>    audio: copy playback stream in sequential order
>    audio: add pcm_ops function table for capture backend
>    audio: revert tests for pcm_ops table
>    audio: restore mixing-engine playback buffer size
>    paaudio: reduce effective playback buffer size
>    dsoundaudio: reduce effective playback buffer size
>    ossaudio: reduce effective playback buffer size
>    paaudio: fix samples vs. frames mix-up
>    sdlaudio: fix samples vs. frames mix-up
> 
>   audio/alsaaudio.c   |   1 +
>   audio/audio.c       | 194 ++++++++++++++++++++++++--------------------
>   audio/audio_int.h   |   9 +-
>   audio/coreaudio.c   |  13 +--
>   audio/dsoundaudio.c |  30 ++++---
>   audio/jackaudio.c   |   5 +-
>   audio/noaudio.c     |   1 +
>   audio/ossaudio.c    |  17 +++-
>   audio/paaudio.c     |  49 ++++++-----
>   audio/sdlaudio.c    |  21 +++--
>   audio/wavaudio.c    |   1 +
>   11 files changed, 199 insertions(+), 142 deletions(-)



Re: [PATCH 00/15] reduce audio playback latency
Posted by Volker Rümelin 2 years, 2 months ago
> On Donnerstag, 6. Januar 2022 10:21:47 CET Volker Rümelin wrote:
>> This patch series reduces the playback latency for audio backends,
>> in some cases significantly. For PulseAudio, the audio buffer is
>> also moved from the QEMU side to the PulseAudio server side. This
>> improves the drop-out safety for PulseAudio.
>>
>> I actually measured the latency reduction with the PulseAudio
>> backend. For the test I used my Linux host configured to play
>> audio with PulseAudio. The guest was a Linux guest, also
>> configured to use PulseAudio.
> I haven't reviewed all the patches yet, but from what I read so far, does that
> mean the additional 3rd buffer is solely for PulseAudio, so for JACK and other
> backends these changes would overall be a degradation, wouldn't they?

No, nothing changes for JACK and it's an improvement for all the other 
backends where I added a buffer_get_free function. The important changes 
are in [PATCH 10/15] audio: restore mixing-engine playback buffer size. 
That patch tries to keep the mixing-engine buffer empty at the end of 
audio_run_out().

I couldn't reduce the playback latency for JACK, because the JACK audio 
buffers are already very small and any further reduction introduces 
playback glitches on my system.

For PulseAudio there is no additional buffer. I only increased the size 
of the server side buffer from 15ms to 46,4ms and added a 
buffer_get_free function. Before this patch series a few ten ms after 
playback started the mixing-engine buffer was full which added 2 * 
46,4ms to the playback latency. With these patches the mixing-engine 
buffer is empty. This looks like the buffer in use was moved from the 
mixing-engine to the PulseAudio server side.

>> Measuring audio latencies is difficult. I played a sine tone in
>> the guest with Audacity and measured the time from releasing the
>> left mouse button until the tone can be heard. A few seconds
>> before the measurement I started playback of an audio file with
>> 10 minutes of silence to fill the audio buffers. The over-all
>> latency can't be used to estimate the playback latency, but it
>> can be used to calculate the playback latency reduction.
>>
>> The measured over-all latency with PulseAudio is around 200ms
>> without these patches and around 135ms with these patches. The
>> difference of 65ms agrees well with the expected value of
>> 46.4ms * 2 + 15ms - 46.4ms = 61.4ms. 46.4ms * 2 is the size of
>> the mixing-engine buffer ("[PATCH 14/15] paaudio: fix samples vs.
>> frames mix-up" explains the factor 2), 15ms is the server side
>> PulseAudio buffer size used before these patches and 46.4ms is
>> the new server side PulseAudio buffer size.
>>
>> Volker Rümelin (15):
>>     audio: replace open-coded buffer arithmetic
>>     audio: move function audio_pcm_hw_clip_out()
>>     audio: add function audio_pcm_hw_conv_in()
>>     audio: inline function audio_pcm_sw_get_rpos_in()
>>     paaudio: increase default latency to 46ms
>>     jackaudio: use more jack audio buffers
>>     audio: copy playback stream in sequential order
>>     audio: add pcm_ops function table for capture backend
>>     audio: revert tests for pcm_ops table
>>     audio: restore mixing-engine playback buffer size
>>     paaudio: reduce effective playback buffer size
>>     dsoundaudio: reduce effective playback buffer size
>>     ossaudio: reduce effective playback buffer size
>>     paaudio: fix samples vs. frames mix-up
>>     sdlaudio: fix samples vs. frames mix-up
>>
>>    audio/alsaaudio.c   |   1 +
>>    audio/audio.c       | 194 ++++++++++++++++++++++++--------------------
>>    audio/audio_int.h   |   9 +-
>>    audio/coreaudio.c   |  13 +--
>>    audio/dsoundaudio.c |  30 ++++---
>>    audio/jackaudio.c   |   5 +-
>>    audio/noaudio.c     |   1 +
>>    audio/ossaudio.c    |  17 +++-
>>    audio/paaudio.c     |  49 ++++++-----
>>    audio/sdlaudio.c    |  21 +++--
>>    audio/wavaudio.c    |   1 +
>>    11 files changed, 199 insertions(+), 142 deletions(-)


Re: [PATCH 00/15] reduce audio playback latency
Posted by Christian Schoenebeck 2 years, 2 months ago
On Sonntag, 9. Januar 2022 18:06:44 CET Volker Rümelin wrote:
> > On Donnerstag, 6. Januar 2022 10:21:47 CET Volker Rümelin wrote:
> >> This patch series reduces the playback latency for audio backends,
> >> in some cases significantly. For PulseAudio, the audio buffer is
> >> also moved from the QEMU side to the PulseAudio server side. This
> >> improves the drop-out safety for PulseAudio.
> >> 
> >> I actually measured the latency reduction with the PulseAudio
> >> backend. For the test I used my Linux host configured to play
> >> audio with PulseAudio. The guest was a Linux guest, also
> >> configured to use PulseAudio.
> > 
> > I haven't reviewed all the patches yet, but from what I read so far, does
> > that mean the additional 3rd buffer is solely for PulseAudio, so for JACK
> > and other backends these changes would overall be a degradation, wouldn't
> > they?
> No, nothing changes for JACK and it's an improvement for all the other
> backends where I added a buffer_get_free function. The important changes
> are in [PATCH 10/15] audio: restore mixing-engine playback buffer size.
> That patch tries to keep the mixing-engine buffer empty at the end of
> audio_run_out().
> 
> I couldn't reduce the playback latency for JACK, because the JACK audio
> buffers are already very small and any further reduction introduces
> playback glitches on my system.

And that's actually my concern. A split 2 buffers -> 3 buffers while 
(approximately) retaining overall latency increases the chance of dropouts.

> For PulseAudio there is no additional buffer. I only increased the size
> of the server side buffer from 15ms to 46,4ms and added a
> buffer_get_free function. Before this patch series a few ten ms after
> playback started the mixing-engine buffer was full which added 2 *
> 46,4ms to the playback latency. With these patches the mixing-engine
> buffer is empty. This looks like the buffer in use was moved from the
> mixing-engine to the PulseAudio server side.



Re: [PATCH 00/15] reduce audio playback latency
Posted by Volker Rümelin 2 years, 2 months ago
Am 10.01.22 um 14:11 schrieb Christian Schoenebeck:
> On Sonntag, 9. Januar 2022 18:06:44 CET Volker Rümelin wrote:
>>> On Donnerstag, 6. Januar 2022 10:21:47 CET Volker Rümelin wrote:
>>>> This patch series reduces the playback latency for audio backends,
>>>> in some cases significantly. For PulseAudio, the audio buffer is
>>>> also moved from the QEMU side to the PulseAudio server side. This
>>>> improves the drop-out safety for PulseAudio.
>>>>
>>>> I actually measured the latency reduction with the PulseAudio
>>>> backend. For the test I used my Linux host configured to play
>>>> audio with PulseAudio. The guest was a Linux guest, also
>>>> configured to use PulseAudio.
>>> I haven't reviewed all the patches yet, but from what I read so far, does
>>> that mean the additional 3rd buffer is solely for PulseAudio, so for JACK
>>> and other backends these changes would overall be a degradation, wouldn't
>>> they?
>> No, nothing changes for JACK and it's an improvement for all the other
>> backends where I added a buffer_get_free function. The important changes
>> are in [PATCH 10/15] audio: restore mixing-engine playback buffer size.
>> That patch tries to keep the mixing-engine buffer empty at the end of
>> audio_run_out().
>>
>> I couldn't reduce the playback latency for JACK, because the JACK audio
>> buffers are already very small and any further reduction introduces
>> playback glitches on my system.
> And that's actually my concern. A split 2 buffers -> 3 buffers while
> (approximately) retaining overall latency increases the chance of dropouts.

No, the 3 * 512 frames JACK buffer improves dropout safety compared to a 
2 * 512 frames buffer. Before my patches I could sometimes hear glitches 
when I switched from and to the QEMU GTK window. With the change to a 3 
* 512 frames JACK buffer, audio playback is glitch free and it comes 
without additional playback latency.

>
>> For PulseAudio there is no additional buffer. I only increased the size
>> of the server side buffer from 15ms to 46,4ms and added a
>> buffer_get_free function. Before this patch series a few ten ms after
>> playback started the mixing-engine buffer was full which added 2 *
>> 46,4ms to the playback latency. With these patches the mixing-engine
>> buffer is empty. This looks like the buffer in use was moved from the
>> mixing-engine to the PulseAudio server side.
>


Re: [PATCH 00/15] reduce audio playback latency
Posted by Volker Rümelin 2 years, 2 months ago
> This patch series reduces the playback latency for audio backends,
> in some cases significantly. For PulseAudio, the audio buffer is
> also moved from the QEMU side to the PulseAudio server side. This
> improves the drop-out safety for PulseAudio.
>

>
> Volker Rümelin (15):
>   audio: replace open-coded buffer arithmetic
>   audio: move function audio_pcm_hw_clip_out()
>   audio: add function audio_pcm_hw_conv_in()
>   audio: inline function audio_pcm_sw_get_rpos_in()
>   paaudio: increase default latency to 46ms
>   jackaudio: use more jack audio buffers
>   audio: copy playback stream in sequential order
>   audio: add pcm_ops function table for capture backend
>   audio: revert tests for pcm_ops table
>   audio: restore mixing-engine playback buffer size
>   paaudio: reduce effective playback buffer size
>   dsoundaudio: reduce effective playback buffer size
>   ossaudio: reduce effective playback buffer size
>   paaudio: fix samples vs. frames mix-up
>   sdlaudio: fix samples vs. frames mix-up
>
>  audio/alsaaudio.c   |   1 +
>  audio/audio.c       | 194 ++++++++++++++++++++++++--------------------
>  audio/audio_int.h   |   9 +-
>  audio/coreaudio.c   |  13 +--
>  audio/dsoundaudio.c |  30 ++++---
>  audio/jackaudio.c   |   5 +-
>  audio/noaudio.c     |   1 +
>  audio/ossaudio.c    |  17 +++-
>  audio/paaudio.c     |  49 ++++++-----
>  audio/sdlaudio.c    |  21 +++--
>  audio/wavaudio.c    |   1 +
>  11 files changed, 199 insertions(+), 142 deletions(-)
>

I used the wrong mail address to send these patches. I'll wait a few 
days for reviews before I send a version 2 series with the correct address.

With best regards,
Volker