[BUG][LNL] Speaker playback start to distort in the middle of playing

[aiChaoSONG]

Describe the bug
In verifying kernel patch #4748, I found that the speaker playback on LNL_RVP_AIOC with RT711 + 2xRT1316 + RT714 codecs is distorted in the middle of the playback. That is, on playback start up, the sound is good, and after playing for a while (about 10s), the sound starts to become distorted.

This issue only happens on LNL_RVP_AIOC, MTL_RVP_AIOC is good.

Below picture is the wave I recorded with my phone:

Normal Sound, zoomed in

Distorted Sound, zoomed in

To Reproduce
aplay -D hw:0,2 -fdat -vvv sine_wave.wav, and play for a while.

Environment

1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
   • Kernel: {topic/sof-dev}, Since the enabling commit, the issue could be reproduced. I tested the latest kernel with ASoC: SOF: Intel: Fix upstream merge issues causing LNL audio quality failures #4748 included, commit id: 0cba055, and our daily kernel from 2023/10/26, commit id: c0dfcde0414.
   • SOF: {main/stable-v2.8}
2. Name of the topology file
   • Topology: {sof-lnl-rt711-l0-rt1316-l23-rt714-l1.tplg}
3. Name of the platform(s) on which the bug is observed.
   • Platform: {LNL_RVP_AIOC}

[ujfalusi]

@aiChaoSONG, this is puzzling for sure.
Can you provide kernel and firmware log (only from playback start to distortion start+10 seconds).

If you stop and re-start the playback right away when it is distorted (not allowing DSP to suspend), will the distortion be gone again and re-appear?

Is it possible to test similar case with HDA setup?

[ujfalusi]

@plbossart, @bardliao, FYI

[aiChaoSONG]

If you stop and re-start the playback right away when it is distorted (not allowing DSP to suspend), will the distortion be gone again and re-appear?

Yes, always good at start, and then becomes distorted.

Is it possible to test similar case with HDA setup?

HDA is good.

@ujfalusi Just work with Bard, and we come to the conclusion that this issue is caused by sdw link aggregation on LNL. We are using one DMA to drive two SDW links, which HW guys said won't work.

When we only use single RT1316, there is not distortion at all.

[plbossart]

So just to be clear, this problem is not related tot the ch_maps addition, right? This must happen even with a kernel prior to the upstream merge, yes?

[plbossart]

also wondering what topology is used. IIRC we tested this with passthrough pipelines in the past.

[bardliao]

So just to be clear, this problem is not related tot the ch_maps addition, right? This must happen even with a kernel prior to the upstream merge, yes?

Right, we had the issue before the ch_maps addition. We didn't notice it because the sound is clear in the first few seconds. BTW, I just tested aggregated speaker feedback today, and it can only capture some random noise. The issue only happens on LNL, but not on MTL.

[bardliao]

also wondering what topology is used. IIRC we tested this with passthrough pipelines in the past.

The topology is

[plbossart]

the speaker feedback aggregation should work just fine even with the hardware-based solution. The potential issue with xruns is only with the copy of the same stream for playback. But on the capture side, the data coming from different links is different so there's no real potential of the data being flushed by the DMA.

FWIW we tested the feedback with the loopback mode, but we never tested actual feedback from a codec. There might be some amplifier configuration issues as well.

We should retry with the passthrough pipelines for now, and re-add the loopback mode just to compare with what we had before.

I do think the firmware-based aggregation is required, but there are probably multiple variables at play here, and it doesn't hurt to try to triangulate the problems in depth.

[bardliao]

BTW, I just tested aggregated speaker feedback today, and it can only capture some random noise.

@plbossart I double checked the speaker feedback aggregation, and it can capture the loopback audio. Sorry for the wrong information above.
However, the audio data are on only 2 channels.

feedback.zip

We may need to work on channel mapping.

[bardliao]

The distortion happens if I test with passthrough a topology, too.

But I can get all 4 channels data with the passthrough topology.

loop_wav.zip

[plbossart]

@bardliao did you mean

"the distortion happens on playback even with a passthrough topology"

"on capture I can get all 4 channels data without distortion with the passthrough topology."

That would mean two separate bugs then,
a) distortion on playback which isn't unexpected due to DMA behavior
b) channel data being corrupted on capture with the regular topology.

IIRC we only tested capture w/ aggregation in passthrough mode on FPGA so it's possible we missed something.

[bardliao]

@bardliao did you mean

"the distortion happens on playback even with a passthrough topology"

Yes

"on capture I can get all 4 channels data without distortion with the passthrough topology."

No, I mean there are audio data on all 4 channels but the audio data are also distorted.
I mentioned that because I can only get 2 channel data with a regular topology.

[plbossart]

ok then we have 3 bugs! Can we split the issue so that playback, capture with passthrough and capture with regular topology are all handled separately?

[bardliao]

ok then we have 3 bugs! Can we split the issue so that playback, capture with passthrough and capture with regular topology are all handled separately?

Yes, it is better to split the issue. But, what is the capture with passthrough issue? The captured data are exactly the same as what we hear from the speaker which is distorted. IOW, if the original audio is "ABCD" and play from the speaker become "A+B-C+D-", then we will also get "A+B-C+D-" from the loopback capture. Sorry if "there are audio data on all 4 channels but the audio data are also distorted." is misleading.

[ujfalusi]

@bardliao, capture == loopback?
If the loopback capture is corrupted at the same time and same way as the playback on the speaker then it indicates that the corruption happens before the loopback point, right?

Can you verify just capture from line or mic that it also starts corrupting?

[plbossart]

Good point @ujfalusi. on FPGA we always used the SoundWire loopback capability since it's best for remote systems. The weakness is indeed that if the playback is borked there's no way to test the capture independently, that requires a dedicated input. @fredoh9 used a separate device to play audio 24/7 in the lab so that we could test capture without loopback.

Something like a TTS app saying the current UTC time followed by a sine wave would be a great way of testing if capture works...

[bardliao]

@bardliao, capture == loopback? If the loopback capture is corrupted at the same time and same way as the playback on the speaker then it indicates that the corruption happens before the loopback point, right?

Yes, I suspect the issue is related to a HW limitation that we can't use HW aggregation. Related FW PR is thesofproject/sof#8361

Can you verify just capture from line or mic that it also starts corrupting?

Headset capture should be fine. @keqiaozhang Could you confirm it? However, the distortion only happens on the SDW aggregation speakers. The capture device is speaker's feedback. No mic or line for the capture pcm.

[bardliao]

I can confirm that the distortion disappeared when we use FW base aggregation mode. I am working on the kernel change now.

[plbossart]

It's not clear if this issue is still current, in daily tests we can see multiple alsa-bat failures, and this specific test show a broken pipe

planresultdetail/37723?model=LNLM_SDW_AIOC&testcase=check-alsabat-headset-capture-599

2024-02-06 22:29:37 UTC [REMOTE_COMMAND] alsabat -Phw:CODEC,0 --standalone -n 240000 -r 48000 -c 2 -f S16_LE -F 599 -k 2.1
2024-02-06 22:29:38 UTC [REMOTE_COMMAND] alsabat -Chw:sofsoundwire,1 -c 2 -r 48000 -f S16_LE -F 599 -k 2.1
Overrun: Broken pipe(-32)
WARNING: Signal too weak!

[bardliao]

@keqiaozhang @aiChaoSONG Can you check if the issue still exists?