Letux Kernel

Issue 883: Port AESS (ABE) driver to current kernel

Reported by Michael Mrozek, Jun 14, 2018

Right now, the AESS driver is not working.
This is a driver that enables hardware support for mixing, 
converting, etc. audio streams.

Without it, the CPU has to do everything, which is a horrible waste 
of CPU power.

Andrej started to work on this, but the current status is unknown:
https://github.com/andrey-utkin/linux/tree/pyra-aess

Comment 1 by Marek Belisko, Jun 22, 2018

I can work on it as I'm familiar with kernel sound subsystem.

Comment 2 by Marek Belisko, Jun 23, 2018

Status: Started
Owner: marekb

Comment 3 by Nikolaus Schaller, Feb 16, 2019

A new version is available which seems to work on OMAP5 but breaks 
boot process on Pandaboard-ES. It even makes the Pandaboard reboot.
Labels: Priority:Critical Priority:Medium

Comment 4 by Nikolaus Schaller, Jul 22, 2019

Has become worse with v5.3-rc1 because kernel APIs have changed and 
some more code had to be disabled.
Labels: Device:Pyra

Comment 5 by daveshah, Jul 23, 2020

What is the current status of this? I can see an aess-v4 branch that 
looks to be based on 5.8-rc6 but I presume this still needs more 
work.

Comment 6 by Nikolaus Schaller, Jul 23, 2020

Status: it is still broken...
I have started to understand what it should be doing and rearranging 
code in better manageable pieces (e.g. firmware loader, more dmic 
etc.). So that it is also easier to test individual additions. This 
is not yet published.

A git revert --no-edit letux-base..letuxaess-v4 makes audio work 
(without aess of course) in low-power mode.

Comment 7 by daveshah, Jul 24, 2020

Is something like this possible on 5.6 (as 5.8 is not much use on 
the Pyra until we fix the panel driver...)? I had a very quick try 
of reverting (with -m 1) the commit "Merge branch 
'letux/aess-v3' into letux-5.6" but that seemed to revert too 
much, and there are no signs of any working sound devices appearing.

Comment 8 by Nikolaus Schaller, Jul 24, 2020

It is possible but difficult. You would have to filter all patches 
that affect the abe-twl6040 driver and revert them.

But a quick solution (I have not tried) could be to

git checkout v5.6.19 -- sound/soc/ti/omap-abe-twl6040.c

This should wipe out the changes for AESS interfacing but keep some 
AESS and McASP related files around. But those should not be used, 
i.e. not harm.

Comment 9 by daveshah, Jul 24, 2020

> git checkout v5.6.19 -- sound/soc/ti/omap-abe-twl6040.c

I tried this briefly and it seems to be most of the solution!

The ALSA devices appear correctly and after enabling the output I 
get good quality audio out of the headphone port using aplay.

Unfortunately the speaker isn't working, all I get out of that is 
very quiet, random noise, roughly following the cadence of what is 
playing, and a faint click when something starts/stops playing.

But at least this gives me something to look into over the weekend! 
(PulseAudio stuff also isn't working yet but that's doubtless just a 
userspace config issue...)

Comment 10 by Nikolaus Schaller, Jul 24, 2020

Great!

The "quiet, random noise, roughly following the cadence of what 
is playing, and a faint click" may be because "low power 
mode" isn't set up.

Check the -lp option of 
https://git.goldelico.com/?p=letux-kernel.git;a=blob;f=Letux/root/twl
;h=3930ae8e9d641fd1056ae8034700b443c66640d9;hb=refs/heads/letux-5.6.y
#l109

At least for my experiments this makes a big difference.

Comment 11 by Nikolaus Schaller, Jul 24, 2020

Long time ago there was some discussion about the 32kHz clock  or 
something for the high-performance mode missing and I think it did 
work in ca. 4.9 or so.
The general problem with handsfree driver code is that the uEVM has 
not wired up the twl6040 pins to any connector. Therefore nobody 
tests it with OMAP5.
Only the PandaBoard ES has wired one channel and I have connected a 
speaker there for cross-checking. But my PandaBoard doesn't boot 
reliable. And clock setup may be different.

Comment 12 by daveshah, Jul 24, 2020

Speakers are now working, was actually my error configuring ALSA. 
Switching from plughw: to hw:, and using the AUDIODEV environment 
variable rather than -D, has made the speakers work!

Thanks for your help!

Comment 13 by Nikolaus Schaller, Jul 24, 2020

For reference, the nice overview is nowadays only available through:

https://web.archive.org/web/20131104232426/http://omappedia.org/wiki/
Audio_Drive_Arch

Some terms and definitions:
"ABE" (Audio Back End) subsystem integrates:
- Peripheral connectivity modules (3 McBSP, 1 McASP, 1 Slimbus, 1 
DMIC, 1 McPDM, HDMI)
- AESS (Audio Engine Sub-system) i.e. a DSP with firmware
- ATC (Audio Traffic controller)
- interconnects, memory, GP timers, watchdog
- ABE power domain can remain always on even when the rest is in OFF 
mode

So it forms an autonomous audio subsystem which for example supports 
handsfree telephony between a modem connected to some McBSP and the 
twl6040 with connected speakers and microphones.

Software is structured in
* codec drivers (e.g. twl6040)
* interface drivers (e.g. McBSP, McASP, McPDM)
* AESS driver and firmware loader
* omap-abe-twl6040 which like a spider in the net initializes 
everything
* DTS entries define the hardware connections for .compatible = 
"ti,abe-twl6040"

The upstream driver sound/soc/ti/omap-abe-twl6040.c is quite simple 
and supports a simple control of the twl6040 but not for AESS.

Comment 14 by Nikolaus Schaller, Jul 28, 2020

Finally I have invested some time to reintegrate and fix most of the 
fragments on top of v5.8-rc so that it:

* adds DTS nodes for ranges and reg mapping
* compiles the code
* still supports non-aess sound like upstream omap-abe-twl6040.c
* loads the aess driver as a separate and independent module
* tries to download aess firmware
* gracefully falls back to upstream omap-abe-twl6040.c functionality 
if aess firmware load fails

The following extensions of the TI code are for further study:
* DMIC 0/1/2 support
* SPDIF
* AESS
* connections between McBSP1,2,3 for FM, Bluetooth, Modem

The tree based on v5.8-rc7 is here:

	














https://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/letux/aess-v5


Some of these patches are cosmetics for omap-abe-twl6040.c and can 
probably already be upstreamed now.

Note that this series contains the earlier attempt and does a REVERT 
to upstream omap-abe-twl6040.c on which the new code is based on. 
The earlier and reverted patches are only for reference because they 
contain code that has not yet been ported.

A full kernel which can run on the Pyra (tested with 
letux_defconfig) is here:

	














https://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/work-aess-new

Created: 2 years 1 month ago by Michael Mrozek

Updated: 13 days 7 hours ago

Status: Started

Owner: Marek Belisko

Followed by: 3 persons

Labels:
Type:Defect
Priority:Critical
Device:Pyra