Hardware Accelerated WebRTC Playback

Yonathan

Hey there,
is there any kind of Image where I get some kind of Hardware Decoding for WebRTC?
At the time I tried:
Using Armbian with this Script (https://forum.armbian.com/topic/9590-s905x-alpha-media-capabilities-testing-script/). Video gets decoded from the CPU but idels at around 50-60% but when transmitting a Video with a lot of changes the delay gets pretty big.

- Also used Ubuntu but the result was terrible

I am using Electron which is based on the Chromium Browser, so thats why I cant use Android.

loverpi

Right now decoding is only supported by gstreamer. Most browser engines do not support the entire stack. There has been an multi-year effort. The pipeline to have it work is as follows:
Kernel Driver: OK
V4L2 M2M: OK
Gstreamer: OK
Wayland: OK
Ozone: Not OK
Chromium: Not OK

Yonathan

Thanks for the answer @loverpi ! 
I came across the article from BayLibre (https://baylibre.com/hardware-accelerated-video-decoding-and-system-load-monitoring/) and there chromium works fine of course with patches. I cant find any starting point to achieve the same.

loverpi

@Yonathan That was a proof of concept demo. There's still a lot of work. At that point, the V4L2 M2M driver was not even complete. Ozone support was recently merged but needs to be continue to be improved. We expect the stack to work by the end of the year.

Yonathan

@loverpi thank you so much for your answer! At the current time I completly rewrote everything in Android and can only test nougat, because oreo has no avaible image for sd (http://share.loverpi.com/board/libre-computer-project/libre-computer-board-aml-s905x-cc/image/android/) is that on purpose?

loverpi

Additionally the VPU has several planes with different capabilities in each plane. The hardware decoders can scale and output the decoded video into certain planes. There's a lot of work between Wayland and the Ozone layers on how to setup these planes to play back videos. Low power devices use these optimizations to save power but the flip side is that the application logic needs to be far more complex.

Yonathan

@loverpi thanks for the additional info! But regarding Android is it possible to get Android 8.0 (Oreo) and support for USB Wlan Sticks? Ok running WebRTC on Android doesnt seem to be better. Is Hardware Acceleration enabled in Android 7.1?

Yonathan

Ok even with turning on software-rendering trough Flutter I cant stream FullHD Video with WebRTC... Im somewhat upset about this... 
Running the same on a Huawei MediaPad T3 works so nicely with only 20% CPU Usage (cant monitor GPU) also cant monitor anything on the given Android Image from your Download-Page. Developermode can be activated but crashes right after you select it.

Yonathan

Ok lets go on with this...
@loverpi So I now use the WebRTC Implementation from Gstreamer and display the video by it but I still dont get any Hardware Acceleration. I run the latest Version of Armbian and my pipeline looks like this:

videotestsrc is-live=true ! x264enc ! h264parse ! rtph264pay config-interval=1 ! queue ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! rtpjitterbuffer


In general Gstreamer doesnt seem to be accelerated in any way...

Yonathan

@loverpi Everything works fine on Windows but when I run armbian and install gstreamer vp8 and vp9 is not accelerated and when I choose h264 with v4l2dec it says it can decode the frame... Could you provide a script where everything should work fine?