How to enable i.MX vpuenc_h264 ?

[l4es]

Dear All,

I'm integrating gstreamer-imx for the iMX8M Plus in Buildroot borrowing package implemented here : https://github.com/boundarydevices/buildroot-external-boundary/tree/master/package/gst1-imx-v2

When verifying the available hardware encoder/decoder plugins, I cannot find vpuenc_h264 :

root@imx-camera:~# dmesg | grep hantro
[ 2.063741] hantrodec 0 : module inserted. Major = 236
[ 2.064220] hantrodec 1 : module inserted. Major = 236
[ 2.065434] hantroenc: HW at base <0000000038320000> with ID <0x80006200>

root@imx-camera:~# gst-inspect-1.0 | grep imx
libav: avdec_simbiosis_imx: libav Simbiosis Interactive IMX Video decoder
imxvpu: imxvpudec_sspark: i.MX VPU Sorenson Spark video decoder
imxvpu: imxvpudec_divx5: i.MX VPU DivX 5 & 6 video decoder
imxvpu: imxvpudec_divx4: i.MX VPU DivX 4 video decoder
imxvpu: imxvpudec_divx3: i.MX VPU DivX 3 video decoder
imxvpu: imxvpudec_cavs: i.MX VPU AVS (Audio and Video Coding Standard) video decoder
imxvpu: imxvpudec_vp6: i.MX VPU VP6 video decoder
imxvpu: imxvpudec_vc1: i.MX VPU VC-1 advanced profile video decoder
imxvpu: imxvpudec_wmv3: i.MX VPU WMV3 / Window Media Video 9 / VC-1 simple profile video decoder
imxvpu: imxvpudec_h263: i.MX VPU h.263 video decoder
imxvpu: imxvpudec_mpeg4: i.MX VPU MPEG-4 video decoder
imxvpu: imxvpudec_mpeg2: i.MX VPU MPEG-1 & 2 video decoder
imxvpu: imxvpudec_webp: i.MX VPU WebP video decoder
imxvpu: imxvpudec_jpeg: i.MX VPU JPEG video decoder
imxvpu: imxvpudec_vp9: i.MX VPU VP9 video decoder
imxvpu: imxvpudec_vp8: i.MX VPU VP8 video decoder
imxvpu: imxvpudec_h265: i.MX VPU h.265 / HEVC video decoder
imxvpu: imxvpudec_h264: i.MX VPU h.264 / AVC video decoder
imxv4l2video: imxv4l2videosink: NXP i.MX V4L2 video sink
imxv4l2video: imxv4l2videosrc: NXP i.MX V4L2 video source
imxaudio: imxuniaudiodec: NXP i.MX uniaudio decoder
imx2d: imxpxpvideotransform: i.MX PxP video transform
imx2d: imxpxpvideosink: i.MX PxP video sink
imx2d: imxipuvideotransform: i.MX IPU video transform
imx2d: imxg2dvideotransform: i.MX G2D video transform
imx2d: imxg2dvideosink: i.MX G2D video sink
imx2d: imxg2dcompositor: i.MX G2D video transform

Could someone help to point out what possible missing dependency or configuration that prevents the vpuenc_h264, please ?

Thank very much in advance and best regards,
Khang

[dv1]

This currently is not possible, since the i.MX8m plus has a different encoder (VC8000E) that is not supported by libimxvpuapi. I'm working on this.

[l4es]

Dear @dv1,

This currently is not possible, since the i.MX8m plus has a different encoder (VC8000E) that is not supported by libimxvpuapi. I'm working on this.

Could you share any progress on this, please ?
Thanks and Best Regards,
Pascal

[dv1]

Progress is slow, because the VC8000E encoder is not trivial to use, and because many other tasks keep me busy. I am trying to get it in for Yocto Kirkstone, but I cannot guarantee it.

[d-lareg]

Hi @dv1,

since I am in the same situation as @l4es (developing with buildroot), I would be happy to get an update. Is the encoding already implemented for GStreamer in Yocto. I am using the linux kernel 5.10.9, do you see any problems if I integrate the Yocto package in my Buildroot pipeline, or is the kernel to outdated?

[l4es]

Hi @d-lareg,

Clarification of VPU plugins :
Prior BSP 5.10.35 : vpuenc_h264 & vpudec_h264
Since BSP 5.10.35 : v4l2h264enc & v4l2h264dec

More detail here : https://community.nxp.com/t5/i-MX-Processors/iMX8MPlus-How-to-mix-two-video-sources-into-one-with-Gstreamer/m-p/1532117

Personally, kernel 5.10.9 seems a little bit outdated even-though it has been stayed quite long time with Buildroot.

[dv1]

@d-lareg @l4es I am currently facing the problem that parts of the encoder actually collide with libimvxpuapi's current design. I am trying to implement basic encoding support, but it would only expose a subset of the VC8000E's full functionality. I estimate this will take me 1-2 months to finish at this point.

[dv1]

@d-lareg @l4es I finally have a working version in git master. It turned out that a combination of hardware problems and driver bugs were causing a ton of confusion, but I can now encode with the VC8000E. h.264 only for now, but everything is prepared for h.265 encoding.

[l4es]

Hi @dv1,

Great news! i am eager to test it, if possible! Could you advice which commit that I could use for the H.264 encoding, please? Or should I wait for some time for having H265 encoding available as well ?
Many thanks.

[l4es]

Hi @dv1,
I found that the implementation was done in https://github.com/Freescale/libimxvpuapi.

Thank you

[l4es]

Hi @dv1,

Just FYI from Gary Bisson of Boundary Devices :

Actually this isn't needed any longer as NXP now provides a userspace daemon that makes the VPU appear as a standard V4L2 device:
https://github.com/buildroot/buildroot/tree/master/package/freescale-imx/imx-vpu-hantro-daemon
With this, you can use the standard v4l2enc plugin of GStreamer without the need for any additional plugin.

But I think we still need your implementation in older BSPs.

[dv1]

The last time I tried the daemon, it was prone to multiple crashes, and did need the NXP fork of GStreamer, because some patches on top of v4l2enc were needed. Perhaps this improved, I'll check, but yeah, this works with older BSPs, and also works with upstream GStreamer.

[l4es]

Dear @dv1,
For using the H264 encoding with the VC8000E of iMX8M Plus, could you advice the commits of gstreamer-imx and libimxdmabuffer that are associated with Freescale/libimxvpuapi@b79a50a, please ?

[dv1]

@l4es I recommend using b69bef1ac976d71d8e9f527d61446d2a7e8c77eb for libimxdmabuffer and a2c0128 for gstreamer-imx .

[l4es]

Hi @dv1,
Thanks very much. Sorry to bother you again, could you share some quick gstreamer command for testing the encoder on iMX8M Plus please?