Feature request: Add hardware acceleration for FFmpeg

[Klench]

I have just started using this software and it seems great. However, using FFmpeg for conversion of large video files can take a lot of time if using CPU only for transcoding. Using e.g. Intel Quick Sync for hardware acceleration would probably speed things up a lot.

[C4illin]

What is missing to make it possible?

[C10udburst]

Isn't passing the /dev/dri to the container enough?

[Klench]

I am not an expert but I assume some hardware acceleration related arguments need to be added to the ffmpeg command:

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

(In addition to passing the device to the container)

[C4illin]

I added FFMPEG_ARGS as en env variable, set it to "-hwaccel qsv" and pass the gpu to the container. Hopefully it works, I dont have an intel cpu but I tested with vulkan.

[broizter]

I added FFMPEG_ARGS as en env variable, set it to "-hwaccel qsv" and pass the gpu to the container. Hopefully it works, I dont have an intel cpu but I tested with vulkan.

When trying to convert something I'm getting this error:

convertx  | ConvertX v0.9.0
convertx  | 🦊 Elysia is running at http://localhost:3000
convertx  | Alpine Linux v3.20
convertx  | GraphicsMagick 1.3.43 2024-03-23 Q16 http://www.GraphicsMagick.org/
convertx  | resvg v0.44.0
convertx  | Bun v1.1.38
convertx  | XeTeX 3.141592653-2.6-0.999996 (TeX Live 2024/Alpine Linux)
convertx  | djxl v0.10.2 [AVX2,SSE4,SSE2]
convertx  | assimp vVersion 5.4 -shared -st (GIT commit 0)
convertx  | pandoc 3.1.13
convertx  | vips-8.15.2
convertx  | ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  | No write access to /.config/calibre using a temporary dir instead
convertx  | Inkscape 1.3.2 (091e20ef0f, 2023-11-25)
convertx  | jobId set to: 8
convertx  | stderr: ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/8/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x15087be6fd80] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/libx265 @ 0x15087a1fb800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/8/BigBuckBunny.h265.
convertx  | Error opening output files: I/O error
convertx  | 
convertx  | Failed to convert ./data/uploads/1/8/BigBuckBunny.mp4 from mp4 to h265 using ffmpeg. error: Error: Command failed: ffmpeg -hwaccel qsv -i "./data/uploads/1/8/BigBuckBunny.mp4"  "./data/output/1/8/BigBuckBunny.h265"
convertx  | ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/8/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x15087be6fd80] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/libx265 @ 0x15087a1fb800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/8/BigBuckBunny.h265.
convertx  | Error opening output files: I/O error
convertx  | 
convertx  | jobId set to: 9
convertx  | stderr: ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x1491cbee8680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/9/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x1491cc22bd00] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x1491cbee8680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/mpeg1video @ 0x1491cbeea800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/9/BigBuckBunny.mpeg.
convertx  | Error opening output files: I/O error
convertx  |

Here is my docker compose:

  convertx:
    user: "701:701"
    image: "ghcr.io/c4illin/convertx:main"
    container_name: "convertx"
    restart: "unless-stopped"
    devices:
      - "/dev/dri:/dev/dri"
    environment:
      - "FFMPEG_ARGS=-hwaccel qsv"
      - "TZ=*****"
      - "JWT_SECRET=*******"
    volumes:
      - "./convertx:/app/data"
    networks:
      - external

Output of ls -l /dev/dri outside the container

crw-rw---- 1 root 701 226,   1 dec  9 11:40 card1
crw-rw-rw- 1 root 701 226, 128 dec  9 11:40 renderD128

And inside the container

crw-rw----    1 root     701       226,   1 Dec  9 11:40 card1
crw-rw-rw-    1 root     701       226, 128 Dec  9 11:40 renderD128

Hardware acceleration is working fine in other containers with the same setup.

[C4illin]

Hardware acceleration is working fine in other containers with the same setup.

Does ffmpeg work with hardware acceleration outside the container? Maybe some more args is needed

[broizter]

Hardware acceleration is working fine in other containers with the same setup.

Does ffmpeg work with hardware acceleration outside the container? Maybe some more args is needed

It should be enough I think. This works outside the container ffmpeg -hwaccel qsv -c:v h264_qsv -i BigBuckBunny.mp4 -c:v h264_qsv -b:v 5M test.mp4

[C4illin]

Ok, if the command works by itself but not in the container with the same input there is only two possibilities: you added the wrong device or the alpine version of ffmpeg doesn't inclue qsv

[C4illin]

Maybe try again and add -loglevel verbose -hwaccel_device /dev/dri/renderD128

[C4illin]

You could also test -hwaccel vaapi

[ektorasdj]

Hello Everyone.

To make it work on my synology with intel celeron J4125 that uses i965 intel video driver (vaapi):
I created a custom dockerfile to install the missing dependencies for the i965 driver to work with Convertx image :

FROM ghcr.io/c4illin/convertx:latest

# Install additional dependencies for VAAPI support
RUN apk add --no-cache \
    gcompat \
    libva-utils 

EXPOSE 3000
ENTRYPOINT [ "bun", "run", "./src/index.tsx" ]

I also downloaded the i965 driver file : i965_drv_video.so and mapped it inside the container.

Compose file:

services:
  convertx: 
    image: custom-convertx:latest #the custom image i build with the dockerfile above
    container_name: convertx
    restart: always
    ports:
      - "3050:3000"
    devices:
      - /dev/dri:/dev/dri
    environment:
      - JWT_SECRET=Secret
      - ACCOUNT_REGISTRATION=false
      - HTTP_ALLOWED=true
      - AUTO_DELETE_EVERY_N_HOURS=24
      - FFMPEG_ARGS=-hwaccel vaapi
      - LIBVA_DRIVER_NAME=i965
      - LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/
    volumes:
      - /volume2/docker/convertx/data:/app/data
      - /volume2/docker/convertx/driver:/usr/lib/x86_64-linux-gnu/dri #here is the i965_drv_video.so driver file
    privileged: true #needed for container to have access to the gpu device of the host

This worked with my synology and converts with hardware accelerated ffmpeg just fine.
If i had to make a request is to include in future images the required libraries and files for vaapi to work without a custom docker image.

In any case i would like to say thank you to the developer since its a great and handy tool!