Possible to enable hardware transcoding? #508

maisun · 2020-05-18T19:10:57Z

Hi,
I'm running homebridge as docker container (oznu/homebridge) on my NAS (QNAP with Intel CPU), I believe the docker image comes with ffmpeg. Is there a way to enable hardware transcoding? I've checked and it seems the HW transcoding is possible:
[~] # ls /dev/dri/
card0 renderD128
Anyone manage to get this to work with NAS? Thanks!

mickgiles · 2020-05-28T22:32:15Z

I have hardware transcoding running on an ubuntu box and I had to jump through a bunch of hoops to do so. The big one was I had to compile my own version of ffmpeg with the Intel transcoder enabled. If you have the Intel transcoder then you will need ffmpeg with --enable-vaapi and --enable-hwaccel=h264_vaapi enabled.
If you can get that far you may need a custom shell script to manipulate the command line. This is what I use...

#!/bin/bash

CMD="/home/user/bin/ffmpeg"

while [[ $# > 1 ]]
do
key="$1"

case ${key} in
    -pix_fmt)
    CMD+=" $1 vaapi_vld"
    shift
    ;;
    *)
    CMD+=" $1"
    ;;
esac
shift
done

exec $CMD ${!#}

And then this is what my config looks like...

"platform": "Camera-ffmpeg",
"videoProcessor": "/home/user/bin/ffmpeg.sh",
"cameras": [
    {
        "name": "Front Camera",
        "videoConfig": {
            "source": "-init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -rtsp_transport tcp -re -i rtsp://USER:[email protected]:80/cam/realmonitor?channel=1&subtype=0",
            "stillImageSource": "-rtsp_transport tcp -re -i rtsp://USER:[email protected]:80/cam/realmonitor?channel=1&subtype=0",
            "maxStreams": 3,
            "maxWidth": 1280,
            "maxHeight": 720,
            "maxFPS": 30,
            "maxBitrate": 1024,
            "vcodec": "h264_vaapi",
            "videoFilter": "format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080 -bf 0"
        }
    },

I hope this helps, I spent a while trying to figure out all of the missing pieces

maisun · 2020-05-30T12:26:36Z

Hi, it's beyond my technical skills but

I have hardware transcoding running on an ubuntu box and I had to jump through a bunch of hoops to do so. The big one was I had to compile my own version of ffmpeg with the Intel transcoder enabled. If you have the Intel transcoder then you will need ffmpeg with --enable-vaapi and --enable-hwaccel=h264_vaapi enabled.
If you can get that far you may need a custom shell script to manipulate the command line. This is what I use...
#!/bin/bash

CMD="/home/user/bin/ffmpeg"

while [[ $# > 1 ]]
do
key="$1"

case ${key} in
    -pix_fmt)
    CMD+=" $1 vaapi_vld"
    shift
    ;;
    *)
    CMD+=" $1"
    ;;
esac
shift
done

exec $CMD ${!#}
And then this is what my config looks like...
"platform": "Camera-ffmpeg",
"videoProcessor": "/home/user/bin/ffmpeg.sh",
"cameras": [
    {
        "name": "Front Camera",
        "videoConfig": {
            "source": "-init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -rtsp_transport tcp -re -i rtsp://USER:[email protected]:80/cam/realmonitor?channel=1&subtype=0",
            "stillImageSource": "-rtsp_transport tcp -re -i rtsp://USER:[email protected]:80/cam/realmonitor?channel=1&subtype=0",
            "maxStreams": 3,
            "maxWidth": 1280,
            "maxHeight": 720,
            "maxFPS": 30,
            "maxBitrate": 1024,
            "vcodec": "h264_vaapi",
            "videoFilter": "format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080 -bf 0"
        }
    },
I hope this helps, I spent a while trying to figure out all of the missing pieces
Hi, it's beyond my technical skills but thanks a lot for the hints!

quartist · 2020-06-13T14:29:38Z

Hi,

Im trying to use hardware transcoding with Jonathon FFmpeg: https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4

just works but not in homebridge-camera-ffmpeg.

my FFmpeg:
ffmpeg version 4.2.2-1ubuntu118.04.york0 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu118.04)
configuration: --prefix=/usr --extra-version='1ubuntu1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100

and my line:

[2020-6-13 14:27:29] [Camera-ffmpeg] Start streaming video from Salon with 1280:720@25fps (299kBit)
/usr/lib/node_modules/homebridge-camera-ffmpeg/node_modules/ffmpeg-for-homebridge/ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_device intel -rtsp_transport http -i rtsp://192.168.20.12:8554/unicast -map 0:0 -vcodec h264_vaapi -pix_fmt yuv420p -r 25 -f rawvideo -tune zerolatency -vf format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080 -bf 0 -b:v 299k -bufsize 299k -maxrate 299k -payload_type 99 -ssrc 5192760 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params /TdaI1eBdmf92HsLykK9QJ1ihhGIHgjXaLbRDk69 srtp://192.168.50.61:61816?rtcpport=61816&localrtcpport=61816&pkt_size=1316 -loglevel debug
ffmpeg version 4.2.2-d1e8be3 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --pkgconfigdir=/build/workspace/lib/pkgconfig --prefix=/build/workspace --pkg-config-flags=--static --extra-cflags=-I/build/workspace/include --extra-ldflags=-L/build/workspace/lib --extra-libs='-lpthread -lm' --enable-static --disable-debug --disable-shared --disable-ffplay --disable-doc --enable-openssl --enable-gpl --enable-version3 --enable-nonfree --enable-pthreads --enable-libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-runtime-cpudetect --enable-libfdk-aac --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable-libspeex --enable-demuxer=rtsp --enable-indev=alsa --enable-outdev=alsa
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100

libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Splitting the commandline.
Reading option '-init_hw_device' ... matched as option 'init_hw_device' (initialise hardware device) with argument 'vaapi=intel:/dev/dri/renderD128'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'vaapi'.
Reading option '-hwaccel_device' ... matched as option 'hwaccel_device' (select a device for HW acceleration) with argument 'intel'.
Reading option '-rtsp_transport' ...
matched as AVOption 'rtsp_transport' with argument 'http'.
Reading option '-i' ... matched as input url with argument 'rtsp://192.168.20.12:8554/unicast'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'h264_vaapi'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '25'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-tune' ...
matched as AVOption 'tune' with argument 'zerolatency'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080'.
Reading option '-bf' ...
matched as AVOption 'bf' with argument '0'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '299k'.
Reading option '-bufsize' ...
matched as AVOption 'bufsize' with argument '299k'.
Reading option '-maxrate' ...
matched as AVOption 'maxrate' with argument '299k'.
Reading option '-payload_type' ...
matched as AVOption 'payload_type' with argument '99'.
Reading option '-ssrc' ...
matched as AVOption 'ssrc' with argument '5192760'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtp'.
Reading option '-srtp_out_suite' ...
matched as AVOption 'srtp_out_suite' with argument 'AES_CM_128_HMAC_SHA1_80'.
Reading option '-srtp_out_params' ...
matched as AVOption 'srtp_out_params' with argument '/TdaI1eBdmf92HsLykK9QJ1ihhGIHgjXaLbRDk69'.
Reading option 'srtp://192.168.50.61:61816?rtcpport=61816&localrtcpport=61816&pkt_size=1316' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option init_hw_device (initialise hardware device) with argument vaapi=intel:/dev/dri/renderD128.

Device creation failed: -12.
Failed to set value 'vaapi=intel:/dev/dri/renderD128' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

[2020-6-13 14:27:29] [Camera-ffmpeg] ERROR: FFmpeg exited with code 1

mickgiles · 2020-06-13T15:15:30Z

That version of ffmpeg doesn't have --enable-vaapi and --enable-hwaccel=h264_vaapi compiled in. You will need to compile your own version off ffmpeg with these options.
This is what I followed to build my version https://gist.github.com/Brainiarc7/eb45d2e22afec7534f4a117d15fe6d89

maisun · 2020-06-13T15:18:57Z

Hi,

Im trying to use hardware transcoding with Jonathon FFmpeg: https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4

just works but not in homebridge-camera-ffmpeg.

my FFmpeg:
ffmpeg version 4.2.2-1ubuntu1~~18.04.york0 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~~18.04)
configuration: --prefix=/usr --extra-version='1ubuntu1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100

and my line:

[2020-6-13 14:27:29] [Camera-ffmpeg] Start streaming video from Salon with 1280:720@25fps (299kBit)
/usr/lib/node_modules/homebridge-camera-ffmpeg/node_modules/ffmpeg-for-homebridge/ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_device intel -rtsp_transport http -i rtsp://192.168.20.12:8554/unicast -map 0:0 -vcodec h264_vaapi -pix_fmt yuv420p -r 25 -f rawvideo -tune zerolatency -vf format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080 -bf 0 -b:v 299k -bufsize 299k -maxrate 299k -payload_type 99 -ssrc 5192760 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params /TdaI1eBdmf92HsLykK9QJ1ihhGIHgjXaLbRDk69 srtp://192.168.50.61:61816?rtcpport=61816&localrtcpport=61816&pkt_size=1316 -loglevel debug
ffmpeg version 4.2.2-d1e8be3 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --pkgconfigdir=/build/workspace/lib/pkgconfig --prefix=/build/workspace --pkg-config-flags=--static --extra-cflags=-I/build/workspace/include --extra-ldflags=-L/build/workspace/lib --extra-libs='-lpthread -lm' --enable-static --disable-debug --disable-shared --disable-ffplay --disable-doc --enable-openssl --enable-gpl --enable-version3 --enable-nonfree --enable-pthreads --enable-libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-runtime-cpudetect --enable-libfdk-aac --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable-libspeex --enable-demuxer=rtsp --enable-indev=alsa --enable-outdev=alsa
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100

libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Splitting the commandline.
Reading option '-init_hw_device' ... matched as option 'init_hw_device' (initialise hardware device) with argument 'vaapi=intel:/dev/dri/renderD128'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'vaapi'.
Reading option '-hwaccel_device' ... matched as option 'hwaccel_device' (select a device for HW acceleration) with argument 'intel'.
Reading option '-rtsp_transport' ...
matched as AVOption 'rtsp_transport' with argument 'http'.
Reading option '-i' ... matched as input url with argument 'rtsp://192.168.20.12:8554/unicast'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'h264_vaapi'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '25'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-tune' ...
matched as AVOption 'tune' with argument 'zerolatency'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'format=nv12|vaapi,hwupload,scale_vaapi=w=1920:h=1080'.
Reading option '-bf' ...
matched as AVOption 'bf' with argument '0'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '299k'.
Reading option '-bufsize' ...
matched as AVOption 'bufsize' with argument '299k'.
Reading option '-maxrate' ...
matched as AVOption 'maxrate' with argument '299k'.
Reading option '-payload_type' ...
matched as AVOption 'payload_type' with argument '99'.
Reading option '-ssrc' ...
matched as AVOption 'ssrc' with argument '5192760'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtp'.
Reading option '-srtp_out_suite' ...
matched as AVOption 'srtp_out_suite' with argument 'AES_CM_128_HMAC_SHA1_80'.
Reading option '-srtp_out_params' ...
matched as AVOption 'srtp_out_params' with argument '/TdaI1eBdmf92HsLykK9QJ1ihhGIHgjXaLbRDk69'.
Reading option 'srtp://192.168.50.61:61816?rtcpport=61816&localrtcpport=61816&pkt_size=1316' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option init_hw_device (initialise hardware device) with argument vaapi=intel:/dev/dri/renderD128.

Device creation failed: -12.
Failed to set value 'vaapi=intel:/dev/dri/renderD128' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

[2020-6-13 14:27:29] [Camera-ffmpeg] ERROR: FFmpeg exited with code 1

I tried the same, it works but I don't know how to make it work for HomeBridge. I run HB based on the docker image oznu/docker-homebridge, it comes with ffmpeg in the image. However that particular version doesn't support HW transcoding. I found FFMPEG from Jellyfin docker image also supports HW transcoding, it would be nice to try replace the FFMPEG with either the Jonathan's version or Jellyfin's version. It is unfortuantely beyond my technical skills :-(

mickgiles · 2020-06-13T15:29:03Z

AH, I see. If you have a version of ffmpeg with vaapi built in you can definitely get this to work with what I have included here. (as long as your CPU is an Intel with hw encoding support)
Take a look at my config. There is a line for videoProcessor that points to a shell script I wrote ffmpeg.sh (also inline).
"videoProcessor": "/home/user/bin/ffmpeg.sh"
Put that shell script somewhere in your home directory and change the path in the config to point to it.
Then change the following line in the shell script to point the the version of ffmpeg you want to use...
CMD="/home/user/bin/ffmpeg"
Then pay close attention to the config I included as it has a lot of required options to get hw encoding working. Specifically the options in source, vcodec, and videoFilter.

quartist · 2020-06-13T19:22:15Z

working.
"cameras": [
{
"name": "Salon",
"videoProcessor": "/home/homebridge/ffmpeg.sh",
"videoConfig": {
"source": "-hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -rtsp_transport http -i rtsp://192.168.20.12:8554/unicast",
"stillImageSource": "-rtsp_transport http -i rtsp://192.168.20.12:8554/unicast -vframes 1 -r 1",
"maxStreams": 2,
"maxWidth": 1280,
"maxHeight": 720,
"maxFPS": 25,
"maxBitrate": 3000,
"debug": true,
"audio": false,
"acodec": "copy",
"vcodec": "copy"
}
}

mickgiles · 2020-06-13T22:30:26Z

Good job!
There is a bug in the latest version of homebridge-ffmpeg-camera that causes still images to not work.
I've logged that bug here #504
There are 2 ways to get around this until an official fix is available

Downgrade to 0.1.17 with sudo npm install -g [email protected]
Install my fork sudo npm install -g --unsafe-perm https://github.com/mickgiles/homebridge-camera-ffmpeg.git --save-dev
Whichever you do, just monitor the bug listed above for when it is fixed to get back on the main branch

maisun · 2020-06-14T08:55:09Z

I guess when you have copy for vcodec it will skip transcoding??

quartist · 2020-06-14T10:30:18Z

Good job!
There is a bug in the latest version of homebridge-ffmpeg-camera that causes still images to not work.
I've logged that bug here #504
There are 2 ways to get around this until an official fix is available

Downgrade to 0.1.17 with sudo npm install -g [email protected]

Install my fork sudo npm install -g --unsafe-perm https://github.com/mickgiles/homebridge-camera-ffmpeg.git --save-dev
Whichever you do, just monitor the bug listed above for when it is fixed to get back on the main branch

still image work in my configuration without videoFilters option.

maisun added the question label May 18, 2020

donavanbecker closed this as completed Jun 23, 2020

washcroft mentioned this issue Nov 10, 2021

Hardware Acceleration Using VAAPI (Intel GPUs) #1223

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Possible to enable hardware transcoding? #508

Possible to enable hardware transcoding? #508

maisun commented May 18, 2020

mickgiles commented May 28, 2020

maisun commented May 30, 2020

quartist commented Jun 13, 2020

mickgiles commented Jun 13, 2020

maisun commented Jun 13, 2020

mickgiles commented Jun 13, 2020

quartist commented Jun 13, 2020 •

edited

Loading

mickgiles commented Jun 13, 2020

maisun commented Jun 14, 2020

quartist commented Jun 14, 2020

Possible to enable hardware transcoding? #508

Possible to enable hardware transcoding? #508

Comments

maisun commented May 18, 2020

mickgiles commented May 28, 2020

maisun commented May 30, 2020

quartist commented Jun 13, 2020

mickgiles commented Jun 13, 2020

maisun commented Jun 13, 2020

mickgiles commented Jun 13, 2020

quartist commented Jun 13, 2020 • edited Loading

mickgiles commented Jun 13, 2020

maisun commented Jun 14, 2020

quartist commented Jun 14, 2020

quartist commented Jun 13, 2020 •

edited

Loading