Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pixiv Ugoria - ffmpeg not working #5421

Closed
subash-ganesan opened this issue Apr 2, 2024 · 4 comments
Closed

Pixiv Ugoria - ffmpeg not working #5421

subash-ganesan opened this issue Apr 2, 2024 · 4 comments

Comments

@subash-ganesan
Copy link

subash-ganesan commented Apr 2, 2024

I'm trying to download pixiv ugoria and convert them to mkv using ugoria-copy postprocessor. I get below error

[gallery-dl][debug] Version 1.26.9 - Executable
[gallery-dl][debug] Python 3.10.12 - Linux-6.6.19-1-MANJARO-x86_64-with-glibc2.39
[gallery-dl][debug] requests 2.31.0 - urllib3 2.2.1
[gallery-dl][debug] Configuration Files ['${HOME}/.config/gallery-dl/config.json', '${HOME}/.gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://www.pixiv.net/en/artworks/117497371'
[pixiv][debug] Using PixivWorkExtractor for 'https://www.pixiv.net/en/artworks/117497371'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): app-api.pixiv.net:443
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/illust/detail?illust_id=117497371 HTTP/1.1" 200 None
[postprocessor.ugoira][debug] using concat demuxer
[pixiv][debug] Active postprocessor modules: [UgoiraPP, MetadataPP]
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=117497371 HTTP/1.1" 200 None
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): i.pximg.net:443
[urllib3.connectionpool][debug] https://i.pximg.net:443 "GET /img-zip-ugoira/img/2024/04/03/01/12/17/117497371_ugoira1920x1080.zip HTTP/1.1" 200 3478644
./gallery-dl/pixiv/88754781 ka2_420/117497371_p0.mkv[postprocessor.ugoira][debug] ['ffmpeg', '-f', 'concat', '-i', '/tmp/tmpyus9rxa8/ffconcat.txt', '-r', '1000/10', '-hide_banner', '-loglevel', 'error', '-c', 'copy', './gallery-dl/pixiv/88754781 ka2_420/117497371_p0.mkv']
ffmpeg: /tmp/_MEIJtV6QY/libgcc_s.so.1: version `GCC_13.0.0' not found (required by /usr/lib/libhwy.so.1)

[postprocessor.ugoira][error] Non-zero exit status when running ['ffmpeg', '-f', 'concat', '-i', '/tmp/tmpyus9rxa8/ffconcat.txt', '-r', '1000/10', '-hide_banner', '-loglevel', 'error', '-c', 'copy', './gallery-dl/pixiv/88754781 ka2_420/117497371_p0.mkv'] (1)

As we can see, it is clearly showing gcc is not found. But I can confirm it is installed

gcc --version  1 ✘
gcc (GCC) 13.2.1 20230801
Copyright (C) 2023 Free Software Foundation, Inc.

It would be helpful if someone could help reolve this

@Hrxn
Copy link
Contributor

Hrxn commented Apr 2, 2024

Something is very off here..

You definitely don't need GCC (a compiler) to simply run ffmpeg.

Does ffmpeg work as expected for you? What is your output for ffmpeg -version, for example?

@subash-ganesan
Copy link
Author

subash-ganesan commented Apr 2, 2024

Here's what i got

ffmpeg -version  8 ✘
ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.1 (GCC) 20230801
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100

@mikf
Copy link
Owner

mikf commented Apr 2, 2024

I can reproduce this error:

ffmpeg: /tmp/_MEIgUt8so/libgcc_s.so.1: version `GCC_13.0.0' not found (required by /usr/lib/libhwy.so.1)

I initially thought this might have been caused by building on an Ubuntu VM (4ccdba8), but even older binaries yield this error:

$ ./old.bin -v --ugoira-conv-copy https://www.pixiv.net/en/artworks/117457000 
[gallery-dl][debug] Version 1.24.5 - Executable
...
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/librubberband.so.2)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libopenmpt.so.0)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libsrt.so.1.5)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libshaderc_shared.so.1)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libSPIRV.so.14)
ffmpeg: /tmp/_MEIf15A6a/libgcc_s.so.1: version `GCC_13.0.0' not found (required by /usr/lib/libhwy.so.1)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libSPIRV-Tools.so)
ffmpeg: /tmp/_MEIf15A6a/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /usr/lib/libSPIRV-Tools-opt.so)

Probably something related to PyInstaller and its bundled libraries.

You definitely don't need GCC (a compiler) to simply run ffmpeg.

"version 'GCC_13.0.0' not found" relates to the GCC version a .so library was built against and requires to load, I think.

@mikf
Copy link
Owner

mikf commented Apr 6, 2024

I found the cause and solution to this by accident when looking through yt-dlp code. PyInstaller modifies LD_LIBRARY_PATH such that external programs try to load PyInstaller libraries instead of system libraries first: https://github.com/pyinstaller/pyinstaller/blob/develop/doc/runtime-information.rst#ld_library_path--libpath-considerations

@mikf mikf closed this as completed Apr 6, 2024
JackTildeD added a commit to JackTildeD/gallery-dl-forked that referenced this issue Apr 24, 2024
* save cookies to tempfile, then rename

avoids wiping the cookies file if the disk is full

* [deviantart:stash] fix 'index' metadata (mikf#5335)

* [deviantart:stash] recognize 'deviantart.com/stash/…' URLs

* [gofile] fix extraction

* [kemonoparty] add 'revision_count' metadata field (mikf#5334)

* [kemonoparty] add 'order-revisions' option (mikf#5334)

* Fix imagefap extrcator

* [twitter] add 'birdwatch' metadata field (mikf#5317)

should probably get a better name,
but this is what it's called internally by Twitter

* [hiperdex] update URL patterns & fix 'manga' metadata (mikf#5340)

* [flickr] add 'contexts' option (mikf#5324)

* [tests] show full path for nested values

'user.name' instead of just 'name' when testing for
"user": { … , "name": "…", … }

* [bluesky] add 'instance' metadata field (mikf#4438)

* [vipergirls] add 'like' option (mikf#4166)

* [vipergirls] add 'domain' option (mikf#4166)

* [gelbooru] detect returned favorites order (mikf#5220)

* [gelbooru] add 'date_favorited' metadata field

* Update fapello.py

get fullsize image instead resized

* fapello.py Fullsize image

by remove ".md" and ".th" in image url, it will download fullsize of images

* [formatter] fix local DST datetime offsets for ':O'

'O' would get the *current* local UTC offset and apply it to all
'datetime' objects it gets applied to.
This would result in a wrong offset if the current offset includes
DST and the target 'datetime' does not or vice-versa.

'O' now determines the correct local UTC offset while respecting DST for
each individual 'datetime'.

* [subscribestar] fix 'date' metadata

* [idolcomplex] support new pool URLs

* [idolcomplex] fix metadata extraction

- replace legacy 'id' vales with alphanumeric ones, since the former are
  no longer available
- approximate 'vote_average', since the real value is no longer
  available
- fix 'vote_count'

* [bunkr] remove 'description' metadata

album descriptions are no longer available on album pages
and the previous code erroneously returned just '0'

* [deviantart] improve 'index' extraction for stash files (mikf#5335)

* [kemonoparty] fix exception for '/revision/' URLs

caused by 03a9ce9

* [steamgriddb] raise proper exception for deleted assets

* [tests] update extractor results

* [pornhub:gif] extract 'viewkey' and 'timestamp' metadata (mikf#4463)

mikf#4463 (comment)

* [tests] use 'datetime.timezone.utc' instead of 'datetime.UTC'

'datetime.UTC' was added in Python 3.11
and is not defined in older versions.

* [gelbooru] add 'order-posts' option for favorites (mikf#5220)

* [deviantart] handle CloudFront blocks in general (mikf#5363)

This was already done for non-OAuth requests (mikf#655)
but CF is now blocking OAuth API requests as well.

* release version 1.26.9

* [kemonoparty] fix KeyError for empty files (mikf#5368)

* [twitter] fix pattern for single tweet (mikf#5371)

- Add optional slash
- Update tests to include some non-standard tweet URLs

* [kemonoparty:favorite] support 'sort' and 'order' query params (mikf#5375)

* [kemonoparty] add 'announcements' option (mikf#5262)

mikf#5262 (comment)

* [wikimedia] suppress exception for entries without 'imageinfo' (mikf#5384)

* [docs] update defaults of 'sleep-request', 'browser', 'tls12'

* [docs] complete Authentication info in supportedsites.md

* [twitter] prevent crash when extracting 'birdwatch' metadata (mikf#5403)

* [workflows] build complete docs Pages only on gdl-org/docs

deploy only docs/oauth-redirect.html on mikf.github.io/gallery-dl

* [docs] document 'actions' (mikf#4543)

or at least attempt to

* store 'match' and 'groups' in Extractor objects

* [foolfuuka] improve 'board' pattern & support pages (mikf#5408)

* [reddit] support comment embeds (mikf#5366)

* [build] add minimal pyproject.toml

* [build] generate sdist and wheel packages using 'build' module

* [build] include only the latest CHANGELOG entries

The CHANGELOG is now at a size where it takes up roughly 50kB or 10% of
an sdist or wheel package.

* [oauth] use Extractor.request() for HTTP requests (mikf#5433)

Enables using proxies and general network options.

* [kemonoparty] fix crash on posts with missing datetime info (mikf#5422)

* restore LD_LIBRARY_PATH for PyInstaller builds (mikf#5421)

* remove 'contextlib' imports

* [pp:ugoira] log errors for general exceptions

* [twitter] match '/photo/' Tweet URLs (mikf#5443)

fixes regression introduced in 40c0553

* [pp:mtime] do not overwrite '_mtime' for None values (mikf#5439)

* [wikimedia] fix exception for files with empty 'metadata'

* [wikimedia] support wiki.gg wikis

* [pixiv:novel] add 'covers' option (mikf#5373)

* [tapas] add 'creator' extractor (mikf#5306)

* [twitter] implement 'relogin' option (mikf#5445)

* [docs] update docs/configuration links (mikf#5059, mikf#5369, mikf#5423)

* [docs] replace AnchorJS with custom script

use it in rendered .rst documents as well as in .md ones

* [text] catch general Exceptions

* compute tempfile path only once

* Add warnings flag

This commit adds a warnings flag

It can be combined with -q / --quiet to display warnings.
The intent is to provide a silent option that still surfaces
warning and error messages so that they are visible in logs.

* re-order verbose and warning options

* [gelbooru] improve pagination logic for meta tags (mikf#5478)

similar to 494acab

* [common] add Extractor.input() method

* [twitter] improve username & password login procedure (mikf#5445)

- handle more subtasks
- support 2FA
- support email verification codes

* [common] update Extractor.wait() message format

* [common] simplify 'status_code' check in Extractor.request()

* [common] add 'sleep-429' option (mikf#5160)

* [common] fix NameError in Extractor.request()

… when accessing 'code' after an requests exception was raised.

Caused by the changes in 566472f

* [common] show full URL in Extractor.request() error messages

* [hotleak] download files with 404 status code (mikf#5395)

* [pixiv] change 'sanity_level' debug message to a warning (mikf#5180)

* [twitter] handle missing 'expanded_url' fields (mikf#5463, mikf#5490)

* [tests] allow filtering extractor result tests by URL or comment

python test_results.py twitter:+/i/web/
python test_results.py twitter:~twitpic

* [exhentai] detect CAPTCHAs during login (mikf#5492)

* [output] extend 'output.colors' (mikf#2566)

allow specifying ANSI colors for all loglevels
(debug, info, warning, error)

* [output] enable colors by default

* add '--no-colors' command-line option

---------

Co-authored-by: Luc Ritchie <[email protected]>
Co-authored-by: Mike Fährmann <[email protected]>
Co-authored-by: Herp <[email protected]>
Co-authored-by: wankio <[email protected]>
Co-authored-by: fireattack <[email protected]>
Co-authored-by: Aidan Harris <[email protected]>
@mikf mikf added the executable label Jun 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants