Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FFMPEG autorotates source images without user control #1260

Closed
dkorolov opened this issue Jan 19, 2023 · 5 comments
Closed

FFMPEG autorotates source images without user control #1260

dkorolov opened this issue Jan 19, 2023 · 5 comments

Comments

@dkorolov
Copy link
Contributor

FFMPEG autorotates images with Orientation EXIF tag. It happens automatically without user control during the first usage of FFMPEG.
In many cases, this does not influence data processing (because of the behavior described below). We need to have ways to control this behavior.
This behavior can give more issues and unexpected results, especially for data import (RealityCapture as an example)
Note: FFMPEG autorotates images with the Orientation EXIF tag only the first time. All images that are already saved with FFMPEG have removed part of the EXIF tags including the Orientation tag. So it does not autorotate already saved images.

The current behavior in the case of using source images :

  1. Currently we use FFMPEG for all folders we created with ns-process-data (all created folders: images. images_2, images_4, images_8). This is normal behavior for image downscaling. For the original size folder (images) that don't require border pixel cropping, we do not expect to use FFMPEG. But we always use it because of the bug described below.
  2. In process_data.py we have bug here .
        # Current code
        num_frames = len(copy_images_list(image_paths, image_dir, verbose))
        # Should be:
        num_frames = len(copy_images_list(image_paths, image_dir, verbose=verbose))

This bug forces to use FFMPEG here with crop_border_pixels=0 in copy_images_list. This is not expected behavior. BUT this is a "lucky" bug that prevents nerfsudio data processing to be broken :) If we just fix it we can get data inconsistency (different image orientation) between the original and downscaled folders. At the current moment all are consistent because we use FFMPEG with the original image in any case.
3. We need to fix the bug above and decide how to control autorotation
4. I made several tests but unfortunately don't find any practical way to switch off autorotation in FFMPEG for images (tested in Windows). Please give me an idea of how to switch autorotation off if anybody have success with this.
5. As a possible option we can control Orientation EXIF tag. We can add a function to remove this tag in copied images. And also function to check Orientation EXIF tag status. PIL can read EXIF or we can use exiftool utility for this as a subprocess.

@machenmusik
Copy link
Contributor

Supposedly there is an option for recent ffmpeg -noautorotate -- does that help?

@dkorolov
Copy link
Contributor Author

For some reason, I don't see this -noautorotate option in my Windows ffmpeg.
Instead, I see:
-autorotate automatically insert correct rotate filters

@dkorolov
Copy link
Contributor Author

Ok , I see in doc:
-autorotate
Automatically rotate the video according to file metadata. Enabled by default, use -noautorotate to disable it.

@machenmusik
Copy link
Contributor

machenmusik commented Jan 20, 2023

Can you try #1261? (applying that change on top of #1055)

@machenmusik
Copy link
Contributor

(Feel free to reopen if PR fix fails in your case.)

lucasthahn pushed a commit to tne-ai/nerfstudio that referenced this issue Jan 26, 2023
chris838 pushed a commit to chris838/nerfstudio that referenced this issue Apr 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants