Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"wobbly rubber sheet" effect due to "slow" top-down scanning of camera? #131

Open
AlainKnaff opened this issue Oct 4, 2024 · 6 comments

Comments

@AlainKnaff
Copy link

Hi,

During a recent trip, I took a couple of videos of animals (marmots, ibexes) with extreme zoom (30x) with my Panasonic DC-TZ202.

As I don't have a steady hand the original videos were shaking quite a bit.

Your tool was godsend, and it could produce videos were the main subject is remarkably steady. Thanks for this wonderful program.

However, the background looks wobbly, a bit as if it was printed on a rubber sheet which was repeatedly pulled apart and released again.

Example:

Original video:
http://alain.knaff.lu/Photos/2024-Barcelonette/Video.orig/P1158994.MP4

Processed video (with vid.stab with shakiness=10 and smoothing=0 , and finally cropped to 960x540):
http://alain.knaff.lu/Photos/2024-Barcelonette/Video/P1158994.MP4

As these are zoom images, lens distortion is unlikely to be a factor (would come into play for wide angle shots)

A friend of mine thinks that this may be due to the finite time it takes the camera to "scan" the video from top down. I.e. topmost lines are scanned slightly earlier than lines at the bottom. So, if the camera is moving down image tend to get compressed vertically, whereas if it is moving up it tends to get extended. Could it be that vid.stab is compensating for purely geometric shake (i.e. translation, rotation, perspective), but not for this time-based artifact?

Thanks,

Alain

@georgmartius
Copy link
Owner

Dear Alain,
yes, that is true. Rolling shutter, as it is called, it not considered in vid.stab, unfortunately. You might want to try the stabilizer in youtube, if that is an option for you. They compensate for that.
Best,
Georg

@AlainKnaff
Copy link
Author

Thanks for your quick reply. Unfortunately youtube is not an option for me. However, thanks for giving me the name of this phenomenon ("rolling shutter"), that way I can start searching more easily for other solutions online :-)

@AlainKnaff
Copy link
Author

Btw, is there any documentation of the file format of transforms.trf available? Maybe something could be done by preprocessing this file after the detect and before the transform step? Thanks

@georgmartius
Copy link
Owner

There is no proper documentation. But even if, it would not help, as the transformation that computed and are applied to compensate are rigid (apply to the whole frame).

@AlainKnaff
Copy link
Author

What do you mean by "rigid"? Linear?
If so, then it would be doable to compensate for first order effects (speed of panning, but not acceleration) which might already be an improvement. Indeed, for this we would only need shear (for horizontal panning) and scaling along one axis (for vertical panning), both of which are linear.

@georgmartius
Copy link
Owner

Yes, it is linear. You are right, that in principle linear transformations would be enough, but these shearings and scalings are not implemented, but might actually not so difficult to implement, but not by changing the transform file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants