WIP, ENH: frame number via compositor #28
Conversation
* Based on discussion in yuxuanzhuang#27, and as an alternative to yuxuanzhuang#14, this is a demo for using external image generation followed by a compositor step to overlay the image that contains nothing but the frame number in it. * Predictably, as recently noted at yuxuanzhuang#21 (comment) this ablates the background coloring that also uses the compositor. It seemed fairly obvious to me that this would happen even as non-expert, if we just naively use the compositor for isolated purposes that try to ignore each other. I guess I can't escape the node connection reality under the hood as much as I'd like to. [ci skip]
ggmolvis/camera.py
Outdated
| width=resolution[0], | ||
| height=resolution[1], | ||
| text=f"frame: {self.frame_number}") | ||
| _composit_frame_label(img_path=img_path) |
There was a problem hiding this comment.
Note that this won't work for movies yet--some/all of the infrastructure may need to be migrated to _update_frame(), which is a bit annoying for design perhaps.
| text: str): | ||
| img = Image.new('RGBA', (width, height), (0, 0, 0, 0)) | ||
| draw = ImageDraw.Draw(img) | ||
| font = ImageFont.truetype("/System/Library/Fonts/Geneva.ttf", 80) |
There was a problem hiding this comment.
This is obviously not portable. Also, to make matters worse, using the default built-in font from PIL doesn't allow custom font size: python-pillow/Pillow#2695
And, of course, the font size may need to be tuned in some cases unless we scale it "automatically" somehow.
| scene.use_nodes = True | ||
| tree = scene.node_tree | ||
| links = tree.links | ||
| tree.nodes.clear() |
There was a problem hiding this comment.
presumably this is what ablates the background composition
| text_height = text_bbox[3] - text_bbox[1] | ||
| x = (width - text_width) // 2 | ||
| y = height - text_height - 40 | ||
| draw.text((x, y), text, font=font, fill=(255, 255, 255, 255)) |
There was a problem hiding this comment.
would probably need a way for user to specify the color of the font, especially if they are also specifying the color of the background
|
I don't think we would need Blender's compositor here. Here is a snippet from existing code where the viewport and annotations images are composited. If the alpha image is regular (ie., not pre-multiplied), the conversion step is also not needed:
Currently, it is the same code that is used for viewport display using |
* Based on reviewer feedback, avoid the need to write image files to disk when compositing in frame numbers. Instead, keep the pixel data in memory and pass directly into Blender compositor workflow.
* Use a pre-render handler to allow compositing of frame numbers into Blender movies. [ci skip]
Maybe there is some confusion, but I'm generally adding features for both at the same time--I only use the image samples for fast iteration/sharing. I've pushed in a commit to use a pre-render handler to allow the composition to propagate to the movie renders as well now. Here is sample movie generated from this branch using a reference (https://github.com/tylerjereddy/render_north_star) render script: https://youtube.com/shorts/uwDHh-y9uBw?feature=share The cross-interference with the background color composition remains, but the frame number insertion seems to be working "ok" now. The other caveats from above remain alongside one additional observation, which is a bit of "bouncing" in the text placement I think. Maybe the math that does the positioning of the text could compensate for that.
That I think I agree on, though the only simplification I've been able to make so far is to stop writing temporary image files and just keep them in memory, which is a nice clean up. I suspect we agree the compositor is more or less needed for both image and video support at this point. Also, I'd be a bit hesitant to treat image and movie generation separately unless there's a really good reason for that--it would be nice if the image could serve as a reliable preview for the movie frames, which may break down a bit if we use separate text placement approaches for both. |
|
Frame number placement also seems "ok" in movies when focal length is varied on this branch, as intended. Most of the other caveats/issues above still apply though. |
Based on discussion in Design choices for Text Annotations #27, and as an alternative to WIP, ENH: SceneObject text overlay #14, this is a demo for using external image generation followed by a compositor step to overlay the image that contains nothing but the frame number in it.
Predictably, as recently noted at ENH: use compositor to set solid background color #21 (comment), this ablates the background coloring that also uses the compositor. It seemed fairly obvious to me that this would happen, even as non-expert, if we just naively use the compositor for isolated purposes that try to ignore each other. I guess I can't escape the node connection reality under the hood as much as I'd like to.
[ci skip]
I'm assuming that there was also some desire to screen capture
blf-placed labels as well and get them into the render in some roundabout way, but the current approach is even cruder and just produces empty images with frame numbers at the bottom using an external lib.For still images, it seems more resilient to changes in focal length:
Focal length 110:
Focal length 80:
Focal length 10: