-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incremental rendering for very large images #77
Comments
You need to be mindful of how much memory rendering such a large image requires. At a minimum, it would require width x height x 4 bytes since the images are 32 bits per pixel. When anti-aliasing is enabled, it requires more than 16x as many bytes because 4 pixels vertically x 4 pixels horizontally are used to calculate a single anti-aliased pixel. At 32 bits (4 bytes) per pixel, 5700x4065x16x4 = 1.38GB. 32 bit processes on Windows have a 2GB memory limit. XaoS also uses other buffers internally that grow with the size of the image, so you are getting very close to the 2GB limit. Using a 64-bit binary on a system with 16GB, I was able to render up to 14000x13000. 14000x14000 crashed with an antialiasing enabled, which would have required over to 11GB for just the anti-aliasing buffer. I confirmed that the call to allocate the AA buffer is failing, so I'm pretty sure this is just a memory limitation issue. Not much we can do to get around that. I would suggest you try rendering the image without anti-aliasing, which would allow you to do 4x the resolution horizontally and vertically. I'm working right now on a new release now, which should hopefully be ready within the next few months. I don't plan to make Windows binaries for 3.6 at this point since a new version is imminent. For the next Windows release, I will offer a 64-bit binary which would allow more memory, but you're still limited by the amount of memory in your computer. There are ways that we could split the image up and render each one to disk and then combine them, but that would be a lot more effort. I will change this issue to an enhancement to do disk-based rendering for large images, but I can't promise when I will get to that, if ever. |
Dear J.B. Langston,
I am very glad that you still put effort in Xaos. Thank you for the clarifications. If time allows, I will try out your suggestion to render the png without anti-aiasing. In any case I look forward to your new Xaos version, later this year. Please notify me when it comes out.
Best regards
Hinrich Ruyter
From: J.B. Langston
Sent: Saturday, May 11, 2019 5:18 AM
To: xaos-project/XaoS
Cc: hruyter ; Author
Subject: Re: [xaos-project/XaoS] Xaos 3.5 crashes with render command (#77)
You need to be mindful of how much memory rendering such a large image requires. At a minimum, it would require width x height x 4 bytes since the images are 32 bits per pixel. When anti-aliasing is enabled, it requires 16x as many bytes because 4 pixels vertically x 4 pixels horizontally are used to calculate a single anti-aliased pixel. At 32 bits (4 bytes) per pixel, 5700x4065x16x4 = 1.38GB. 32 bit processes on Windows have a 2GB memory limit. XaoS also uses other buffers internally that grow with the size of the image, so you are getting very close to the 2GB limit.
Using a 64-bit binary on Linux with a system with 16GB, I was able to render up to 14000x13000. 14000x14000 crashed with an antialiasing enabled, which would have required over to 11GB for just the anti-aliasing buffer. I confirmed that the call to allocate the AA buffer is failing, so I'm pretty sure this is just a memory limitation issue. Not much we can do to get around that. I would suggest you try rendering the image without anti-aliasing, which would allow you to do 4x the resolution horizontally and vertically.
I'm working right now on a new release now, which should hopefully be ready within the next few months. I don't plan to make Windows binaries for 3.6 at this point since a new version is imminent. For the next Windows release, I will offer a 64-bit binary which would allow more memory, but you're still limited by the amount of memory in your computer.
There are ways that we could split the image up and render each one to disk and then combine them, but that would be a lot more effort. I will change this issue to an enhancement to do disk-based rendering for large images, but I can't promise when I will get to that, if ever.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@kanurag94 Please check if some entries at https://stackoverflow.com/questions/3286186/what-c-library-allows-scaling-of-ginormous-images can be useful. Maybe libpng is not the way to go, I am unsure. |
@kanurag94 These are my results when experimenting with the example given by @hruyter:
|
Actually |
Thanks
I get a segfault always. Although rendering from file menu works.
For me this is working on xpf files inside xaos examples |
Not sure what you mean. If I try this: |
I meant using GUI for rendering is working. So I tried to find what's wrong while using cli Line 168 in 6df9acc
About the original issue: I get : Line 498 in 6df9acc
I am able to reproduce error message |
It seems XaoS needs a whole redesign to support big images that exceed memory limits. |
@kanurag94 I suspect overflow! Perhaps it is as simple as making bestprice a long instead of an int (and of course MAXPRICE = LONG_MAX)... but I suspect various other _int_s in that code would have to become _long_s! I will try this soon, but wanted to pass the idea on in case somebody else has time first. Also, many years ago I used jemalloc with XaoS because XaoS is a heavy malloc() user ... has anyone made that work lately? I have not been able to figure out the incantation to get it through qmake. |
@trathborne Thanks. I had this idea earlier, I have tried replacing best price int -> long and MAXPRICE to LONG_MAX. That didn't work. Probably replacing other _int_s may do the trick. I'll check.
This is interesting. I think jemalloc hasn't been tried out in XaoS earlier. I guess we may be able to run pre linking commands for export LD_PRELOAD = -jealloc -- this isn't neat though. |
@kanurag94 thanks for checking the LONG_MAX rabbithole. Good luck! As for jemalloc, I recall that vefore the Qt days it was just a matter of messing around in some .h and Makefiles. Today I started from https://github.com/jemalloc/jemalloc/wiki/Getting-Started and updated
should go into which QMAKE_ variable, and it looked like the options made it into the compile and link commands, but |
XaoS's ability to render extremely large images is limited by the amount of memory available on the computer. Rendering the image to disk incrementally would allow us to get around this limitiation.
Original Report
BACKGROUND: Trying to generate a XAOS poster, I stumbled over an xpf file, where XAOS 3.5 crashes under windows XP at an even quite low size. So far I had experienced error messages related to control buffer memory, when the desired resolution was above about 15.000 x 11.000 pixels. My desired size would be 42.520 x 30.324 pixels, taking partly advantage of the 2.880 x 1.440 dots per inch resolution of an Epson Stylus Pro 7890, for a XAOS poster of 75 x 50 cm.
STRANGE BEHAVIOUR: With the xpf file below, XAOS crashes with the render command, when the desired size is only 5.700 x 4.065 pixels or higher. (Up to size 5.600 x 3.993, rendering works fine, taking about 12 minutes, resulting in a 22 MB png file.)
QUESTIONS: Would those crashes and the Buffer error messages disappear wit XOS 3.6?
When could your XAOS fans expect XAOS 3.6 for Windows?
XPF FILE: This is the XPF file I wanted to generate a poster of:
;Position file automatically generated by XaoS 3.5
; - a realtime interactive fractal zoomer
;Use xaos -load to display it
(initstate)
(filter 'anti #t)
(filter 'palette #t)
(palette 2 7362 0)
(formula 'mandel)
(angle 242.865)
(maxiter 15730)
(view -1.98540931206269E -5.46966061929803E-08 5.36595249570754E-11 5.36595254454941E-11)
THANKS for all your efforts and the impressing results.
Hinrich Ruyter
The text was updated successfully, but these errors were encountered: