Pixinsight file sizes?

Well the 1st thing is the processed files immediately grow from 16bit to 32bit. You really do not want to change that back to 16bit or your combined results are going to really suffer.  At least 32bit is really needed to be able to properly render the color or grayscxale depth. Then the 2x drizzle doubles once again!

I use a qhy294mono pro and raw files are 4164x2795 unbinned.  That is 27.745 MB or 181.96 mb. Processed subs grow to 45.487MB. Drizzle that 2X and there you are at 91MB.

Simple math.  Drizzle 2x2 and you increase size 4x.  After doing all of your processing you can rescale the image to get it back under control size wise.  I usually do a 50% rescale before uploading to Astrobin.  That gets the image back under 25MB.

That is exactly how it works. 2x drizzle quadruples to file size. Have you considered deleting the Calibration and registraton files I suspect 99.9% of most mortal users never need. These are the system generated sub-folders under the working folder that I use a script to delete immediately after the WBP completes. 
Calibrated
Registered
cosmetized
Logs
There also a lot of superfluous files created in the master folder alongside the actual drizzle files you actually want to work with e.g. LN_* etc
I use a script to move the ones I do not want to a JUNK folder on another drive and rename the ones I want to something much shorter.
e.g. 
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono_drizzle_2x_autocrop.xisf 
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono_drizzle_2x_autocrop.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono_drizzle_2x_autocrop.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono_autocrop.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono_autocrop.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono_autocrop.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono_drizzle_2x.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono_drizzle_2x.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono_drizzle_2x.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono.xisf
LN_Reference_Light_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono.xisf
LN_Reference_Light_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono.xisf
LN_Reference_Light_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono.xisf

masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Red_mono_drizzle_2x_autocrop.xisf  -> RED.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Green_mono_drizzle_2x_autocrop.xisf -> GREEN.xisf
masterLight_BIN-1_3388x2712_EXPOSURE-180.00s_FILTER-Blue_mono_drizzle_2x_autocrop.xisf -> BLUE.xisf
Also when you open those files they contain an unwanted weighting component I a pretty sure most people have no use for other than to perhaps direct cropping.  So I close the extra weightings files and then use this in the Process Console to just save. The XISF files  size halves.

save * –nomessages –noverify

Another neat trick is to use an Image Container that contains a list of open views by their Image Identifier and drag that list of images to a saved instance of the ViewID process that has been configured to change the long winded Indentifier to something more obvious and readable that is 100% consistent with the new file name. That Image Container can also be dropped on a Gradient Correction (*new) or GraxPert process icon to quickly get those jobs done. My next step is to repeat for BlurXTerminator for Star Correction (only) and I only use Non Stellar correction later on the combined image.

var P = new Script;
P.filePath = "$PXI_SRCDIR/scripts/mschuster/ViewIdReplace/ViewIdReplace.js";
P.md5sum = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
P.parameters = [ // id, value
   ["version", "1.4"],
   ["pattern", "_integration_autocrop"],
   ["replacement", ""]
];
P.information = "";

All this does is save a huge amount of time and diskspace.
Seriously someone should write a Cleanup script or they should add an option to the WBP to allow these extra disk hogs to be automatically removed.
I am an Excel VBA developer so I use that. 
There may be users who like to dive down the rabbit hole of logs and calibration files after WBP but I suspect for most users they just want to get stuck into the end results and get on with it. Maybe someone has shares in drive manufacturers or has a panic attack if those files get deleted but there is allways therapy available.

Good Luck
Cheers
Ian

You also might be saving the rejection maps as part of the masters.  Check the settings

Hi Sean
My .fits subframes are each ~120meg (16 bits using a ASI 6200mm camera which is 62 megapixel binning 1x). normally integrate the images  and save them as 32bit floating point .xsif files which come out to ~239meg each.  When I combine the integrated LRGB frames they can get large  ~ 720meg each.  So it does use a lot of space.  Are you setting the 64 bit option in the integration as this would double the size.  When you use 2x drizzle that will quadruple the image size which is what is probably what is causing the size increase.  

Since I keep most of the calibrated, Registered and integrated files   I  will eat up 50 to 100gig  or more per image. So In my case I bought an 8 port NAS server and put 8 eight tera byte drives into it but keep 1 on hot standby and  use the NAS in RAID 6 for double redundancy so get around 40 terabytes available.  I also keep separate backups on regular 18 terabyte drives which I purchased when they went on sale.

Jon Rista:
You might need 64-bit float...it depends on how many subs you are stacking. For a master dark, I would think 32-bit would be fine, but for deep light stacks, sometimes you want to use 64-bit. But that would only apply to the final stack, not the intermediates...for most of the intermediates you'll generally want to use 32-bit float.

I was wondering which criteria can be used to estimate when it's better to switch to 64-bit.
Would I be able to see it in the streched image? Effects like posterisation?

It is highly doubtful you would need to use 64-bit float outside of teh final integration itself. Your intermediates shouldn't need to be 64-bit. 

As for how to know when to switch to 64-bit... Well, first, as far as I gathre now, WBPP seems to use 64-bit by default, which IMO is a mistake. That would indeed balloon data file size by 4x from 16-bit raw FITS files or whatever source data you have. (Drizzling is NOT needed here to make the intermediates 4x larger, all that is needed is 64-bit float!!)

I used to have a formula for calculating when I was stacking enough subs to warrant 64-bit. Its been a while and I don't remember it off the top of my head. I usually switched once I got up to a couple hundred subs or more.

Another reason to switch to 64-bit is if you are doing HDR processing, and combining short exposures with longer exposures. You'll most likely need 64-bit in those cases. I do know that PI now supports these PSF based weighting algorithms, and I don't yet know enough about them to know if that might do some kind of HDR-like scaling. If so, then you might need 64-bit if you use that as well.

Key thing though, is you really should NOT need 64-bit files for everything...only the final integration, unless you are using a ton of files for your masters.

How do i set wbpp bit depth?  I can’t even figure out how to check what it’s set to. 

last integration just crashed most of the way through. 8 hours in. Task manager wouldn’t even end the pixinsight task :/ stupid windows 11. 

i have 160 frames of 91mb each. Masters (without drizzle) are coming out to 360mb each.  It seems excessive. I have an i71300k with 32g ram and 3 ssd drives.

Yeah i have a 4070ti and cuda enabled. It really helps with blurx. 

i still don’t know why my masters are so big.

Looks like it’s using a 32bit float wile running ¯\_(ツ)_/¯

It’s actually a 14bit sensor. 

asi294mm pro bin1

fits are 91mb each. 

master lights (and darks and flats) are 365mb each. 

i reset wbpp to default and it’s still the same. No drizzle. 

¯\_(ツ)_/¯

Sean Mc,

The 294MM is a 47MP camera which, at 2bytes (16 bits) per pixel, should give you over 90MBytes per .fits file. Regardless of the dynamic range of the image itself (you may have selected gain to give you a 12 bit or 14 bit image for instance), whatever computer is controlling your rig will create a 16 bit file. That's the size of file that computers are designed to operate with.

I have a 294MC Pro which gives me a resolution of 4144*2822 (bin 2), and an image size of 11.7MP -  comes out at just over 22 MBytes. But in your case with the MM version you must be using the unlocked bin1 mode which gives you a resolution of 8288*5644 pixels....right? Hence your file size is 4 times my file size.

Pixinsight increases your image dynamic range from 16 bits to 32 bits so that when stretching and editing your images you get better resolution and avoid problems like "posterization" of your data. That doubles the size of your files. So, expect your .xisf files to be more than ~180MBytes. You can configure PI to create 64 bit images if you feel your target needs even higher dynamic range (like the Orion Nebula for instance) - in that case you'd end up with ~360Mbyte files. Make sure you don't have "Create 64 bit image" selected in WBPP.

Then, if you drizzle, that quadruples the size of your file. Question - do you need to use drizzle? Drizzle only helps you when you are undersampling. By that I mean you are "undersampling" if your sensor pixel size is bigger than your scope optical resolution. Since the 294MM in bin1 mode pixel size is 2.315um (very small) you might find that drizzle doesn't yield any benefit to your image. This will save you quite a lot of memory if you stop using drizzle which can also be turned off in WBPP. FYI, James Lamb has made a great video which explains optical versus sensor resolution: (1091) Telescope Resolution, FWHM, and HFR - YouTube

Finally, if you have selected "Create Rejection Files" in WBPP Integration settings then your masters will increase in size by a factor of 3. 

If you are really short of computer memory then consider using your camera in bin2 mode to reduce the raw file size, and make sure that "Create 64 bit images" isn't ticked, untick "Create RejectionFiles" and stop using drizzle. Delete intermediate files of old image processing and consider getting yourself some sort of archiving system to free up space.

For storing data I personally only keep the raw files and delete everything else once I have finished processing an image. Before starting image processing I always check that I have 200GB of memory available. I use two 8 TB regular disk drives for archiving to clear files off my fast M2 SSD drives.

I hope this helps!! Good Luck

Chris Ashford