Pixinsight Error with Image Integration WBPP/FBPP

Was there any error message displayed in the console, which might give a clue to the exact problem? It's hard to guess without that. Here are a few thoughts:

1. Check if there is enough memory & disk space.

2. Use Blink to look at the registered frames, maybe something went wrong during calibration (as you said there were a few lights freezing the process?).

3. Also, look through the file list of the registered frames and check, if some have an impossible low size (almost empty->broken file).

4. Try running ImageIntegration on the registered frames, to see if the problem occurs again.

Derek Vasselin:
Do any software-savvy people out there know why Pixinsight WBPP and FBPP (tried with both) fail during the image integration phase?

I can't figure this out and I don't know where to even start looking. I'm using raw files for flats, lights, darks, and biases.

All my lights look fine. I'm grouping them by night to match flats associated with each night.

The whole process works fine up until the integration part. It gets hung up there for hours (like 6+) and eventually fails.

Looks like there might be 5 lights that were holding it up, but I have no idea which or what the issue is? Can't find any data on this.

I managed to run WBPP with other targets just fine, so not sure what is different here.

Here is the process log: ProcessLogger.txt

Here is the exact part of the log I'm having issues:


******************** IMAGE INTEGRATION ********************
Group of 271 Light frames (271 active)
SIZE  : 6248x4176
BINNING  : 1
Filter   : NoFilter
Exposure : 300.00s
Keywords :
Mode     : post-calibration
Color    : RGB

Rejection method auto-selected: Generalized Extreme Studentized Deviate
*** Warning: ImageIntegration failed.
!!! Error: Warning: Master Light file was not generated.
***********************************************************

Regarding the problem you are raising, I have not been given the case but I think you should report this possible BUG to the PIXINSIGHT developers within the pix forum, I am sure you will receive a response regarding it.

WBPP produces logs which should show everything that would have flashed by in the Process Console. That should tell you what went wrong. It is very unusual for something to go wrong in the integration step, though.  Usually, if something is going to go wrong it happens earlier in the process.

As to the pedestal of 47 and dark over subtraction... 

Needing a pedestal to be added is NOT an indication of dark over subtraction, or at least not an indication that you have done anything wrong. Rather, it is an indication that PixInsight is preventing over subtraction. There is always some variability from one pixel to the next in an individual light frame. This is due to the combination of thermal noise, read noise, and shot noise. In most modern cameras, thermal noise is extremely low, often less than one electron per pixel in even a multi minute exposure. Read noise is likewise quite low in a modern camera, often on the order of just three or four ADU. As a result, shot noise is often the largest noise component. That's variability in the number of photons actually coming from a given bit of sky in a given exposure. Especially if you are shooting narrow band, such that sky glow is heavily suppressed, you might be looking at quite limited signal from "empty" regions of the sky. While most pixels will have at least some signal from the sky after a minute or two of exposure, the poisson distribution of light can easily allow for some pixels that receive essentially no light at all on a given sub. Once you subtract out the master dark, it is not weird for some number of pixels to actually be driven negative just due to random variations. Keep in mind you have many millions of pixels, so there will actually be a handful that are five or more sigma outside the normal range.  To me, a pedestal of 47 being added in suggests either your sub exposures might be a touch short for your sky conditions, or that your pedestal is just a bit low, but it isn't an indication of a problem. I tend to see pedestals of 0-3 ADU added on my broadband images and pedestals of 0-20 added in on my narrowband images. The fact that PixInsight needs to make this adjustment is, at most, an indication of a very, very minor problem. Honestly, I don't see it as a problem at all, though. PixInsight keeps track of the added pedestal, so it doesn't throw anything off in the arithmetic. A pedestal added by PI is really no different than just raising your offset a tiny bit. The only real issue would be if you were using a software application that didn't make this sort of adjustment. Certainly, the fact that some of your pixels would be driven to zero without an offset would have nothing to do with image integration failing.

Check your logs to see what error was reported. That will help us diagnose.

- Jared

8 GBs is really few for PixInsight...I'm surprised you didn't have problems earlier, I have 16 and my Mac is close to dying every WBPP run.
You can look here and here for tips.