Back to forums

Help Me Understand Why So Many Rejected Frames

Pleiades Astrophoto PixInsight Processing OIII Deep Sky Sh2-2
16 replies•772 views
Robert Khoury avatar
Robert Khoury avatar

Apologies for the bad phone screenshot. I was stacking my Oiii frames for Thor's helmet and noticed more than half of my subs were rejected in WBPP. I use the same settings based off an Adam Block tutorial I watched last year and usually on average will have between 1-3 subs rejected. I'm not sure what is happening here.

I manually check each sub for issues then run the stack through Sub Frame Selector and remove subs outside the averages for FWHM, eccentricity, stars and median (even though I dont fully understand this one to be honest). 

I took half this stack in December with temps in the 20's-30's and the other half last night where it got into the 10's at some point. Not sure if the reference frame was so strong (possibly one from December where seeing was excellent) its throwing most of this stack out from last night as seeing was not great until around midnight. 

I did process the image with ~60 Ha frames and ~20ish Oiii subs and it came out ok but definitely feel like it's not as great as I would have hoped. Our weather isn't looking too awesome over the next month so felt last night was my chance to finish this project. I'm functional in Pixinsight but by no means super knowledgeable about the ins and out of the software.
Helpful Engaging
Christian Großmann avatar
Christian Großmann avatar
Hi,

I'm not sure I can help with this. But I once had a similar problem with Deep Sky Stacker a long time ago. The reason for my problem was the contrast of the images. Because OIII often emits a very faint amount of light, the subs of my target were much darker than the usual subframes of other objects. If the captured contrast is too low, I can imagine that PI has some problems (as DSS did back then). It somehow depends on your target and your exposure time. My solution was to increase the exposure time (or maybe the gain as well) and everything worked fine. Maybe it's worth to look at the histogram to see, how far left the image content is compared to other OIII subs of maybe another target.

I'm not sure if this is helpful to you, but at least it's worth to have a look.

CS

Christian
Helpful Supportive
Jure Menart avatar
Jure Menart avatar
Hi @Robert Khoury

Can you check the bad frames rejection section in Log and maybe publish it here? There you should see the normalized weights (even though you said you already checked with Subframe Selector). Once it happened to me that all the weights of rejected frames were simply 0 (but images were OK in Subframe selector). But even if it's not zero it might give you (or forum here) some clues?

I don't exactly understand what happened to my lights back then (as I am always using the same gain and offset in camera), it seems that WBPP had some issues and calibration failed (I think it was one full set of frames for one filter, but I can't remember details, maybe it was set of frames of one night?). I solved it with setting the 'output pedestal setting' to some manual value (can't remember exactly the value) as auto value seems to cause issues during calibration:


Hope this helps a little bit, keep us updated
Supportive
Cody Looman avatar
Cody Looman avatar
I recently processed some data from Thor's helmet aswell and experienced the same issue. 47 out 130 frames in the Oiii filter were rejected. On visual inspection all of those frames seemed fine. I'll have to rerun them and tweak some settings in WBPP. I'll let you know if I have any success.
Concise Supportive
Jan Erik Vallestad avatar
Jan Erik Vallestad avatar
Often there are subtle differences, it can be hard to tell by looking at the images without properly measuring them with subframeselector or checking the logs to see why they were excluded. 

You could possibly tune WBPP in order to let the lesser frames be stacked as well, or simply stack them manually in Pixinsight. I'm not 100% sure but if you add better data along the way sometimes WBPP can exclude new frames as well. At least that's what I've found lately. Maybe the FWHM (or one of the many other variables) is varying too much?

If you can't see any explanation in the logs I would at least load the subs here:
Michael Stevenson avatar
Michael Stevenson avatar
Like others here mentioned, I also had this issue and the frames looked fine. I read about a workaround that worked for me in this instance - change the minimum weight in the integration parameters for lights to zero. I don't know if this is the correct way, but it works.
Robert Khoury avatar
Robert Khoury avatar
Christian Großmann:
Hi,

I'm not sure I can help with this. But I once had a similar problem with Deep Sky Stacker a long time ago. The reason for my problem was the contrast of the images. Because OIII often emits a very faint amount of light, the subs of my target were much darker than the usual subframes of other objects. If the captured contrast is too low, I can imagine that PI has some problems (as DSS did back then). It somehow depends on your target and your exposure time. My solution was to increase the exposure time (or maybe the gain as well) and everything worked fine. Maybe it's worth to look at the histogram to see, how far left the image content is compared to other OIII subs of maybe another target.

I'm not sure if this is helpful to you, but at least it's worth to have a look.

CS

Christian

Thank you Chrisitan for the insight. From my initial screening the frames looked very similar and the nebula showed up quite nicely. But I agree, should probably check the histograms and see if there were any issues. I've stuck with Unity gain and 300 second subs for nebula imaged from my Bortle 5 backyard and this really is the first time I've noticed such a huge rejection in WBPP

CS

Robert
Robert Khoury avatar
Robert Khoury avatar
Jure Menart:
Hi @Robert Khoury

Can you check the bad frames rejection section in Log and maybe publish it here? There you should see the normalized weights (even though you said you already checked with Subframe Selector). Once it happened to me that all the weights of rejected frames were simply 0 (but images were OK in Subframe selector). But even if it's not zero it might give you (or forum here) some clues?

I don't exactly understand what happened to my lights back then (as I am always using the same gain and offset in camera), it seems that WBPP had some issues and calibration failed (I think it was one full set of frames for one filter, but I can't remember details, maybe it was set of frames of one night?). I solved it with setting the 'output pedestal setting' to some manual value (can't remember exactly the value) as auto value seems to cause issues during calibration:


Hope this helps a little bit, keep us updated

*Thank you Jure, this would be the log file found after stacking correct? I can look through it some more and see if I can find any info and get it uploaded here later when I get home.

I do utilize pedestals when stacking, I generally stick with a number of 100 and was still getting the rejections. Maybe I could try auto and see what happens. I also use the same offset and gain settings
Robert Khoury avatar
Robert Khoury avatar
Cody Looman:
I recently processed some data from Thor's helmet aswell and experienced the same issue. 47 out 130 frames in the Oiii filter were rejected. On visual inspection all of those frames seemed fine. I'll have to rerun them and tweak some settings in WBPP. I'll let you know if I have any success.

Yes please do!
Robert Khoury avatar
Robert Khoury avatar
Michael Stevenson:
Like others here mentioned, I also had this issue and the frames looked fine. I read about a workaround that worked for me in this instance - change the minimum weight in the integration parameters for lights to zero. I don't know if this is the correct way, but it works.

That crossed my mind as well as I set it to 0.5. I'll give it a go and see what happens. Thank you!
Well Written
Robert Khoury avatar
Robert Khoury avatar
Often there are subtle differences, it can be hard to tell by looking at the images without properly measuring them with subframeselector or checking the logs to see why they were excluded. 

You could possibly tune WBPP in order to let the lesser frames be stacked as well, or simply stack them manually in Pixinsight. I'm not 100% sure but if you add better data along the way sometimes WBPP can exclude new frames as well. At least that's what I've found lately. Maybe the FWHM (or one of the many other variables) is varying too much?

If you can't see any explanation in the logs I would at least load the subs here:

Thank you Jan, I'll probably tweak the minimum weight setting as I set it to 0.5 for my stacking. I'm under the assumption the frames I captured in December are that much better than January as seeing or some other factor was probably better then.
Michael Stevenson avatar
Michael Stevenson avatar
Robert Khoury:
That crossed my mind as well as I set it to 0.5. I'll give it a go and see what happens. Thank you!


I think that .5 is VERY high, I believe it defaults to .05.
Robert Khoury avatar
Robert Khoury avatar
Michael Stevenson:
Robert Khoury:
That crossed my mind as well as I set it to 0.5. I'll give it a go and see what happens. Thank you!


I think that .5 is VERY high, I believe it defaults to .05.

Crap ok maybe too aggressive then for this data. I think this may potentially be the core problem then.
Robert Khoury avatar
Robert Khoury avatar
Alright everyone, ran the stack again but with a minimum weight of 0.05 vs 0.5 under image integration and ZERO rejected frames. I was too aggressive for this data. Thank you all for taking time to respond and offer insight. I've learned more about WBPP from this experience. I was reliant on tutorial videos and really need to sit down and learn more about the software (one of these days, lucky to even image with little ones running around but hey, at least the toddler is showing interest in space and the hobby 😉).
Respectful Engaging Supportive
Jure Menart avatar
Jure Menart avatar
*Thank you Jure, this would be the log file found after stacking correct? I can look through it some more and see if I can find any info and get it uploaded here later when I get home.

* Hi Robert,

Great to hear you find it out and works now

To answer about your log for next time (or somebody else) - WBPP script produces folder 'logs' parallel to calibrated, master and registered folders in the folder you set as output in WBPP script. In the logs folder you have log file(s) based on date/timestamp when you run the process (see example below) and is generally very informative about all steps of the WBPP script. You have also section about frame rejection (you can see example with some frames accepted and some rejected, the normalized weight and the limit for rejection - default is set to 0.050 as you also set it now ):
Helpful
scottdevine avatar
scottdevine avatar
Robert Khoury:
Alright everyone, ran the stack again but with a minimum weight of 0.05 vs 0.5 under image integration and ZERO rejected frames. I was too aggressive for this data. Thank you all for taking time to respond and offer insight. I've learned more about WBPP from this experience. I was reliant on tutorial videos and really need to sit down and learn more about the software (one of these days, lucky to even image with little ones running around but hey, at least the toddler is showing interest in space and the hobby 😉).



Robert, 

I'm glad you asked and got all these replies. I had a similar issue and was able to resolve based upon this post.  It's the great thing about this community. Lots of advice. Thank you!
Well Written Respectful Supportive
Robert Khoury avatar
Robert Khoury avatar
Robert Khoury:
Alright everyone, ran the stack again but with a minimum weight of 0.05 vs 0.5 under image integration and ZERO rejected frames. I was too aggressive for this data. Thank you all for taking time to respond and offer insight. I've learned more about WBPP from this experience. I was reliant on tutorial videos and really need to sit down and learn more about the software (one of these days, lucky to even image with little ones running around but hey, at least the toddler is showing interest in space and the hobby 😉).



Robert, 

I'm glad you asked and got all these replies. I had a similar issue and was able to resolve based upon this post.  It's the great thing about this community. Lots of advice. Thank you!

I'm very glad this thread was able to help you! It was such a mystery at the time but in hindsight made complete sense as to what was happening. 

CS - Robert
Well Written Respectful Supportive