Confusing results coming from WBPP

17 replies880 views
Craig Dixon avatar
Attached is a screenshot showing two different stacks from WBPP, each stacked with exactly the same settings (including the same master dark and master flat dark). Each stack is from two different nights but the same scope, camera, filter, etc (Rig is in an observatory so nothing changed between nights).

Image on left: 51x 300 second lights - Night 1
Image on right: 65x 300 seconds lights - Night 2

Both images just have an STF applied - nothing more. Why is there such a difference in colour? Why is the noise considerably worse in the right image that has a longer integration time?

The flats on the left image also haven't calibrated properly, which is also concerning as I do flats every morning after a session.

I'm really confused about all of this. Is anyone able to shed any light on this?

TIA.
Well Written Engaging
Craig Dixon avatar
andrea tasselli avatar
First off, the image on the right cannot be compared with the one on the left. One has been properly calibrated and the other not. Secondly, I suspect that if both were calibrated properly the apparent noise would be in the same league. Ditto for color (beside that fact that neither has been properly color-calibrated). The question is really why the one on the left wouldn't calibrate.  Maybe parasitic lights nearby?
Helpful Insightful Concise
Craig Dixon avatar
That’s why it’s so confusing. WBPP ran in exactly the same way on both sets of data. The data is identical in that it was shot with the same gear from the same location, under the same conditions just two nights apart. WBPP produced no errors so I can’t figure out why there is such a difference.
Daniele Borsari avatar
Hi,

The difference in noise and colour is caused by the STF that stretches the right image more than the one on the left.

This is because the right image has a flatter background and the STF can afford to stretch more without clipping the black point. This results in a noisier image (but with more signal displayed) and in a green bias (due to the fact that colors aren't calibrated and that the green channel has probably more signal).

In the first night's image, the background isn't even (you may have had problems in shooting flats or you may have forgotten to add biases/dark flats in WBPP) and the STF can't do a more aggressive stretch without clipping the blacks.

Performing a background removal (DBE, GradientCorrection, Graxpert, etc. etc.) and then a STF on both masters will probably lead to very similar images.

This is my opinion according to my knowledge, I hope it helps.

Daniele
Helpful Respectful
Craig Dixon avatar
Okay, thanks. So it sounds like there is nothing to worry about but I need to figure out why the flats for the first image didn’t work properly then
Robert Žibreg avatar
It's auto stretch issue
Kevin Boucher avatar
Hi Craig,

 I'm in an airport waiting to come home after my solar eclipse trip to Austin but I agree with one of the other replies that it looks like the image on the left did not calibrate at all. So my question would be did you only create calibration frames for one of the two nights and if you did did you set up your grouping keywords so that it would know to find the calibration frames and apply them for both nights?

As I don't have Pixinsight  in front of me I can't give you any detailed instructions to check this out… but that's what I would pursue at this time. You should see this in your WBPP log.

Kevin
Helpful
Dan Kokinda avatar
Craig Dixon:
Okay, thanks. So it sounds like there is nothing to worry about but I need to figure out why the flats for the first image didn’t work properly then

I think you need to start there.  You can't make a comparison at all unless you work out any known discrepancies, which you know of at least this one.

Also, is it possible there were slight differences in the atmosphere, such as thin high clouds on the first night that may have diffused things?  Parasitic lights as someone else suggested?  But, as Daniele mentioned, because the images are different, STF will behave differently, possibly magnifying and discrepancies.
Adam Block avatar
Ok… so I have an educated guess that assuming facts not yet in evidence.
Is the masterflat that you used on BOTH evenings…was it constructed prior to the image on the right or from the session of the image on the right (which is better calibrated)? 

1. If yes, then your optics are changing… you are physically changing the conditions during acquisition so the flats will need to be taken matching each session. THIS IS TYPICAL unless you really touch nothing with a permanent setup.
2. If no, and you are using an OLD master flat and it correctly calibrated NEW data but not the older data… this is very hard to explain unless you really made adjustments in some non-expected way.

-adam
HR_Maurer avatar
I'm not the big expert in WBPP, but did you use local normalization, and if so - did you check the local normalization master?
Dustin Gazz avatar
It looks like 1 was taken before the meridian and the other after. If you did not rotate your camera or open your image train, use the same flats for both.
Well Written Concise
Clayton avatar
Over the last 2 night the seeing has been very different. Tuesday night for us on the East Coast of Oz we had clear skies, but really bad seeing, Wednesday night was awesome! My subs for both of those nights look like yours, the bad seeing night looks dim, good seeing everything is there.
Mike Sample avatar
Since your sessions are under the same conditions, I would not stack the two separately. The statistics work better when you feed it more frames. WBPP's weighting would not be as effective when stacking sessions separately. 

Personally, I never rely on STF other than judging what workflow is appropriate. In your case, I would run DynamicCrop, DBE, GradXpert, ImageSolver, SPCC, BlurXterminator, and NoiseXterminator. After that, I start stretching with HistogramTransfer stopping before saturating very many stars. I would then separate the stars from the target and further stretch the target with GeneralizedHyperbolicStretch. I use CurveTransformation to further stretch the stars.

I would not be concern with either result you posted, but you should be stacking all the lights in one WBPP pass.
Helpful
Craig Dixon avatar
Thanks for all of the replies. There is quite a lot to go through here so I'll expand on my initial post above. Have 10.5 hours on M109, shot over three nights. I initially stacked all of the data together, using groups on WBPP but noticed the calibration problems in the left image above. I stacked the data again separately by session to investigate the problem and discovered that the problems are coming from the data from night one.
Kevin Boucher:
Hi Craig,

 I'm in an airport waiting to come home after my solar eclipse trip to Austin but I agree with one of the other replies that it looks like the image on the left did not calibrate at all. So my question would be did you only create calibration frames for one of the two nights and if you did did you set up your grouping keywords so that it would know to find the calibration frames and apply them for both nights?

As I don't have Pixinsight  in front of me I can't give you any detailed instructions to check this out... but that's what I would pursue at this time. You should see this in your WBPP log.

Kevin

I used a master dark and master flat dark from my library but did take flats after each session and I did use grouping in WBPP to make sure the correct flats were applied to each night's data.
Dan Kokinda:
Craig Dixon:
Okay, thanks. So it sounds like there is nothing to worry about but I need to figure out why the flats for the first image didn’t work properly then

I think you need to start there.  You can't make a comparison at all unless you work out any known discrepancies, which you know of at least this one.

Also, is it possible there were slight differences in the atmosphere, such as thin high clouds on the first night that may have diffused things?  Parasitic lights as someone else suggested?  But, as Daniele mentioned, because the images are different, STF will behave differently, possibly magnifying and discrepancies.

The seeing wasn't great for each of the nights I imaged this target but I open all light frames in ASI FITS Viewer and manually delete any subs that have clouds or poor stars. This way, every sub that goes into WBPP has been manually visually inspected. Some frames will naturally have better signal than others due to passing loud and seeing but the worst frames are manually removed and my understanding is that WBPP then assigns weights to each sub to account for this.
Adam Block:
Ok... so I have an educated guess that assuming facts not yet in evidence.
Is the masterflat that you used on BOTH evenings...was it constructed prior to the image on the right or from the session of the image on the right (which is better calibrated)? 

1. If yes, then your optics are changing... you are physically changing the conditions during acquisition so the flats will need to be taken matching each session. THIS IS TYPICAL unless you really touch nothing with a permanent setup.
2. If no, and you are using an OLD master flat and it correctly calibrated NEW data but not the older data... this is very hard to explain unless you really made adjustments in some non-expected way.

-Adam

I took flat 20 frames for each session using an LED flat panel and the auto flats feature in ASIAir. My setup is in an observatory so after night one, I removed the dew shield, put the LED panel on top of the scope, took my flats, replaced the dew shield and then moved the scope to it's parked position and closed the roof. The next night, I opened the roof, turned the power on and ran the same plan from ASIAir again. None of the imaging train was changed so it shouldn't be necessary to do new flats between sessions but I do this as a matter of caution. The only change in optics would be the very slight change in focus between sessions as well as throughout the imaging sessions (Using a ZWO EAF set to refocus every hour or ever 2 degrees Celsius change in temperature). My understanding is that very slight changes in focus shouldn't cause problems with flats.
I'm not the big expert in WBPP, but did you use local normalization, and if so - did you check the local normalization master?

Local normalisation was used in WBPP but I'm not sure how to check the master file. The log doesn't show any errors here. When I use WBPP, I load up all the lights, flats, master dark and master flat dark, then group the images by night using the date. I select the "Maximum quality with no compromised" preset, check the diagnostics button and then run the stack.
Dustin Gazz:
It looks like 1 was taken before the meridian and the other after. If you did not rotate your camera or open your image train, use the same flats for both.

Both imaging sessions were started at roughly the same time and included a meridian flip about 2 hours in. A lot of pre-flip subs might have been deleted from that stack due to cloud so that might be why WBPP has outputted the file 180 degrees flipped but WBPP handles the orientation of pre and post-flipped images well. Either way, the rotation of the camera relative to the scope isn't changed so the flats would still be in the same orientation in relation to any dust/vignetting.

Mike Sample:
Since your sessions are under the same conditions, I would not stack the two separately. The statistics work better when you feed it more frames. WBPP's weighting would not be as effective when stacking sessions separately. 

Personally, I never rely on STF other than judging what workflow is appropriate. In your case, I would run DynamicCrop, DBE, GradXpert, ImageSolver, SPCC, BlurXterminator, and NoiseXterminator. After that, I start stretching with HistogramTransfer stopping before saturating very many stars. I would then separate the stars from the target and further stretch the target with GeneralizedHyperbolicStretch. I use CurveTransformation to further stretch the stars.

I would not be concern with either result you posted, but you should be stacking all the lights in one WBPP pass.

I've only stacked the two nights separately to troubleshoot and to find find out which data set is causing the problems. The intention is to figure this issue out and stack all three nights together. I agrees with your workflow but at the moment, I'm just relying on an STF to check the data for issues before I move on. applying an STF has allowed me to see some calibration problems, which need fixing first.
Helpful
Craig Dixon avatar
Interestingly, I just stacked the data from night one using the flats from night 2 and it's looking absolutely fine. This is great in that I've been able to fix the problem but it does leave me wondering what the problem actually was and how I can avoid it in the future. The two sets of flats should have been identical as the imaging train hadn't changed.

 I also wonder if this is related to my other post here:

https://www.astrobin.com/forum/c/astrophotography/deep-sky-processing-techniques/odd-error-with-flats-and-possible-rotation/
Well Written Engaging
andrea tasselli avatar
Obviously it did change. Likely the way you take flats otherwise the 2nd night's flats wouldn't have been useful for the 1st night's lights.
James avatar
Your image on the left looks like it didn't have flats applied.


Maybe you missed a setting.  For example, if this was taken with a osc (asi533mc?) then the "cfa images" check box for the flats on the calibration tab needs to be checked.  If it's not checked, the flats will be ignored.  I forget to enable that check box all the time. 

Glad you got the issue sorted out.
Well Written Helpful Insightful Respectful Concise Supportive
Related discussions
Please advise regarding C8 Flats not calibrating dust motes
Hi everyone Yesterday night I imaged M63. 2600MC Pro, Celestron C8 with 0.63x reducer, OAG-L and 174MM Mini All using NINA. Flats taken with my regular Geoptik Manual Flat Panel. Lights: 85 subs, 300s, Gain 100, Offset 20, Temp -5C I took my set of F...
Master flat dark issues could cause color differences between calibrated stacks.
7 days ago
Darks, Flats, Bias created with Voyager, Lights created with SGP WBPP rejects with the error....
No matching Master Bias, Master Dark and Master Flat found. The following image parameters are all the same. QHY600M Mode set to Mode_3 ( 'Extend Fullwell 2CMS') Binning Exposure length Gain Offset Filter The only significant difference I can...
WBPP stacking with master dark/flat frames; relevant to calibration frame matching.
Jul 31, 2022
Flats adding to images instead of subtracting in PixInsight WBPP
Astro Friends, I have been imaging for about 5 years and have been successful using refractors and mono imaging. I have branched out this year and now have an SCT and mono setup. I am seeking help with my flats which seem to add themselves to my imag...
Flat calibration problems in WBPP can produce unexpected color variations between sessions.
Jan 2, 2026
What would cause flats to not correct properly?
Pix Insight WBPPQHY600m – CDK14 + ReducerFilters are 50mm with filter masks, the Astonomiks even have a built-in mask (plus additional plastic masks).Chroma filters have the same patternSpike-a-flat panelThere are no light leaks - daytime darks = nig...
WBPP confusing results; author using WBPP for calibration troubleshooting.
May 23, 2022