why did my flats not correct this gradient?

I've just processed a session in Astro Pixel Processor, but despite calibrating with flats, I still ended up with a HUGE gradient. Any ideas as to why?

Here is the stacked result and the master flat

What camera do you use?
A ZWO ASI2600MCpro? When yes ->Have you bought the camera in 2021/2022?
Have a look at this forum thread: https://www.astrobin.com/forum/c/astrophotography/other/asi-2600-mc-pro-bright-white-circle-in-subs/

Stuart Taylor:
I've just processed a session in Astro Pixel Processor, but despite calibrating with flats, I still ended up with a HUGE gradient. Any ideas as to why?

Here is the stacked result and the master flat

Can you send me your light frames?
I´ll have a look at them...

Bernhard Suntinger:
What camera do you use?
A ZWO ASI2600MCpro? When yes ->Have you bought the camera in 2021/2022?
Have a look at this forum thread: https://www.astrobin.com/forum/c/astrophotography/other/asi-2600-mc-pro-bright-white-circle-in-subs/

Interesting. Yes, I purchased the 2600MC Pro last year. I am aware of the potential for oil leak (there has been a discussion on Cloudy Nights), but I don't see any oil leak in my camera.

fyi all frames were shot at -15C and binned 2x2 - I also used an Optolong L Pro filter

I'll upload the lights someplace and message you. Thanks

I'm having a similar issue with my new setup right now as well. It's an AstroTech 115 triplet with the ASI294mono camera and Astronomik Deep Sky LRGB filters. At first I thought it was dew, but it doesn't happen with the NB Astronomik filters in the same EFW, or when I switched out the filters for my original ZWO set. I'm in the process now of collecting some sample images for the tech support at Highpoint, but as of now we still don't have an answer.

Bernhard Suntinger:
Can you send me your light frames?
I´ll have a look at them...

I can't seem to find a personal message function in Astrobin. Is there a way I can send you a link to my lights?

You can send me the link.

I had a similar problem with a 135mm f2 lens.

What is the exposure time of your flats? There is an interesting series of videos by Adam Block where he deals with overcorrections of flats happening. He mentions that also undercorrections (as in my and probably your case) may occur and he claims that very short exposure times for flat files with CMOS sensors may introduce such errors. Adam recommends that exposure times for flat files should be well above one second to avoid potential issues.

For details watch these videos:

Do Your CMOS Images Look like THIS?? (Part 1) - YouTube
Do Your CMOS Images Look like THIS?? (Part 2) - YouTube
Do Your CMOS Images Look like THIS?? (Part 3) - YouTube

Good luck and clear skies
Wolfgang

I had a similar problem with a 135mm f2 lens.

What is the exposure time of your flats? There is an interesting series of videos by Adam Block where he deals with overcorrections of flats happening. He mentions that also undercorrections (as in my and probably your case) may occur and he claims that very short exposure times for flat files with CMOS sensors may introduce such errors. Adam recommends that exposure times for flat files should be well above one second to avoid potential issues.

For details watch these videos:

Do Your CMOS Images Look like THIS?? (Part 1) - YouTube
Do Your CMOS Images Look like THIS?? (Part 2) - YouTube
Do Your CMOS Images Look like THIS?? (Part 3) - YouTube

Good luck and clear skies
Wolfgang

this is interesting. I did not know this. However, I just checked my flats and the exposure time was 2.63s (I try to keep the brightness of my flats panel low-ish). But this is useful to know! Thanks

Stuart Taylor:
Bernhard Suntinger:
Can you send me your light frames?
I´ll have a look at them...

I can't seem to find a personal message function in Astrobin. Is there a way I can send you a link to my lights?

You find it here:

Clear skies
Wolfgang

so this is the final processed image from this sequence. It took a lot of work to remove the gradient.

Not great result, but ok.

Pinwheel Galaxy with 20cm SCT

Stuart Taylor:
this is interesting. I did not know this. However, I just checked my flats and the exposure time was 2.63s (I try to keep the brightness of my flats panel low-ish). But this is useful to know! Thanks

Not sure it's true. My flats are significantly shorter than 1s and never had the issue. The issue is in the use of flat panels.

These are your stakked images (without flat correction).
I applied Dynamic Background Extraction (DBE) in PixInsight and did extreme histrogram-stretching with ScreenTransferFunction.
->The result seems rather to be a problem with your reducer distance or flatframes.
I think that the dark circle in the middle of the image comes from the circumstance that it isn´t possible to set allignment points in the middle of the spiral galaxy.
-->And so the calculated backgroundlevel in DBE differs to the reality.

I recently ran into a similar problem with APP and the reason was kind of surprising. When I took the flats (and darkflats) after the session I forgot to give APP the filter information. Thus, the fits header was not containing my l-enhance filter. When loading lights and flats in APP, it notices this information and does not assign those flats to the lights automatically, because it seems to not be a match from the fits header. So you have to choose "use for all channels" in the pop-up window when loading your flats and all other correction frames.

andrea tasselli:
Stuart Taylor:
this is interesting. I did not know this. However, I just checked my flats and the exposure time was 2.63s (I try to keep the brightness of my flats panel low-ish). But this is useful to know! Thanks

Not sure it's true. My flats are significantly shorter than 1s and never had the issue. The issue is in the use of flat panels.

I would be reluctant to challenge Adam Blocks statements. He did not claim that the problem must occur for any Camera/lens combination. The statement is that it can occur.

Clear skies
Wolfgang

andrea tasselli:
Stuart Taylor:
this is interesting. I did not know this. However, I just checked my flats and the exposure time was 2.63s (I try to keep the brightness of my flats panel low-ish). But this is useful to know! Thanks

Not sure it's true. My flats are significantly shorter than 1s and never had the issue. The issue is in the use of flat panels.

I would be reluctant to challenge Adam Blocks statements. He did not claim that the problem must occur for any Camera/lens combination. The statement is that it can occur.

Clear skies
Wolfgang

This is not a problem with the IMX571 sensor of the ASI2600MC. Bias there is stable with short exposures and the sensor has a big linear range.

It is known to occur with the Panasonic sensor of e. g. the ASI1600 and with the IMX294 due to unstable bias in short exposures. It may be an issue in the IMX492, too, but I have no problems taking well-correcting sky flats with that sensor and exposures <1s and using bias. In the IMX183 the huge amp glow will show up if you try bias correction for flats of several second lengths, so here you need either short exposures or flat darks. So, in general, Adam Block’s advice is sound: Use flat darks and 2-4s exposures for your flats.

As to the OP’s problem:
I’ld first try to calibrate and stack manually if you have access to a software that lets you do it. I can also do that if you give me access to the data. If that doesn’t work I’ld suggest a light leak around the filter or from the back of the filter wheel/drawer.

Bernhard Suntinger:
These are your stakked images (without flat correction).

Thanks for trying this Berhnard.

Björn Hoffmann:
I recently ran into a similar problem with APP and the reason was kind of surprising. When I took the flats (and darkflats) after the session I forgot to give APP the filter information. Thus, the fits header was not containing my l-enhance filter. When loading lights and flats in APP, it notices this information and does not assign those flats to the lights automatically, because it seems to not be a match from the fits header. So you have to choose "use for all channels" in the pop-up window when loading your flats and all other correction frames.

This is interesting. But I don't think it would apply to my situation because I am shooting with OSC and a L Pro filter (which was kept on for flats also). So all stacking calibration etc was with exactly the same optical train.

Torben van Hees:
I can also do that if you give me access to the data.

Thanks. I'll message you with the link

Stuart Taylor:
This is interesting. But I don't think it would apply to my situation because I am shooting with OSC and a L Pro filter (which was kept on for flats also). So all stacking calibration etc was with exactly the same optical train.

It's not about your optical train. I meant telling the NINA flat assistant that there is a filter at all. Example: You take your lights with the L-Pro. Then this information will be stored in the fits header. When you then take your flats, but forget to tell NINA that there is the L-Pro, this information will not be written into the fits header. APP later thinks that the files don't match and it will not assign the master flat to your lights, if you don't force it to do so.

Björn Hoffmann:
You take your lights with the L-Pro. Then this information will be stored in the fits header.

I don't think there is any info about filter in my FITS headers. There is no way the camera could know this. I simply screw the filter into the focal reducer

I found the reason for the failure: You have different acquisition parameters for your lights and calibration frames: The lights are binx2, CFA, the master dark is binx1 no CFA, and the flats and bias are binx1, CFA. I've calibrated a few of your lights with a bias and a flat resampled to the correct size and they calibrate fine. The thing is to take new flats, bias and darks with the same parameters as your lights, and then calibrate again. Here's a calibrated master from about 10 frames.

masterLight_BIN-2_EXPOSURE-35.00s_FILTER-NoFilter_RGB_1.jpg

Just to confirm what Torben has said .. that is definitely the problem. I found the same issues of size discrepencies and bayer matrix incompatibilities looking at your master light and flat that you uploaded. The flat shows up as a grayscale 8 bit 6248x4176, the light shows up as a 24 bit RGB 3146x2165.

Andy Wray:
Just to confirm what Torben has said .. that is definitely the problem. I found the same issues of size discrepencies and bayer matrix incompatibilities looking at your master light and flat that you uploaded. The flat shows up as a grayscale 8 bit 6248x4176, the light shows up as a 24 bit RGB 3146x2165.

Ok, then the flats aren‘t even of the same format, because the ones I tested were RGB, but wrong size. I didn‘t check for that. Stuart, you need to make sure you do not vary these exposure parameters (binning, cfa) between the lights and calibration frames. If I may advise, stick to binx1 with this camera - there is no real benefit to binx2 that you can not more easily realize in postprocessing except smaller image files. But you don‘t have bad data there - so just reshoot flats, darks and bias for this set and you‘re ready to go - will turn out much better than the „salvaged“ result without calibration.

Ken Rockelein:
I'm having a similar issue with my new setup right now as well. It's an AstroTech 115 triplet with the ASI294mono camera

@Ken Rockelein I can't unfortunately lend a hand to OP, but in your case I might be able to lend some advice. I also have the 294MM as well as the 294MC. In case you haven't tried this yet, I have found that the 294MM (and MC) really, really don't play nice with short exposure calibration frames, especially when trying to calibrate flats/darks etc with bias.

I have scrapped using bias frames completely and moved towards dark flats instead. I used to use APT's flats tool but now manually test flat exposures to find an avg. of around 32K with at LEAST a 1 second exposure. Obviously adjusting brightness is a big deal here given the length of those flay exposures.

Scrapping bias and instead using Darks and DarkFlats and Flats with longer exposures, my calibrations have been absolutely perfect. TL;DR, the 294's really hate sub 1 second calibration frames (ESPECIALLY in narrowband) so dropping bias and going for longer exposure flats might be worth trying.

Connor Kessler:
I also have the 294MM as well as the 294MC. In case you haven't tried this yet, I have found that the 294MM (and MC) really, really don't play nice with short exposure calibration frames, especially when trying to calibrate flats/darks etc with bias.

I don't find the same in my calibration procedure, at all. I still use very short flats, 0.1s to 0.5s. With or without darkflats as opposed to bias. Given that CMOS darks are not scalable I'd venture that dropping the bias subtraction operation and replace it with the dark-flats makes sense operationally.

If I may add my experience: Longer sky flats also work perfectly with the QHY294M. Exact match of dark exposure to the flats is not needed (now, that would be tedious with sky flats…). The amp glow and dark current is low enough that I can calibrate, say, 10s flats with 2s darks. Basically you use them as a „long bias“.

Related discussions