Black Pits -- What are they and how to remove?

Pleiades Astrophoto PixInsight Processing Narrowband Deep Sky Artifacts
19 replies•484 views
Jeff Ridder avatar
Jeff Ridder avatar
Captured some Horsehead and Flame in narrowband the other night. Each of my integrated frames -- Ha, OIII, and SII have black pits everywhere. And they're correlated across the three filters. Same pits in each image, so it's not a particular filter. They also exist in the raw subs, but much fainter (obviously). Calibrated with a new set of (50 each) darks, flats, and flat-darks. These pits aren't in the calibration files. I also collected with these filters on a previous night on another target -- no pits.

1) What are they? Since they don't show up in my flats, I'm guessing not dust. It was a very cold night, possibly frost on the sensor window?

2) Anything I can do to remove them? I use PixInsight. Tried various tricks with Cosmetic Correction on the calibrated subs -- no change.

This is the SII integrated image. The black pits are the same in the other two and in the same locations.
Well Written Helpful Engaging
David Redwine avatar
David Redwine avatar
Frost on the sensor window should have a more diffuse edge, the edge of these spots seems too sharp.  I'm guessing frost on the actual sensor surface itself, but that is just a guess.

Try regenerating the desiccant.
Honeycakehorse avatar
Honeycakehorse avatar
Hey Jeff,
i had smaller black dots across my image and some time ago.
The integration process was my problem, when i did it manually step by step. I can't explain the exact reason but it resolved when i switched to the weighted batch preprocessing tool…
If i can remember correctly, it had something to do with the calibration of darks.
Maybe just try a different integration method and compare the results before changing setup routines.
clear skies,
honeycakehorse
Jeff Ridder avatar
Jeff Ridder avatar
Aaaargh! Just found another night's worth of data with the same affliction. Black pits that correlate across filters. BUT, these pits don't correlate with the pits from a different night. Homing in on frost being the issue.

@Honeycakehorse  -- thanks for the suggestion. I'll give that a try.
Honeycakehorse avatar
Honeycakehorse avatar
hmm, do you use dew heater?
​​​​​​
(nevertheless i would check out an alternative integration smile)
Jeff Ridder avatar
Jeff Ridder avatar
Honeycakehorse:
hmm, do you use dew heater?
​​​​​​
(nevertheless i would check out an alternative integration ;))

The ASI1600MM-Pro has desiccant pellets inside the sensor window. I've refreshed them (per above suggestion) and ASI claims you don't need a dew heater. I'm thinking they might be wrong.
Honeycakehorse avatar
Honeycakehorse avatar
This might be the case but it doesn't protect the telescope itself from dew, right?
Well Written Respectful
Jeff Ridder avatar
Jeff Ridder avatar
Honeycakehorse:
This might be the case but it doesn't protect the telescope itself from dew, right?

Should have mentioned -- I have dew straps on the OTA.
David Redwine avatar
David Redwine avatar
Dew on the OTA usually just results in low contrast.
Well Written
David Redwine avatar
David Redwine avatar
BTW  pixel rejection such as Winsorized Sigma Clipping only rejects  data that is an outlier for a particular pixel stack.  If this artifact is in all the subs it will not be rejected by the clipping function during integration.

You might try the Canon banding script.
Helpful Insightful Concise
Jeff Ridder avatar
Jeff Ridder avatar
David Redwine:
BTW  pixel rejection such as Winsorized Sigma Clipping only rejects  data that is an outlier for a particular pixel stack.  If this artifact is in all the subs it will not be rejected by the clipping function during integration.

You might try the Canon banding script.

Thanks for the tip. Was unaware of this script. Tried it, but no improvement. I think I'm down to paint by number or re-shoot.
Bob Lockwood avatar
Bob Lockwood avatar
Just a thought,

I think the bigger picture hear is finding where and why the spots are there too begin with, not to just process them out. I think the spots are to small to be anything other then on the sensor or the window, and if it’s from dew and has dried, the sensor or window may need to be cleaned, if that’s possible.
Jeff Ridder avatar
Jeff Ridder avatar
Bob Lockwood:
Just a thought,

I think the bigger picture hear is finding where and why the spots are there too begin with, not to just process them out. I think the spots are to small to be anything other then on the sensor or the window, and if it’s from dew and has dried, the sensor or window may need to be cleaned, if that’s possible.

Thanks, and I agree. Given that they show up on subs from some nights and not on others, I think I've determined that it is frost. The fact that the spots are in the same location across multiple filters on the same night, but then in a different location on another night, seems to support the frost hypothesis. I've cleaned both the sensor and the window, and refreshed the desiccant inside the chamber. Hoping that takes care of the problem. Just in case it doesn't, I've ordered the ZWO dew heater which, like all other astronomy gear, is on backorder until Spring. ;)
Well Written Insightful Respectful Concise
Arun H avatar
Arun H avatar
Jeff -

Assuming it is dew/frost, it may not be inside the sensor chamber (which is sealed) but on the outer window surface. I had an issue with this same camera of dew. I initially thought it was on the sensor and microwaved the desiccant tablets. This made no difference. The solution was to stick a resistance heater on the outer surface of the camera where it mates with the filter wheel - this one:

https://astronomy-imaging-camera.com/product/anti-dew-heater

That surface is obviously just above a very cold sensor and so does get cold. It is heated by waste heat from the TEC, but since the TEC does not generate much heat in winter, that heating can be ineffective. ZWO knows this is an issue, and newer cameras come with this pre-installed. Not the ZWO ASI 1600MM Pro though. So if your desiccant tablet refresh doesn't work, install this heater.
Well Written Helpful Insightful
Jeff Ridder avatar
Jeff Ridder avatar
Jeff -

Assuming it is dew/frost, it may not be inside the sensor chamber (which is sealed) but on the outer window surface. I had an issue with this same camera of dew. I initially thought it was on the sensor and microwaved the desiccant tablets. This made no difference. The solution was to stick a resistance heater on the outer surface of the camera where it mates with the filter wheel - this one:

https://astronomy-imaging-camera.com/product/anti-dew-heater

That surface is obviously just above a very cold sensor and so does get cold. It is heated by waste heat from the TEC, but since the TEC does not generate much heat in winter, that heating can be ineffective. ZWO knows this is an issue, and newer cameras come with this pre-installed. Not the ZWO ASI 1600MM Pro though. So if your desiccant tablet refresh doesn't work, install this heater.

Thanks, but I beat you to it! ;) Found that and ordered it but, like all other astro gear, it is on back order. So, hoping the desiccant refresh gets the job done. Will find out later this week (I hope).
Michael J. Mangieri avatar
Michael J. Mangieri avatar
Jeff -

Assuming it is dew/frost, it may not be inside the sensor chamber (which is sealed) but on the outer window surface. I had an issue with this same camera of dew. I initially thought it was on the sensor and microwaved the desiccant tablets. This made no difference. The solution was to stick a resistance heater on the outer surface of the camera where it mates with the filter wheel - this one:

https://astronomy-imaging-camera.com/product/anti-dew-heater

That surface is obviously just above a very cold sensor and so does get cold. It is heated by waste heat from the TEC, but since the TEC does not generate much heat in winter, that heating can be ineffective. ZWO knows this is an issue, and newer cameras come with this pre-installed. Not the ZWO ASI 1600MM Pro though. So if your desiccant tablet refresh doesn't work, install this heater.

I've added this dew heater to my 1600 and my frost/dew issues have gone away.  Recommended addition to the ASI1600mm.
Paolo Manicardi avatar
Paolo Manicardi avatar
I remember a IRIS tutorial to extract the flat-field from lights images.
You can give it a try to avoid trash lot of integration time, or find a way to replicate the procedure with PixInsight

http://www.astrosurf.com/buil/iris/tutorial2/doc91_us.htm
Alan Brunelle avatar
Alan Brunelle avatar
Sounds like you have identified your problem.

in the future, if you have your camera dismounted, it is not too difficult to just plug it in and cool it down to your operating temperature for a while. If there is frosting on the protective window or sensor, you should be able to see it under a bright light.

Also frost will often nucleate on dust or dirt particles, either on the sensor cover glass or the window. So cleaning can still help.
Helpful Concise
AwesomeAstro avatar
AwesomeAstro avatar
Although I've never had frost on the imaging sensor (only the corrector plate, which looks nothing like this because the problem is very, very close to the focal point to be this clear on your image), it seems that the general consensus here is that this is the issue.

While you're waiting for your new equipment, you can test a few other things without needing to image. First off, did you take your flats on a different day than the imaging session, or did you take them right afterwards? If I take flats on a different day, dust in my train definitely moves, making those flats fail to remove the dust motes. If you took them the same night, then they should at least slightly help with this dust or frost. If you took them later on, neither of these problems would be calibrated out with those flats.

It's also worth noting that even though very obvious in a final stack, the presence of these motes in the flats isn't always obvious, because they hardly have the same amount of integration time. You should debayer (if applicable) your master flat, use backgroundneutralization, and then hit it with automatic background extractor. Now use the screentransferfunction, and it will reveal, very aggressively, any pits or motes in your master flat. This can help with the investigation too, since you won't always see these imperfections in the master flat even if they're there. You certainly can't pick them out in individual flat frames, at least usually.
Helpful Insightful
Related discussions
Dark Flats, NB filter and Pixinsight
I've recently started using a Pegasus Flatmaster to take my flats. In NINA I'm using dynamic brightness mode and taking Narrow Band Flats of 2 seconds. On a night where I use Ha and SII for example, I take 25 Flats on each filter, NINA then h...
Discusses flat calibration issues with narrowband filters and dynamic brightness mode.
Sep 19, 2022
Why Sii frames have this ?
Hi, Why my Sii master frame have this weird black ring and how I can remove it? is it because of light pollution or issues with flat frames? this is total integration of 270x180" I'm using Antlia filters 3nm, the frames were calibrated with ...
Addresses black artifacts in SII frames and potential flat frame problems.
Dec 22, 2022
Please don't skimp on your flats
Two days ago I was trying out my new OAG and decided to do a quick capture of M81/M82. I thought: oh well, I haven't changed that much, so I just used old flats and darks. What was I thinking? I ended up with weird dust bunnies, vignetting and gr...
Emphasizes importance of proper flat calibration to avoid image artifacts.
Apr 18, 2022
Calibration help
I'm running into a problem when trying to calibrate my Ha frames in Pixinsight. I'm using an ASI178MM-Cool which I know has some serious amp glow at longer exposures but up until this point, darks have been able to remove it quite successfull...
Black pits in narrowband frames may relate to calibration issues with darks.
Sep 19, 2019