Question about Flats and Multiple Image Sessions - Forums

Topic starter

Apr 16, 2023

Hi!

Last night I planned on shooting 100 180" subs of M101. But around midnight, the clouds rolled in making guiding impossible so I shot 30 flats and called it a night.

My question is this: If the weather doesn't allow me to continue imaging this object for another 3 or 4 nights, I am going to bring my gear into the house rather than leave it outside for days or even weeks.

When I setup and reshoot another 100 subs or so and take new flats, which flats do I use for calibration? The ones in the first session or the ones in the 2nd? I want to collect around 10-12 hours of data and am unsure how to proceed with flat calibration over multiple nights. I assume the dust bunnies will move around…

Thanks!
Jerry

Well Written Engaging

Roman Pearah

Apr 16, 2023

Jerry Gerber:
The ones in the first session or the ones in the 2nd?

Yes.

You calibrate the lights from each session with the master flat for that session. Calibration is done to each separate light sub before anything else. It's not done to combinations of lights. So you use the master flat that is appropriate.

Well Written Helpful Concise

Piotr Janicki

Apr 16, 2023

Dust bunnies moving around is unlikely. These things stick with a capital S. A more valid concern is new ones entering the train. So it doesn't hurt to redo flats.

I find it to be the best practice to clean everything REAL good, isopropyl alcohol, cotton balls, the whole nine yards. And dust proof everything as best you can. Certain spots in the train are dust magnets.

If you do a good job cleaning and dustproofing the image train, you'll be able to reuse flats for months on end, not forever tho sińce eventually something will get in.

Helpful

Roger Redcat

Apr 16, 2023 · Edited: Apr 16, 2023

I have only ever done 16 x flats with same amount dark flats for my setup always had perfect results.
Recently started Bias and x 40 of these.
I keep my kit on patio 24/7 double covered and Aok for 3 years now so can do these on the next days.
Roger.

Roger Redcat

Apr 16, 2023

Not a good idea to keep cleaning the lens as most dust bunnies will be inside the train including filters and sensors, always possibilities of scratches so best to take new flats each time you set up , which is the generally accepted advice.
Roger

Patrick Graham

Apr 16, 2023

If you're going to keep the same orientation of the camera relative to the scope and to the filters, then you should be able to use the flats from your first session in later sessions. The issue develops when you break down the equipment and set up again at a later date. The orientation of the imaging train will be off, no matter how diligent you are about getting the orientation exactly as it was before. It's always good practice to reshoot the flats with the new set up, even if your using the same equipment.

Patrick

Well Written Helpful Insightful Concise

Jerry Gerber

Topic starter

Apr 16, 2023

OK, no problem taking flats after each session if I take apart the setup and re-set it back up again and keeping same subs if I keep the setup the same without taking it apart.

My question is this: Let's say I shoot 60 subs and then I calibrate and integrate them into a master light. When I got out the next time lets say I shoot 100 subs of the the same object. Do I calibrate and integrate all the new subs into a new master light and then combine the two master lights?

What I am not sure of is if I am using 2 sets of subs of the same object and 2 sets of flats, at what point do I integrate all of the subs into a master light?

Jerry

Patrick Graham

Apr 16, 2023

I think it would be easier to just have 1 master light by combining the 60 subs from the first session with the 100 from the second session in your example. You would have a better signal to noise ratio (cleaner image). I don't think PI would integrate 2 separate light masters. Probably save yourself some time and frustration by waiting until you have all 160 ready to calibrate and integrate. Of course, experimentation will determine which is better for your case. Are you using the WBPP script?? You can upload both sets into the respective folders and WBPP will do it all.

Patrick

Helpful

Roman Pearah

Apr 17, 2023

Jerry Gerber:
OK, no problem taking flats after each session if I take apart the setup and re-set it back up again and keeping same subs if I keep the setup the same without taking it apart.

My question is this: Let's say I shoot 60 subs and then I calibrate and integrate them into a master light. When I got out the next time lets say I shoot 100 subs of the the same object. Do I calibrate and integrate all the new subs into a new master light and then combine the two master lights?

What I am not sure of is if I am using 2 sets of subs of the same object and 2 sets of flats, at what point do I integrate all of the subs into a master light?

Jerry

Don't stack stacks. Integrate all of the calibrated frames at one time, if you can help it.

Well Written

Joe Linington

Apr 17, 2023

Patrick Graham:
I think it would be easier to just have 1 master light by combining the 60 subs from the first session with the 100 from the second session in your example. You would have a better signal to noise ratio (cleaner image). I don't think PI would integrate 2 separate light masters. Probably save yourself some time and frustration by waiting until you have all 160 ready to calibrate and integrate. Of course, experimentation will determine which is better for your case. Are you using the WBPP script?? You can upload both sets into the respective folders and WBPP will do it all.

Patrick

Why does everyone assume that everyone is using PI?

You should integrate the whole project as one master light. Astro Pixel Processor, PixelInsight and DSS all have multisession support that will let you match flats to lights. Siril can do it but it’s a bit more advanced. APP and Siril will integrate separate master lights, I’m sure PI can too but you are better to do one large integration.

Jerry Gerber

Topic starter

Apr 17, 2023

Roman Pearah:
Jerry Gerber:
OK, no problem taking flats after each session if I take apart the setup and re-set it back up again and keeping same subs if I keep the setup the same without taking it apart.

My question is this: Let's say I shoot 60 subs and then I calibrate and integrate them into a master light. When I got out the next time lets say I shoot 100 subs of the the same object. Do I calibrate and integrate all the new subs into a new master light and then combine the two master lights?

What I am not sure of is if I am using 2 sets of subs of the same object and 2 sets of flats, at what point do I integrate all of the subs into a master light?

Jerry

Don't stack stacks. Integrate all of the calibrated frames at one time, if you can help it.

That make sense. But if I shoot one set of images one night with flats, and then let's say I return to that same object a week later to shoot more subs, if I use the master flat from the first session, the dust patterns may be different from the 2nd session, and if I use the master flat from the 2nd session, the dust pattern might be different from the first night! Hmm, there's got to be a "best practice" solution...

Engaging

Joe Linington

Apr 17, 2023

What software are you using to stack?

Jerry Gerber

Topic starter

Apr 17, 2023

Joe Linington:
What software are you using to stack?

Pixinsight

David Webster

Apr 17, 2023

IF, you will not be removing or rotating your image train, and you are covering your scope with a cap, you really only need the flats at the end.
You're going to be integrating images from different nights. You're not going to lose any dust particles. You may attract a few. Hence the first flats would then be useless and or pointless!

Jerry Gerber

Topic starter

Apr 17, 2023 · Edited: Apr 17, 2023

David Webster:
IF, you will not be removing or rotating your image train, and you are covering your scope with a cap, you really only need the flats at the end.
You're going to be integrating images from different nights. You're not going to lose any dust particles. You may attract a few. Hence the first flats would then be useless and or pointless!

Hi David,

The problem is that I did remove my image train as I store my gear in the house if I am not going to be using it for several days. So though i understand the first flats would be useless after the 2nd session, I suppose the 2nd set of flats may not take care of the dust on the first subs shot on a previous night. Not sure how to resolve this...yet.

Piotr Janicki

Apr 17, 2023

Jerry Gerber:
David Webster:
IF, you will not be removing or rotating your image train, and you are covering your scope with a cap, you really only need the flats at the end.
You're going to be integrating images from different nights. You're not going to lose any dust particles. You may attract a few. Hence the first flats would then be useless and or pointless!

Hi David,

The problem is that I did remove my image train as I store my gear in the house if I am not going to be using it for several days.

But do you HAVE to break it down?
Its an old picture from before I dust/light proofed the EFW but it makes the point across. This is how I store mine.

I break it down into individual sections, eg. optics, mount, counterweights, tripod. Etc. But I don't dissasemble everything completely, I just take the scope off the saddle and put it away, then break apart the mount into more manageable pieces and store that away too.

Concise

Jared Willson

Apr 17, 2023

Jerry Gerber:
Roman Pearah:
Jerry Gerber:
OK, no problem taking flats after each session if I take apart the setup and re-set it back up again and keeping same subs if I keep the setup the same without taking it apart.

My question is this: Let's say I shoot 60 subs and then I calibrate and integrate them into a master light. When I got out the next time lets say I shoot 100 subs of the the same object. Do I calibrate and integrate all the new subs into a new master light and then combine the two master lights?

What I am not sure of is if I am using 2 sets of subs of the same object and 2 sets of flats, at what point do I integrate all of the subs into a master light?

Jerry

Don't stack stacks. Integrate all of the calibrated frames at one time, if you can help it.

That make sense. But if I shoot one set of images one night with flats, and then let's say I return to that same object a week later to shoot more subs, if I use the master flat from the first session, the dust patterns may be different from the 2nd session, and if I use the master flat from the 2nd session, the dust pattern might be different from the first night! Hmm, there's got to be a "best practice" solution...

In PixInsight (which you said you are using) it is possible to group your flats into two separate groups by day. Then, you do the same thing with your light frames. So you would end up with:

1 master dark (if you are using darks)
1 master dark-flat or master bias (whichever you are using)
2 separate master flats (one from each of your two sessions)
2 separate groups of lights, one from the first night and one from the second night

PixInsight will then:
- subtract your single master bias/master dark-flat from each individual flat
- integrate your two calibrated flat groups into two separate master flats
- subtract your single master dark from each individual light frame
- divide each individual light frame by the appropriate master flat (first night of lights will use the first night master flat, 2nd will use the 2nd)
- integrate the now properly calibrated lights as a single stack

You can do all this in Weighted Batch Preprocessing in a single batch. Read up on how "grouping" works in PixInsight to see how to force it to recognize the lights from different nights as needing different flats applied.

- Jared

Well Written Helpful Insightful

Jonny Bravo

Apr 17, 2023

Use the "Keyword" function in WBPP. I use this all the time.

Create directories like this:

DATE_20230401
DATE_20230408
DATE_20230415
...

Under each of those directories, put that session's lights and flats (and dark flats if you use them). If you're using biases, then no need to put them under each session unless of course you've changed the gain/offset/temp, in which case each directory would need the full set of lights, darks, flats, biases/darkFlats.

You'll end up seeing something like this:

In my particular case, I shot LRGB data on the 12th and then spent the 13th capturing only L. I took flats for each session as well, thus the "Flats" and "Darks" tab contain a similar breakdown as shown in the screenshot above. Now, if you look at the Calibration tab, you'll see that everything is properly calibrated, using the date's appropriate data:

Finally, if you look at the Post-Calibration tab, you'll see that all the calibrated lights are then stacked into single masters per channel:

Click that Run button and go busy yourself with other things while WBPP works its magic.

Well Written Helpful Engaging Supportive

Dave Rust

Apr 17, 2023

There's a large amount of good information here.

While all of this guidance does indeed describe proper procedures, I prefer to not over-think it. In general, I use the same calibration frames, including flats, for most of a season (maybe 2 months). I even use the same files if I rotate the camera. Despite the fact that this is not a best-practice, my images come out just fine. My scope's circular gradient must be pretty symmetrical.

The only time I reshoot flats is when I discover new dust shadows after stacking. I just return to my scope in the garage and shoot new flats, then restack.

But, then again, I'm not over-handling or disassembling the gear, so there's a low likelihood that new dust has been introduced after wheeling the scope inside (it's on a dolly).

I even use straight RGB flats on images shot through filters with good results.

I have success this way, but I am not recommending you do the same. I'm just saying that using older flats can be pretty forgiving until new dust shows up.

Well Written Helpful Insightful Respectful Engaging

Arun H

Frequent contributor

Apr 17, 2023

Dave Rust:
While all of this guidance does indeed describe proper procedures, I prefer to not over-think it. In general, I use the same calibration frames, including flats, for most of a season (maybe 2 months). I even use the same files if I rotate the camera. Despite the fact that this is not a best-practice, my images come out just fine. My scope's circular gradient must be pretty symmetrical.

It really does depend on your individual configuration.

I have re-used narrow band flats for months.

LRGB flats are a different story if imaging from a light polluted site. It just takes one instance of a dust particle having moved that your flat does not correct for you to get into the habit of regularly taking flats.

This entire thread also made me realize how much WBPP seems to have made people not realize that you can do calibration step by step in PI or what is happening behind the scenes when you do WBPP, which is what we used to do before WBPP.

If you want to calibrate a set of lights with a specific master flat - simply use the Image Calibration Process in PI to create a set of calibrated frames. Calibrated frames are valuable to store - I've combined data from different scopes simply by registering and integrating calibrated frames.

Helpful Insightful

ZoboZodiac

Apr 17, 2023

I think from what you’re saying that you might be getting confused with whether you have to merge the flats or apply the same calibration across all the images, but you can actually do it separately.

So let’s say you have 60 images from the first night, and you take flats and darks etc. and then you go out another night and take 100 images with a new set of flats and darks.

For the first set of 60 lights you apply the flats (and darks etc) from the first night to those lights. This gives you 60 calibrated lights, but you don’t stack them together yet.
For the second set of 100 lights you apply the flats from the second night and end up with 100 calibrated lights.

Finally you then have 160 calibrated lights, and these are what you stack together at the end.

Helpful Concise