PI: Blink or not to Blink?

JohnHen
28 replies1.1k views
JohnHen avatar
Blink has been (or still is?) a must in the PI workflow. My question is: Do you use Blink or do you leave it up to the weighing/selection/registration algorithms of WBPP to do the job? The algorithms are getting better and better and not using Blink might save quite some time. Looking forward to hear about your opinions and experience (and if you could even share results with or without to support your statements that would be even better).
Thx, John
Well written Respectful Engaging
Paul Deeter avatar
I use blink to weed out the obviously bad (really bad) frames prior to WBPP.  I don't feel this step takes that much time.  However, I'm not capturing 500+ light frames either.  Just my two cents.  Clear skies, Paul.
Well written Concise
Jesco avatar
I use Blink, too.

Ideally I would like to use SubframeSelector also to screen out outliers with low star count (hazy, marginally cloudy subs) or bad FWHM and eccentricity (>0.6). Nowadays I spend 10-20hrs on a single subject which would allow for more rigid screening. But that’s not really possible within the WBPP workflow.
Well written Insightful Respectful Concise
JohnHen avatar
Paul Deeter:
I use blink to weed out the obviously bad (really bad) frames prior to WBPP.  I don't feel this step takes that much time.  However, I'm not capturing 500+ light frames either.  Just my two cents.  Clear skies, Paul.

*
Paul, thanks for your experience.
Let me add from my side: I have often 500+ subs where time of the Blink procedure really matters.
CS, John
andrea tasselli avatar
Never blinked smile
John Hayes avatar
Sometimes I might have as many as 800 x 130MB files and I consider blinking to be an essential tool to weed out frames that weighting and filters might not deal with properly.   Passing high thin clouds or other effects can cause issues that shouldn't be included in the stack.   The blink tool is my number one most used tool in PI.

- John
Well written Insightful Respectful Concise
Reg Pratt avatar
I'll use blank to weed out periods of clouds and then let APP do the rest. Running 2 scopes at short subs means thousands of subs at the end of each session. Far too many to blink it all.
JohnHen avatar
John Hayes:
Sometimes I might have as many as 800 x 130MB files and I consider blinking to be an essential tool to weed out frames that weighting and filters might not deal with properly.   Passing high thin clouds or other effects can cause issues that shouldn't be included in the stack.   The blink tool is my number one most used tool in PI.

- John

Dear John,
I also have full frame where a single sub is 120MB and besides the # of subs the file sizes makes it even more cumbersome.
So, ]very interesting to hear that you value Blink to be essential.
Thx, John
JohnHen avatar
Reg Pratt:
I'll use blank to weed out periods of clouds and then let APP do the rest. Running 2 scopes at short subs means thousands of subs at the end of each session. Far too many to blink it all.

Interesting point to use Blink partially for periods of clouds. Thx,
JanHeremans avatar
First blink to cancel subs with clouds and plane trails , then followed by subframeselector with criteria : eccentricity , FWHM, stars and noise.
jewzaam avatar
I blink for big changes due to clouds or for any  bad guiding that sneaks past my first scripted "toss things for bad RMS" (which is captured by NINA in my filenames).  Also a very bright satellite or airplane trail I've found will not always integrate out, so I tend to toss those.
Dimitris Kavallieratos avatar
Using blink as others have said, to exclude the obviously bad frames that would be a waste of time trying to integrate them. I find that even with a big number of frames, it is a fairly fast method. Subframe selector will exclude the less obvious ones with the right formula and small weight assignment or with a simple approach just weighting by the number of stars for example.
Helpful
JohnHen avatar
I find that even with a big number of frames, it is a fairly fast method.


*
Depends: If you have 500+ subs and each with 120MB (full frame) then even the loading into Blink is not negligible assumung an 'average' PC is used.
CS, John
Dimitris Kavallieratos avatar
I find that even with a big number of frames, it is a fairly fast method.


*
Depends: If you have 500+ subs and each with 120MB (full frame) then even the loading into Blink is not negligible assumung an 'average' PC is used.
CS, John


Indeed, you could play around with SS formulae and conditions to auto exclude low weight assigned frames and see what works. Although, if you have invested time to capture that many frames it is almost inevitable to invest a LOT of time for pre and post also, just to be sure you are taking full advantage of your dataset
JohnHen avatar
Indeed, you could play around with SS formulae and conditions to auto exclude low weight assigned frames and see what works. Although, if you have invested time to capture that many frames it is almost inevitable to invest a LOT of time for pre and post also, just to be sure you are taking full advantage of your dataset


I like that idea. CS, John
andrea tasselli avatar
Kidding aside I don't blink because it is far easier to sort subs according to number of stars detected and reject those with an obvious drop in detected ones (clouds coming through...), as shown here:



You can do other sorting for fwhm, weight, eccentricity and so on and drop them from the stack as needed.
JohnHen avatar
andrea tasselli:
Kidding aside I don't blink because it is far easier to sort subs according to number of stars detected and reject those with an obvious drop in detected ones (clouds coming through...), as shown here:



You can do other sorting for fwhm, weight, eccentricity and so on and drop them from the stack as needed.

*
I agree. That is another effective way of (manual) selection.
CS, John
Marcelof avatar
I usually use Blink  to discard obvious bad frames, but mostly looking for something unusual like a nova or an asteroid (so far, nothing smile ).

But I leave the job of selecting the best ones to Subframe Selector.
TurtleCat avatar
I’ll join the chorus on using blink for egregious problems in a sub. I don’t discard subs just because of an unround star or a jet trail, though. As long as I have enough subs the rejections will fix the problem sub and I still get the benefit of the added signal.
Ben Koltenbah avatar
An interesting thread.  I'll chime in with my two cents (now four cents with inflation) worth:

I Blink.  I blink often and early.

With so many subs to sift through, I need to quickly flip through them to first remove the egregious (see P.S. below) ones with airplanes, errant clouds, owls, wife turning on the backyard light, etc.

At times past, I have attempted to skip the Blink visual check and solely use SubframeSelector (SS), and I was surprised to find that some of the frames SS passed were those that I would definitely have rejected.  With the many parameters available on which to rack, stack and sort the frames, this is not surprising.  I am finding, however, that the newer parameters like PSFSignalWeight and such are doing a better job than some of the older ones I used before.

Given my past issues with SS, I usually open a second instance of PI on which I load the same list of files into Blink as I have into SS on the main window.  This allows me to do a visual spot check on the frames while I adjust the selection criteria.

All the Best,
Ben

P.S.  Whoops!  The last guy used "egregious", so I'll change mine to "painfully obvious to the most casual, distracted of observers".
Helpful Engaging Supportive
JohnHen avatar
Ben Koltenbah:
An interesting thread.  I'll chime in with my two cents (now four cents with inflation) worth:

I Blink.  I blink often and early.

With so many subs to sift through, I need to quickly flip through them to first remove the egregious (see P.S. below) ones with airplanes, errant clouds, owls, wife turning on the backyard light, etc.

At times past, I have attempted to skip the Blink visual check and solely use SubframeSelector (SS), and I was surprised to find that some of the frames SS passed were those that I would definitely have rejected.  With the many parameters available on which to rack, stack and sort the frames, this is not surprising.  I am finding, however, that the newer parameters like PSFSignalWeight and such are doing a better job than some of the older ones I used before.

Given my past issues with SS, I usually open a second instance of PI on which I load the same list of files into Blink as I have into SS on the main window.  This allows me to do a visual spot check on the frames while I adjust the selection criteria.

All the Best,
Ben

P.S.  Whoops!  The last guy used "egregious", so I'll change mine to "painfully obvious to the most casual, distracted of observers".

*
Hello Ben, thanks for sharing your experience.
' ... parameters like PSFSignalWeight and such are doing a better job than some of the older ones ...'
That is a noticeable point.
Also like your practice of opening a second PI instance.
Thx, John
Chris White- Overcast Observatory avatar
Ben Koltenbah:
An interesting thread.  I'll chime in with my two cents (now four cents with inflation) worth:

I Blink.  I blink often and early.

With so many subs to sift through, I need to quickly flip through them to first remove the egregious (see P.S. below) ones with airplanes, errant clouds, owls, wife turning on the backyard light, etc.

At times past, I have attempted to skip the Blink visual check and solely use SubframeSelector (SS), and I was surprised to find that some of the frames SS passed were those that I would definitely have rejected.  With the many parameters available on which to rack, stack and sort the frames, this is not surprising.  I am finding, however, that the newer parameters like PSFSignalWeight and such are doing a better job than some of the older ones I used before.

Given my past issues with SS, I usually open a second instance of PI on which I load the same list of files into Blink as I have into SS on the main window.  This allows me to do a visual spot check on the frames while I adjust the selection criteria.

All the Best,
Ben

P.S.  Whoops!  The last guy used "egregious", so I'll change mine to "painfully obvious to the most casual, distracted of observers".



I blink.  Ben hit on the reason why....  SS often passes images that I would reject and rejects images I would pass.  Analysis software for FWHM in particular is confused by astigmatism and eccentricity.  So unless you have a perfectly corrected field for the entire chip (which most of us do not have) a visual inspection seems to be the most robust.  Even with my poor eyes, I can blink through images very fast to determine roundness and shape.  I use a very fast scope, so I might easily have 500 or more subs in a given project.  I always run through blink twice.  First pass is zoomed out to see if clouds rolled through.  Second pass is at 2:1 to look at star shape.  I can blink through a couple hundred images in just a few minutes.  Anything I miss will be an outlier for rejection algorithms...

Subframe Selector takes a long time, and I dont trust the results... I never use SS.
Helpful Insightful Engaging
JohnHen avatar
Chris White:
Ben Koltenbah:
An interesting thread.  I'll chime in with my two cents (now four cents with inflation) worth:

I Blink.  I blink often and early.

With so many subs to sift through, I need to quickly flip through them to first remove the egregious (see P.S. below) ones with airplanes, errant clouds, owls, wife turning on the backyard light, etc.

At times past, I have attempted to skip the Blink visual check and solely use SubframeSelector (SS), and I was surprised to find that some of the frames SS passed were those that I would definitely have rejected.  With the many parameters available on which to rack, stack and sort the frames, this is not surprising.  I am finding, however, that the newer parameters like PSFSignalWeight and such are doing a better job than some of the older ones I used before.

Given my past issues with SS, I usually open a second instance of PI on which I load the same list of files into Blink as I have into SS on the main window.  This allows me to do a visual spot check on the frames while I adjust the selection criteria.

All the Best,
Ben

P.S.  Whoops!  The last guy used "egregious", so I'll change mine to "painfully obvious to the most casual, distracted of observers".



I blink.  Ben hit on the reason why....  SS often passes images that I would reject and rejects images I would pass.  Analysis software for FWHM in particular is confused by astigmatism and eccentricity.  So unless you have a perfectly corrected field for the entire chip (which most of us do not have) a visual inspection seems to be the most robust.  Even with my poor eyes, I can blink through images very fast to determine roundness and shape.  I use a very fast scope, so I might easily have 500 or more subs in a given project.  I always run through blink twice.  First pass is zoomed out to see if clouds rolled through.  Second pass is at 2:1 to look at star shape.  I can blink through a couple hundred images in just a few minutes.  Anything I miss will be an outlier for rejection algorithms...

Subframe Selector takes a long time, and I dont trust the results... I never use SS.

*
Hi Chris, thanks for chiming in and confirming that SS is not doing what would be obvious when doing manually.
Interesting you state that even with 500 subs you blink through in 2 rounds in just a few minutes. I have serious problems with 500+ subs at full frame (120MB). I use an iMac with 64GB RAM.
Thx, John
John Hayes avatar
I typically use SubFrameSelector first to sort my raw, uncalibrated images using three criteria.  First, I set an upper FWHM limit.  Second, I set an upper Eccentricity limit, usually around 0.5 (or 0.55 if I'm desperate.). Third, I set a lower limit on Stars.  If there are less than 20 stars, I know that something went wrong and it's going to be an empty frame.  There are times under poor conditions when I might gather as many as 200, 150, 150, 150 subs in LRGB. The SFS process will usually reduce the total number down to half that–or less.  Then, I blink through each individual channel–and I don't load all of those huge, 120MB images at once!  I normally limit each blink session to no more than about 50 frames and it doesn't take all that long to work through all of my data on my MacBook M1 Max.  The blink session is most useful for weeding out frames that are ruined by passing clouds.  High thin clouds can create a large bright glow around the brighter stars that I don't want mixed into the stack.  Clouds can also produce frames that have a very low overall signal.  Weighting can deal with this situation but in most cases, the number of frames affected by this problem is low enough that I just toss them out.  Sometimes I catch instances of double imaging when a wind gust strikes the scope or other weird and unexpected problems.  

The number one reason to blink is that subtle problems that aren't eliminated at the beginning of the process can show up in strange ways at the end where they are MUCH more difficult to address.  My goal is to always start the processing process with the cleanest possible set of integrated data.

John
Well written Helpful Insightful Engaging Supportive
Chris White- Overcast Observatory avatar
Hi Chris, thanks for chiming in and confirming that SS is not doing what would be obvious when doing manually.
Interesting you state that even with 500 subs you blink through in 2 rounds in just a few minutes. I have serious problems with 500+ subs at full frame (120MB). I use an iMac with 64GB RAM.
Thx, John




Well, I load my files into Blink.  Then click the histogram stretch to apply a unique stretch to every frame.  This takes the most amount of time... a few minutes on large data sets.

The blink process itself is pretty fast.  I scroll through and spend maybe half a second on each image.  So to go through a few hundred images is just a few minutes.  The human eye is very good at seeing "problems" like background issues caused by clouds or out of round stars.   I fly through and only stop to check off images that are a problem.  Then I select all of those problem images afterwards and move to a reject folder. 

I just blinked through 100 images this morning.  i had to reject about 25 due to clouds.  To go through twice took less than 5 minutes total.
Helpful Engaging
Related discussions
Stacking different resolution images in WBPP
Hi guys, I am taking images using a dual setup with DSLRs that produce different resolution images. The preprocessing is done in WBPP. Here are some screenshots showing my configuration: Post-Calibation tab: WBPP creates separate integrations for eac...
Discusses WBPP preprocessing workflow, directly relevant to author's question.
Oct 3, 2024