Back to forums

New Script: PhotometricContinuumSubtraction -- Pixinsight

Pleiades Astrophoto PixInsight Processing Narrowband Deep Sky
Charles HagenJohn HayesKevin MorefieldMark Petersen
•48 replies•5k views
Charles Hagen avatar
Charles Hagen avatar
Description:
I have just released a new Pixinsight script, PhotometricContinuumSubtraction or 'PCS'. This script takes your registered narrowband and broadband integrations and using a photometric routine automatically calculates the ideal continuum subtraction weights and performs the subtraction, resulting in very speedy, perfectly subtracted images with no trial and error or manual adjustments. Additionally, you can feed it your linear starless masters and it will perform the appropriate subtraction using the photometrically calculated values. This supersedes my previous color calibration routine for continuum subtraction as it is both faster and more performant, especially in the edge cases with very bright targets or targets with a lot of continuum leakage.

You can also save out instances of the script with your nominal image identifiers pre-loaded. This allows you to apply the script without opening the script dialog. For instance, I always name my H-alpha and Red channels "Ha" and "Red" respectively. I can save off an instance icon and apply it to any image and it will automatically preform the continuum subtraction on images with matching identifiers.

The Pixinsight repository (available below) includes this script as well as one other currently (ApplySTF - I will be making another post shortly). I plan on making more tools in the future. You can find them under Scripts > NightPhotons > [ ]. I hope you all find this useful. Please let me know if you have any thoughts, questions or requests!

Thanks and clear skies,
Charlie

Repository:
https://raw.githubusercontent.com/charleshagen/pixinsight/main/updates/

Source code: https://github.com/charleshagen/pixinsight/
Website: https://www.nightphotons.com/
Astrobin: https://www.astrobin.com/users/jimmythechicken/

Screenshots and Examples:

Graphical Interface


M81 & M82 Example [Before and After]
​​​​​

Helpful Engaging Supportive
Mark McComiskey avatar
Mark McComiskey avatar
Super excited to try this out. I’ve been using your methodology manually. Thanks for this.
Francesco Meschia avatar
Francesco Meschia avatar
Charlie – I just tested your script on an image I’m processing at the moment. It works perfectly, saves a ton of time, and I really like the fact that it uses quantitative methods to achieve its goal.
Francesco
Well Written
Charles Hagen avatar
Charles Hagen avatar
@Francesco Meschia That is great to hear! I am very glad it seems to be working well for you! Hopefully I will find the time to update my website with all the new stuff soon.

-Charlie
Well Written Respectful Supportive
Rowan Prangley avatar
Rowan Prangley avatar
Amazing work Charlie.
​​​​Thank you for your time creating these scripts!
Well Written Respectful Supportive
Charles Hagen avatar
Charles Hagen avatar
Rowan Prangley:
Amazing work Charlie.
​​​​Thank you for your time creating these scripts!

Of course! Hopefully it will benefit you all as much as it has benefitted me!
Mau_Bard avatar
Mau_Bard avatar
Thank you Charles for publishing your work! I have been applying a couple of times the manual procedure for continuum subtraction you well described on your site and I am looking forward to check your new script.
The starless option sounds a non trivial feature as well!
Respectful Supportive
John Hayes avatar
John Hayes avatar
Thanks for developing this tool!  It sounds really useful and I can’t wait to try it.

John
Well Written
Kevin Morefield avatar
Kevin Morefield avatar
Well, that was easy.  Many thanks!  First attempt looked at least as good as what I was getting manually.

Kevin
Charles Hagen avatar
Charles Hagen avatar
Kevin Morefield:
Well, that was easy.  Many thanks!  First attempt looked at least as good as what I was getting manually.

Kevin

Excellent! Glad to see it worked for you. Cant wait to see what you make with it! 
-Charlie
Richard Mak avatar
Richard Mak avatar
Thank you!
Bill McLaughlin avatar
Bill McLaughlin avatar
Would like to try it and PI "View installed updates" shows it as installed but there is no entry for Nightphotons under scripts and nothing under all scripts. Latest Windows PI version fully updated PI shows no updates available and restart does not fix it.
Charles Hagen avatar
Charles Hagen avatar
Bill McLaughlin:
Would like to try it and PI "View installed updates" shows it as installed but there is no entry for Nightphotons under scripts and nothing under all scripts. Latest Windows PI version fully updated PI shows no updates available and restart does not fix it.

Go to Scripts > Feature Scripts > Add , Then select the folder C:\Program Files\PixInsight\src\scripts (or wherever your pixinsight is installed). Press "Select Folder" and it should populate your scripts. Let me know if that works for ya!
-Charlie
Well Written Helpful Supportive
Bill McLaughlin avatar
Bill McLaughlin avatar
OK Thanks that got it. I guess I had forgotten that step in installing scripts.
Kevin Morefield avatar
Kevin Morefield avatar
I assume for OIII we would do a blend of the G and B and then run the script on the blended BG master?  And is there an empirically derived ideal split between G and B?
Charles Hagen avatar
Charles Hagen avatar
Kevin Morefield:
I assume for OIII we would do a blend of the G and B and then run the script on the blended BG master?

My recommendation would be to use purely green when subtracting Oiii - usually the cuts on green filters are high enough to avoid any Hbeta signal. Blue will include that signal. Subtracting that Hbeta from an image that doesn’t contain any may result in over subtraction or large holes. This is especially true on objects like the squid that have faint Oiii emission but high Ha and Hb.

it gets a little more complicated with Sii for the same reason, given that red will often be dominated by Ha. I will detail my solution in the article that I eventually write for this process.
Helpful Insightful Respectful
Mark Petersen avatar
Mark Petersen avatar
I love the concept and the fact that it will deliver consistent results.  But I caution people in assuming that this will deliver "perfectly subtracted" images.  From conversations with my scientist friends, the only way to do perfect CS is through the use of off-band filters, wide-band filters that contain no narrowband wavelengths.  For O3 CS in particular, it's really a can of worms without using off-band filters.  It depends heavily on the subject itself.  Some things might contain a lot of blue continuum for example, so subtracting green only would be a problem.  But with those caveats in mind, I applaud the effort.  A script like this that uses photometric info is an improvement over what has been a vague process that uses "touchy feely"  fudge factors.
Well Written Helpful Insightful Respectful Engaging Supportive
Charles Hagen avatar
Charles Hagen avatar
Mark Petersen:
I love the concept and the fact that it will deliver consistent results.  But I caution people in assuming that this will deliver "perfectly subtracted" images.  From conversations with my scientist friends, the only way to do perfect CS is through the use of off-band filters, wide-band filters that contain no narrowband wavelengths.  For O3 CS in particular, it's really a can of worms without using off-band filters.  It depends heavily on the subject itself.  Some things might contain a lot of blue continuum for example, so subtracting green only would be a problem.  But with those caveats in mind, I applaud the effort.  A script like this that uses photometric info is an improvement over what has been a vague process that uses "touchy feely"  fudge factors.

You're right in that I should be careful with how I describe it, 'perfection' is perhaps overselling it a bit haha. There is of course no way to get a truly perfect continuum subtraction, especially using broadband data. However, this script should find the optimal solution (or very close to it) for the approximation that is allowed by using broadband filters as a continuum reference. I would be very curious to see how it performs with a more appropriate reference too - I've not had the opportunity to mess around with the more specialized continuum references as you described. If you have some of that data and would be willing to run it through PCS I would love to see the results!

Thanks,
Charlie
Well Written Insightful Respectful Engaging Supportive
Mark Petersen avatar
Mark Petersen avatar
Charles Hagen:
Mark Petersen:
I love the concept and the fact that it will deliver consistent results.  But I caution people in assuming that this will deliver "perfectly subtracted" images.  From conversations with my scientist friends, the only way to do perfect CS is through the use of off-band filters, wide-band filters that contain no narrowband wavelengths.  For O3 CS in particular, it's really a can of worms without using off-band filters.  It depends heavily on the subject itself.  Some things might contain a lot of blue continuum for example, so subtracting green only would be a problem.  But with those caveats in mind, I applaud the effort.  A script like this that uses photometric info is an improvement over what has been a vague process that uses "touchy feely"  fudge factors.

You're right in that I should be careful with how I describe it, 'perfection' is perhaps overselling it a bit haha. There is of course no way to get a truly perfect continuum subtraction, especially using broadband data. However, this script should find the optimal solution (or very close to it) for the approximation that is allowed by using broadband filters as a continuum reference. I would be very curious to see how it performs with a more appropriate reference too - I've not had the opportunity to mess around with the more specialized continuum references as you described. If you have some of that data and would be willing to run it through PCS I would love to see the results!

Thanks,
Charlie

Hi Charles,  I'd love to do this comparison too and hope to obtain an off-band filter soon.  But it might be awhile before I will be able to share the results.  I'll definitely reach out to you when I have something to share though.  In the meantime your script is the only CS method I will be using, I love the concept.  One question - I browsed through the javascript and see the scale factor that is being used in the pixelmath, but I was unclear in how the scale factor was obtained from dynamicPSF.   I always use dynamicPSF to examine eccentricity and FWHM, but in this instance is it peak brightness info that is obtained from dynamicPSF?
Well Written Respectful
Charles Hagen avatar
Charles Hagen avatar
Mark Petersen:
Charles Hagen:
Mark Petersen:
I love the concept and the fact that it will deliver consistent results.  But I caution people in assuming that this will deliver "perfectly subtracted" images.  From conversations with my scientist friends, the only way to do perfect CS is through the use of off-band filters, wide-band filters that contain no narrowband wavelengths.  For O3 CS in particular, it's really a can of worms without using off-band filters.  It depends heavily on the subject itself.  Some things might contain a lot of blue continuum for example, so subtracting green only would be a problem.  But with those caveats in mind, I applaud the effort.  A script like this that uses photometric info is an improvement over what has been a vague process that uses "touchy feely"  fudge factors.

You're right in that I should be careful with how I describe it, 'perfection' is perhaps overselling it a bit haha. There is of course no way to get a truly perfect continuum subtraction, especially using broadband data. However, this script should find the optimal solution (or very close to it) for the approximation that is allowed by using broadband filters as a continuum reference. I would be very curious to see how it performs with a more appropriate reference too - I've not had the opportunity to mess around with the more specialized continuum references as you described. If you have some of that data and would be willing to run it through PCS I would love to see the results!

Thanks,
Charlie

Hi Charles,  I'd love to do this comparison too and hope to obtain an off-band filter soon.  But it might be awhile before I will be able to share the results.  I'll definitely reach out to you when I have something to share though.  In the meantime your script is the only CS method I will be using, I love the concept.  One question - I browsed through the javascript and see the scale factor that is being used in the pixelmath, but I was unclear in how the scale factor was obtained from dynamicPSF.   I always use dynamicPSF to examine eccentricity and FWHM, but in this instance is it peak brightness info that is obtained from dynamicPSF?

Here's the general flow: detect the stars in the broadband image, filter on those, select the brightest stars that fall below the flux threshold, then I instantiate DynamicPSF twice to get the flux values on those stars between the two images; the narrowband and broadband. Then I go through them and match the pairs and build an array of flux ratios - so to answer your question, I am using specifically the flux here rather than the peak values. The peaks would be unreliable as the signal is discretized and the star core may exist over multiple pixels, may have worse FWHMs, etc. The flux is a more reliable metric. Then I simply find the median value of the ratio array to be used as the scalar in the PixelMath. That represents the 'normal' transformation that would best match the star fluxes which are assumed here to be exclusively continuum signal.  Hope that helps!
Helpful Insightful
Mark Petersen avatar
Mark Petersen avatar
Hi Charles,  yes that makes sense.  Thanks for the details.
Well Written Respectful
Francesco Meschia avatar
Francesco Meschia avatar
Could PI's new Spectrophotometric Flux Calibrator be used to do the flux calculations for you?
Well Written
Charles Hagen avatar
Charles Hagen avatar
John Hayes:
Thanks for developing this tool!  It sounds really useful and I can’t wait to try it.

John

Of course, it is my pleasure! I found it quite fun to develop, I have some more ideas that might be useful too, perhaps there will be a couple more soon.  I would love to see your results if you end up giving it a try! 
-Charlie
Well Written Respectful
URSAMO avatar
URSAMO avatar
My heart is playing drums now, looks so promising and time saver! Thank you so much for your help sell it for 59.99 usd then smile
Charles Hagen avatar
Charles Hagen avatar
As much as I would love to make money off of this, I think it is more worthwhile to keep tools like this free for the community! (There is a donate button in the top right of my website though, if you'd like to support my work ) I hope you enjoy the tool!
Well Written Respectful Supportive