Cosmic Clarity Satellite Removal Tool - Forums

Topic starter

Nov 23, 2024 · Edited: Nov 23, 2024

Satellite trails are becoming more and more of a problem in our sub exposures. Don't throw out the exposure because of satellite trails, remove the trails from the image!!

I am sure everyone here has examples of a satellite trail making its way to our master stack. Our stacking programs are good, but as more and more satellite trails are in your stack, the rejection algorithms have a harder time removing them. Let Cosmic Clarity Satellite Removal do the heavy lifting of removing the satellite trail in the sub exposure first so the stacking rejection algorithm can handle the full stack much better.

My new program has real time monitoring of a folder such that as your telescope downloads new sub exposures it can be running the satellite trail removal on them so in the morning all your subs are ready for processing.

Please watch the video to learn the ins and outs of the program. Version 1 is available right now!

https://youtu.be/v4QgwMZeShU

https://www.setiastr.../cosmic-clarity

Drive Folder for the program: here

Helpful Engaging

Adam Block

Nov 23, 2024

It appears there may be some rule breaking here.
Uncalibrated frames will have an electronic signature that the AI will not be able to reproduce in the replacement of satellite trails.
E.g. There is a hot pixel (in a trail) that normally would be subtracted as part of calibration.
Wouldn't it be best to apply the removal to calibrated frames? To suggest to operate on uncalibrated data seems like it is asking for trouble and certainly breaks the best practice of removing instrumental effects/signatures before further processing (of any kind).

-adam

Helpful Insightful

ScottF

Frequent contributor

Nov 23, 2024

Does it save the original unaltered files?

Franklin Marek

Topic starter

Nov 23, 2024 · Edited: Nov 23, 2024

Adam Block:
It appears there may be some rule breaking here.
Uncalibrated frames will have an electronic signature that the AI will not be able to reproduce in the replacement of satellite trails.
E.g. There is a hot pixel (in a trail) that normally would be subtracted as part of calibration.
Wouldn't it be best to apply the removal to calibrated frames? To suggest to operate on uncalibrated data seems like it is asking for trouble and certainly breaks the best practice of removing instrumental effects/signatures before further processing (of any kind).

-adam

even within the slice almost nothing is altered except the trail, as it is trained to keep pixel values unaltered. Remember, slices that dont have a trail are completely unaltered. If you are very concerned with hotpixel or other subtraction worries then yes perform this in batch mode on your calibrated frames

Franklin Marek

Topic starter

Nov 23, 2024

Does it save the original unaltered files?

it doesnt do anything with the original images, those stay in the folder.

Franklin Marek

Topic starter

Nov 23, 2024

DEBUG Version available for Windows users experiencing errors. Please use this to help me troubleshoot your system. https://drive.google.com/file/d/10JFmKVB7RPexO2UeLpEFKry78dnGjP9i/view?usp=drive_link mirror site: https://drive.google.com/file/d/1RdN-oNkWN3kWb8x_zloR23pQnWk-hTm5/view?usp=drive_link

D. Jung

Nov 24, 2024

Thanks for another great contribution. Really appreciate your scripts and tools.

Well Written Respectful Supportive

Michael Jarvis

Nov 24, 2024

This is somewhat tangential to this discussion thread. Does anyone have detailed instructions for downloading and using Cosmic Clarity in the MACOS environment, especially within Pixinsight? I have an M1 MacBook Pro. All other PI scripts (e.g. all other Seti Astro features) work well but Cosmic Clarity is either very slow or doesn't run at all. Should I just use the CC Suite in standalone mode?

Well Written Engaging

Rostokko

Nov 24, 2024 · Edited: Nov 24, 2024

Michael Jarvis:
This is somewhat tangential to this discussion thread. Does anyone have detailed instructions for downloading and using Cosmic Clarity in the MACOS environment, especially within Pixinsight? I have an M1 MacBook Pro. All other PI scripts (e.g. all other Seti Astro features) work well but Cosmic Clarity is either very slow or doesn't run at all. Should I just use the CC Suite in standalone mode?

In my experience (M3) CC is quite slow to start up - it does take half a minute or more for me; the startup is so slow that you may be tempted to think it's hung at times.
Once it runs, it runs reasonably fast, and it does take full advantage of the GPU.
I have seen the same behavior when running it in standalone mode, I don't think it has anything to do with the PI integration. I *guess* it's possibly related to requiring the Rosetta translation.
Once I got used to that, it's not a big deal for me; the quality of the final result typically justifies that extra "startup hesitation".

Edited: I actually think the startup slowness is due to the fact that @Franklin Marek is building the Mac binaries as what Mac classifies as "Unix Executable File"; I wouldn't be surprised if that makes Rosetta 2 skip caching, or if it forces them through a different translation path which introduces the described behavior.

Well Written Helpful Insightful Engaging Supportive

Michael Jarvis

Nov 24, 2024

Rostokko, Thanks for your feedback. I think you are having better luck than me. CC-Sharpen took 10 min to complete within PI. I just am not sure that I have done things correctly. I am relatively new to PI (~4 months) but all other processes and scripts work with no issues. I have all the RC-Astro plugins so this is not mission critical. I am sure I am being boneheaded at some level. thanks.

Rostokko

Nov 24, 2024 · Edited: Nov 24, 2024

Michael Jarvis:
Rostokko, Thanks for your feedback. I think you are having better luck than me. CC-Sharpen took 10 min to complete within PI. I just am not sure that I have done things correctly. I am relatively new to PI (~4 months) but all other processes and scripts work with no issues. I have all the RC-Astro plugins so this is not mission critical. I am sure I am being boneheaded at some level. thanks.

How long of those 10 minutes is due to the startup behavior vs. the actual processing? After starting the processing in PI, click on the terminal window which should be visible in the dock, and check when you actually see the "Seti Astro" logo printed out. The time between PI start and when that logo is printed in the terminal window is what I am referring to as the "startup phase". Then you should be able to see the 1 or 3 channels being processed with a percentage number.

Well Written Helpful Insightful

Michael Jarvis

Nov 24, 2024

Thanks. I will give it a try.

David Fercho

Nov 25, 2024

I haven't been able to run it successfully on MACOS at all. I am using a 2020 Intel iMac running Sequoia. When I start the executable, it takes about 1 minute to bring up the GUI. And then the app crashes a few seconds after I browse to the input and output folders and click 'Batch Process Input Folder'.

Well Written

Franklin Marek

Topic starter

Nov 25, 2024

Can you share what error the terminal window is giving you, that would be helpful.

Michael Jarvis

Nov 25, 2024

I think it's related to some sort of shell error on my MacBook Pro. See this string: to use zsh, please run `chsh -s /bin/zsh`

Rostokko

Nov 25, 2024

Michael Jarvis:
I think it's related to some sort of shell error on my MacBook Pro. See this string: to use zsh, please run `chsh -s /bin/zsh`

Which MacOS version are you running?

Michael Jarvis

Nov 25, 2024

Sequoia 15.1.1

Rostokko

Nov 25, 2024

Michael Jarvis:
Sequoia 15.1.1

Interesting; I'm still running Sonoma and I don't see that; I wonder if it's related. But I'm not sure about how CC (or Python underneath) relies on zsh - if it does...

Michael Jarvis

Nov 25, 2024

I had the same issue when I was running Sonoma and thought I would give it another go with the new OS. No luck. I will download everything again later today and post the terminal error messages. Thanks again for everyone's help.

Franklin Marek

Topic starter

Nov 25, 2024

The zsh warning shouldn't be the issue. there is something else going on. The text from the terminal window would be a big help for sure.

David Fercho

Nov 25, 2024

Franklin Marek:
Can you share what error the terminal window is giving you, that would be helpful.

Here is the terminal output:

Seti_Satellite_abort.txt

Michael Jarvis

Nov 25, 2024

this is what I get in the terminal. PI just spins for 10 min or more. Then I force quit.Last login: Mon Nov 25 09:52:20 on ttys000 The default interactive shell is now zsh.To update your account to use zsh, please run `chsh -s /bin/zsh`.For more details, please visit https://support.apple.com/kb/HT208050./Users/michaelf.jarvis/Documents/CosmicClaritySharpenDenoise_MACOS/setiastrocosmicclarity_denoisemac –denoise_strength 0.90 –denoise_mode full Michaels-MacBook-Pro:~ michaelf.jarvis$ /Users/michaelf.jarvis/Documents/CosmicClaritySharpenDenoise_MACOS/setiastrocosmicclarity_denoisemac –denoise_strength 0.90 –denoise_mode full Killed: 9Michaels-MacBook-Pro:~ michaelf.jarvis$

Rostokko

Nov 25, 2024

David Fercho:
Seti_Satellite_abort.txt

That's odd, the certificate of that URL seems perfectly valid: https://www.sslshopper.com/ssl-checker.html#hostname=https://download.pytorch.org/models/resnet18-f37072fd.pth

Well Written

Rostokko

Nov 25, 2024

Michael Jarvis:
PI just spins for 10 min or more

Yeah, zsh is irrelevant here. It looks like the executable starts and then "sits there", as it does for me; but for me it "snaps out of it" after less than a minute, while for you apparently it never does - or it doesn't in any tolerable amount of time. Processing hasn't even started yet there.

Rostokko

Nov 25, 2024 · Edited: Nov 25, 2024

David Fercho:
Seti_Satellite_abort.txt

That's odd, the certificate of that URL seems perfectly valid: https://www.sslshopper.com/ssl-checker.html#hostname=https://download.pytorch.org/models/resnet18-f37072fd.pth

Are you sure your network isn't blocking pytorch.org? Some companies are picky about allowing download from open source python or node.js repositories...
You could try downloading it manually in a temp folder with:
curl -o resnet18-f37072fd.pth https://download.pytorch.org/models/resnet18-f37072fd.pth

Concise