New scripts for PixInsight: ExtensionToolset

16 replies535 views
Astrogerdt avatar
Hi everyone,

I have developed ExtensionToolset, a collection of scripts to support image processing and analysis in PixInsight.

Features included in ExtensionToolset:

AdditiveColorCalibration: A high-accuracy and robust additive color calibration tool based on a background preview. In contrast to current tools, it is more robust in case the background preview contains
stars, a common situation in widefield imaging. It is not intended to replace SPCC or other tools, it should be considered as yet another tool capable of handling one specific problem case better than others.
BTM: The Brightness and Transparency Meter is a tool that allows you to measure the night sky brightness in magnitudes per square arcsecond and the transparency in percent from a single
calibrated image and a few acquisition details. Along with these values, it also calculates the USQ (Unified Sky Quality) index, which is a measure of the quality of the night sky further
explained later in the post.
USQCalculator: The Unified Sky Quality index Calculator allows you to calculate the USQ index based on already known night sky brightness and transparency values. This makes it easier
to compare conditions across multiple places and photographers.
GradientStrength: for a given series of images, you can calculate the gradient strength score for each image to automatically select the best subset of images for the generation of a
LocalNormalization reference image. This is purely based on gradient strength inside the images, intended to simplify the gradient in the integrated image, not based on SNR.
ExtensionToolsetOverview: in case you need a quick high-level overview over all features, release notes or other important information, all that is bundled here.

Installation:
For installation, add this repository to your PixInsight update repository list: https://gerriterdt.github.io/ExtensionToolset-release/

You can manage your update repository list under Resources -> Updates -> Manage Repositories -> Add.
After adding the repository, check for new updates and follow the instructions.

Usage:
Once installed, you can find the scripts in the Script menu under the entry "ExtensionToolset".

Each script includes a standardized description at the top, explaining the use case, preconditions, and results of the script.
For a full documentation of each script, you can refer to the tooltips of each individual element on the user interface of the script.

Known limitations:
- The measurement of the night sky transparency often yields accurate results, but in case you use images with a very small FOV, a low SNR, or a combination of both, the results become less accurate. Generally, larger FOV images with a better SNR are beneficial for increased accuracy. I hope to improve this in the future.

Further details about the USQ:
Since the USQ index is a new concept, I want to explain it in more detail.

Most of the time, it is hard to compare the sky quality for imaging purposes across different nights or locations. Clearly, darker skies are better, but how much better are they actually, and how much better is a dark sky if it is not as transparent as a brighter sky?

The USQ index solves this problem by combining both metrics of quality into a single index. The USQ index is a value between 0 and 100, where 0 is the worst possible sky quality and 100 is the best possible sky quality. It is proportional to the SNR of a well‑exposed image taken under these conditions.

In practice, this means that if you have calculated a USQ index of 20 for location A and a USQ index of 60 for location B, then you can expect a well‑exposed image taken at location B to have an SNR of 60 / 20 = 3 times the SNR of a well‑exposed image taken at location A with all other conditions being equal. That means that, at location A, you have to expose 3^2 = 9 times as long for the same SNR compared to location B.

Of course, there are some limitations here, but I have tried to make them as small as possible:

The USQ index is only defined for night‑sky brightness between 16.0 and 22.5 magnitudes/arcsec^2. Values outside this range are technically possible, but will leave the [0, 100] range.
While the USQ index is proportional to the SNR of well‑exposed images, it is not of great help for images that are not well‑exposed. Additionally, the proportionality is valid for most of the possible values, but it is not proportional for values very close to the boundaries, i.e. a SQM of exactly 16.0 is not perfectly in the proportional range, as the USQ will be 0. This is, however, of low practical relevance, since the boundaries are pretty wide, and we generally aim to have well-exposed images anyway.
The SNR that I have used here is the SNR of large objects, not of small objects like stars. The effects of seeing are not included in the USQ index.
I have the strong hope that the USQ index can be established as a standard measure for night‑sky quality, improving the community’s ability to compare locations and nights with each other, fostering a better understanding of the quality of images by other astrophotographers and what can be achieved by changing locations.

Troubleshooting & Feedback:
If you encounter any issues or have suggestions, feel free to reach out to me through any of the forums where I have published the initial announcement. Feedback and feature requests are always welcome!

Also, I will continue to develop more features in the future, further enhancing the ExtensionToolset. If you have any ideas or suggestions, let me know - I will be happy to consider them.

For reference, here is the original announcement in the PixInsight forum: https://pixinsight.com/forum/index.php?threads/new-scripts-extensiontoolset.26110/#post-173931

CS Gerrit
Helpful Engaging
Tony Gondola avatar
Sounds useful. Any plans for doing these for Siril as well?
Well Written Respectful
Astrogerdt avatar
Glad to know you like the idea ;)

As for Siril or any other image processing application: no, I don't have plans to develop ExtensionToolset for any other image processing platform. 
While the idea of supporting more platforms in interesting and would definitely reach more astronomers, there are some technical issues involved. I know of no common way to write scripts for PixInsight and any other processing platform, which would mean I have to re-implement every feature in different runtime environments. Additionally, I rely on the PSF fitting routines and other details of PixInsight, which are different for other platforms too. 

I can't out rule that I will re-consider this in the future, but I wouldn't count on that ever happening, unfortunately. 

CS Gerrit
Helpful Insightful Respectful
andrea tasselli avatar
Alas, it doesn't seem to exist: 

https://gerriterdt.github.io/ExtensionToolset/: Network error. Response code: 404
Astrogerdt avatar
Did you open the link in a browser or add it to the update repositories in PixInsight

Opening it in the browser will yield a 404, but downloading the updates in PixInsight should work using that link. There is just nothing to show on the page.
I just re-installed all updates on my system, including this one, and it worked for me.

Let me know if you continue to see this issue.

CS Gerrit
Concise
Michael Nemetz avatar
Did you open the link in a browser or add it to the update repositories in PixInsight

Opening it in the browser will yield a 404, but downloading the updates in PixInsight should work using that link. There is just nothing to show on the page.
I just re-installed all updates on my system, including this one, and it worked for me.

Let me know if you continue to see this issue.

CS Gerrit

Doesn't seem to work as a repo. 404
andrea tasselli avatar
Michael Nemetz:
Did you open the link in a browser or add it to the update repositories in PixInsight

Opening it in the browser will yield a 404, but downloading the updates in PixInsight should work using that link. There is just nothing to show on the page.
I just re-installed all updates on my system, including this one, and it worked for me.

Let me know if you continue to see this issue.

CS Gerrit

Doesn't seem to work as a repo. 404

Ditto.
Astrogerdt avatar
Found the error on my side, I somehow copied the wrong link into the announcement message..... Sorry for the inconvenience!

Here is the correct link: https://gerriterdt.github.io/ExtensionToolset-release/

I edited the initial post to prevent further confusion.

CS Gerrit
Helpful Concise
andrea tasselli avatar
Unable to download repository information.

https://gerriterdt.github.io/ExtensionToolset-release/: Parsing repository update information: Parsing Signature element (line=1 offset=0): Unknown code signing identity 'GerritErdt'
Astrogerdt avatar
Have you updated PixInsight in the last week?

PixInsight can parse unknown CPD IDs, since I have been approved pretty recently you probably don't have the current cps list that includes me. 

If that is the case, you can fix this by removing my repo, updating everything else, adding my repo and updating again. 

CS Gerrit
Concise
andrea tasselli avatar
Doesn't work
Astrogerdt avatar
Any more information? Can't really help if I don't know what's going on….
andrea tasselli avatar
Does it depends on PI's build? Because I'm testing this on 1.8.9-3 build 1612.
Astrogerdt avatar
Yes, that can be the issue here. 

I currently set it to versions >1.9. 
The reasoning is that I have done many tests with the star matching routine I implemented and am pretty certain of its accuracy and robustness, which depends on both plate solving and PSF fitting. 

I know that there have been some big changes to plate solving in 1.9 and I think also some to psf fitting, so I didn't want to risk false results with older versions. And I thought that enough people were using 1.9 to not make this decision an issue. 

I have to publish an update anyways tomorrow, so I can consider changing that. But I can't guarantee that everything will work perfectly, there might be some subtle changes that are hard to detect. 
​​​​​​
CS Gerrit
Helpful
Astrogerdt avatar
Hi, 

 
I have just released an update with a bug fix, that should fix an issue reported on the PixInsight forum. 
Additionally, I have changed the version requirement to include 1.8.9 and onwards. 
However, I want to point out basically all testing I have done has been done on 1.9 onwards. I know of no difference that would cause major issues, which is why I will give it a try, but I don't have done extensive testing for that version. 

 

I hope that this solves the issues. 

Also, I will still post about updates here for some time, but in the long term, I will provide all relevant information over the PixInsight forum, in the announcement thread. Here is the link: https://pixinsight.com/forum/index.php?threads/new-scripts-extensiontoolset.26110/

 

CS Gerrit
andrea tasselli avatar
Same error again using that repo.
Brian Diaz avatar

hi

Sounds great thgis script, i will try next time

thank you

CS

Brian