RC Astro's BlurXTerminator - A Deconvolution Breakthrough? Now posted in the Tips & Techniques section of my website.

Very nicely done Pat!

John

Very nice write-up, Pat.  I had a nice chuckle about the section on The Conspiracy Camp.  Although retired now like you, I worked with AI experts and tested the technology in my life science field.  So, I got to learn how it works and many of the principles behind the technology.  Unfortunately, there is still a big learning curve as many still view it as a "black box".

Superb and helpful write up Pat. Especially the mono info!

Bob

Bruce Donzanti:
Very nice write-up, Pat.  I had a nice chuckle about the section on The Conspiracy Camp.  Although retired now like you, I worked with AI experts and tested the technology in my life science field.  So, I got to learn how it works and many of the principles behind the technology.  Unfortunately, there is still a big learning curve as many still view it as a "black box".

Hi Bruce,

Well this makes me chuckle too. Did you find out that neural network models are no longer black boxes and that they became scrutable? Please flatten the learning curve for the rest of. I already gave the example of the 2018 FICO Explainable machine learning challenge link The winning team was from IBM. The challenge to explain these models was based on this : "[...] the black box nature of machine learning algorithms means that they are currently neither interpretable nor explainable." The winner of the Fico recgonition award from the same competition, a computer scientist at Duke University recently published an article about Why black box machine learning should be avoided for high-stakes decisions in a Nature journal ; she and the Journal still view them as a black box... The article pretty much sums up the problems we face when trusting black box models, with interesting consequences that apply to our case (photography being of course more trivial than the other cases).

In another review she gives a clear definition of the problem : A black box machine learning model is a formula that is either too complicated for any human to understand, or proprietary, so that one cannot understand its inner workings.

Blur XT is both: 1) it's a very complicated unintelligible model (StarXT and Starnet have between 20-30 million parameters) 2) no details about its testing and validity are available because of commercial proprietary isssues.

Bogdan

Hi Patrick,

It is an interesting post summarizing the documentation in an intelligible way and it includes a very useful part on NN and learning.

You pretty much summed up the game change in a very funny way: ""Remember my 9-part deconvolution series ? Well just forget all of that crap and buy BlurXTerminator. The End"  I also loved the reply from Loran Hughes about your future video about it : Does YouTube accept videos under 10 seconds?

There is not much to say though about how to use it (hence Loran's joke) : it is a slider that you push more or less on a linear image you did not manipulate. It is the AI that does it. That's why you don't need to watch the 9 part series or have profound knowledge about PSF's or deconvolution. The AI will do it better than you or me. And this is what users report, and I am not talking about beginners, but very good astrophotographers : your workflow is significantly simplified and shortened. Because according to its name, it deblurs and it replaces not only the deconvolution part (which a lot of people skip anyway, including me when the gains are minimal like on narrowband), but the sharpening part too. Once stretched there's not much sharpening to do. And the AI can produce details that are better than the final result (as we could see in various posted examples), not only better than the RL deconvolution result.

However, your presentation is pretty unfair when talking about the "pushback". Of course this tool produces realistic, better than human results with practically no noise; it's gonna be popular. Popular does not mean valid though. You're making two fallacies : a Straw Man, by presenting critics as either some envious people attached to their processing skills and Poisoning the Well, by painting critics of AI based sharpening as some ignorant conspiracy theorists. Yes, there were critical replies that clearly showed some did not understand how it works or how NN work. But there also were a majority of fans that had no clue either, but since it improves their images and makes them sharp and pretty, they concluded it works. That's ok though?

The main issue with any black box AI model is trust and its validity. I posted the link to the 2022 article above. Blur XT is an AI statistical model and like any model it has an error component (this applies to all statistical models, regressions etc. AI based or not). The AI stops when there is no longer an improvement in the loss function, but that does not mean that somehow we have hit truth. It just does not descend anymore. It does not imply that it is error free in all cases and produces only true results. That's just a naive generalization.  So, explaining how NN work does not answer the question of what is the accuracy of this particular sharpening AI model. If you come to the Emergency room with chest pain, I can explain to you how logistic regression models are working, but that does not mean that the my particular model I used for your diagnosis works.

Some naively assume that BlurXT only "reveals" true details. That's simply Begging the Question. Could be the case for some regions of the image, could be wrong for other regions. A new version appeared because it produced false star colors or double stars in the periphery, including a L only version that won't interfere with color. The model was wrong for these aspects in some cases (in others it worked well). How do you establish how accurate it is for the rest of the details? It was easy to spot when the colors change or that coma becomes a double star. How do you decide if the model is wrong for the rest of the details and for which ones? That is the trust problem, not the "conspiracy" problem.

Best regards,

Bogdan

John Hayes:
Very nicely done Pat!

John

Thank you so much John!

CS,
Pat

Bruce Donzanti:
Very nice write-up, Pat.  I had a nice chuckle about the section on The Conspiracy Camp.  Although retired now like you, I worked with AI experts and tested the technology in my life science field.  So, I got to learn how it works and many of the principles behind the technology.  Unfortunately, there is still a big learning curve as many still view it as a "black box".

Thank you, Bruce!   I found working with Neural Networks fascinating. After years of first principles-based science and statistical modeling, I was intrigued by the whole machine-learning network approach.  I worked with it for about a year and a half and also played with alternative concepts, like genetic algorithms that learn through an evolutionary process that included modeled mutation rates. 

Fascinating stuff - it was always a rush seeing desired behaviors falling into place through the learning processes!

In researching for this article, I discovered there are python libraries that make it easy to implement and train these various architectures and I am tempted to play with it again.

All the best,
Pat

Bob Rucker:
Superb and helpful write up Pat. Especially the mono info!

Bob

Thank you, Bob!

Pat

Bogdan Borz:

Bruce Donzanti:
Very nice write-up, Pat.  I had a nice chuckle about the section on The Conspiracy Camp.  Although retired now like you, I worked with AI experts and tested the technology in my life science field.  So, I got to learn how it works and many of the principles behind the technology.  Unfortunately, there is still a big learning curve as many still view it as a "black box".

Hi Bruce,

Well this makes me chuckle too. Did you find out that neural network models are no longer black boxes and that they became scrutable? Please flatten the learning curve for the rest of. I already gave the example of the 2018 FICO Explainable machine learning challenge link The winning team was from IBM. The challenge to explain these models was based on this : "[...] the black box nature of machine learning algorithms means that they are currently neither interpretable nor explainable." The winner of the Fico recgonition award from the same competition, a computer scientist at Duke University recently published an article about Why black box machine learning should be avoided for high-stakes decisions in a Nature journal ; she and the Journal still view them as a black box... The article pretty much sums up the problems we face when trusting black box models, with interesting consequences that apply to our case (photography being of course more trivial than the other cases).

In another review she gives a clear definition of the problem : A black box machine learning model is a formula that is either too complicated for any human to understand, or proprietary, so that one cannot understand its inner workings.

Blur XT is both: 1) it's a very complicated unintelligible model (StarXT and Starnet have between 20-30 million parameters) 2) no details about its testing and validity are available because of commercial proprietary isssues.

Bogdan

Hi Bogdan,

You raise a good point - the massive connections of some models made them and aspects of the technology inscrutable.  Having said that, I am finding that, to a certain extent,  Convolutional Neural Network models add layers of feature detectors and pooling.  While weights and the learning process are still complex, I find the fact that the hidden layer is becoming more and more engineered to enhance the semantic knowledge of key elements that the network is learning.  This suggests that some hidden layers are becoming more understandable and, therefore, the weights are becoming more meaningful to human minds. 

I have no problem with a Network improving an image or reducing noise.    I  might not be as trusting if it were driving a pace-maker,,,

Thanks Much,
Pat

Bogdan Borz:
Hi Patrick,

It is an interesting post summarizing the documentation in an intelligible way and it includes a very useful part on NN and learning.

You pretty much summed up the game change in a very funny way: ""Remember my 9-part deconvolution series ? Well just forget all of that crap and buy BlurXTerminator. The End"  I also loved the reply from Loran Hughes about your future video about it : Does YouTube accept videos under 10 seconds?

There is not much to say though about how to use it (hence Loran's joke) : it is a slider that you push more or less on a linear image you did not manipulate. It is the AI that does it. That's why you don't need to watch the 9 part series or have profound knowledge about PSF's or deconvolution. The AI will do it better than you or me. And this is what users report, and I am not talking about beginners, but very good astrophotographers : your workflow is significantly simplified and shortened. Because according to its name, it deblurs and it replaces not only the deconvolution part (which a lot of people skip anyway, including me when the gains are minimal like on narrowband), but the sharpening part too. Once stretched there's not much sharpening to do. And the AI can produce details that are better than the final result (as we could see in various posted examples), not only better than the RL deconvolution result.

However, your presentation is pretty unfair when talking about the "pushback". Of course this tool produces realistic, better than human results with practically no noise; it's gonna be popular. Popular does not mean valid though. You're making two fallacies : a Straw Man, by presenting critics as either some envious people attached to their processing skills and Poisoning the Well, by painting critics of AI based sharpening as some ignorant conspiracy theorists. Yes, there were critical replies that clearly showed some did not understand how it works or how NN work. But there also were a majority of fans that had no clue either, but since it improves their images and makes them sharp and pretty, they concluded it works. That's ok though?

The main issue with any black box AI model is trust and its validity. I posted the link to the 2022 article above. Blur XT is an AI statistical model and like any model it has an error component (this applies to all statistical models, regressions etc. AI based or not). The AI stops when there is no longer an improvement in the loss function, but that does not mean that somehow we have hit truth. It just does not descend anymore. It does not imply that it is error free in all cases and produces only true results. That's just a naive generalization.  So, explaining how NN work does not answer the question of what is the accuracy of this particular sharpening AI model. If you come to the Emergency room with chest pain, I can explain to you how logistic regression models are working, but that does not mean that the my particular model I used for your diagnosis works.

Some naively assume that BlurXT only "reveals" true details. That's simply Begging the Question. Could be the case for some regions of the image, could be wrong for other regions. A new version appeared because it produced false star colors or double stars in the periphery, including a L only version that won't interfere with color. The model was wrong for these aspects in some cases (in others it worked well). How do you establish how accurate it is for the rest of the details? It was easy to spot when the colors change or that coma becomes a double star. How do you decide if the model is wrong for the rest of the details and for which ones? That is the trust problem, not the "conspiracy" problem.

Best regards,

Bogdan

Hi Bogdan,

Thanks for your thoughtful post. 

You are, of course, right, that there may be performance issues with BlurXTerminator, especially in its early stages of development.  And I am sure there are points of valid concern that could cause some thoughtful pushback. 

Maybe I was a bit unfair in my characterizations. 

But that was primarily driven by feedback that I thought was unfair and not thoughtful -  I saw more of that than I could stomach. 

And while it is possible that BlurXT could cause some artifacts in some cases - I have seen little evidence of that in my tests so far.  That's not to say it's not there, or I won't ever see it.

But I do know that traditional deconvolution also could cause distortions - and often did!  It is much easier to create distortions. with the traditional tool,  than it is to get good results!  It is one of the reasons that it is so hard to learn and use.  It takes a lot of experience and finesse to get good results and avoid causing distortions or artifacts. 

It seems to me that I get better results easier with BlurXT - so I value it. 

The biggest advantage of digital imaging and image processing is that you can change any aspect of an image. 
The biggest disadvantage of digital imaging and image processing is that you can change any aspect of an image. 

It's a two-edged sword. 

And as you point out - ANY algorithm - whether rule-based, statistical, or ML in nature, attempts to solve a problem - but does so with an associated error rate. As the astrophotographer, it is on you to use these tools and decide if they contribute to your vision of where you want to take your image. 

I have no problem with folks rejecting a tool because they don't like the performance or disagree philosophically with how it works.    I do have a problem with people inventing reasons to push back, and that was what I was responding to. 

It could well be a trust problem in many cases.   As you say, "It was easy to spot when the colors change, or that coma becomes a double star. How do you decide if the model is wrong for the rest of the details and for which ones?".  I think that is true of almost any advanced algorithm that tries to do sharpening, noise reduction,  or any other transform. 

To be honest, I don't really care what technology BlurXTeterminator uses.  I like the results it gives me - to my eye, they are superior to what I have been able to achieve with traditional deconvolution - despite the time and effort I put into it.

I took some time to describe Neural Network Technology because I wanted to introduce the concept of the learning process - and that just because you might use Hubble and JWST data in the learning process - it does not mean that the tool is literally replacing your pixels with Hubble pixels.  

But to your point, I think I will go back and amend the article and add a third category - " An Issue of Trust."  I think you have brought up a valid point, and I thank for doing that. 

All the best,
Pat

Excellent article and review of BlurX.  I too think it is great, and I believe only the start of more AI-based processes.

I just thought you were a bit dismissive of some concerns expressed on using BlurX to sharpen images.   I am continually asked - "are your doing astronomy, or just creating pretty pictures?"  My honest answer, as a scientist, is "it's complicated".

Foundational to deconvolution (even MORE foundational than the PSF itself) is the principle of superposition.  It is the "conservation of mass/energy" or the 1st law of thermodynamics in science, or the material/energy balance foundation of all engineering.    All of the theory and principles behind deconvolution fall back on the superposition principle, and is why deconvolution, if it is to be called deconvolution, must be done on linear images so that all the brightness adds up and none is generated or eliminated just to conform with expectations.   

If you are an astronomer, you should mistrust any  process that violates the superposition fundamental whether it is AI based or otherwise.   If you are skillful at performing deconvolution, that is a skill that is very useful for doing astronomy through a camera or any other instrument.   For example, the spectrum of light must "add up" to the total light recieved in a linear manner.    I wouldn't be at all surprised if deconvolution was performed on the JWST image that shows the "sparkle" galaxy.  The scientific law is the law, and you don't mess with the law.

If you are a photographer, BlurX indeed does an spectacular and astounding job a making your picture look better and creating awe-inspiring results.   It is doing this by deblurring your picture so that your nebulosity and stars look more like the nebulosity and stars on the images it was trained on.   It is completely uninterested in whether the image is linear or not.   It has a great advantage over deconvolution in that, by recognizing and correcting for some of the pitfalls of the math sort-cuts used to perform true deconvolution iteratively while dealing with confined/limited brightness, large contrast, and random noise,  it creates a picture that your brain finds more pleasing and considers more realistic.   However, there is a danger that the JWST "sparkle" galaxy might not appear, the sparkles eliminated, or maybe something else.

If you are like most astrophotographers and have a foot in both camps (like myself who learns about space through their camera, and yet enjoys creating nice pictures) then it is important to know what your are doing.   I already use AI for star removal and noise reduction and have come to terms with when they do a great, good, or poor job at either recognizing stars/noise or backfilling what the space looks like behind stars or underneath the noise.   I also recognize that this makes my image somewhat "made up" by the AI based on its training, but I still use them.   The same will go for BlurX - I will use it when necessary to make my images look better, but only when necessary and not completely trust what it shows me unless it can be verified by other images done with sound scientific/math principles.   so I guess I mess with the superposition sometimes, but only when the cause is good.

Thanks for your helpful review.

Pat, the video is an excellent companion to your article and well worth the time. I've watched 3 other videos on the new tool by other astrophotographers which were helpful but seemed focused on OSC cameras. Since I'm a mono imager, your workflow examples were very helpful. This tool has been a game changer since traditional deconvolution was hit or miss (mostly miss) for me. Thanks again for the well presented information.

CS
Bob