AI based tools and application in present and future AP

Hello Björn,

Thank you for your excellent response.   

Fundamentally your explanation captures the key paradigm shift within AP.   AP started with hand guided telescopes and film DSLR's.   Much effort was done to enhance film sensitivity, then autoguiding enabling long exposures, then CCD and calibration routines, then low noise CMOS all looking to increase signal to noise of actual photon to sensor records.   Digital processing generally incorporates signal flow steps are reversible (i.e. data not destroyed) but often pushed to the limit of generating artifacts whether via over sharpening or deconvolution artifacts.   The result was the continuous development of better and better mounts, sensors, scopes, imaging locations and software.

It is widely accepted that some image manipulation such as cosmetic corrections, clone stamping, and similar processes which locally modify data are used for known issues in fidelity.    The reason for changing the underlying data is justifiable and explainable.

However, as you noted AI can take it a step further.  Not only is the underlying data changed it is also augmented by a guess.   The quality of the guess is often based on learning algorithms and teaching procedures that need several references… these references might include public data (such as HST), or could be made by categorizing vast databases of images on the Web (which may or may not be public domain / subject to copy right).   Certainly it would be possible to develop an AI routine that could make even the simplest datasets rival those of Hubble if proper weighting and priority to attributes was learned through the training process.   Other learning methods may be object independent and focused on the underling structures of gas flow and less obtrusive but still result in augmented detail.   However, a core tenant within AI application is that these are enhancements, not based on the data collected and generally irreversible in the final product.       As noted, today's AI tools are relatively harmless.  However, the craft is advancing quickly.

Thanks,

Hello Björn,

Thank you, corrected.   

To be clear,  I understand that within any hobby it is natural for someone to experiment, to try new tools, and to develop a deeper understanding of their craft.  AP has several paths that one can take and often each person will pursue many different paths throughout their experience.   Everyone is free to pursue their interests as they see fit as some are driven to create art, others driven for competition, and many driven by the satisfaction of hard work and desire to learn.  There are no one-size fits all guidelines that can be applied.

However, I feel in the digital age that each person has a responsibility to ensure the transparency of their public works.   As AI applications become stronger further blurring reality and expectations it is necessary to be clear about what is being represented and to avoid mischaracterizations when possible.

Thanks!
Jonathan

I concur that everyone needs to decide what their objective is.   If it's art, then pretty much everything is fair game.   If not, then some constraints should apply.

For my 2 cents, I'm not interested in the art.  Even if I could train a NNet to "fill in the holes" of my data with known / missing Hubble data, that wouldn't excite me.    What IS interesting about this hobby is to struggle with my gear (as it is), and try by effort and skill to extract as much real info of as high a quality as I can.    It's the journey to high quality via time, effort, and learned skill(s) that appeals.   I suspect that I'm in good company there.  

While the new AI technology is novel, I think it's longevity could be suspect.   After the initial "wow" factor wears off, the realization of added details that aren't real might take some luster off.   Also, taken to extremes, an even newer technology could come along that just fills in or replaces lower quality data from JWST data.   At that point, why bother taking images at all?    I'd rather know that every photon I caught and processed wound up represented in my images as best as I could get them (but real).    I can get Hubble (and JWST future) images via other means.

In that regard, I would prefer anything that can "generate" detail to be considered art, and be separated from other images that remain closer to science products.   I know good judges can tell when Topaz has been used (for example), but it would probably be better to require folks to divulge use of those products so it's clear.

Cheers,  Doug

I am a bit confused as to what the question is  or what is being questioned in this thread.  I'm not sure if it  is AP post processing freedom beyond certain parameters.  No parameters have been set that I know of.  Certainly no clear ones in the post above.  OR, is it the advancement of  software for AP post processing that makes that task easier referred to in this thread as IA.

I would like to comment on both of the above questions.

The Hubble images  (HST)have been referred to a number of times, as the example of imaging as science,  in the above comments.  As an academic (non-astronomy) I had the opportunity to meet Zoltan Levay, the primary image processor for the Hubble images for years (now retired).  Zolt was kind enough to view my mediocre images. He was responsible for the famous Pillars of Creation image along with many many others.  He talked about  the Hubble pallet and CCD processing techniques.  I think judging by the comments above,  Zolt's processing techniques may not meet the above commenter's  imaging processing criteria.

I would like to explain why I think that.  For one, The Hubble Images we see are not done for scientific purposes.  They are done for Public Relations purposes.  NASA being a US government funded agency is vulnerable to political whims and needs to maintain a high positive  profile among the US citizenship.  And YES, they do need to be PRETTY.  However, ALL the data used for the Hubble images  that the public sees comes from Hubble research projects.  But, we don't see that.  Most of the data is taken from narrow band since that is the most useful for research.  Zolt was challenged with gathering data from several research projects to put enough data together to compose an image.   Sometimes and image will be missing data on certain area in the composition.  Zolt would go to land based telescope to gain the missing data.  YES, some of the Hubble images are not 100% Hubble.  And none of it was taken by Zolt.

Color does not exist.  Not as we think of it anyway.  It is a human evolutionary development  to interpret  certain light wave lengths.  Color only exist within our perception.  Different species will interpret it differently, such as  particular insects.  The Bayer array was designed to duplicate our human color perception.
All cameras do not transfer wave lengths in the same way.  Some are more sensitive  to certain wave lengths. Most CCD/CMOS camera or Anti-Blooming and do not/can not measure star magnitude accurately.

I am a retire Art Professor and Artist.  Astrophotography is not ART.  It is CRAFT.  Believe me, you do not want Astrophotography to go to "Art".  There you might see a blank black page with the word Galaxy written on it.  The artist states , " It is what ever you perceive it to be".  Pushing the colors beyond natural is not Art, it is bad Craft.  Or, just bad taste as a social esthetic construct may evaluate. 

I prefer to purchase my own equipment at some expense.  Learn the acquisition and processing procedures and software.  I do my own acquisition and respect others that do also.  BUT, I don't make my own scopes.  I don't machine my own mounts or focuser.  i don't build my won cameras or write my software.  I didn't build my computers.  For all of that equipment,  I relied on others more knowledgeable and talented that me.    AND, I can't express enough how thankful  I am to those that made the fine equipment I own.  No one does this alone.

I have looked at a number of images on this site.  I have been very impressed with a great deal of them,  BUT, I haven't see any science research.  However,  some imagers do get involved in research projects.  I just saw an ad today requesting imagers help in a project.  I personally have never been interested spectrography.   I think wee know the great contribution Christopher Go as made with his Jupiter images.  But, I feel it's just fine if your images are primarily for your own pleasure.

Lynn K.

Thank you Johnathan. That is very different than my original take on this thread.    You are correct. It is already happening.  I'm sure many will take advantage of it's ability.  One comment I hear a lot is. " I really don't want to invest that much time/effort".

One of the real attraction of astrophotography to me is knowing the photons that went down my telescope tube and hit my sensor was traveling for thousand or maybe millions of years and just now landed on my chip.  I am then able to convert them to electron and digitize them into a recognizable image.
The whole process is simply amazing that it can happen in by backyard. 

I prefer to know that what I am seeing in the final image, is directly connected to that process.  In art we call it "Integrity of the Materials".  I know that my astro  images are not linier and has been manipulated.  Being a painter and working to a degree with 400 year old techniques, I am comfortable with that manipulation.  But I want it to be of my own doing.  In the astro-images, as in some of the areas of my painting, I try to be as accurate as possible to the source.   But,  I know they will never be.  Unlike the astro-images, I can directly compare the 3D source to the  2D reproduction, and see where my personal interpretation has altered in the paintings.   With the Astro-images I only have the data and how my Photoshop skills stretch the data.  I have no true reference to compare.  I often will look (Google) at other Asro-images of the same object, only to see a large variety of interpretations.  I may go to the Hubble image, but it too is an interpretation.

It seems that what you are suggesting is that AI software will average all those Google (or otherwise) images and plug in and average for that particular pixel or pixels. Truly democratic or generic.  I don't use PixInsight, but I have heard it has a process were it can do a plate solve and with a data base apply the correct star color to all the stars in your image.  Maybe that data is more scientifically derived.  I don't know.  I will have to admit, since I can only do Narrow Band and have difficulty with star color that star color fix is appealing.

Lynn K.

Lynn K:
suggesting is that AI software will average all those Google (or otherwise) images and plug in and average for that particula

Hello Lynn,  close but not quite correct.  The process I imagine is that AI will look at your image and based on the detail, colors, attributes “guess” at what you were trying to achieve. That would set the path on first few branches in the neural net (or other algorithm) instructing a guessed goal.   Then knowing the “target/goal” the algorithm will identify similar image attributes (from the dataset, Google, web etc) known to be good and also reject those known to be not good (the training step).   The algorithm is then able to quantify and model the best attributes of other images and make a guess of how to add the detail into your own.    It is not directly copying but filling in pixel by pixel based on neighbors in your image and referring back to what was learned in the good examples of the dataset to guess what that pixel value should be to maximize contrast or add high level frequency for apparent detail.     The algorthitms learn how the image should look on a mathematical sense but fills in the gaps when it thinks your image is too soft, too much noise, not enough resolution, out of focus or has artifacts from over sharpening or deconvolution rings on a pixel by pixel basis.    

The resulting image will look like it started with your data, make it clear and sharp but a large amount of what you see is a guess that is known to be aesthetically pleasing (hard to know it is false because it looks great) but in the end is not based on the actual data collected.

What is above is an example of a more intrusive form of AI.   I want to be clear that there are also “good” AI tools that can be made that also make other processing steps easier (okay to replace tedious manual work if it is clear what happens).  For example Masks or Auto settings for sharpening or deconvolution (even identify different settings on a sub sampling level 64x64 pixels) to maximize what is really there (at limit of producing artifacts) but in my opinion should stop short of making up data.

Painting is a CRAFT, the results of which are ART. AP is a CRAFT and the results are ART. Give 10 people the same data set and you'll get 10 unique results. That is ART. At NASA everyone does image production the same way, using the same tools. That is CRAFT. Perhaps that is a subtle distinction

If adding detail is frowned upon, what about noise removal programs….. if some form of AI was involved, would that be frowned upon or embraced.

Re:  "If adding detail is frowned upon, what about noise removal programs"….

I think we're all wrestling a bit with the new AI technology.   The AI noise reduction approach doesn't seem to offend quite as much because it's (mostly) just lowering the background.   It's still a cheat, but a less offensive cheat (where the "answer" should have been to increase integration time).   AI sharpening seems to be where there's more friction because to achieve the goal, fake details are generated and added to the "signal".   For many of us, we're trying to avoid drifting too far towards that nebulous (pun intended) "art" boundary.    In that sense, a "less is more" compromise seems to work.    A lot of us are experimenting with AI techniques but watching results closely, keeping effects on a "tight leash".    Personally, I have found myself oscillating between more and less usage as I fine tune monitoring those effects.   It should be interesting in time to see how accepted these techniques become.

Hi Bjorn,   We're in total agreement about your point of "Why are you doing AP", but maybe not quite about how well AI technologies become accepted.   If the majority in the hobby are only going for pretty pictures (whether via competition or not), then it's likely your conclusion about quick acceptance of AI technologies will pan out.   If on the other hand, most will come to appreciate (now or in the future) a closer synergism to the science (as do visual observers who appreciate what's implied by those fuzzy eyepiece images), then maybe AP will only flirt with this "sexy"/artsy technology before returning to something closer to science appreciation.     Regardless, I believe separation is warranted (especially for competition), as there is a fairly clear boundary to avoid crossing between science appreciation and art.

Use of new methods including AI strikes me as acceptable if documented; and, there is little consistency in what constitutes a controversial method, which varies with the intended audience and use of the image.

Personally I treat my Astrobin gallery as much as an on-line journal to myself, to document my AP journey and see over time how my style evolves, as to inform others. Both are important. Review of prior images' metadata helps me plan upcoming imaging goals, and to learn over time from mistakes. The metadata record (I hope) assists others in their imaging plans, perhaps if only to avoid my mistakes!

I do not post images for their scientific value and pursue the hobby for pure enjoyment, and am happy to try new techniques even if still "controversial". Even in peer reviewed scientific journals, departures from the norms of methodology are accepted as long as they are described in detail and carry a professionally reasonable scientific justification. The criteria of peer reviewed publication are strict and differ from those applied when professional astronomical organizations are communicating with the general public, because the audience and communications intents differ. For example, consider all the artistic latitude taken by NASA/ESO herein,

https://youtu.be/dmKBBzVHyUA

I have read elsewhere that in many Hubble and ESO images *all* the raw star data has been cosmetically replaced from libraries of artificial PSF images of the correct spectral fluxes for the filter sets used[1]. Depending on the downstream audience and intended use of these images, that's either reasonable or not. It won't be reasonable to many pixel peepers on Astrobin yet it is still a professionally acceptable technique.

To communicate to the public, an artistic component for AP seems justified. Like it or not, each astrophoto expresses both nature and something about the imager who created it. 

DaVinci's Mona Lisa is not a science class anatomy lesson and we do not exclude it from the category of accepted fine art because the artist failed to render the skin pores and hair follicles accurately: the gestalt matters.

Ansel Adams' Zone System photography is lauded for its depiction of nature, not rejected as "unnaturally contrasty" not in keeping with how the scene actually looks when standing there oneself.

My metadata and comments disclose any use of some new or unconventional methodology, for later reference by myself and to inform others. In this sense I am happy to give AI tools a try with considerable skepticism, as I mentioned in the image comments to,

https://www.astrobin.com/q81or1/B/?nc=user

Certainly I don't do it for unfair advantage in contesting, and disabled contesting for my gallery some time ago as I have other concerns about the rules and methodology. I do AP from the viewpoint of a retired engineer for whom new approaches in image processing and computation are technically interesting and worth experimentation. Progress in image processing won't evolve over time otherwise.

If the wider Astrobin community likes or dislikes one of my images, I see that in the Likes and infer it from the comments. But in the end I do this hobby for the sheer pleasure of being under the night sky and in pursuit of a personal exploration of the universe, for keeping mentally sharper in my elder years (because AP is hard), and to share the beauty of nature with my family, friends and the general public.

---Edit/Update:

1. Regarding PSF library star substitution in Hubble images see:

https://youtu.be/7Dy0CyUCaPs