Error Tolerance: Reacquiring a target after an interruption - Forums

Topic starter

Frequent contributor

Apr 5, 2021

I manually start my system each night and its not uncommon for some weather event during the night to cause the observatory roof to close for some period of time before it reopens. That kind of interruption is the kiss of death for my imaging sessions–and it's been happening a lot lately. Last night the roof closed for around 40 minutes (shortly after I went to sleep) before reopening for the rest of the night. Between my mount alignment and the flexure in my OTA, my system cannot hold a guide star within the guide box for more than a short period of time so when the roof closes, my mount gets lost pretty quickly. That meant the loss of a sizable portion of the night under good imaging conditions…and this behavior is starting to drive me really crazy.

I run my system with MaximDL with FocusLock running continuously to maintain focus and APCC-Pro to control my mount. MaximDL is remarkable in that it has pretty much zero tolerance for errors. It knows when a guide star is lost but there are no options to do anything about it to recover. I've talked to the folks at DL about this and they simply have no desire to implement any kind of error recovery options. The attitude that they've expressed to me is that if something fails, well…that's just too bad. I find that attitude particularly frustrating given all the bugs that I constantly have to work around in that code.

Obviously there are other software options but the tricky part is getting FocusLock to work with them. So the first thing that I'm wondering is if anyone has written any macros for MaximDL to handle data interruptions or other errors. On the surface, it doesn't sound like it should be all that hard but I've never looked into what capabilities that MaximDL has for macro control. Next, I'm wondering if ACP Expert might be able to deal with this. It's pricy but it would also be a great way to schedule operations as well.

Anyway, I'm open to suggestions. I'll like to figure out how to better maximize my imaging time.

John

Helpful Engaging

Dave Erickson

Apr 5, 2021

Hi John,
I use a completely different set of software and suffer the same issues when clouds roll in, so not much to offer except I feel your pain.
Can you have the system send a text or email to awaken you when a critical event occurs?

Regards,
Dave

Well Written Respectful Supportive

David Redwine

Apr 5, 2021

Take a look at Lunitico's product I don't have it but it is advertised to address your problem.

CS

https://shop.lunaticoastro.com/product/gns-good-night-system/

John Hayes

Topic starter

Frequent contributor

Apr 5, 2021

Hi John,
I use a completely different set of software and suffer the same issues when clouds roll in, so not much to offer except I feel your pain.
Can you have the system send a text or email to awaken you when a critical event occurs?

Regards,
Dave

Thanks Dave. Yeah, I could do that but A) my wife will get really grumpy and B) that kind of defeats the advantage of having an automated system. Lately, it would be waking me up multiple times a night!

This just can't be all that hard to fix. The software knows when guiding fails. It just needs a set of commands that a user can select to respond--including a way to specify how often to try to re-acquire the object. It's almost enough to make me want to start programming again!

John

Well Written

John Hayes

Topic starter

Frequent contributor

Apr 5, 2021

David Redwine:
Take a look at Lunitico's product I don't have it but it is advertised to address your problem.

CS

https://shop.lunaticoastro.com/product/gns-good-night-system/

Thanks David. I'll check it out...

John

Kevin Morefield

IOTD Staff

Apr 6, 2021

CCDAutoPilot will re-platesolve, and re-search for the guide star when the star is lost. But as you say, It most likely will not work with your ONAG - though I do not know that explicitly.

Dave Erickson

Apr 6, 2021

John Hayes:
It's almost enough to make me want to start programming again!

Hi John,
Agreed, it takes a lot to get me to fall into that sink hole... But I have significant time invested in my automation software, customized for each dome and imager.
I am currently redoing the small dome instruments and software...

Dave

Ara Jerahian

Apr 6, 2021

Hi John-

Have you looked at Voyager? Many folks use it for remote observatory automation primarily for its robustness and fail proof capabilities.

You should PM @Tim Hutchison here on AB. He just did a presentation on his automated observatory setup on TAIC. His setup runs his automation continuously, day and night, although of course his Voyager logic raises a false flag when it’s daylight, but it’s always running. All he does is enter the targets he wants imaged and corresponding sequence details for that target, and that’s it. Voyager and a few Python scripts he’s written does the rest.

He seems like a genuinely good man who may be willing to help, if you so desire.

CS, Ara

Helpful Respectful Engaging Supportive

Tim Hutchison

Apr 6, 2021

Thank you, Ara, for the kind words regarding my presentation.

John - I'd be more than happy to help you solve your problem. I am running at Deep Sky West as well and re-acquiring a target after a weather event is no problem. You can easily do this with the functionality of Voyager right out of the box. No special programming required. You can even keep using maxim for camera control. The only thing I am not using is FocusLock, but I don't see that as being an issue. I'm confident you can continue to use it. Feel free to reach out if you want to discuss it.

Best.
Tim

Well Written Respectful Concise Supportive

John Hayes

Topic starter

Frequent contributor

Apr 6, 2021

Thanks guys! Voyager has been recommended both here and through PM so I'll definitely give it a serious look. Tim, I'll go check out your AIC presentation as well. I definitely need to move my software control into this century so I appreciate all the good advice.

- John

Well Written Respectful

Tim Hutchison

Apr 6, 2021

Happy to help if there is anything you'd like to bounce off of me!

(This is probably the only area of astrophotography where I might be able to answer YOUR question! :laughing-1

Take care!
Tim.

Gabe Shaughnessy

Apr 7, 2021

Another vote for Voyager. It can run sundown to sunup with full control over what events trigger suspend/exit. It's a great package for running your setup in a completely automated way. I have it running both of my imaging rigs every night.

Gabe

Well Written Concise

Michel Makhlouta

Apr 7, 2021

I don't have an observatory to comment on the roof part, but SGP has some decent recovery built in and easy to configure.

I usually set 3-4 targets per night (I have a limited FoV) with start and end times. At astro dark, it cools the camera, centers the object, focuses the camera, starts guiding and go through the sequence. Once its time is up, it moves to the next one, centering/focusing/guiding/imaging and at astro dawn, it parks the mount,, stops guiding, warms the camera and just sits there until I wake up to cover the scope or dismantle my setup. I am sure it can open and close roofs as well.

Now for the recovery. It has some pretty advanced stuff if you connect weather sensors and whatnot. But you can also simply set it to try to recovery every X minutes for Y minutes before it gives up. I usually set it for 10 minutes retries and a total of 6 hours. Every 10 minutes, it tries to recenter and move on, if it fails, it waits another 10 minutes. At the 6 hours mark, or the "end limit", it gives up and runs the end of sequence stuff I mentioned earlier.

Helpful Engaging

John Hayes

Topic starter

Frequent contributor

Apr 7, 2021

Thanks Michael. I have SGP and I originally intended to install it on the 20”, but in the interest of expediency, I started commissioning the scope with the same software that I have on the 14”. I know that old software and I just got lazy. I’ve noticed that SGP has a lot more error recovery options and it looks great. Unfortunately, trying to get PHD2 and FocusLock to talk to each other was wasting too much clear sky time so I put it aside. I’m finally getting to the point where I can pivot to resolving my software control software so I’ll have to give it a second look. I’ll look at both Voyager and SGP for both scopes. I’d ultimately love to completely dump Maxim but it turns out to be necessary for building a sky model with the Planewave so I can’t totally abandon it.

John

Jerry Macon

May 23, 2021 · Edited: May 23, 2021

Hi John,
Sorry to hear you are having so much trouble with weather dome closures and failure to recover.

I have completely solved this problem with NINA.
The new sequencer in NINA is so amazing, you can program it to do practically anything you want it to do.

I used SGP for about 7 years, from the very beginning, until 18 months ago when I shifted to NINA, never to go back.
My experience is that NINA is far more stable than SGP has ever been, and the feature set blows NINA, and every other product out of the water.

Night after night I now get absolute successful recovery of target/focus after my dome reopens from a weather triggered closure.
For years I would lose many night's of imaging because SGP could not recover.

How do I do it with NINA?
One key requirement: your dome/weather control must be a separate program. In my case I wrote a control program in the AutoIT Scripting language which monitors my weather station and completely controls the dome. So it has no interaction whatsoever with NINA. Sounds like you have a similar dome control setup.

NOTE: NINA does have complete dome control at this time. However, weather recovery is currently being worked on, so not available.

So how have I programmed NINA to recover from any weather/dome interruption?
Simple:
In the programmable sequencer I issue a command "Center After Drift" about 4 times an hour. When issued, it runs a plate solve on the last image in a separate thread (so no imaging time is used). If this differs from my target's coordinates by more than my allowed error, it automatically runs a Center command which re-centers on the target.

So NINA just takes images all night long without any interruptions, other than some re-center operations if needed because either:
1) my mount drifted enough (I don't guide), or
2) if I were guiding with PHD2 and it lost the guide star but found a bogus star and moved my mount all over the place (a common failure of PHD2), or
3) weather closed my dome

Then when weather clears and dome opens again, NINA on the 15 minute interval I have set, discovers it is no longer centered due to the call to "Center After Drift", and recenters my mount on the target.

This beautiful behavior has been working great for me for months. Just last night I started imaging at 10pm, weather was bad, not getting any good images, but plate solves and focus runs worked fine, and it wasn't overcast so the dome stayed open, so I went to bed. At 2AM the weather cleared, my dome opened, and shortly thereafter I was getting perfect images of Abell74.

I also do a auto-focus run about 2 times an hour, and those always work after a weather failure.
Lots of images on bad nights are bad. They are so easy to find. I embed the HFR and STAR COUNT in the file name (plus TEMP and FOCUSPOSITION).
Looks like this:
Abell74_Blue_20.00s_3.22_348_23482.00_17.90_2020-09-14_0002.xisf

John, if you want to go into more detail on this feel free to private msg me or a call.
Jerry

Luca Marinelli

May 24, 2021

That's a nice way to handle recovery, Jerry - thanks for explaining it in detail!

Am I correct that with your method to handle recovery you need to ensure that the roof can close with the telescope in any position and not only when it is parked? I am a strong believer in this physical configuration even in my backyard observatory as I never want to walk out to find the roof jammed against the telescope, but some people choose to live more dangerously.

Also, does NINA support FocusLock?

John, with all of SGPs shortcomings, recovery is one of the things it does quite flexibly (including after the roof closes and re-opens). SGP v4.660 has fixed bugs in the recover on safe conditions process and works well even when multiple targets are lined up in a sequence (previously it required user input if multiple targets were selected, which completely defeated the purpose of automated recovery). I use it routinely and now it works well. NINA and Voyager can use native drivers for ZWO and QHY cameras, while SGP is limited to the ASCOM driver. With the latest version of QHY, ZWO ASCOM drivers and SGP 64bit v4 beta, there is now very little difference in the available settings and information recorded to FITS header between ASCOM and native drivers. NINA can read the humidity sensor in the chamber of the QHY268M (I don't know if the QHY 600M has the humidity sensor), while SGP cannot.

Luca

Helpful Insightful Respectful Engaging

Jerry Macon

May 24, 2021 · Edited: May 24, 2021

Yes, Luca, your dome/roof must operate on it's own and be able to close, regardless of mount position. In fact, with my approach, the mount never parks until the end of session. NINA just operates as if the sky is always open. So whether it is clouds or a closed roof that occurs, it does the same recovery process. Works really well.

When I quit using SGP, the recovery code was not working effectively, and they had been working on getting it to work for over a year. I and others just finally gave up. SGP is a good and effective product, it just has too many shortcomings compared to NINA for me. The programmable sequencer in NINA is a godsend.

Another reason I found NINA superior is it is dramatically faster in everything it does. Downloading and process images took about 5 times longer with SGP than NINA. I would like to know how long the current version of SGP takes to download and process stats on one of the 62 mbit images from my ASI6200MM.
I could not get SGP working with my 6200 after I got it so I could not test this myself.

This slowness is always a PITA, but a real killer for trying to do lucky imaging.
Specifically, NINA could take and process 1 second images in a total of 2.2 seconds (1.2 second overhead). SGP took 7 seconds for the same. Overhead: 1.2 versus 6, 5 to 1. This was an older camera before the 6200.

And SGP no longer supports any native camera drivers.

I am checking to see if NINA supports FocusLock. Will keep you posted.

Helpful