×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

"Dome parking / Telescope parking" policy

I'm actually thinking that this whole discussion highlights that this management of what shutdown first should be completely left out of the drivers. The INDI drivers should be minimal and do their job. Right now, we have communication between drivers that are trying to perform some complex interlocking shutdown procedure. This should be definitely left out to clients to handle in my opinion. Maybe the whole thing should be removed from INDI drivers.

Nevertheless, there is ONE specific drivers that's whole purpose is to protect these drivers, it's the Watchdog driver. Currently, the watchdog driver performs a complete shutdown if it loses connection with the client. I suppose this driver can be extended to snoop weather data as well. This way, you don't really have to use Ekos (if you want to) to perform shutdown and just let the watchdog driver handles it for it. It would remove all the complex interlocking scenarios we have to account for. Such options could be left out the clients, whether that client in INDI Watchdog driver, or Ekos or CCDCiel ...etc.
The following user(s) said Thank You: Eric
3 years 8 months ago #56740

Please Log in or Create an account to join the conversation.

  • Posts: 554
  • Thank you received: 138
I see your point but isn't the whole Indi snoop system designed to allow drivers to interact at the driver level? This seems to be useful for interactions such as telescope/dome synchronising.

Responding to weather also seems to be in this category because it is the sort of thing that should be running all the time, regardless of what applications are running.

What we have at present seems to be partially implemented in indi, and partially implemented in the Ekos Observatory and Scheduler modules. This is where I think it has difficulties. It is difficult to know where and how to set it up. It also seems only to be fully active when a sheduler task is running.

What we have in the drivers at the moment seems close. but what I'd do is make each driver only responsible for actions to it's own hardware, something like this:
The Dome driver snoops on weather and telescope and has a dome policy property which specifies when the dome/roof can be parked. This manages the dome parking depending on the snooped states of the weather and telescope.
The Telescope driver snoops on the weather and dome. The only property is needs is a boolean to say if it must park when the weather is bad. It doesn't need to wait for anything. When the weather reports bad it parks.
The Weather driver handles generating the warnings and alerts. If delays to prevent false alerts because of transient events are reuired these are implemented here because this driver is aware of the weather properties it is monitoring. For example it may want to delay because of a cloud event but definitely not if it starts raining.
That's it.
When a weather alert is raised the telescope sees this and parks. The dome/roof also sees this. It can close immediately if its policy allows or it will wait until the telescope is parked as well before parking.

Ekos need not be involved at all although there's noting to stop it handling things for itself, in particular the routine close down at the end of a session.

The main change seems to be to add weather snooping to the telescope and the associated telescope park if the weather is unsafe. The dome driver may not need changing at all.

Not sure how Jasem's watchdog driver should be involved, it seems to be a combination of a driver and a client. When it's trigggered it sends park commands to the drivers. It could have weather snooping added, becoming a fourth way to manage shut down.
The following user(s) said Thank You: Jasem Mutlaq
3 years 8 months ago #56743

Please Log in or Create an account to join the conversation.

I am working now on the updated Watchdog driver, which is indeed driver/client combination. I believe with this driver, it removes the whole orchestration complexity required for this shutdown procedure due to weather. I will share the driver once done.
3 years 8 months ago #56744

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
I think the complexity here comes that there is not THE dome and THE telescope, but many variants. And on top of it, quite complex decisions to be taken how to act. Do we need a threshold before reacting on alerts, how do we react on warnings and that in combination with all the variants whether a dome and its shutter should wait for the scope being parked.

My conclusion is, we need to separate orchestration of devices from lower level device control. Being it the watchdog, the observatory module, the scheduler or something entirely separate - everything has its ups and downs.

Going ahead and increasing the dependencies between the single drivers is the wrong way. Drivers need to be 100% robust, therefore we should leave the "stupid" and accept, that they cannot resolve every possible situation and leave the more sophisticated controls to the layer above.

Wolfgang
3 years 8 months ago #56745

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Fully agreed! If we find here a common understanding, how it should be, I will go ahead and implement it. But currently, I do not see an agreement how it should be.

I personally tend to a logic where the observatory has a "Start" and a "Stop" button which both have a flexible way to graphically configure the start and stop procedures. If this is in place, I would replace the startup and shutdown procedures from the Scheduler by a simple option to call "Start" and "Stop" in the Observatory.

So please give me feedback if this is the right direction - or share your own ideas - so that we can move forward.

Cheers
Wolfgang
3 years 8 months ago #56746

Please Log in or Create an account to join the conversation.

Ok I pushed the updated driver. This also requires updated Ekos since the client watchdog now uses seconds instead of minutes to be consistent to weather watchdog. So to test, both INDI+KStars must be updated to latest GIT.

Now the watchdog driver has a Client and Weather watchdogs that can toggled. Previously, it was only client watchdog which would trigger shutdown after communication loss with client. Now I added weather watchdog as well.



Regarding mount locking, it's added in Options



So the watchdog driver should be now sufficient without any external clients (Ekos..etc). The watchdog is a driver/client hybrid so it can orchestrate this without requiring any complex inter-driver communication. Please test this and report back any issues. If all goes well, I believe we ought to remove the weather handling in dome class completely. I even think the mount/dome locking policies can be removed and moved to the watchdog driver. The watchdog driver can monitor both drivers and enforce the policy if it detects a violation. But this is another issue, let's tackle it one at a time.
3 years 8 months ago #56755
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Looks really nice! What about if we give the Watchdog a UI representation in EKOS but leaving the entire controlling logic inside the watchdog driver? And we should then be so consequent and remove all dome and weather related stuff from the scheduler as well.
3 years 8 months ago #56756

Please Log in or Create an account to join the conversation.

  • Posts: 554
  • Thank you received: 138
Is Indi only expected to run with Ekos? I thought that Indi was a generic astro hardware control standard, much as ASCOM is.

The Watchdog looks good, am pulling and rebuilding. Keeping this away from people who don't need it is a good idea.

I'd keep the Watchdog entirely separate from Ekos so it becomes available to other applications.

I'd be in favour of removing the snooping from the dome and telescope base drivers although the dome may need to continue to snoop on the telescope to implement dome sync. One thing, are there other clients that are depending on the current dome and telescope park interactions?

How complex are the scope/dome interactions in reality? In some cases the scope must be parked before the dome/roof is parked and the dome/roof unparked before unparking and moving the telescope but that's it isn't it? One boolean - park dome/roof first or not.

Managing alerts and warnings with appropriate delays seems to me to belong in the driver that generates the alerts and warnings. With ASCOM we had safety monitors with one property - Safe. We could have multiple monitors for example a SafeToOpen monitor to manage opening and closing the observatory and a SafeToObserve monitor to manage the observing process. SafeToObserve could go false on thin cloud or twilight, SafeToOpen may need heavy cloud or strong winds. With the warn and alert system I'd use warn to manage observing and alert to manage the observatory.

As for agreement full agreement is probably not possible. Somebody is sure to have a system that this can't cope with. Something that works for the majority of people is probably the best that can be done. There is always the option of turning the WDT off.
3 years 8 months ago #56789

Please Log in or Create an account to join the conversation.

No the watchdog is completely independent from Ekos. The CLIENT watchdog must have explicit support from clients though. i.e. the client must actively renew the heartbeat. Dome/Telescope should snoop on each other for slaving and maybe prevent inadvertent collisions only (though this latter part can be delegated to Watchdog as well). So I'd say we need to remove any unnecessary interactions and if we get consensus on the watchdog, the weather handling should be removed from the Dome class.
3 years 8 months ago #56794

Please Log in or Create an account to join the conversation.

  • Posts: 311
  • Thank you received: 42
Tried the WatchDog with simulators.

Attempted to remove the roof snooping on the weather so it would not be involved in detection. The set on the weather
field did not move the right field value into the left field. Should confirm in some way that with different weather delay thresholds between the roof and the watchdog there is only a single weather detection and park request.

The WatchDog continues to show a green status after the weather alert is raised.

First test was with telescope policy set to ignore dome, the dome policy set to ignore telescope.
The WatchDog to park both which it does. As before without the WatchDog, the mount and the roof park together.
The WatchDog outputs the "Shutdown" complete message when the roof completes parking, the telescope is still in the process of parking.

Second test was with the Dome Policy set to telescope locks. The behavior is as before without the WatchDog the telescope parks and the roof blocks with warning messages at the indi level. When the telescope has parked, the lock is removed,the roof remains open. When the roof is later manually closed the WatchDog outputs Shutdown complete and the status light on the Shutdown line turns from yellow to green. When the weather alert clears the WatchDog outputs message.

This meets expectations of detecting weather alerts and being able to effect park commands.
3 years 8 months ago #56864

Please Log in or Create an account to join the conversation.

  • Posts: 554
  • Thank you received: 138
AIUI the idea of the new Watchdog is that it can manage the telescope/dome/ROR parking in bad weather with no intervention by the the telescope and dome/ROR drivers. The intent is that the existing auto park behaviour in the telescope and dome base drivers is removed. This means:
Telescope Dome policy in the telescope driver must be set to none.
In the ROR driver the Dome Telescope policy is set to none and AutoPark is set to Disabled.
The Ekos Observatory module has the alert and warning actions unchecked.
The scheduler need not be running.

In the WDT simulator main tab:
Weather threshold is set to 0 so that the close down happens immediately.
The trigger is set to Weather.
Shutdown has Park Dome and Park Telescope checked.
In the WDT Simulator Options tab:
The Mount Policy is set to "Mount locks. Dome must wait for mount to park before it can start the parking procedure."

Set the weather to unsafe (I start it raining).
This works as expected. The Telescope starts to park and when it reports parked the ROR starts to park. The times in the various logs bear this out. I assume that Park also closes the shutter.
Stop the rain, unpark ROR, unpark Scope.

Just a thought, should the scope be allowed to unpark when the ROR is parked? If that should be inhibited then the telescope will still need to snoop on the dome and check the dome park status. The Dome policy would still need to be available because not all users require this.

Set the Mount policy (in the WDT) to "Mount is ignored. Dome can start parking without waiting for mount to complete parking. "
Start the rain:
The telescope and ROR start to park at the same time as expected.

The telescope can be unparked and slewed to an object with the ROR parked and the weather unsafe.
Setting the Telescope Dome Policy to "Dome parking policy set to: Dome locks. This disallows the scope from unparking when dome is parked" will prevent this so it looks as if the dome snooping needs to be retained and the policy still be available for the ignore and dome locks states at least.

This seems pretty close. It seems easier to set up the observatory behaviour in the WDT than in multiple places, the only outstanding thing is preventing telescope unpark and slews when the roof is closed, that is set in the telescope driver.

I've not looked at how Ekos copes with an observatory that's closed down but even if it keeps going the equipment should be safe. Ekos can monitor the weather and do it's own orderly shut down.

Opening and starting or resuming a sequence when the weather improves probably belongs in Ekos.
3 years 8 months ago #56897

Please Log in or Create an account to join the conversation.

Great analysis Chris. So yes as you correctly pointed out, the watchdog driver now does not manage whether parking/unparking is permitted given the policy. I think this is next step. Maybe this should pave the way to remove all the interlocks logic from dome/mount drivers and just let Watchdog manage these for those who are worried about this situation. This certainly simplifies the drivers quite a bit and they can concentrate on their core functionalities instead.

Consequently, when the user selects "Mount locks" then that means it should prevent mount from unparking if the dome is parked. Since now this is implemented in the mount driver, the unparking itself is prevented from happening in the first place. However, if we move this functionality to Watchdog, then the mount will unpark normally. However, upon detecting this, the watchdog driver would then force a park command to the mount to revert the unparking action. It's not ideal of course, but if we consider moving this logic completely outside, then this is how I see it panning out.
The following user(s) said Thank You: Gonzothegreat
3 years 8 months ago #56898

Please Log in or Create an account to join the conversation.

Time to create page: 0.733 seconds