×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Re:Guiding remains suspended

  • Posts: 1185
  • Thank you received: 370
I stepped into a problem last night where PHD2 remains suspended after focusing finished. The Guide module assumed PHD2 would be in state GUIDING while PHD2 remained suspended. Here is the extract from the logs:
<code>[2020-04-09T01:04:27.017 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: request: "{\"id\":5783,\"jsonrpc\":\"2.0\",\"method\":\"set_paused\",\"params\":[true,\"full\"]}"
[2020-04-09T01:04:27.018 CEST DEBG ][ org.kde.kstars.ekos.capture] - setFocusStatus: 5
[2020-04-09T01:04:27.018 CEST INFO ][ org.kde.kstars.ekos.focus] - "Capturing image..."
[2020-04-09T01:04:27.243 CEST DEBG ][ org.kde.kstars.ekos.scheduler] - Focus State "In Progress"
[2020-04-09T01:04:27.285 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: event: "{\"Event\":\"Paused\",\"Timestamp\":1586387067.284,\"Host\":\"star-hunter\",\"Inst\":1}\r\n"
[2020-04-09T01:04:27.285 CEST INFO ][ org.kde.kstars.ekos.guide] - "PHD2: Guiding Paused."
[2020-04-09T01:04:27.288 CEST DEBG ][ org.kde.kstars.ekos.capture] - Guiding state changed from "Guiding" to "Suspended"
[2020-04-09T01:04:27.288 CEST INFO ][ org.kde.kstars.ekos.guide] - "Guiding suspended."
[2020-04-09T01:04:27.291 CEST DEBG ][ org.kde.kstars.ekos.scheduler] - Guide State "Suspended"
[2020-04-09T01:04:27.294 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: response: "{\"jsonrpc\":\"2.0\",\"result\":0,\"id\":5783}\r\n"
[2020-04-09T01:04:27.436 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: event: "{\"Event\":\"GuideStep\",\"Timestamp\":1586387067.435,\"Host\":\"star-hunter\",\"Inst\":1,\"Frame\":1987,\"Time\":4304.212,\"Mount\":\"INDI Mount [Avalon StarGo]\",\"dx\":-0.044,\"dy\":0.088,\"RADistanceRaw\":0.025,\"DECDistanceRaw\":0.096,\"RADistanceGuide\":0.025,\"DECDistanceGuide\":0.096,\"RADuration\":23,\"RADirection\":\"West\",\"DECDuration\":29,\"DECDirection\":\"South\",\"StarMass\":127352,\"SNR\":237.03,\"HFD\":4.64,\"AvgDist\":0.12}\r\n"
[2020-04-09T01:04:27.436 CEST DEBG ][ org.kde.kstars.ekos.capture] - Guiding state changed from "Suspended" to "Guiding"
[2020-04-09T01:04:27.437 CEST INFO ][ org.kde.kstars.ekos.guide] - "Guiding resumed."
</code>

The critical point is at 01:04:27.436 where PHD2 reports a guiding step although it is suspended. This has the consequence that the EKOS guiding module thinks PHD2 is in the guiding mode and sets the status to GUIDING. Here is the code snippet:
<code>void Guide::setAxisDelta(double ra, double de)
{
//If PHD2 starts guiding because somebody pusted the button remotely, we want to set the state to guiding.
//If guide pulses start coming in, it must be guiding.
if(guiderType == GUIDE_PHD2 && state != GUIDE_GUIDING)
setStatus(GUIDE_GUIDING);
</code>

As mentioned in the comments the intention behind this is that PHD2 has changed its state, but I think this assumption is at least too general. I personally tend to remove this statement, since from my point of view this is at least the wrong place to change the state.

Any thoughts about this?

As a workaround, I suggest avoiding to suspend guiding during focussing, until this problem is fixed.

- Wolfgang
Last edit: 3 years 11 months ago by Wolfgang Reissenberger.
3 years 11 months ago #51723

Please Log in or Create an account to join the conversation.

IIRC, this statement was added there *specifically* as a workaround for PHD2 not correctly reporting state, so it is inferred from the axis delta. Question is, was PHD2 suspended? if yes, why was it sending GuideStep after Paused?
3 years 11 months ago #51725

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Yes, guiding was suspended by the Focus module for focusing. I have no idea why PHD2 made this move after pausing. Here is the extract from the PHD2 debug log:
01:04:27.247 00.002 3014541328 evsrv: cli 0x2311860 request: {"id":5783,"jsonrpc":"2.0","method":"set_paused","params":[true,"full"]}
01:04:27.247 00.000 3014541328 SetPaused type=2 isPaused=0 exposurePending=1
01:04:27.247 00.000 3014541328 Guider::SetPaused(2)
01:04:27.283 00.036 3014541328 Status Line: Paused
01:04:27.285 00.002 3014541328 evsrv: cli 0x2311860 response: {"jsonrpc":"2.0","result":0,"id":5783}
01:04:27.286 00.001 3009007664 Move returns status 0, amount 23
01:04:27.287 00.001 3009007664 MoveAxis(S, 29, ABG)
01:04:27.434 00.147 3009007664 Move returns status 0, amount 29
01:04:27.434 00.000 3009007664 move complete, result=0
As you could see it receives a set_paused request and completes it. And just a couple of milliseconds afterwards it has the MoveAxis() call. It might be the case that this is a concurrency issue where a move has been started before pausing but finished afterwards.

So I would say it is a glitch of PHD2, sure. But do we still need this state correction in the guiding module? Do we have a test case where this is (or was at least) necessary?

Wolfgang
3 years 11 months ago #51726

Please Log in or Create an account to join the conversation.

I believe this was added by Robert to fix such issue. A workaround would be perhaps to ignore such requests if they are sent within a very narrow time frame from the paused command? say 100ms or less? it's not ideal I know. Another issue is perhaps to bring this to the attention of PHD2 developers and they might be able to do something about it.
3 years 11 months ago #51727

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Hm, a workaround for a workaround, I would like to avoid it. I'll reach out for the PHD2 team, maybe they can help.

But generally, I do not like this workaround, I would like to get rid of it. But I would like to reconstruct the test cases that led to this workaround.

- Wolfgang
The following user(s) said Thank You: Eric
3 years 11 months ago #51728

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic Re:Guiding remains suspended

I think there should be more fault tolerance management in the Guider module specifically for situations where something is changed on PHD2 by the end-user (or whatever source). If the end-user does stop guiding in PHD2 then the Guider module should not simply reflect the change but make sure guiding restarts. There are several elements, this one or connecting devices, where the current Guider code doesn't remain the real controller. Of course, this could be handled by the Scheduler, through the Guider, instead.

-Eric
3 years 11 months ago #51730

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
This code part has been introduced within an entire set of enhancements for PHD2 guiding from Oct 2, 2019 .

I'll check it with Robert whether we can reconstruct the test cases for this specific snippet.

- Wolfgang
3 years 11 months ago #51750

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
The analysis from PHD2 is clear. this behavior is due to the concurrency of parallel threads. The guide step has been issued before the pause command arrived, but has completed after pausing. See here the discussion on the PHD2 group .

I'm afraid we need to fix it on EKOS side...

- Wolfgang
3 years 11 months ago #51751

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
Yep, as you guys correctly surmised, we really need that code there because all too frequently, PHD2 will start guiding again and we don't get a notification that it did start guiding, we just start getting guide steps again. That might be because somebody pressed the guide button in PHD2 or some other reason. I left the comment there for exactly this reason, and so people would know why I did it. The issue is that sometimes JSON messages are not sent and sometimes they are not received, so we have to be sure that the code is robust. If it starts sending guide steps again, even if it didn't tell us that it was going to be guiding again, we have to assume it is in guiding mode.

Jasem might have a good idea, maybe we could ignore guide steps for a certain number of milliseconds after the pause was issued.
3 years 11 months ago #51754

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Do we have the option to ask PHD2 about its status? That would be very helpful if we detect behavior that is unexpected so that we can re-sync the status between PHD2 and EKOS.
3 years 11 months ago #51756

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
So here is the page where the API for PHD2 is described in great detail: github.com/OpenPHDGuiding/phd2/wiki/EventMonitoring
That is what I used when I redesigned the PHD2 support in KStars.

To answer your question, it says "The AppState notification is only sent when the client first connects to PHD2 (see #Initial_Messages). If an application would like to maintain an up-to-date AppState status, it will need to update its notion of AppState by handling individual notification events"

The way the PHD2 support was first designed, KStars would only change its guide state when it received a notification event. But the notifications did NOT always come in. Whether the JSON document didn't make it intact or it was never sent by PHD2, more times than I would like, the state wouldn't change to guiding.

The only way I could make it more robust was to still try to use the notification events when they do come in, but if it starts sending guide steps and it wasn't notified that PHD2 started guiding, we just have to assume that it did start guiding and didn't tell us.
The following user(s) said Thank You: Eric
3 years 11 months ago #51757

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
What about using the "get_app_state" method? Due to the documentation it returns the AppState and we could distinguish Stopped, Selected, Calibrating, Guiding, LostLock, Paused and Looping.
3 years 11 months ago #51763

Please Log in or Create an account to join the conversation.

Time to create page: 0.186 seconds