×
INDI Library v1.8.5 Released (19 Apr 2020)

April 2020 release of INDI Library v1.8.5 introduces new drivers while providing fixes and improvements to existing devices and core framework.

Re:Guiding remains suspended

1 month 3 weeks ago 1 month 3 weeks ago by sterne-jaeger.
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51723
I stepped into a problem last night where PHD2 remains suspended after focusing finished. The Guide module assumed PHD2 would be in state GUIDING while PHD2 remained suspended. Here is the extract from the logs:
<code>[2020-04-09T01:04:27.017 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: request: "{\"id\":5783,\"jsonrpc\":\"2.0\",\"method\":\"set_paused\",\"params\":[true,\"full\"]}"
[2020-04-09T01:04:27.018 CEST DEBG ][ org.kde.kstars.ekos.capture] - setFocusStatus: 5
[2020-04-09T01:04:27.018 CEST INFO ][ org.kde.kstars.ekos.focus] - "Capturing image..."
[2020-04-09T01:04:27.243 CEST DEBG ][ org.kde.kstars.ekos.scheduler] - Focus State "In Progress"
[2020-04-09T01:04:27.285 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: event: "{\"Event\":\"Paused\",\"Timestamp\":1586387067.284,\"Host\":\"star-hunter\",\"Inst\":1}\r\n"
[2020-04-09T01:04:27.285 CEST INFO ][ org.kde.kstars.ekos.guide] - "PHD2: Guiding Paused."
[2020-04-09T01:04:27.288 CEST DEBG ][ org.kde.kstars.ekos.capture] - Guiding state changed from "Guiding" to "Suspended"
[2020-04-09T01:04:27.288 CEST INFO ][ org.kde.kstars.ekos.guide] - "Guiding suspended."
[2020-04-09T01:04:27.291 CEST DEBG ][ org.kde.kstars.ekos.scheduler] - Guide State "Suspended"
[2020-04-09T01:04:27.294 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: response: "{\"jsonrpc\":\"2.0\",\"result\":0,\"id\":5783}\r\n"
[2020-04-09T01:04:27.436 CEST DEBG ][ org.kde.kstars.ekos.guide] - PHD2: event: "{\"Event\":\"GuideStep\",\"Timestamp\":1586387067.435,\"Host\":\"star-hunter\",\"Inst\":1,\"Frame\":1987,\"Time\":4304.212,\"Mount\":\"INDI Mount [Avalon StarGo]\",\"dx\":-0.044,\"dy\":0.088,\"RADistanceRaw\":0.025,\"DECDistanceRaw\":0.096,\"RADistanceGuide\":0.025,\"DECDistanceGuide\":0.096,\"RADuration\":23,\"RADirection\":\"West\",\"DECDuration\":29,\"DECDirection\":\"South\",\"StarMass\":127352,\"SNR\":237.03,\"HFD\":4.64,\"AvgDist\":0.12}\r\n"
[2020-04-09T01:04:27.436 CEST DEBG ][ org.kde.kstars.ekos.capture] - Guiding state changed from "Suspended" to "Guiding"
[2020-04-09T01:04:27.437 CEST INFO ][ org.kde.kstars.ekos.guide] - "Guiding resumed."
</code>

The critical point is at 01:04:27.436 where PHD2 reports a guiding step although it is suspended. This has the consequence that the EKOS guiding module thinks PHD2 is in the guiding mode and sets the status to GUIDING. Here is the code snippet:
<code>void Guide::setAxisDelta(double ra, double de)
{
//If PHD2 starts guiding because somebody pusted the button remotely, we want to set the state to guiding.
//If guide pulses start coming in, it must be guiding.
if(guiderType == GUIDE_PHD2 && state != GUIDE_GUIDING)
setStatus(GUIDE_GUIDING);
</code>

As mentioned in the comments the intention behind this is that PHD2 has changed its state, but I think this assumption is at least too general. I personally tend to remove this statement, since from my point of view this is at least the wrong place to change the state.

Any thoughts about this?

As a workaround, I suggest avoiding to suspend guiding during focussing, until this problem is fixed.

- Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
knro
Administrator
Administrator
Posts: 7992
Karma: 51
Re:Guiding remains suspended #51725
IIRC, this statement was added there *specifically* as a workaround for PHD2 not correctly reporting state, so it is inferred from the axis delta. Question is, was PHD2 suspended? if yes, why was it sending GuideStep after Paused?

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51726
Yes, guiding was suspended by the Focus module for focusing. I have no idea why PHD2 made this move after pausing. Here is the extract from the PHD2 debug log:
01:04:27.247 00.002 3014541328 evsrv: cli 0x2311860 request: {"id":5783,"jsonrpc":"2.0","method":"set_paused","params":[true,"full"]}
01:04:27.247 00.000 3014541328 SetPaused type=2 isPaused=0 exposurePending=1
01:04:27.247 00.000 3014541328 Guider::SetPaused(2)
01:04:27.283 00.036 3014541328 Status Line: Paused
01:04:27.285 00.002 3014541328 evsrv: cli 0x2311860 response: {"jsonrpc":"2.0","result":0,"id":5783}
01:04:27.286 00.001 3009007664 Move returns status 0, amount 23
01:04:27.287 00.001 3009007664 MoveAxis(S, 29, ABG)
01:04:27.434 00.147 3009007664 Move returns status 0, amount 29
01:04:27.434 00.000 3009007664 move complete, result=0
As you could see it receives a set_paused request and completes it. And just a couple of milliseconds afterwards it has the MoveAxis() call. It might be the case that this is a concurrency issue where a move has been started before pausing but finished afterwards.

So I would say it is a glitch of PHD2, sure. But do we still need this state correction in the guiding module? Do we have a test case where this is (or was at least) necessary?

Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
knro
Administrator
Administrator
Posts: 7992
Karma: 51
Re:Guiding remains suspended #51727
I believe this was added by Robert to fix such issue. A workaround would be perhaps to ignore such requests if they are sent within a very narrow time frame from the paused command? say 100ms or less? it's not ideal I know. Another issue is perhaps to bring this to the attention of PHD2 developers and they might be able to do something about it.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51728

A workaround would be perhaps to ignore such requests if they are sent within a very narrow time frame from the paused command? say 100ms or less? it's not ideal I know. Another issue is perhaps to bring this to the attention of PHD2 developers and they might be able to do something about it.

Hm, a workaround for a workaround, I would like to avoid it. I'll reach out for the PHD2 team, maybe they can help.

But generally, I do not like this workaround, I would like to get rid of it. But I would like to reconstruct the test cases that led to this workaround.

- Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10
The following user(s) said Thank You TallFurryMan

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
TallFurryMan
Platinum Boarder
Platinum Boarder
Posts: 873
Karma: 13
More
Re:Guiding remains suspended #51730
I think there should be more fault tolerance management in the Guider module specifically for situations where something is changed on PHD2 by the end-user (or whatever source). If the end-user does stop guiding in PHD2 then the Guider module should not simply reflect the change but make sure guiding restarts. There are several elements, this one or connecting devices, where the current Guider code doesn't remain the real controller. Of course, this could be handled by the Scheduler, through the Guider, instead.

-Eric

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51750
This code part has been introduced within an entire set of enhancements for PHD2 guiding from Oct 2, 2019 .

I'll check it with Robert whether we can reconstruct the test cases for this specific snippet.

- Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51751
The analysis from PHD2 is clear. this behavior is due to the concurrency of parallel threads. The guide step has been issued before the pause command arrived, but has completed after pausing. See here the discussion on the PHD2 group .

I'm afraid we need to fix it on EKOS side...

- Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2253
Karma: 22
More
Re:Guiding remains suspended #51754
Yep, as you guys correctly surmised, we really need that code there because all too frequently, PHD2 will start guiding again and we don't get a notification that it did start guiding, we just start getting guide steps again. That might be because somebody pressed the guide button in PHD2 or some other reason. I left the comment there for exactly this reason, and so people would know why I did it. The issue is that sometimes JSON messages are not sent and sometimes they are not received, so we have to be sure that the code is robust. If it starts sending guide steps again, even if it didn't tell us that it was going to be guiding again, we have to assume it is in guiding mode.

Jasem might have a good idea, maybe we could ignore guide steps for a certain number of milliseconds after the pause was issued.

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51756
Do we have the option to ask PHD2 about its status? That would be very helpful if we detect behavior that is unexpected so that we can re-sync the status between PHD2 and EKOS.

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2253
Karma: 22
More
Re:Guiding remains suspended #51757
So here is the page where the API for PHD2 is described in great detail: github.com/OpenPHDGuiding/phd2/wiki/EventMonitoring
That is what I used when I redesigned the PHD2 support in KStars.

To answer your question, it says "The AppState notification is only sent when the client first connects to PHD2 (see #Initial_Messages). If an application would like to maintain an up-to-date AppState status, it will need to update its notion of AppState by handling individual notification events"

The way the PHD2 support was first designed, KStars would only change its guide state when it received a notification event. But the notifications did NOT always come in. Whether the JSON document didn't make it intact or it was never sent by PHD2, more times than I would like, the state wouldn't change to guiding.

The only way I could make it more robust was to still try to use the notification events when they do come in, but if it starts sending guide steps and it wasn't notified that PHD2 started guiding, we just have to assume that it did start guiding and didn't tell us.
The following user(s) said Thank You TallFurryMan

Please Log in or Create an account to join the conversation.

1 month 3 weeks ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 497
Karma: 6
More
Topic Author
Re:Guiding remains suspended #51763
What about using the "get_app_state" method? Due to the documentation it returns the AppState and we could distinguish Stopped, Selected, Calibrating, Guiding, LostLock, Paused and Looping.

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

Time to create page: 0.222 seconds