×

INDI Library v2.0.7 is Released (01 Apr 2024)

Bi-monthly release with minor bug fixes and improvements

Read More...

trouble with eqmod mount disconnecting

  • Posts: 535
  • Thank you received: 109

Replied by Jim on topic trouble with eqmod mount disconnecting

Last night, with logging on from the start, everything worked without issue again. So far the correlation would be, always run full logging B)

Will leave it on for it to happen again.
6 years 1 month ago #24202

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic Re:trouble with eqmod mount disconnecting

Ah ! I got the exact same issues as you all with my HEQ5. I had disconnects in the past, with indi from June '17 built from source, but I would relate those to cables and stuff getting in the way of rotation. Plus, the mount would complain about the voltage stability, because it was dropping under 11V too often.

However when I rebuilt the January version of INDI, and proceeded to observe during 5 consecutive nights under 0°C end of February, I started to observe weird disconnection issues precisely like you describe.

The mount and its eqmod driver would work flawlessly during one full night. Mount was parked but powered the whole day, as well as the INDI server. The following night I would have those disconnections without any warning.

Specifically, I remember imaging M42 for 20min, then switching to the Bubble nebula, to the west. The mount disconnected on its slew to the target. Incidentally, it crashed the Atik driver at the same time. Both drivers would restart, but the eqmod driver wasn't able to reconnect to the mount.

At that moment, I restarted the whole indiserver, and for whatever reason, the eqmod driver was able to reconnect the mount. I tested slewing around and it was working. Then I slewed back to Bubble, and the driver lost the mount again with the same messages, with Atik crashed too, roughly at the same coordinates location.

To recover from this situation, I had to turn the mount off and back on again. No amount of effort on indiserver or connection cables would restore functionality.

I'm really puzzled with this issue, my current opinion is that this is a regression, but there are several parameters playing together that are difficult to separate.

-Eric
Last edit: 6 years 1 month ago by Eric.
6 years 1 month ago #24254

Please Log in or Create an account to join the conversation.

  • Posts: 152
  • Thank you received: 28

Replied by Petar Milevski on topic Re:trouble with eqmod mount disconnecting

Hi Jasem just to report back, last night was again a bit of a disaster. indi_eqmod_telescope performed ok with no disconnects this time, but the internal guider was not happy and my RPi3 indiserver lost wifi connection to my client straight after guiding stopped, which ultimately shut everything down. So i had to restart the rpi3 on numerous occasions.

As i said i would do, I logged everything, see attached files.

regards
6 years 1 month ago #24268
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic trouble with eqmod mount disconnecting

I'm checking another feature (repeat options in the scheduler), but I just had four full days of testing with EQMod Mount and CCD Simulator in my basement.

Not a single disconnection, although test coordinates were spread randmoly over the whole hemisphere.

Quite a few unrelated issues that I'll be working on (scope hitting pier for 4 hours is one of them :S, as well as non-functional watchdog) but nothing at all related to the disconnection problem. I'll add other drivers, increase the size of the test vector and test again in the coming days.

-Eric
The following user(s) said Thank You: Jim
6 years 3 weeks ago #24513

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic trouble with eqmod mount disconnecting

I'm now at 7 days of testing. I'm not scheduling jobs 100% of the time, but at least the setup is powered and indiserver is running 24/24.
No disconnections as clear as what was reported were encountered.

I had one short occurrence of read/write glitches which got notified as popups in Ekos, but I wasn't able to relate them to anything in the log :huh:

I had one crash in indi_eqmod_telescope, probably possibly maybe related to a memory leak because indiserver died from an oom directly:
Child process 15952 died
2018-03-28T20:05:50: Driver indi_eqmod_telescope: stderr EOF
2018-03-28T20:05:50: Driver indi_eqmod_telescope: restart #1
2018-03-28T20:05:50: fork: Cannot allocate memory
2018-03-28T20:05:50: good bye

If it makes sense to anyone, here are the last words of indi_eqmod_telescope, nothing really different from the usual 1-second exchange:
2018-03-28T20:05:47: EQMod Mount: [SCOPE] Compute local time: lst=8.39228421 ( 8:23:32.22) - julian date=2458206.33721594
2018-03-28T20:05:47: EQMod Mount: [COMM] dispatch_command: ":j1", 4 bytes written
2018-03-28T20:05:47: EQMod Mount: [COMM] read_eqmod: "=65737C", 8 bytes read
2018-03-28T20:05:47: EQMod Mount: [SCOPE] GetRAEncoder() = 8156005
2018-03-28T20:05:47: EQMod Mount: [COMM] dispatch_command: ":j2", 4 bytes written
2018-03-28T20:05:47: EQMod Mount: [COMM] read_eqmod: "=7F8E9E", 8 bytes read
2018-03-28T20:05:47: EQMod Mount: [SCOPE] GetDEEncoder() = 10391167
2018-03-28T20:05:47: EQMod Mount: [SCOPE] Current encoders RA=8156005 DE=10391167
2018-03-28T20:05:47: EQMod Mount: [ALIGNMENT] Status: Mnt. Algnt. NORTH Date 2458206.337216 encoders RA=8156005 DE=10391167 Telescope RA 15.010909 DEC 79.889322
2018-03-28T20:05:47: EQMod Mount: [ALIGNMENT]  Direction RA(deg.)  -99.279375 DEC 79.889322 TDV(x -0.028307 y 0.173253 z 0.984470)
2018-03-28T20:05:47: EQMod Mount: [ALIGNMENT] Failed TransformTelescopeToCelestial: Scope RA=15.0109 Scope DE=79.889322, Aligned RA=15.010909 DE=79.889322
2018-03-28T20:05:47: EQMod Mount: [COMM] dispatch_command: ":f1", 4 bytes written
2018-03-28T20:05:47: EQMod Mount: [COMM] read_eqmod: "=301", 5 bytes read
2018-03-28T20:05:47: EQMod Mount: [COMM] dispatch_command: ":f2", 4 bytes written
2018-03-28T20:05:47: EQMod Mount: [COMM] read_eqmod: "=011", 5 bytes read
2018-03-28T20:05:47: EQMod Mount: [SCOPE] GetRAPeriod() = 6
2018-03-28T20:05:47: EQMod Mount: [SCOPE] GetDEPeriod() = 6
2018-03-28T20:05:47: EQMod Mount: [SCOPE] CheckMotorStatus() : Axis = 1
2018-03-28T20:05:47: EQMod Mount: [DEBUG] IsRARunning() = false
2018-03-28T20:05:47: EQMod Mount: [SCOPE] CheckMotorStatus() : Axis = 2
2018-03-28T20:05:47: EQMod Mount: [DEBUG] IsDERunning() = true

As it was mentioned ealier, it's maybe important to note is that indi_eqmod_telescope has full logs in my test. This swamps the log with 21-line blocks every 1 second, and may alter the realtime environment.

-Eric
6 years 3 weeks ago #24632

Please Log in or Create an account to join the conversation.

  • Posts: 535
  • Thank you received: 109

Replied by Jim on topic trouble with eqmod mount disconnecting


I honestly have not had skies good enough to do anything since my last post. I did want to mention that when I had no problems for 2 nights in a row, I, too, was running full logging on the mount. 2 data points is not a pattern, but might be the start of one?
6 years 3 weeks ago #24633

Please Log in or Create an account to join the conversation.

Replied by Jasem Mutlaq on topic trouble with eqmod mount disconnecting

well, I'll check today and run full logs and monitor the memory performance. I don't need good weather, it can be done in broad daylight :D
6 years 3 weeks ago #24648

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic Re:trouble with eqmod mount disconnecting

Indeed, my system's been capturing many targets from my basement during the last few days :) as a side note, work on the scheduler is progressing. You are right @knro, this is quite a complex beast!

-Eric
The following user(s) said Thank You: Jose Corazon
6 years 3 weeks ago #24650

Please Log in or Create an account to join the conversation.

Replied by Jasem Mutlaq on topic Re:trouble with eqmod mount disconnecting

I have made changes to the scheduler a few days ago that would re-evaluate all jobs anytime evaluateJobs is called. Just FYI if you are not aware of that. That fixed problems with the "Messier Marathon" scheduler list that I received.
The following user(s) said Thank You: Eric, Jose Corazon
6 years 3 weeks ago #24652

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic Re:trouble with eqmod mount disconnecting

I saw this too, and was about to check the reason. With a 30-item scheduler job it takes ages for the evaluation on my low-power VM :) But I don't want to hijack that thread, I'll report my findings in the other thread.

-Eric
6 years 3 weeks ago #24654

Please Log in or Create an account to join the conversation.

Replied by Jasem Mutlaq on topic Re:trouble with eqmod mount disconnecting

There were indeed few memory leaks that are now fixed in GIT.
6 years 2 weeks ago #24669

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301

Replied by Eric on topic Re:trouble with eqmod mount disconnecting

Thanks. My setup has been running for three weeks without power interruption. During these two weeks, there have been three indiserver crashes induced by indi_eqmod_telescope, always while slewing or tracking. None of these crashes had the disconnection messages. All crashes did damage the functioning of the camera driver, which was used during the tests. The focuser and guider drivers were connected but not actively used, and never suffered from the driver crash (except that they died along).

I'll pick the changes in indiserver and let you know the outcome. With my changes in the scheduler it's time to test jobs auto-scheduled over multiple nights :)

About leaks, I suggest we set a static code analysis of the tree periodically. I know Synopsis/Coverity provides such service free of charge for open-source projects.

-Eric
6 years 1 week ago #24912

Please Log in or Create an account to join the conversation.

Time to create page: 0.287 seconds