Hi Eric and Jasem,

I came across a new idiosyncracy in the scheduler: When I program a specific target and I set it to start ASAP it will work fine ONLY if I start the scheduler BEFORE astronomical twilight allows it to run (it goes to sleep until then). The same settings will no longer work once the earliest startup time defined by astronomical twilight has past. In that case the scheduler will tell me that the job is invalid, because startup time has passed (by anything between 0 and 3 seconds, depending on how fast the scheduler reacts to the push of the start button).

I would think that all that is required is to program a 10 second delay, i.e. add 10 s to the actual time at the moment the scheduler is started when set for ASAP. The only workaround I have found so far is to manually program a startup time with a delay added, but for that I need to sit at the computer and it is laborious to do if I need to restart the scheduler for any number of reasons, like the focuser failing or the guide module not finding a guide start right away.

Attached the log from 2 nights ago, not sure how informative they are. I had to add manual startup times between 2215h and ~2240h local time, because of the above described past startup time issue.

Please let me know your thoughts,

Jo

PS: Attachment was too large, here link for download: www.dropbox.com/sh/x6qa5fp1pv54i3x/AABWK...Zu-oxlNawATBt7a?dl=0

I saw that, yes, but I thought I had fixed the issue in kstars-bleeding, while 2.9.6 shows the problem.

Because this is a "surprise" for the end-user that jobs are rescheduled when clicking the start button (schedule should be stable), I wanted to periodically reschedule jobs. That would have worked around this issue, but there are a few problems with this so I postponed the change.

-Eric

OK, that explains it. I am still running 2.9.6 on the Zotac I used last night. That run went fine, but at the end I found another problem: I had checked 'Repeat schedule 2 more times' instead of 'Sequence completion', because I saved the images to a folder that already contained images from a prior run and I did not want the sequence to stop once it had reached the maximum number of exposures specified in the M27.esq file. As M27 crossed the minimum specified altitude (43 degrees) the scheduler stopped the sequence and searched for the next job, which was again M27 but for the next night.
But instead of shutting down, the scheduler merely went to sleep. It did NOT park the mount, instead it continued to guide until I manually shut down the mount.

2018-07-22T04:57:02 Job 'M 27' scheduled for execution at Sun Jul 22 22:12:59 2018. Parking the mount until the job is ready.
2018-07-22T04:57:02 Job 'M 27' is selected for next observation with priority #5 and score -1000.
2018-07-22T04:57:02 Job 'M 27' has a total score of -1000
2018-07-22T04:57:02 Job 'M 27' is scheduled to start at 22/07/18 22:12 where its altitude is 46.9 degrees.
2018-07-22T04:57:02 Job 'M 27' has a total score of -1000
2018-07-22T04:57:02 Astronomical twilight rise is at 04:57:00, set is at 22:12:00, and current time is 04:57:02
2018-07-22T04:57:01 Job 'M 27' is aborted.
2018-07-22T04:57:01 Job 'M 27' current altitude (42.9871 degrees) crossed minimum constraint altitude (43 degrees), marking aborted.
2018-07-22T04:56:59 Job 'M 27' is selected for next observation with priority #5 and score -1000.
2018-07-22T04:56:59 Job 'M 27' has a total score of -1000
2018-07-22T04:56:59 Job 'M 27' target altitude is 43 degrees at 22/07/18 04:56 (score -1000).
2018-07-22T04:56:59 Job 'M 27' is scheduled to start at 22/07/18 04:55 where its altitude is 43.2 degrees.
2018-07-22T04:56:59 Job 'M 27' has a total score of -1000
2018-07-22T04:56:59 Job 'M 27' target altitude is 43 degrees at 22/07/18 04:56 (score -1000).
2018-07-22T04:56:59 Astronomical twilight rise is at 04:57:00, set is at 22:12:00, and current time is 04:56:59
2018-07-22T04:56:58 Job 'M 27' is aborted.
2018-07-22T04:56:58 Job 'M 27' current altitude (42.9977 degrees) crossed minimum constraint altitude (43 degrees), marking aborted.
2018-07-21T22:21:40 Job 'M 27' capture is in progress...
2018-07-21T22:21:40 Job 'M 27' guiding is in progress.
2018-07-21T22:20:29 Starting guiding procedure for M 27 ...
2018-07-21T22:20:29 Job 'M 27' repositioning is complete.
2018-07-21T22:20:28 Job 'M 27' alignment is complete.
2018-07-21T22:17:53 Job 'M 27' is capturing and plate solving.
2018-07-21T22:17:53 Job 'M 27' focusing is complete.
2018-07-21T22:16:58 Job 'M 27' is focusing.
2018-07-21T22:16:58 Job 'M 27' is restarting its focusing procedure.
2018-07-21T22:16:58 Warning! Job 'M 27' focusing failed.
2018-07-21T22:13:27 Job 'M 27' is focusing.
2018-07-21T22:13:27 Job 'M 27' slew is complete.
2018-07-21T22:13:06 Job 'M 27' is slewing to target.
2018-07-21T22:13:03 Mount already unparked.
2018-07-21T22:13:00 Scheduler is awake. Jobs shall be started when ready...
2018-07-21T22:11:53 Sleeping until observation job M 27 is ready at Sat Jul 21 22:13:00 2018...
2018-07-21T22:11:53 Job 'M 27' is selected for next observation with priority #5 and score -1000.
2018-07-21T22:11:53 Job 'M 27' has a total score of -1000
2018-07-21T22:11:53 Astronomical twilight rise is at 04:57:00, set is at 22:12:00, and current time is 22:11:53
2018-07-21T22:11:52 Astronomical twilight rise is at 04:57:00, set is at 22:12:00, and current time is 22:11:52
2018-07-21T22:06:01 Job evaluation complete.
2018-07-21T22:06:01 Job 'M 27' is scheduled to start at 21/07/18 22:12 where its altitude is 46.1 degrees.
2018-07-21T22:06:01 Job 'M 27' has a total score of -1000
2018-07-21T22:06:01 Job 'M 27' estimated to take 13h 06m 15s to complete.
2018-07-21T22:06:01 Job 'M 27' 250x180" requires a dither procedure.
2018-07-21T22:06:01 Astronomical twilight rise is at 04:57:00, set is at 22:12:00, and current time is 22:06:00


This is another potential unintended consequence of rescheduling: Failure to park the mount and shutting down at twilight. Twilight should override rescheduling and force parking and shutdown.

Here again a link to the log file from last night.

www.dropbox.com/sh/rqntzv5fk1u2m1m/AABcP...2X_6W2I9xG4FBya?dl=0

I hope it helps. Thanks for the great work, Eric! I collected a full run of 104 x 180s exposures last night, Failure to shutdown and park was the only hitch this time.

Best

Jo

I...believe...that problem should also be fixed in kstars-bleeding, but it's been more than one month since I pushed that and my memory got reset by holidays the release of 2.9.6 took me a bit short on patches...

-Eric

As a side note, and because this is visible in your test, my objective was to set a target for capture at the beginning of a week, mark it for infinite loop and let the scheduler arrange the observation. Though there are still issues, that's a basic fire and forget feature, and the majority of the work from there is to be able to mix other jobs in between. I'd like to somehow implement the feature "do whatever you want, but capture X hours of this target".

-Eric

Thanks Eric, that makes perfect sense.
However, I think the mount should respond to the instructions to park itself and switch off the cameras when the scheduled run ceases because of twilight. It should then shut down and wait in the park position for the next scheduled run. It actually does exactly that when I use the timed schedule during the night between targets. It just doesn't do it when I program the target to be imaged ASAP. At least not in the current stable 2.9.6 version.
The situation I encountered this morning was that the guide module was still running and the telescope was approaching the horizon. In addition, the cameras were still active. The guide cam for sure only returned white frames by the time I found the scope.
Again, no griping here, just testing in the real world and reporting unexpected behavior.
Best
Jo

OK, an update on a few more small things I found with the scheduler:

Last night it worked fine overall, but there were a few more issues: First, I had two jobs, M27 and NGC281. Both were set to start ASAP, but after M27 was finished, the scheduler decided to wait. Since it was 11 PM, I did not want to hang around to find out whether it might eventually decide to continue, I instead stopped it, deleted the M27 portion and restarted it. It then ran fine for the rest of the night.

That is, until this morning. I had deselected the twilight button, because I wanted to collect images between 5.30 and 6 am here (with the light pollution astronomical twilight is meaningless) and the scheduler actually finished the exposure sequence shortly after 6 am, but then it failed to park the mount and shut down. Instead, it went into waiting for the next night, all the while tracking continued. It would be great if one could instruct the scheduler to shut down, if the next target is scheduled for the next night.

Another thing I noticed is that the scheduler does not set the filter wheel to the next sequence, but that previous filters remain selected in the focuser and in the solver. That leads to excessive refocusing and filter changes before the actual sequence is being started (up to 3 times). I wonder whether one could instruct the focuser to first change the filter to the next one in sequence, then focus with that, then have the solver run with that as well before image capture starts. That way, only one refocusing run would be required.

Lastly, it would be great to have a button that allows me to select when the guide module should recalibrate. Right now, it does not automatically stop and recalibrate if there is either a manual capture sequence running, even after that is stopped by the scheduler, or if the guide cam is set on loop. For a while there were a few buttons in Ekos that allowed me to instruct the guide module to recalibrate every time when it was called up, but those buttons are gone again. It would be great to have them back!

Otherwise, it worked just fine last night. As always, check out the images under my profile.

Best

Jo

PS: Log file too large, you can download it here: www.dropbox.com/sh/xdzd7t0an51372j/AAAfJ...iMuaUnVc4k29XYa?dl=0


Scratch point number one, I just realized that the reason why the scheduler did not continue to the next step was because the altitude limit was not yet reached. My mistake,

Thanks for all these reports!

Currently twilight distance is fixed. I plan to allow customisation for high latitude observers and for narrowband imagers. It will also avoid using tricks like you did, only to end up bitten by another condition that did or dit not trigger.

When two jobs are too far away, the scheduler is supposed to either wait or park the setup. In your situation you said that the scheduler was waiting for the next night. Was it waiting because more frames remained to be captured? I now expect the scheduler to abort, then reschedule the job to the next night. Then you said that the scheduler entered a wait state. It was "sleeping"? If you enable "Preemptive Shutdown", the scheduler parks the setup if the next job is far away. If not, it lets the setup unmodified and just waits. Note that this second behavior will be changed soon so that scheduler stops the mount movement while it is waiting. For obvious security reasons, the scheduler should also shutdown the observatory in a few situations.

Thanks for reporting the filter issue too, we'll check that behavior.

-Eric

"Was it waiting because more frames remained to be captured?"

No, the run was finished, it reported 'Complete", but it did not enter shutdown mode and failed to park on its own. The mount kept on tracking the object.

It was not a big deal, since I was awake anyway and manually shut everything down, and the previous night the shutdown sequence worked just fine when the twilight box was checked. As I wrote, a relatively minor issue.

When it comes to tricks, I have a whole bag full of those|



Thanks for everything you are doing here, Eric!

I'll just keep reporting what I find.

Jo

I found a situation where the Scheduler might get hung doing nothing after a capture. I don't know if that could be what you experienced.

There is a "funny" bug, registered as bugs.kde.org/show_bug.cgi?id=398301, which makes all your captures attempts slew to the wall source position when you configure that option in the calibration pop-up for flat captures. The problem is that once this calibration option is set, it is saved with the sequence file, so it's very easy to unknowingly copy-paste the setting everywhere. If you don't readily spot it, this is disastrous for captures, of course.

In that situation, the Capture module doesn't send the notification of the end of the capture to the Scheduler, which hangs.

Interesting! Not sure this applies to me, I did not use the wall setting in the calibration file, however, I told the mount through KStars (not through the Flat calibration setting in the capture module) to slew to a region close to the Zenith while there was still daylight during early dusk and that's where I tried to take the flats.

I used the same 'Flats.esq' file I had set up there the next day and the flats still hang.

The problem was gone when I used the new nightly build as Jasem had suggested, but I also generated a new 'Flats.esq' file.

Would that trigger the same bug?

That should be fairly easy to spot: to set the Wall mode, you need to provide AltAz coordinates. From then on, even attempting to capture a light frame will have the mount slew to the wall source coordinates. When the Scheduler loads a sequence file that was saved while wall mode was set in the calibration pop-up, it will first slew to the target, then slew to the wall source, then calibrate-capture something, then hang doing nothing.

What is not easy to spot is the fact that the calibration setting changed to Wall mode when you load a sequence that has this setting. This can easily propagate to subsequent sequence files you prepare.

-Eric