Welcome, Guest
Username: Password: Remember me
14 Nov 2018
Glad to announce of release of INDI Library v1.7.5 on 2018-11-14. A few drivers were added in this release as we continue to improve & stabilize existing drivers.
Read More...
  • Page:
  • 1
  • 2

TOPIC: D16429 - Testers needed for Scheduler changes

D16429 - Testers needed for Scheduler changes 3 weeks 2 days ago #31700

  • TallFurryMan
  • TallFurryMan's Avatar Topic Author
  • Away
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 624
  • Karma: 13
  • Thank you received: 181
Hello fellow Ekos users,

We would need a hand testing the rewrite of the Scheduler planning algorithm.

Until it is merged, the changes are documented at phabricator.kde.org/D16429 . Please have a look there and feel free to comment in this thread, we are trying to maintain use cases to avoid regressions. Those use cases will be integrated to the documentation.

For reference, Linux users who would like to rebuild their KStars binary with those changes should use the command "arc patch D16429" to apply the differential to their tree. This will require package "arcanist" to be installed, and a security check will be requested when executing the command with explanations on how to proceed.

-Eric
The following user(s) said Thank You: El Corazon

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

D16429 - Testers needed for Scheduler changes 4 days 14 hours ago #32390

  • TallFurryMan
  • TallFurryMan's Avatar Topic Author
  • Away
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 624
  • Karma: 13
  • Thank you received: 181
Hello,

Thanks, this differential is now merged and will be available in the next iteration of builds. For reference on the changes:

Summary

This is a full rewrite and verification of all steps of evaluation of a list of scheduler jobs with the following existing constraints:
  • Dark sky, which schedules a job to the next astronomical dusk/dawn interval.
  • Minimal altitude, which schedules a job up to 24 hours away to the next date and time its target is high enough in the sky.
  • Moon separation, combined with altitude constraint, which allows a job to schedule if its target is far enough from the Moon.
  • Fixed startup date and time, which schedules a job at a specific date and time.
  • Culmination offset, which schedules a job to start up to 24 hours away to the next date and time its target is at culmination, adjusted by an offset.
  • Amount of repetitions, eventually infinite, which allows a job imaging procedure to repeat multiple times or indefinitely.
  • Fixed completion date and time, which terminates a job at a specific date and time.
The UI now displays target altitudes at the current simulation time, and at startup and completion times. It also displays the lead time or gap time between jobs to help with imaging time optimization. Warnings appear in some situations for the user to detect problematic situations, such as a job completing under the minimal altitude.

The rewrite starts by estimating the imaging duration of each job, and eventually reorders them based on option "Sort Jobs by Altitude and Priority".
The reordering algorithm sorts jobs by altitude, considering a setting target is higher than a rising target.
  • When the option is enabled, targets will be automatically sorted whenever jobs are evaluated, using the current simulation time for altitude calculations.
  • When the option is not enabled, a button is available to reorder jobs manually, using the first job startup time for altitude calculation.
This reordering helps to optimize imaging time when preparing a schedule with multiple targets.

The rewrites then proceeds to sequence jobs based on their order in the list, resolving each constraint one after the other.
The sequencing algorithm makes sure there is a minimal gap between jobs, as set in option "Lead Time".
Note that when executing, Scheduler will immediately start a job planned away from current simulation time by an offset smaller than that lead time.
The sequencing algorithm will attempt to resolve all constraints ten times before considering those can't be achieved.
There is no limit to the number of jobs that can be inserted in the queue, but the bounded number of attempts effectively poses a restriction to automated scheduling.

Known bug: The algorithm is not entirely optimized yet, and adding observations may slow the reactivity of the user interface on low-end systems while evaluating.
Known bug: The updated values of the Ekos options are taken into account at the next evaluation of the job list, and not automatically after the options pop-up is dismissed. As there is no periodic evaluation of the job list, the end-user needs to click the "refresh" icon to trigger a new evaluation with the updated settings.

TEST PLAN

Use case: start the observation of a target when its altitude is higher than X degrees
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Enable the "Minimum Altitude" constraint and specify a minimum altitude the target should be visible at from the KStars geographic location before starting the observation. The constraint is enabled by default with a value of 15 degrees.

Target observation will be then scheduled in the interval between the moment it rises higher than the minimum altitude, to the moment it sets lower that the minimum altitude plus 3 degrees. The altitude cut-off is there to avoid the situation where a scheduler job execution is started and aborted after a few minutes because its preparation steps - focus, alignment and guiding - delayed the capture step.

Known bug: You can set any altitude in [15,90] in the user interface. If you disable the constraint, the minimum value of 0 degrees is enforced. Although the code managing the constraint will only enforce the numerical limit when its value is higher than -90 degrees, the user interface will not allow an minimum altitude lower than 15 degrees, and the scheduler code will not allow a target to be lower than 0 degrees for observation.

Known bug: The altitude cut-off value considered when the target sets cannot be customized.

Use case: start the observation of a target when its angle to the Moon is greater than X degrees
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Enable the "Moon Separation" constraint and specify a minimum angle the target and Moon should be separated from, as seen from the KStars geographic location, before starting the observation. The constraint is disabled by default. Target will be then scheduled on the day that separation is higher than the minimum angle.

Known bug: The implementation of this constraint is currently embedded into the altitude constraint, and prevents the altitude check from succeeding if the angle is not large enough. However, the altitude constraint will only look for a suitable date and time in the next rise and set cycle of the target, thus 24 hours at maximum. As the separation with the Moon will not change much during that time, the job will often end up invalid.

Use case: start the observation of a target during night time only
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Enable the "Twilight" constraint. The constraint is enabled by default.

Target observation will be then scheduled in the interval between the astronomical dusk to the astronomical dawn, minus the lead time set in the Ekos options. The lead time cut-off is there to avoid the situation where a scheduler job execution is started and aborted after a few minutes because its preparation steps, focus, alignment and guiding, delayed the capture step.

Known bug: The twilight constraint uses the astronomical dawn and dusk of the current simulation day, even if the target is scheduled to a date that is different. Though there is not much change to that interval day after day, it will cause unexpected delays or lost imaging time when an automated observatory is set to sleep for a few days.

Use case: start the observation of a target when it is culminating
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Change the start-up conditions to the "Culmination" constraint, and specify a positive or negative offset in minutes to the culmination time of the target from the KStars geographic location before starting the observation. The start-up constraint is set to "as soon as possible" by default.

Target observation will then be scheduled to start at the next culmination date and time, adjusted by the specified offset.

Known bug: Because the culmination calculation has a certain level of incertitude, it may happen that the schedule is not stable when evaluated at different date and time values: when re-evaluated later, the job may be found starting after the effective adjusted culmination, and pushed to the next day as jobs cannot be scheduled sooner than they are currently occurring at during an evaluation. To resolve this situation, increase the lead time in the Ekos options to relax the precision of the job start-up time.

Use case: start the observation of a target at a fixed time
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Change the start-up conditions to the "Start At" constraint, and specify a date and time at which starting the observation. The start-up constraint is set to "as soon as possible" by default. The date and time is local to the KStars geographic position.

Target observation will then be scheduled to start at the indicated date and time.

Known bug: Because the real conditions in which the jobs execute may make the duration of the previous job longer than expected, it may happen that some jobs are aborted before starting because the start-up time is passed at the moment they are started. It is currently not possible to customize the duration of the preliminary steps of a job, and a job scheduled to start at a fixed time will not accept to start at a different time. To partially resolve this situation, increase the lead time in the Ekos options to relax the precision of the job start-up time and help with the schedule of prior jobs.

Use case: finish the observation of a target at a fixed time
If the scheduler job to modify was created previously, double-click its line in the job table to enter edit mode. If the job is being created, the constraint can be set immediately.

Change the completion conditions to the "Finish At" constraint, and specify a date and time at which completing the observation. The completion constraint is set to "finish sequence" by default. The date and time is local to the KStars geographic position.

Target observation will then be scheduled to repeat until the indicated date and time. Because the real conditions in which the job execute may make the duration of that job longer than expected, using a fixed repeat count may result in a completion time later than planned. Using a fixed completion time ensure the following job will start at the expected fixed time.

Use case: change the order of observation of targets
If an observation target is currently edited, exit the edition mode by clicking on the "check" icon.

Open the Ekos option, and disable option "Sort Jobs per Altitude and Priority" in the Scheduler tab.

Select an observation target by clicking on its line in the job table. That line is highlighted as a result, and the state of icons "up" and "down" above the job table comply with the following rules:
  • If the selected job is the first of the list, the "up" icon is disabled.
  • If the selected job is the last of the list, the "down" icon is disabled.
  • If the selected job is the only one in the list, the "up" and "down" icons are disabled.
  • In other situations both "up" and "down" icons are enabled.
Clicking the enabled "up" button will swap the selected job with its previous sibling. Clicking the enabled "down" button will swap the selected job with its following sibling.

The schedule of the observation targets following and including the swapped jobs will re-evaluate automatically after the operation.

Use case: sort observations of targets per their altitude
If an observation target is currently edited, exit the edition mode by clicking on the "check" icon.

Open the Ekos option, and disable option "Sort Jobs per Altitude and Priority" in the Scheduler tab.

Examine the start-up time of the first observation target of the job table. When the "sort" icon is clicked, the algorithm computes the altitudes of all observation targets, reorders them per their computed altitude, then re-evaluates the schedule of all jobs.

The sort algorithm orders altitude from set to rise:
  • If target A is setting and target B is rising, then A will be sorted before B.
  • If target A is lower than target B, but is setting, then A will be sorted before B.
  • If target A is lower than target B, but is rising, then A will be sorted after B.

Use case: systematically sort observations of targets per their altitude
If an observation target is currently edited, exit the edition mode by clicking on the "check" icon.

Open the Ekos option, and enable option "Sort Jobs per Altitude and Priority" in the Scheduler tab. When that option is enabled, evaluation of targets will systematically reorder observations. The algorithm computes the altitudes of all observation targets _at the current simulation time it is executed_, reorders them per their computed altitude, then re-evaluates the schedule of all jobs.

The sort algorithm orders altitude from set to rise:
  • If target A is setting and target B is rising, then A will be sorted before B.
  • If target A is lower than target B, but is setting, then A will be sorted before B.
  • If target A is lower than target B, but is rising, then A will be sorted after B.
The following user(s) said Thank You: knro, xsnrg, wvreeven, El Corazon

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

D16429 - Testers needed for Scheduler changes 4 days 14 hours ago #32391

  • stash
  • stash's Avatar
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 140
  • Karma: 3
  • Thank you received: 38
Thats a lot of effort :-)
The following user(s) said Thank You: TallFurryMan

Please Log in or Create an account to join the conversation.

RPI3 Ubuntu 16.04 / AMD desktop Kstars under Ubuntu 16.04 Mounts :azeq6 ,SWAZGoTo

RPI3 Fedora testing out on AMD desktop Fedpra 28 - running kstars 2.9.4 , Indilib 1.7.4 ?????

D16429 - Testers needed for Scheduler changes 4 days 12 hours ago #32396

Great work, Eric. Many thanks! I’ll give it a try this weekend, weather permitting of course!

Please Log in or Create an account to join the conversation.

Modded Canon EOS 700D with a WO ZS 71 ED on a SW HEQ5-Pro /
QHY9 CCD and filter wheel with an 8" RC on a SW EQ6-R

D16429 - Testers needed for Scheduler changes 4 days 10 hours ago #32398

Great work Eric! I will give this a go in about 1 hour. But the changes are sweeeeeeeeet! Very excited to try this on!

Please Log in or Create an account to join the conversation.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

D16429 - Testers needed for Scheduler changes 4 days 10 hours ago #32399

That sounds FANTASTIC Eric!
Unfortunately, the weather here is not cooperating for the next few days, so feedback from what may be your most persistent and keenest "customer" when it comes to the scheduler will unfortunately have to wait a few days.
Nonetheless, thank you for making us all such a wonderful X-mas present!
I will get to it as soon as the weather cooperates again.
Best wishes
Jo

Please Log in or Create an account to join the conversation.

D16429 - Testers needed for Scheduler changes 3 days 2 hours ago #32429

I tried the Scheduler tonight and had mixed results. First of all, I set up a job for NGC 891 which was to culminate at 22:57 CET (though it really culminated at 22:29 but I entered my site location wrong) at my location and I set the job to start 10 minutes after culmination. And surely, at 23:07 the job started. It unparked the mount and ... started an exposure! No tracking, no slewing to the object, no guiding, nada. Here's a screenshot of the scheduler:



I have also attached the log file. Please note that I hadn't enabled the scheduler logging so I did that and then scheduled a new job and started that. Let me know if you want more concise logs and I'll repeat with the logging enabled from the start. In any case, the behaviour the second time was exactly the same: no tracking, no slewing, no guiding.


Wouter
Attachments:
The following user(s) said Thank You: knro

Please Log in or Create an account to join the conversation.

Modded Canon EOS 700D with a WO ZS 71 ED on a SW HEQ5-Pro /
QHY9 CCD and filter wheel with an 8" RC on a SW EQ6-R

D16429 - Testers needed for Scheduler changes 2 days 17 hours ago #32438

Before I forget, I had to actively choose the correct profile in the upper left part of the sequence screen. I would expect the correct profile to be automatically selected based on the current Ekos configuration.


Wouter

Please Log in or Create an account to join the conversation.

Modded Canon EOS 700D with a WO ZS 71 ED on a SW HEQ5-Pro /
QHY9 CCD and filter wheel with an 8" RC on a SW EQ6-R

Re:D16429 - Testers needed for Scheduler changes 2 days 15 hours ago #32441

  • TallFurryMan
  • TallFurryMan's Avatar Topic Author
  • Away
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 624
  • Karma: 13
  • Thank you received: 181
"proceeding directly to capture stage because only calibration frames are pending". Would you care to provide the sequence file too? There might be another bug related to sequence management that needs fixing.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:D16429 - Testers needed for Scheduler changes 2 days 14 hours ago #32444

Here you go, Eric.


Wouter

EDIT wait, wrong file. I'll upload it shortly.

EDIT 2: OK it was the right file but I added the scheduler list as well
Attachments:

Please Log in or Create an account to join the conversation.

Modded Canon EOS 700D with a WO ZS 71 ED on a SW HEQ5-Pro /
QHY9 CCD and filter wheel with an 8" RC on a SW EQ6-R
Last Edit: by wvreeven.

Re:D16429 - Testers needed for Scheduler changes 1 day 16 hours ago #32470

  • TallFurryMan
  • TallFurryMan's Avatar Topic Author
  • Away
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 624
  • Karma: 13
  • Thank you received: 181
@wvreeven Thanks. Unfortunately for your report, the scheduler had the expected behavior: your sequence file is requesting flat frames. Flat frames do not require any step, so all of them are bypassed when Scheduler starts. Capture module will eventually slew to a wall Alt/Az target, but Scheduler will only capture.
This said, we need a better UI both in the Scheduler and in the Capture modules, as it is easy to think modifications were applied while the UI was actually in addition mode.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:D16429 - Testers needed for Scheduler changes 1 day 16 hours ago #32471

Eric, thanks for the investigation and I am soooo sorry for this! I'll pay better attention to the sequence I create for use with the Scheduler. Unfortunately I will be travelling today and after I am back on Wednesday the weather forecast gets bad so I am not sure when I will be able to test again.


Wouter

Please Log in or Create an account to join the conversation.

Modded Canon EOS 700D with a WO ZS 71 ED on a SW HEQ5-Pro /
QHY9 CCD and filter wheel with an 8" RC on a SW EQ6-R
  • Page:
  • 1
  • 2
Time to create page: 0.334 seconds

Login



3rd Party

Choose from the numerous 3rd party INDI drivers to suit your needs!

Got Problem?

Check out the FAQ, the forum, and the bug tracking system to resolve any issues you might have!
You can also subscribe to INDI newsletter and development mailing lists to get the latest updates on INDI!


Gallery

Replica

Why INDI

Replica