×
INDI Library v1.8.1 Released (09 Sep 2019)

Monthly maintenance release INDI v1.8.1

New Scheduler Iteration

4 months 2 weeks ago
TallFurryMan
Platinum Boarder
Platinum Boarder
Posts: 737
Karma: 13
More
Topic Author
New Scheduler Iteration #40669
Hello all,

In 2018, Scheduler saw a large rework of d-bus communication and planning algorithms, as well as many fixes that improved its robustness. I'd like to share my views on the next iteration in 2019:

1. Observatory module

This was discussed in another thread . This module will gather everything related to starting up, monitoring and shutting down the Observatory setup. No need to have a fixed observatory: startup/shutdown scripts, CCD cooler and mount unparking/parking are relocated there, and Scheduler will request the module to just start and stop. Weather safety measures for fixed observatories will be handled there: notifications to Scheduler to handle the situation inside the schedule, and cases of emergency shutdown managed directly and independently from Scheduler. Later on, Observatory could host a script library instead of a single startup/shutdown pair, and an interface for viewing security cameras.

Wolfgang (@Sterne-Jaeger), along with many other fixes in Ekos, initiated the implementation of the Observatory module. The module now lives in kstars/ekos/observatory . Wolfgang needs another developer owning a fixed observatory to take on the implementation of weather safety and roof management. Feel free to contact him.

2. Settings serialization

Each Ekos module should receive serialization support for its settings. Currently, Scheduler has serialization to .esl files, and Capture has serialization to .esq files. Those files are XML databases backing up and restoring the content of the UI displayed in the tabs. We need that feature extended to other Ekos tabs: Align, Focus and Guide. Additionally, in an effort to clarify their contents and ease manual modification and external generation, those serialization files should be structured hierarchically in the same way as current tab UIs are. Special care should be taken to version the format of the serialization files, so that no data would be misinterpreted.

With these serialized settings, Ekos modules should communicate with a simplified d-bus interface. From clients: "load" settings, "start" configured activity, "stop" running activity. From modules: global and/or specific "status" notifications. To change a setting remotely, d-bus clients would update fields of their local settings and issue a new load request. Modules would reply with their updated settings as status notifications. We would need to design this in the philosophy of a REST resource management API.

We should also support loading JSON and YAML files in addition to XML files. This would only be a matter of changing the serialization backend, but there should be a robust structure validation.

3. New Scheduler interface

Supported by the settings presented in the previous section, Scheduler should improve its control facilities and provide an interface summarizing the entire configuration of the observation session. The full observation plan would be a list of target observations, themselves a list of steps to execute, embedding the serialized settings of the Ekos modules. The very rough and preliminary UI draft below should give an idea of the objective.



At the root level, Scheduler should plan the observation targets just like it does now, with enough information about altitude, rising and setting movements, startup and completion, duration and lead time. For each observation target, in a second level, Scheduler should present scheduling restrictions applied to the target (not shown in the screenshot unfortunately), and an enumeration of the steps taking place during this particular observation. Those steps would match the Track/Align/Focus/Guide checkboxes we have now, but that new interface should allow inserting any number of them, in any order. This should support the example workflow "Job Crescent Nebula: Track to Sadr, Align and Focus in Lum with 2s exposure with a single star, then Track to NGC6888, Guide and Capture in Ha 5 times for 480s". That example workflow would be serialized in a single file with settings either embedded or referring to user templates for generic steps.

The "Schedule..." button should be a drop-down action list to add targets either from current kstars coordinates, named objects or existing serialized observation plans. The "Add..." button should be a drop-down action list to add Track/Focus/Align/Guide/Capture steps, either from blank templates or existing user settings saved from Ekos modules, embedded or linked to. The "Mosaic..." button should give access to the excellent existing mosaic tool. The "Load..." and "Save..." buttons should import and export full observation plans or single target observation plans.

Going further, the interface should allow modification of all serialized settings fields with in-tree controls. However, this may prove difficult for touch devices, so we'll have to investigate the accessibility of such a design.

4. Testing

We have a large number of invaluable and knowledgeable end-users using Ekos, testing features and reporting issues. But obviously you, reader, should be able to rely on the capabilities of Ekos, instead of taking part in a full-scale test.

Because we were hit by many regressions in the past years, we really have to spend time writing automated tests for all these new developments as well as for the legacy code. New developments should proceed test-driven, and legacy code should progressively be protected by module-level tests. This would require mocking Ekos modules just above d-bus, and writing current-behavior tests in order to prevent unexpected changes from biting us.

We should also discuss the possibility of integrating new features to Ekos using runtime flags. A special option pane would be used to enable or disable in-development features, in order for willing users to try them out before they are integrated to the application workflow. The new interface of Scheduler would be a good candidate, as it would force the development to proceed in non-breaking iterations.

---

Well, thanks for reading up to here. Thanks for your patience, and feel free to comment and argue!
-Eric

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access
Attachments:
The following user(s) said Thank You Herrhausen, pcxz, fenriques, sterne-jaeger, Deil, Spartacus

Please Log in or Create an account to join the conversation.

4 months 3 days ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 375
Karma: 5
More
New Scheduler Iteration #40995
Many thanks Eric for bringing the Scheduler ahead!

Each Ekos module should receive serialization support for its settings. ... With these serialized settings, Ekos modules should communicate with a simplified d-bus interface.

Very good point! We need to find a balance what is controlled by the Scheduler and what remains under control of other modules. We would end up in a configuration hell if we reflect all single controls of other modules in the Scheduler.

Supported by the settings presented in the previous section, Scheduler should improve its control facilities and provide an interface summarizing the entire configuration of the observation session. The full observation plan would be a list of target observations, themselves a list of steps to execute, embedding the serialized settings of the Ekos modules.

Much appreciated! The current list based view has reached its limits. With a tree based view we could show more details without losing the overview.

We should also discuss the possibility of integrating new features to Ekos using runtime flags. A special option pane would be used to enable or disable in-development features, in order for willing users to try them out before they are integrated to the application workflow. The new interface of Scheduler would be a good candidate, as it would force the development to proceed in non-breaking iterations.

YES! The current approach with one master and nightly builds is great for adding small increments. But redesigns like this one simply need time to reach their maturity.


As already offered: happy to participate!

Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

Time to create page: 0.432 seconds