This is good news. I will go back through logs and see if there is anything. For sure though, the restart did not work and there were no calibration frames in the sequence.

Question: What would cause the Stop INDI button to not stop the INDI processes?

I am wondering if something happened within INDI, as shortly after the above happened I had to do a sudo pkill indi on the RPI3. Should this occur again I will check for this, and see if it corrects.

Thank you

I could reproduce this problem this night with kstars 2.9.1 under Windows:
- created a sequence with 10x45s lights.
- created a scheduler job on M42, start asap, repeat job 18 times, with the sequence above and align/focus/track modules but no guiding.

When started, scheduler says there are "only calibration frames pending", bypasses all modules, proceeds to capture, although there are no calibration frames configured in the sequence. When switching to the next loop on the same job, does the same (double check needed on logs to be precise on this point).

As a result of the lack of alignment, capture drifts slowly out of frame over the three hours. Tracking was left unmodified during that period, and I will check about focusing.

If I got the configuration right, I should be able to share logs about this issue.

-Eric

Forgot to mention that I pressed "start" in the scheduler, then had to "pause" and reset its jobs, because of I had the output path wrong, before pressing "start" again. This is indeed why I post in this thread.

-Eric

Please post your esq and esl files.

"M42.zip" has the sequence and scheduler XML files, "logs.txt" has the log, until the termination of the job. Jump to line 857 for the actual scheduler start and the unexpected message, and notice the previous manipulations. I removed the "SBL::fillToMag" spam for clarity. This is kstars 2.9.1 Build: 2018-01-11T10:39:55Z on Windows.

-Eric

Just to bump this issue with a picture This is what the capture resulted in (stacked, only stretched, no calibration frames), although only 25 minutes of the three-hour run could be used. And 45 seconds of exposure is too much for M42

-Eric

@knro I was looking at the code of the scheduler, and it appears there might be a missing test that could explain the symptom I've seen.

Line 3987 in scheduler.cpp, light frames are set required if the job is a FRAME_LIGHT, plus either a completed count or a completion type to FINISH_LOOP. Wouldn't FINISH_REPEAT be needed here too?

-Eric

Good catch. Well, can you make the change and test?

Good point I've never delved into kstars development more than looking at the code tree on github. Let me check what I can do.

-Eric

So I probably nailed down where the problem was with my last post, but actually it is not the root cause of the issue. In fact, that problem might not be considered as an issue per se, except that it has side effects which cause confusion.

The option "remember job progress" is counting the number of captures relevant to the sequence being run (that is, a count of existing fits files prefixed with the right folder assembly), and uses that number to check if the job is complete or not, and how many captures it still needs.

If I add multiple identical jobs to the scheduler list, and "remember job progress" is engaged, the algorithm of the scheduler will consider all jobs immediately finished as soon as the first job is finished. This happens because all jobs are identical, therefore the same set of captures is checked to determine whether the job is completed or not.

But if you try to reproduce, it's not the case. It is due to an additional issue: the job captures are only enumerated while estimating the duration of the jobs, and only for jobs that are NOT estimated yet.

By the way, I could also observe that modifying a job (by changing the sequence file for instance) should reset the duration, but actually does not: only a new job will arrive with no estimated duration.

Therefore to reproduce the actual issue, one scenario is to create a job with a sequence, start the scheduler and run it to the end, duplicate the job into a new one, and start the scheduler again. No module will get triggered, only captures, and the wrong message about calibration will appear in the logs.

Because this is, well, a set of intertwined issues, this will be a bit more involved to fix it's not even clear what needs fixing. Option "remember job progress" is clearly not compatible with job duplication, but having multiple sequences with realignment and focus might be interesting for prioritizing. Focus could be configured with the checkbox "refocus if HFR...", but what about alignment? That makes guiding obligatory, but it is optional nonetheless. And meridian flip? Or perhaps the way the scheduler currently works is simply too flexible for its own good.

In any case I may start by adding logs which clarify the situation for the end-user, reminding how the scheduler will address jobs. I could then fix the duration estimation, which should trigger for any change, or at least reset when the job is modified. Then probably I could add one level of folders between the scheduler output and the sequence output, say the job index (not very bright though), in order to avoid jobs mixing up when counting captures. Perhaps from there I could add a new button aside the mosaic macro, which would enable users to generate multiple jobs to capture a certain total exposure on a particular object.

Any other ideas on this?

-Eric

Thanks for looking in into this. There are more issues reported here: bugs.kde.org/show_bug.cgi?id=389738

These are partially resolved as well. One issue I partially resolved recently is when running identical jobs and it used to mark them all as complete since it checks the file system for completed files and viola all requested files within the job are there so it is marked as complete. However, now it subtracts the file system # from each job sequentially. Similar, the scheduler sends the signature (prefix) of a job with the number of jobs to resume from. Suppose you have 2 identical scheduler jobs, each captures 5 images. When the scheduler runs, and it finds that 8 images exist on the file system for this particular signature, it marks the first job as complete but then proceed to the 2nd one and runs it. It then informs capture module that 3 images are already completely and that the sequence should start after that.

Changing the sequence file should result in resetting the estimation again, hopefully this is a simple fix. I already thought of the job index but as you can check in the bug report above, this is problematic when it is time to examine the data later on, so instead I approached it differently. So yeah, the scheduler is a complex beast but that is totally expected. There are probably more cases where it fails. For me, I usually just have a single job with a single sequence file so I don't suffer much!

Sorry I forgot to check the KDE bugzilla!

OK that's clear about the jobs. I think the duplication of jobs should be kept for those of us whom run short unguided captures.

From what I understand from your post, when the scheduler is started anew, and that multiple duplicated jobs are in the list, the algorithm will "stack" the captures onto the jobs, starting with the first, whatever what happened during the last session.

I propose to add a column indicating the progress for each job. If jobs are duplicated, the first run will for instance say "10/10, 6/10, 3/10, 0/10", whatever the reason, and the second run will consolidate this into "10/10, 9/10, 0/10, 0/10". Then with that visual feedback, I will make sure "evaluate jobs" is redistributing the jobs that way, just like when opening the scheduler for the first time.

Probably I should enforce this evaluation anytime a job is modified, but that's debatable. There's a problem with this: we may have jobs that are duplicates in terms of captures, but are different in terms of which module is enabled and which is not (too much flexibility). We will still be distributing the captures to the first jobs with no regard for what option is enabled and what option is not. That's what will happen we we start the scheduler anew, of course. We can probably consider that a limitation of the current structure and live with it. I could find a way to warn the user about this: same sequence file, same target, but different options.


-Eric