That looks good!

Now I wonder if another night proceeding with this schedule / job group it would also 'just keep going'.

For me, the issue occured recently again, twice in a night.
I have the logs, but would any dev still like to help out please?

If you have 'Repeat until terminated' checked for all your tile jobs then the same schedule should continue to collect images for all tiles.

If you have verbose logs to share, please post links to them along with your description of what problem you're reporting in as much detail as possible. I've alerted Wolfgang who is more familiar with what I think you're describing.

Hy

Second night running with exactly the same 4 jobs, repeat until terminated enabled for all 4 jobs.

Yesterday there was no issue, but I purposely didn't "touch" any of the files during capture. I still suspect that as soon as the timestamp or other file properties of the saved FITS file changes, the process starts acting weird.

Please can someone from the developers explain what file check is happening before/after saving of each FITS file?

The code simply checks to see if there are files with the right file names in the right directory on disk and counts them. It does not look in detail at those files, just checks the file names

No offense intended at all, but perhaps an error has crept into the code somewhere.

Last night I started exactly the same 4 jobs as 2 nights earlier. And while nothing went wrong last time, this night I notice that job 2 took too many lights and stuck.

• Job 1 did run 5 times x 15 = 75 lights
• Job 2 did run 5 times and continues = 81 lights (!)
• Job 3 did run 4 times x 15 = 60 lights
• Job 4 did run 4 times x 15 = 60 lights

I don't understand at all why job 2 was not stopped after the 75th recording and why job 3 was not started after job 2 was done.

The monitor screen also looks strange:



Each job has its own color, but occasionally this color disappears. For clarity, I have given the jobs a red number in the screenshot above to illustrate this. Why is this happening?

In the logs you can see at 2023-09-12 05:30 AM that job 2 continuous (batch #18) in stead of passing the torch to job 3. For all completeness, these are the relevant log files:

@Fitchie,

It is much better if you submit log files instead of scraping the text from the log windows. There is a lot more info sent to the file than is displayed in the window. Also, one can tell the geography and the software being used, etc. Make sure the log files are set to verbose and check all Ekos modules. Usually those files are too large to post directly, so just gzip them and upload them to Google Drive or Dropbox and send a link.

However, in this case, I don't see any problem to look into (other than the color issue), so at this point I don't think logs are necessary. The mosaic tile-switching seems to have worked very well for you until about 9 minutes before it shut down for astronomical dawn. It wouldn't shock me if the system didn't switch jobs a few minutes before it was going to shut down anyway for daylight.

Hy

Yes,
Here the are:
drive.google.com/file/d/1esVKadDDPfeHZBh.../view?usp=drive_link

And for good measure, the scheduler file:
drive.google.com/file/d/1W8igeXwrIX2ZDeq.../view?usp=drive_link

Thanks a lot and hopefully this helps make EKOS better!

One thing I've did is edit the individual jobs to use a different sequence, as I wanted to use different length lights than when i created the mosaic.
I can't imagine this should matter, but alas here you know.

Cheers,
Bart

Bart

Can you please clarify/repeat, since this is a long thread, what the issue is in the logs you’re submitting?

Thanks for submitting them
Hy

Sure Hy! It was from this session:

It later occurred again, however there are no logs. apparently logging was not enabled 'by default' I'll check next next sessions and make sure it's logging.

@Hy

The mosaic tile-switching seems to have worked very well for you until about 9 minutes before it shut down for astronomical dawn. It wouldn't shock me if the system didn't switch jobs a few minutes before it was going to shut down anyway for daylight.

Does that mean that the scheduler itself determines when the next job can no longer be completed, it will not be started and the current job will continue to be repeated until the end?

That sounds nice at first glance, but in practice it makes little sense with a mosiac. During integration, the same number of lights are required per tile. So the excess pictures are actually pointless and will not be used.

Apart from that, this behavior is very confusing, which is why this thread is becoming so long. Is there a way to 'force' the scheduler to do exactly what is set?

I'm afraid the scheduler is not that smart as we expect.

In the previous case it seems that no new job was started anymore due to daylight restrictions. But the following screenshots I saved from previous nights of inconsistent behavior show some rather random results.

Bart, Fitchie,
I would like to dive into the problems you are reporting here. But the thread is quite long and I'm not sure what exactly happened you are talking about.

As far as I understand is, that you both experience that jobs sometimes don't create the expected number of frames, right?

But please be so kind and share in one post
1. The log files
2. The scheduler file and the capture sequence file(s) you are using
2. And please explain exactly which problem you experience

@Bart: from the last log file you sent it seems like only Part-2 has captured frames (besides captures of M31, which I think is irrelevant here). What exactly is the problem in that section?
@Fitchie: I haven't seen any log file from you, so please be so kind and post them altogether with the points I mentioned above.

Cheers
Wolfgang