×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Unable to open sequence queue file - after already imaging several targets

  • Posts: 19
  • Thank you received: 1
Hi, i'm using the schedule tool and it works, but maybe in 1 in every 5 targets I get and error saying "Unable to open sequence queue file '/home/astroberry/scripts/0pint5s+1s+2s+5s_all_x50.esq' and Ekos crashes out.

What is causing this problem, why would it take hundreds of subs on several targets and then suddenly say it can't access the file (usually when it is getting to the end of the target capture) when every target has used the exact same .esq file?

KStars 3.4.3 running on a raspberry pi 3 on Astroberry.

Thanks in advance.
3 years 4 months ago #62370

Please Log in or Create an account to join the conversation.

  • Posts: 19
  • Thank you received: 1
Should it be of any use in the last few examples it has failed around 118 of 125 or 119 out of 125 captures on the target which fails. So after doing x50@0.5s ok, then x25 at 2s ok, then 43@5s or 44@5s then fail.
Last edit: 3 years 4 months ago by James.
3 years 4 months ago #62372

Please Log in or Create an account to join the conversation.

Please share your esq and esl files.
3 years 4 months ago #62387

Please Log in or Create an account to join the conversation.

  • Posts: 19
  • Thank you received: 1

Hi, please see below - note the .esq file has 30 targets.

2020-11-03.esl file:
 

and 0point5s+1s+2s+5s_all_x50.esq file:
<?xml version="1.0" encoding="UTF-8"?>
<SequenceQueue version='2'>
<CCD>ZWO CCD ASI120MM-S</CCD>
<FilterWheel>--</FilterWheel>
<GuideDeviation enabled='false'>2</GuideDeviation>
<Autofocus enabled='false'>0</Autofocus>
<RefocusOnTemperatureDelta enabled='false'>1</RefocusOnTemperatureDelta>
<RefocusEveryN enabled='false'>60</RefocusEveryN>
<Job>
<Exposure>1</Exposure>
<Binning>
<X>1</X>
<Y>1</Y>
</Binning>
<Frame>
<X>0</X>
<Y>0</Y>
<W>1280</W>
<H>960</H>
</Frame>
<Temperature force='false'>5.7</Temperature>
<Type>Light</Type>
<Prefix>
<RawPrefix></RawPrefix>
<FilterEnabled>0</FilterEnabled>
<ExpEnabled>1</ExpEnabled>
<TimeStampEnabled>1</TimeStampEnabled>
</Prefix>
<Count>50</Count>
<Delay>0</Delay>
<FITSDirectory>/home/astroberry/images</FITSDirectory>
<UploadMode>0</UploadMode>
<FormatIndex>0</FormatIndex>
<Properties>
</Properties>
<Calibration>
<FlatSource>
<Type>Manual</Type>
</FlatSource>
<FlatDuration>
<Type>Manual</Type>
</FlatDuration>
<PreMountPark>False</PreMountPark>
<PreDomePark>False</PreDomePark>
</Calibration>
</Job>
<Job>
<Exposure>2</Exposure>
<Binning>
<X>1</X>
<Y>1</Y>
</Binning>
<Frame>
<X>0</X>
<Y>0</Y>
<W>1280</W>
<H>960</H>
</Frame>
<Temperature force='false'>5.7</Temperature>
<Type>Light</Type>
<Prefix>
<RawPrefix></RawPrefix>
<FilterEnabled>0</FilterEnabled>
<ExpEnabled>1</ExpEnabled>
<TimeStampEnabled>1</TimeStampEnabled>
</Prefix>
<Count>25</Count>
<Delay>0</Delay>
<FITSDirectory>/home/astroberry/images</FITSDirectory>
<UploadMode>0</UploadMode>
<FormatIndex>0</FormatIndex>
<Properties>
</Properties>
<Calibration>
<FlatSource>
<Type>Manual</Type>
</FlatSource>
<FlatDuration>
<Type>Manual</Type>
</FlatDuration>
<PreMountPark>False</PreMountPark>
<PreDomePark>False</PreDomePark>
</Calibration>
</Job>
<Job>
<Exposure>5</Exposure>
<Binning>
<X>1</X>
<Y>1</Y>
</Binning>
<Frame>
<X>0</X>
<Y>0</Y>
<W>1280</W>
<H>960</H>
</Frame>
<Temperature force='false'>5.7</Temperature>
<Type>Light</Type>
<Prefix>
<RawPrefix></RawPrefix>
<FilterEnabled>0</FilterEnabled>
<ExpEnabled>1</ExpEnabled>
<TimeStampEnabled>1</TimeStampEnabled>
</Prefix>
<Count>50</Count>
<Delay>0</Delay>
<FITSDirectory>/home/astroberry/images</FITSDirectory>
<UploadMode>0</UploadMode>
<FormatIndex>0</FormatIndex>
<Properties>
</Properties>
<Calibration>
<FlatSource>
<Type>Manual</Type>
</FlatSource>
<FlatDuration>
<Type>Manual</Type>
</FlatDuration>
<PreMountPark>False</PreMountPark>
<PreDomePark>False</PreDomePark>
</Calibration>
</Job>
</SequenceQueue>

A few other points of note it consistently does it, so restarts don't matter, it will still crash out after a few targets have completed. By the end of yesterday it was doing it after two good targets then fail. No other machine accessing files via the astroberry created share, and about 20GB+ free on the memory card. Also I'm not sure if it is related but when I tried to move over files to my laptop it came up with an error on some of the transfers which were the last files written before a fail and subsequent restart - it would not let me delete those from windows, but could delete from the PI file browser ok. No issue with the content of the file.

Thanks in advance!
3 years 4 months ago #62403

Please Log in or Create an account to join the conversation.

  • Posts: 19
  • Thank you received: 1
This evening i've done some more testing. I created some .esl and .esq files using Ekos (the ones above I generated in excel) and the same happens.

The first was 4 targets = 600 images. Then I added some more targets, saved it down and ran again. But after 5 images it failed (so 605 images successful since EKos was started).

I then started it again and if my maths are correct it has failed again after another 605 images. Feels like too much of a conscience. Is Ekos keeping some sort of log of what it has done as I note on EKos restart scheduler knows how many images it has taken of each target or is it scanning the image directory?
3 years 4 months ago #62431

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301
Sounds like it could be related to invent.kde.org/education/kstars/-/issues/38, but not quite. There is no network share access from the point of view of Ekos, correct?

-Eric
3 years 4 months ago #62452

Please Log in or Create an account to join the conversation.

  • Posts: 19
  • Thank you received: 1

Could be yes, but I'm not 100% sure. The error message window looks the same as in the screenshot after it has crashed out but maybe the actual error text shown before the crash is different. It doesn't let you click the OK button on the message box as soon as it pops up and then a few seconds later you get the crash.

KStars/Ekos is running locally on the PI with all hardware connected locally and I'm looking at it on remote desktop (which happily stays connected and responding fine all evening without issue). The network share "shouldn't" (famous last words!) be having any impact I would speculate. I've also tried sequence files created on the EKos schedule window and ones I created via text editor in case it was a permissions issue on that file and it happens in both cases.

I've not noticed any issues with the file numbering as I'm doing a max of 50 images for each exposure length so the number in the file name is resetting as expected when the next exposure length begins. I also have the timestamp in the file name and the gap between images is never <1.2 second (min 1s exp plus transfer between camera and pi time). The access violation is on the .esl sequence queue file.
Last edit: 3 years 4 months ago by James.
3 years 4 months ago #62453

Please Log in or Create an account to join the conversation.

  • Posts: 1029
  • Thank you received: 301
In the Ekos options, there is a checkbox named "Remember job progress". Could you uncheck that and try?

-Eric
3 years 3 months ago #64292

Please Log in or Create an account to join the conversation.

  • Posts: 33
  • Thank you received: 7
Hi

Did you get this solved?  I have been running KStars/EKOS on my main observatory for a year or so on and off. 

I just set a new pi up in the house which is managing the capture for an AllSkyCam.  The observatory pi never had this issue, but I have seen it almost every night for the new Pi. 

Observatory is decommissioned just now after house renovation but will be back in service in the next few weeks. 

The details:

I set up a 1x60s image sequence to capture an image from the All Sky Camera - it's a Starlight Express Occulus.  Then I set up a schedule with a non-setting target (Polaris) and switch off the align/focus/guide options.  I also have Telescope Simulator running - maybe this is not necessary?  I can't remember why I added it to my profile. 

The sequence is set up to run until cancelled and the twilight control enabled with suitable offsets so I can capture close to sunset/sunrise.  The schedule crashed on nights 1 and 2 fairly early on with the error:

"Unable to open sequence queue file:" then the path to the sequence file.

I realised there might have been some contention on the .esq file so changed the sequence setup to be 60 x 60s to prevent the .esq file being read too often and this worked for 1 night only.  It failed again last night quite early on.  Funny thing though there was a gap of an hour between the last and next to last file captured (saved).  I'm attaching the INDI log files from last night.   

I just noticed that file named 072 is used for the last 60 of the captures (I think).  Unfortunately I overwrote the .esq file when I re-set things up just now but they are very straightforward so I am attaching the new one I just created - I am sure it is the same, and I'm attaching the .esl as well. 

It does look to me as though there is some issue with the counting of the files captured so far, or perhaps in saving the latest file from the camera since 072 is never advanced.

Many thanks for a fabulous product Jasem.

Iain 

Edit:  I have samba running sharing the astroberry root, and also a program called SyncThing (which is also excellent) - this shares all files in certain directories to another computer as they appear.  I have changed the configuration in SyncThing to only check for new files every hour (was watching the directories using notification from the filesystem previously) - will report back if this makes any difference after a day or two.  Reason I did not think this was an issue at first was due to the logging pointing to the .esq file not being able to be read, but now I am not so sure having looked at the log more closely...
Last edit: 2 years 7 months ago by Iain Melville. Reason: Updated with details of samba and SyncThing.
2 years 7 months ago #74531
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 33
  • Thank you received: 7
Same thing last night, made it through 450 odd images then this:

[2021-08-17T04:48:01.228 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_163'"
[2021-08-17T04:48:01.228 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_248'"
[2021-08-17T04:48:01.228 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_227'"
[2021-08-17T04:48:01.228 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_399'"
[2021-08-17T04:48:01.229 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Frame map summary:
[2021-08-17T04:48:01.229 BST DEBG ][ org.kde.kstars.ekos.scheduler] -   "/home/astroberry/Pictures/AllSky/Zenith/Light/Zenith_Light" : 474
[2021-08-17T04:48:01.230 BST INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'Zenith' 60x60\"  completed its sequence of 0 light frames."
[2021-08-17T04:48:01.234 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'Zenith' is configured to loop until Scheduler is stopped manually, has undefined imaging time."
[2021-08-17T04:48:01.234 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Capture State "Capturing"
[2021-08-17T04:48:01.279 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:48:01: Driver indi_sx_ccd: sxClearPixels: libusb_control_transfer -> OK"
[2021-08-17T04:48:01.280 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:48:58.154 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:48:58: Driver indi_sx_ccd: sxClearPixels: libusb_control_transfer -> OK"
[2021-08-17T04:48:58.154 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:49:01.154 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:49:01: Driver indi_sx_ccd: sxLatchPixels: libusb_control_transfer -> OK"
[2021-08-17T04:49:01.155 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:49:01.905 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:49:01: Driver indi_sx_ccd: sxReadPixels: libusb_control_transfer -> OK"
[2021-08-17T04:49:01.906 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:49:01.943 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Uploading file. Ext: fits, Size: 2903040, sendImage? Yes, saveImage? No "
[2021-08-17T04:49:02.016 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] BLOB transfer took 0.0609353 seconds "
[2021-08-17T04:49:02.016 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Upload complete "
[2021-08-17T04:49:02.053 BST DEBG ][           org.kde.kstars.indi] - Image received. Mode: "Normal" Size: 2903040
[2021-08-17T04:49:02.063 BST INFO ][           org.kde.kstars.indi] - "FITS" file saved to "/home/astroberry/Pictures/AllSky/Zenith/Light/Zenith_Light_475.fits"
[2021-08-17T04:49:02.107 BST INFO ][   org.kde.kstars.ekos.capture] - "Download Time: 0.09 s, New Download Time Estimate: 0.08 s."
[2021-08-17T04:49:02.207 BST INFO ][   org.kde.kstars.ekos.capture] - "Received image 55 out of 60."
[2021-08-17T04:49:02.274 BST DEBG ][                       default] - WARNING: Phonon::createPath: Cannot connect  Phonon::MediaObject ( no objectName ) to  Phonon::AudioOutput ( no objectName ).
[2021-08-17T04:49:02.288 BST INFO ][   org.kde.kstars.ekos.capture] - "Capturing 60.000-second  image..."
[2021-08-17T04:49:02.298 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Capture State "Image Received"
[2021-08-17T04:49:02.299 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "Searching in path '/home/astroberry/Pictures/AllSky/Zenith/Light', files 'Zenith_Light*' for prefix 'Zenith_Light'..."
[2021-08-17T04:49:02.311 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_399'"
[2021-08-17T04:49:02.312 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Frame map summary:
[2021-08-17T04:49:02.313 BST DEBG ][ org.kde.kstars.ekos.scheduler] -   "/home/astroberry/Pictures/AllSky/Zenith/Light/Zenith_Light" : 475
[2021-08-17T04:49:02.314 BST INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'Zenith' 60x60\"  completed its sequence of 0 light frames."
[2021-08-17T04:49:02.327 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'Zenith' is configured to loop until Scheduler is stopped manually, has undefined imaging time."
[2021-08-17T04:49:02.328 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Capture State "Capturing"
[2021-08-17T04:49:02.339 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:49:02: Driver indi_sx_ccd: sxClearPixels: libusb_control_transfer -> OK"
[2021-08-17T04:49:02.339 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:49:59.288 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:49:59: Driver indi_sx_ccd: sxClearPixels: libusb_control_transfer -> OK"
[2021-08-17T04:49:59.289 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:50:02.288 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:50:02: Driver indi_sx_ccd: sxLatchPixels: libusb_control_transfer -> OK"
[2021-08-17T04:50:02.289 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:50:03.038 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:50:03: Driver indi_sx_ccd: sxReadPixels: libusb_control_transfer -> OK"
[2021-08-17T04:50:03.039 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:50:03.078 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Uploading file. Ext: fits, Size: 2903040, sendImage? Yes, saveImage? No "
[2021-08-17T04:50:03.151 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] BLOB transfer took 0.0680964 seconds "
[2021-08-17T04:50:03.152 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Upload complete "
[2021-08-17T04:50:03.180 BST DEBG ][           org.kde.kstars.indi] - Image received. Mode: "Normal" Size: 2903040
[2021-08-17T04:50:03.183 BST INFO ][           org.kde.kstars.indi] - "FITS" file saved to "/home/astroberry/Pictures/AllSky/Zenith/Light/Zenith_Light_476.fits"
[2021-08-17T04:50:03.226 BST INFO ][   org.kde.kstars.ekos.capture] - "Download Time: 0.07 s, New Download Time Estimate: 0.08 s."
[2021-08-17T04:50:03.318 BST INFO ][   org.kde.kstars.ekos.capture] - "Received image 56 out of 60."
[2021-08-17T04:51:00.390 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:51:00: Driver indi_sx_ccd: sxClearPixels: libusb_control_transfer -> OK"
[2021-08-17T04:51:00.391 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:51:03.392 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:51:03: Driver indi_sx_ccd: sxLatchPixels: libusb_control_transfer -> OK"
[2021-08-17T04:51:03.392 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:51:04.139 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2021-08-17T03:51:04: Driver indi_sx_ccd: sxReadPixels: libusb_control_transfer -> OK"
[2021-08-17T04:51:04.140 BST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2021-08-17T04:51:04.162 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Uploading file. Ext: fits, Size: 2903040, sendImage? Yes, saveImage? No "
[2021-08-17T04:51:04.223 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] BLOB transfer took 0.0446538 seconds "
[2021-08-17T04:51:04.223 BST DEBG ][           org.kde.kstars.indi] - SX CCD SuperStar : "[DEBUG] Upload complete "
[2021-08-17T04:51:04.276 BST DEBG ][           org.kde.kstars.indi] - Image received. Mode: "Normal" Size: 2903040
[2021-08-17T04:51:04.283 BST INFO ][           org.kde.kstars.indi] - "FITS" file saved to "/home/astroberry/Pictures/AllSky/Zenith/Light/Zenith_Light_476.fits"
[2021-08-17T04:51:04.336 BST INFO ][   org.kde.kstars.ekos.capture] - "Download Time: 0.11 s, New Download Time Estimate: 0.08 s."
[2021-08-17T04:51:04.438 BST INFO ][   org.kde.kstars.ekos.capture] - "Received image 57 out of 60."
[2021-08-17T04:51:04.454 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Capture State "Image Received"
[2021-08-17T04:51:04.455 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "Searching in path '/home/astroberry/Pictures/AllSky/Zenith/Light', files 'Zenith_Light*' for prefix 'Zenith_Light'..."

Seems something is up with the file counting:

[2021-08-17T04:49:02.311 BST DEBG ][ org.kde.kstars.ekos.scheduler] - "> Found 'Zenith_Light_399'"
[2021-08-17T04:49:02.312 BST DEBG ][ org.kde.kstars.ekos.scheduler] - Frame map summary:

Only one file is found/logged at this point.  After this the system chooses the same file name twice (475) then 476 twice then soon after it crashed with the same error reported.  Full log attached.  Interestingly after this fail to count files, it seems to recover but something has upset it by this point.  I might have to start looking at the code.





 
2 years 7 months ago #74544
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 7
  • Thank you received: 1
I too am seeing this error and would like to know a fix and what I can provide to help resolve.

I am running astroberry.io on RPi 4 with 4GB on a 128 GB SD Card.  I have networking enabled to allow remote access for SMB and VNC.

I saw this in the version since Jan 2021, and it used to have a popup dialog box as shown in the attached image.

I upgraded in August 2021 and now the error is no longer in the dialog but is in the message log on the bottom of the screen.  It causes "reset of the file counter to be xyz_<datetime>_001.fits for all files after the error".  It has Status for all jobs shown as "Invalid" when it was already loaded and running for several of the images.

Like the original poster, I create jobs with many targets (> 10 per night), but this is not a requirement.

I had thought this was related to targets in the job "dropping below the horizon" as the job typically runs for 8 hrs (all night) and the first targets drop below 0deg, but this is not a required condition.

I could not find the pic, but will certainly get the error tonight and will add to the post.
 
The following user(s) said Thank You: Iain Melville
Last edit: 2 years 6 months ago by Scott Bushman. Reason: Add image showing error in log window and invalid for items
2 years 6 months ago #75453
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 33
  • Thank you received: 7
Hi Scott

Sorry to hear you might be experiencing the same issue - I'm not sure we are both seeing the exact same issue but it sounds like we possibly are.

I was waiting to hook up an external drive to see if this might be due to reading/writing to the SD card. Sadly it appears that this is not the case. I am easily able to reproduce the issue by running a schedule specifying unlimited images (I set the exposure length to 1s). I'm using the SX camera driver with an all sky cam so my schedule is far simpler than yours by the sound of it. I got the attached issue after 322 1s subs this time. The actual number of subs before the issue varies for me.

I'm happy to put some time/effort into trying to solve this.  I will confirm that I have the latest version (might switch to nightly builds) and reconfirm the issue.  Then I guess the next step might be to try and run kstars in a debug session which I have read about elsewhere in the forum.

Cheers
Iain
2 years 6 months ago #75469
Attachments:

Please Log in or Create an account to join the conversation.

Time to create page: 1.463 seconds