×

INDI Library v1.9.9 Released (30 Nov 2022)

Bi-monthly INDI Library released with new drivers and bug fixes.

Raspberry image download duration?

  • Posts: 314
  • Thank you received: 34
Is this normal that it take ~1 minute to preview fits image from my DSLR? From the point "Exposure done, downloading image..." it needs 50 seconds for the image to arrive in the fitsviewer, previewing/sequence native image takes ~ 30 seconds. I guess this is due the image conversion is done on the raspi, cant this be done on the client side?
--= human, without Windows™ =--
pls excuse my bad english! :)
7 years 3 months ago #4750

Please Log in or Create an account to join the conversation.

  • Posts: 226
  • Thank you received: 88
I have a 350D and, when Long Exposure Noise reduction is set and I take a 30s exposure, it actually takes a dark to subtract it, so the whole process takes more than one minute. For 20s exposure it seems not to be the case. I do not know your DSLR, maybe you could look into that direction. Actually it depends on when the "Exposure done" message is displayed, if just after the exposure time when the shutter closes, that could be the reason.
7 years 3 months ago #4755

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
I dont think this is an noise reduction issue of the camera (Canon700D). I normally disable this stuff when doing raw. If I connect the camera directly to the ekos client it takes only about 6 seconds to display the fits preview. And all exposures where just 5 seconds long. I think the "Exposure done" is always displayed after the image was taken and the shutter is closed. I may just screwed up some code or compiler optimisations. Tomorrow I will try to convert a Canon raw directly on the raspi through dcraw and see how long this takes.
--= human, without Windows™ =--
pls excuse my bad english! :)
Last edit: 7 years 3 months ago by pauledd.
7 years 3 months ago #4758

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
I tried local connection of my DSLR again:
Downloading image preview from "Exposure done, downloading image..." to display in fits-viewer takes:
Ekos remote-connected to localhost: 6 seconds
Ekos remote-connected to raspberry: 50 seconds

on the raspberry I can see the 50 seconds consist of
~18 seconds for the "dcraw -c -t 0 -4 -D /tmp/indi_YxjwXp" command (cpu 100%)
~34 seconds for ??? (cpu 15%)
rest for transfering via network
--= human, without Windows™ =--
pls excuse my bad english! :)
Last edit: 7 years 3 months ago by pauledd.
7 years 3 months ago #4765

Please Log in or Create an account to join the conversation.

  • Posts: 226
  • Thank you received: 88
I think the 34 seconds corresponds to the transform of the image in fits format (if you select fits and not native) and then to the encoding in base64. Maybe you could play with the fits/native switch and the compress/raw one and see if there are some differences. The fits transform may require floating point computations (if it is still in float format), compressing the transferred image may reduce the base64 encoding time (even f it adds some compression time).
But the 15%CPU is astonishing indeed as the computations above are intensive. So maybe there is a 30s timeout somewhere in the network transfer. Or a local write to the SD card. Just some thoughts.
7 years 3 months ago #4768

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
I recorded a video while I was previewing from raspi where you can see cpu usage does not even reach 30% after the dcraw stuff.
I also think that has to do with the fits transform rather than network/sdcard. On my raspi the /tmp is in tmpfs so this should not be the matter and network activity is only in the last 7 seconds where the image comes in.

Do you know which part of the code does the fits transform? Maybe I missed some compiler optimisation flags.
If there is someone out there that has the same waiting time I would finally say I have to live with that fact but without confirmation I always feel I did something wrong.

PS. Ive just got an idea, I will try astroberry distro image and test if there is an difference
--= human, without Windows™ =--
pls excuse my bad english! :)
7 years 3 months ago #4773

Please Log in or Create an account to join the conversation.

  • Posts: 226
  • Thank you received: 88
I forgot to mention that after downloading the image from dcraw, transform to fits, eventually compress it, the driver sends the BLOB data to the server via a pipe. And your video shows ~15% activity for both driver and server starting around 35 seconds. There is an increasing server activity before that (the pipe fills up), then the server jumps to 20%, it may base64 decode the packets from the pipe as it receives them (should look in the code). So on my mind the bottleneck is in the pipe transmission of the data. You may try with compression to reduce the size of the transferred BLOB, and try native format to test if fits transform is the point.
The code which does the fits transform lies in the cfitsio lib. The INDI code is in the UpdateExposure function in indiccd.cpp , which calls uploadFile to send/save the data. The effective transfer to the server is made in the IDSetBLOB function in indidriver.c .
7 years 3 months ago #4776

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
fits/compress 90 seconds
native/raw 25 seconds
native/comress 35 seconds

For now I will try to stick with native as much as possible until someone else complains. The code is to hardcore for my understanding :blush:
--= human, without Windows™ =--
pls excuse my bad english! :)
7 years 3 months ago #4777

Please Log in or Create an account to join the conversation.

  • Posts: 226
  • Thank you received: 88
I made the test with a pi under raspbian using pre-compiled libraries. The results are very similar, and you may be lucky as, in my case, FITS+compress takes more than 4 minutes. I checked hardware floating point acceleration with readelf for gphoto-ccd driver and libcfitsio3, both were ok. So the raspberry pi is quite slow, and the information given by ps should be interpreted more carefully. But the compress time I got above still seems strange to me.
The following user(s) said Thank You: pauledd
7 years 3 months ago #4793

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
Thank you very much for testing, good to know that my setup seems to be not faulty :). Could it be that you used an older raspberry? I use an raspberry pi 2 which could explain the faster compression. Wouldn't it be technical possible to do everything after the "image download" from the camera on the clients side , on the machine where kstars is running? (apart from the fact to rewrite code :whistle: ).
--= human, without Windows™ =--
pls excuse my bad english! :)
Last edit: 7 years 3 months ago by pauledd.
7 years 3 months ago #4794

Please Log in or Create an account to join the conversation.

If you do not need to preview the image, simply set the "Upload" to local (in the GPhoto driver) so that the images are saved on the Raspberry PI (and maybe you can use a viewer to view them later at your own discretion). But at least you'll save all the time for dcraw + base64 encoding + compression + network transfer.
Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info
7 years 3 months ago #4795

Please Log in or Create an account to join the conversation.

  • Posts: 314
  • Thank you received: 34
Good idea I will try that. What about the focuser/astrometry? Do they work with local-only mode? Because yesterday most of the time I spent on the focuser with every iteration capture it took about one minute and for all about 10 minutes to focus properly.
--= human, without Windows™ =--
pls excuse my bad english! :)
7 years 3 months ago #4797

Please Log in or Create an account to join the conversation.

Time to create page: 0.518 seconds