Welcome, Guest
Username: Password: Remember me
25 Jul 2018
Glad to announce of release of INDI Library v1.7.4 on 2018-07-25. A few drivers were added in this release as we continue to improve & stabilize existing drivers.
Read More...
  • Page:
  • 1

TOPIC: EKOS imaging pipeline?

EKOS imaging pipeline? 6 months 1 week ago #24749

I'm looking at setting up a pipeline for Speckle Interferometry. In a nutshell, I would like to pre-process images as the frames are taken. Conceptually this would be from a remote site. We take as many as 10,000 frames per target and while small, that's a lot of data.

Not sure how to even search for what I am asking for, but want the save to disk function in the observatory and the scheduling to be done in my office. That is EKOS scheduler is running on my system in the office, but the data is being collected and reduced at the observatory. Once processed the result is about the size of a single frame. This is of interest to anyone doing high cadence imaging.

Can this be done now? If so a hint on how would be nice. Also a pointer to where you would recommend hacking in the cuda code for the processing :-)

Thank you,

Please Log in or Create an account to join the conversation.

Last Edit: by Gregj888.

EKOS imaging pipeline? 6 months 1 week ago #24756

Greg, this is quite interesting. It is possible, but might require more work to get done. In the Capture module, there is a "script" field which you can execute after each image is captured so it can then process the image. Currently, it does not pass any argument nor it sets any environment variables about the last captured frame, but this could be added in. Is this what you're looking for?

Please Log in or Create an account to join the conversation.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

EKOS imaging pipeline? 6 months 1 week ago #24758

knro, I can make that work in theory, but it dosn't do what I am looking for.

Think remote observatory with slow data connection. We shoot about 5000 frames per target and 10+ targets an hour. We can fill a terabyte drive in 3 nights. So the goal is to collect the raw frames in the observatory and do the Speckle reduction (Fourier based) on the fly (GPU) returning just the Speckle results. That will give us close to a 5000 to 1 reduction in data needing to be transferred. We also need to put the files into a reasonable directory structure and Ideally store each target in a FITs cube instead of a frame per file.

I know that's a lot and I'll take a shot at it. Since the first note I've looked at the code and have an idea what to do.

TIt looks like EKOS does the actual file save, correct? That means that EKOS would need to be running on the observatory computer. If all that is correct, then splitting the scheduler from the capture module would be the Ideal way forward. That is, let the capture module run under the server, and direct it with the scheduler via the server.

Hope that makes sense?

I'm hoping to get the cuda/GPU portion working, probably with Open Astro, but CCDCiel is also a possibility as it uses INDI for the cameras from what I can tell.

Thank you,

Please Log in or Create an account to join the conversation.

EKOS imaging pipeline? 6 months 1 week ago #24759

INDI driver can save the files at the observatory side. The scheduler can either be run remotely or on site. I think you need to write your own script/software to monitor the directory where INDI dumps the data and then perform your own processing there.

Please Log in or Create an account to join the conversation.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

EKOS imaging pipeline? 6 months 1 week ago #24761

Jasem,

That may be the best option. In fact the more I think about it, running the scheduler on the observatory hardware seems like a more robust solution (WAN disconnect). Nice thing about INDI is there are a number of options here. The local file processor was the initial plan, but for several reasons, doing it as part of the capture process looks like a better option.

I have some homework to do... Thank you again.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1
Time to create page: 0.110 seconds

Login



3rd Party

Choose from the numerous 3rd party INDI drivers to suit your needs!

Got Problem?

Check out the FAQ, the forum, and the bug tracking system to resolve any issues you might have!
You can also subscribe to INDI newsletter and development mailing lists to get the latest updates on INDI!


Gallery

Replica

Why INDI

Replica