×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed

  • Posts: 2876
  • Thank you received: 809
For the last couple of months I have been playing around with an idea. First my idea was to combine Sextractor and Astrometry.net to avoid using python. That worked very well and so I build it in as an option in KStars. Not only did it work without python, it was often faster than traditional astrometry solving. But there are many parameters. So I started trying to explore and play around with the parameters and found it easier to do in a separate testing program instead of in KStars so that I could really focus on the problem. Then after I got it up and running, I got a little bolder. I thought it might be a cool idea if we could make astrometry.net into a library and get it natively running inside an Internal Library in KStars on Windows Mac and Linux, instead of an external set of programs that only works on certain operating systems all kludged together. So over the last couple of months, I have done exactly that. I don't believe it is done yet, but I think we are ready for some rigorous testing to start evaluating how well it works, try to make it better, figure out what parameters are effective and which ones are useless, and see what parameters should be added. Right now, for some images, the SexySolver is faster than any of the other methods, but for others it is not. It should by definition be faster because it doesn't have to save any temporary files along the way, nor does it need to call any external programs. So I think if we can fix this up, it should be much more efficient.

Here is my project: github.com/rlancaste/sexysolver-tester



To try this out, you don't need any programming knowledge, you just need some astronomy photos, a computer on which astrometry.net index files have been downloaded, and if you want to compare to external astrometry.net, sextractor, astrometry.net, or ANSVR, they would need to be installed, but that is optional. You just need to download the solver tester program and start trying it out.

Here is the download link to the Windows and Mac installers for the latest release version:
github.com/rlancaste/sexysolver-tester/releases/tag/0.8

What kind of feedback do I need? Well several things:
- Which parameters seem to help solve images faster and more accurately?
- For sextracting, which parameters really seem to help make the source extraction better?
- Which parameters don't seem to help too much?
- Which parameters should not be exposed to the users because they could mess up the solve?
- What additional parameters would we need to add?

For any programmers out there:
- If you can take a look at my code and help optimize it in any ways that would help
- Any Sextractor, SEP, or astrometry.net experts could look at my SexySolver code to see if it is accurate or can be improved
- I had to make a bunch of changes to astrometry to get it to work on Windows. It mostly works now on windows, but if there is any way to make my changes better, that would help.

Once we get this refined, it should be able to greatly enhance Plate Solving, Guiding, and Focusing algorithms and could be used to support other cross platform QT based programs that could run on Astrometry.

Thanks for your help,

Rob
The following user(s) said Thank You: Jasem Mutlaq, Gonzothegreat, Radek Kaczorek, Alfred, Fabio Papa, Wouter van Reeven, Wolfgang Reissenberger, Craig, Brian, Peter Sütterlin and 2 other people also said thanks.
3 years 11 months ago #52637
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 1957
  • Thank you received: 420
Very nice Rob! Thanks very much for doing this. I tried it out on an image of Proxima Centauri that I took tonight. I noticed that when using the internal astrometry.net solver and aborting it, the astrometry.net process is not stopped. I had to kill it by hand. And as I have noticed with plate solving in KStars before (both with sextractor and astrometry.net), NOT using the scale is MUCH, MUCH faster than when using the scale.

Wouter
3 years 11 months ago #52643

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
Thanks for trying it, and please try it some more. The more info we get the better.

For cancelling the internal solver, astrometry.net has a method that monitors for the creation of a “cancel” file. That is what I implemented here instead of trying to just kill the thread. What I read is that this is a more graceful of stopping the search and can avoid some side effects that could occur if you just kill it. Note that it doesn’t stop immediately when you do it this way, it sometimes takes a few seconds. If that way doesn’t work well I can try another way, but it said this way was better.

For the scale, are you sure your scale numbers in the image are right? I think it makes it go faster when the scale is right but makes it take much longer if the scale is wrong. I have some images that I took where the scale is right and some where it was wrong that I took with KStars. I don’t know what I did differently when I took them, I haven’t figured it out yet.

Thanks,

Rob
3 years 11 months ago #52666

Please Log in or Create an account to join the conversation.

  • Posts: 1957
  • Thank you received: 420
Rob, the astrometry.net processes (two of them because I tried twice and aborted both) kept on running for minutes and didn't stop even when I quit SexySolver. I noticed because they both were using 99% CPU time and the fans of my macBook started to kick in. I had to kill the processes manually.

As for the scale, I noticed several months or possibly over a year ago that astrometry.net is a lot faster without using scale even if the scale is more or less right. More or less in this case meaning perhaps a few arc minutes off on a scale of 2.5 degrees. Ever since this is the first thing I disable when I (re)install KStars and Ekos. I'll try with a more or less scale again and will let you know the results.


Wouter
3 years 11 months ago #52673

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
Oh I might have misunderstood what you mean when you said "internal astrometry.net solver". Do you mean the "Internal SexySolver" Setting in SexySolver or one of the other settings using the KStars "internal" copy of astrometry.net? I had thought you meant the internal SexySolver.

Because the method that uses the internal sexysolver is actually my library version of Astrometry.net built into SexySolver and if you quit the program it can't still be running because it is actually a separate thread in the program itself not a separate process, so if you quit the SexySolver program, it can't still be running. But if you run any of the other solvers in the list, they start an external process. That external process is called "astrometry-engine" On a Mac, typically they will use the KStars "internal" build of astrometry.net, since that is already on your computer, but it can also use homebrew if you have that. But if you see a process called astrometry-engine, that is the "external" solver in SexySolver.

Sorry about the confusion. If you mean that the external astrometry.net solver did not stop running when you hit abort, then that is a different issue and I will definitely look into that. If it doesn't stop, then it might not stop in KStars either since I used the same method to stop that one, a kill command. Maybe we should use both the kill command and and cancel file.
3 years 11 months ago #52678

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
One trick you could try would be to solve it once, note the scale, and enter a range that includes that
3 years 11 months ago #52685

Please Log in or Create an account to join the conversation.

  • Posts: 1957
  • Thank you received: 420
Sorry, my messages have not been clear. What I meant is:
- the internal astrometry.net solver keeps running in the background when I cancel it in the SexySolver app, even when I quit SexySolver. This is not the case in Ekos.
- I do not use scale in Ekos nor in SexySolver independently of my using internal astrometry.net or sextractor. I never use the online astrometry.net

I have tried to reproduce astrometry.net still running but can't at the moment. I'll try later again.
3 years 11 months ago #52686

Please Log in or Create an account to join the conversation.

  • Posts: 185
  • Thank you received: 28
This seems to be a lot faster than the internal solver for me. The slowest of 14 solutions took 3.5s and the quickest took 0.9s. The average time was 1.3s, and the median time was 1s. All times rounded to the nearest 1/10th second.

I have attached the results table for information.

File Attachment:

File Name: Sexy_Solve..._RDB.zip
File Size:1 KB
The following user(s) said Thank You: Craig
3 years 11 months ago #52702
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
Thank you very much for the results. Did you try tweaking any settings to see which ones would make it solve faster?
3 years 11 months ago #52704

Please Log in or Create an account to join the conversation.

  • Posts: 185
  • Thank you received: 28
I just loaded and solved the images without any adjustments. The images were all fits files captured in EKOS. The test was essentially what I would use for a "Load and Slew".
3 years 11 months ago #52705

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
So with this program you can simulate the exact same setup you use in Ekos with load and slew so that we can compare the times. I am assuming you do not have the "use sextractor" checkbox checked in KStars? If that's true, you can select the Classic Astrometry.net from the combo box next to the solve button so that you can compare the time. If you do have that box checked, then it is a different option in the combo box, the "Ext Sextractor, Ext Solver" option. That way we can see the difference.

I think due the slight variation that we get in solving times with the same settings, it would be good to have an option to run a certain number of trials and calculate the average time instead of having to do each one and then figure it out. I will add that option
3 years 11 months ago #52707

Please Log in or Create an account to join the conversation.

  • Posts: 185
  • Thank you received: 28
To the best of my knowledge, I don't have the use sextractor box ticked. I re-ran the solver with Classic Astrometry.net for the same targets. I have both sets of information from SexySolver as csv files and an ods file with both in the attached zip file. The ODS file is sorted by RA and then DEC, and it becomes obvious that the internal solver was run on the same target twice.

File Attachment:

File Name: Sexy_Solve...0425.zip
File Size:22 KB


All solutions were performed on my "pierside" computer, an Intel NUC i5-8259U processor with 16GB RAM running Ubuntu 19.10 64-bit. The classic solver took 5-10x the time of the internal solver.
3 years 11 months ago #52710
Attachments:

Please Log in or Create an account to join the conversation.

Time to create page: 2.963 seconds