×
INDI Library v1.8.7 Released (21 Nov 2020)

Nov. 2020 Release brings significant improvements to streaming and recording videos. Support for new drivers including Estatto focuser and Pegasus falcon rotator.

New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed

1 month 1 week ago
DerPit
Platinum Boarder
Platinum Boarder
Posts: 566
Karma: 1
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #61991
When I try to compile the current stellarsolver my compiler (GCC 10.0) complains about missing includes in stellarsolver/sep/extract.cpp (#include <cstdio>) and stellarsolver/sep/lutz.cpp (#include <cstdlib>), else sprintf and malloc are not declared....

As for kstars, should I use 'master' or 'extractor' for testing?

openSUSE Tumbleweed KStars git INDI git
GPDX+EQMOD, CEM60EC, ASI1600+EFW+EAF+ASI290 mini

Please Log in or Create an account to join the conversation.

1 month 1 week ago
Cerro Torre
Senior Boarder
Senior Boarder
Posts: 78
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62004

rlancaste wrote:
I think I have your M65 image, correct? Was this on a Pi or some other computer? I can see if I get a crash. I don't get crashes when using my main computer, but it has much better resources than the PI does.


I am using an Intel NUC with 4 physical cores (8 logical threads) and 16 GB of RAM.

lhoujin wrote: The new solver really can solve fast! But I'm not exactly sure how fast as I did not see a time to solve stat in the output. That would be useful for determining optimal profile or custom settings.


Already added by the kstars team, thanks!

Powered by

GNU / Linux
KDE neon
KStars | EKOS | INDI

and some cheap hardware

Please Log in or Create an account to join the conversation.

1 month 1 week ago
Cerro Torre
Senior Boarder
Senior Boarder
Posts: 78
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62005
Meant as feedback for the developers:
Internal SEP | Internal Solver | 4-ParallelSmallScale (no downscale)

M57 solves in 4.44 seconds
Iris Nebula solves in 3.50 seconds
M65 solves in 2.84 seconds

Good work!

Powered by

GNU / Linux
KDE neon
KStars | EKOS | INDI

and some cheap hardware

Please Log in or Create an account to join the conversation.

1 month 1 week ago 1 month 1 week ago by schwim.
schwim
Expert Boarder
Expert Boarder
Posts: 150
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62007
Hi Rob,

I finally had a chance to do some testing with StellarSolver. I'm running a build from kstars commit 15ba3c90a1ec4724a28b43cfa69fa1fb6693242a and SS commit b8ce7148a1bad6f9dd4f5d8b4c4ef95396db4a3c.

I had been using ASTAP. I switched the solving method to "Internal solver" and left all settings as-is. I then ran Load and Slew and it just worked out of the box - no problems, generally. However, I did find a failure case that I'll share here.

BTW - when testing with the simulators I left the Solver Action as "Nothing" but the slew still occurred. Also, after the unwanted slew, the Load and Slew button was now greyed out. To recover it I had to stop/start the indi services.

The following image solves with no problem, even though the image has some problems including mystery stars.

https://www.dropbox.com/s/20kh08zb0xhc1y5/ic63_Light_L_900_secs_2020-10-18T21-35-21_001.fits?dl=0

This one works as well, without the mystery stars:

https://www.dropbox.com/s/4lx6wqjxu40lll6/ic63_Light_L_900_secs_2020-10-18T21-52-38_002.fits?dl=0

However, this one of the same field does not solve. The main differences I can determine is the median signal of this frame is higher than the other two and the image is rotated 180deg from the others.

https://www.dropbox.com/s/cfxuw6xdln8qjc5/ic63_Light_L_900_secs_2020-10-20T05-10-04_040.fits?dl=0

I've attached the log file with Alignment logging turned on. Also, below is from the log window on the alignment tab. Interesting that it says it needs 19.7GB of RAM. Is that true?

The optics are a corrected Dall-Kirkham at focal length of ~1700mm. Pixel scale is 6um.

Greg
2020-10-22T09:40:09 Solver Failed
2020-10-22T09:40:09 internal pixel buffer full
2020-10-22T09:40:00 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:40:00 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:40:00 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:40:00 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:39:35 Solver Failed
2020-10-22T09:39:35 Solver was aborted, timed out, or failed, so no solution was found
2020-10-22T09:39:35 Starting Internal StellarSolver Astrometry.net based Engine. . .
2020-10-22T09:39:35 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:35 Image width 640 pixels; arcsec per pixel range 0.93949 1.76154

2020-10-22T09:39:35 Scale range: 10.0212 to 18.7898 arcmin wide

2020-10-22T09:39:35 Set odds ratio to solve to 1e+9 (log = 20.7233)

2020-10-22T09:39:34 Configuring StellarSolver
2020-10-22T09:39:34 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:34 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:39:34 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:34 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:39:34 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:39:34 Image received.
2020-10-22T09:39:29 Capturing image...
2020-10-22T09:39:23 Slewing to target coordinates: RA (00h 59m 30s) DEC ( 61° 05' 18").
2020-10-22T09:39:23 Solution coordinates: RA (00h 59m 30s) DEC ( 61° 05' 18") Telescope Coordinates: RA (12h 45m 39s) DEC ( 90° 00' 00")
2020-10-22T09:39:23 WCS information updated. Images captured from this point forward shall have valid WCS.
2020-10-22T09:39:23 Solver RA (14.55265) DEC (60.97575) Orientation (0.22974) Pixel Scale (0.73429)
2020-10-22T09:39:23 Field parity: pos

2020-10-22T09:39:23 Field rotation angle: up is 0.229738 degrees E of N
2020-10-22T09:39:23 Pixel Scale: 0.734292"
2020-10-22T09:39:23 Field size: 55.0635 x 44.0628 arcminutes
2020-10-22T09:39:23 Field is: (-196.752, 50.8996) deg from search coords.
2020-10-22T09:39:23 Field center: (RA H:M:S, Dec D:M:S) = (00:58:12.637, +60:58:32.700).
2020-10-22T09:39:23 Field center: (RA,Dec) = (14.5527, 60.9758) deg.
2020-10-22T09:39:23 Solved with index:  4109
2020-10-22T09:39:23 Number of Matches:  11
2020-10-22T09:39:23 Solve Log Odds:  72.9335
2020-10-22T09:39:23 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Starting Internal StellarSolver Astrometry.net based Engine. . .
2020-10-22T09:39:22 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Scale range: 0.582152 to 0.873228 arcsec/pixel

2020-10-22T09:39:22 Set odds ratio to solve to 1e+9 (log = 20.7233)

2020-10-22T09:39:22 Configuring StellarSolver
2020-10-22T09:39:22 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Stars Found after Filtering: 50
2020-10-22T09:39:22 Keeping just the 50 brightest stars
2020-10-22T09:39:22 Removing the stars with a/b ratios greater than 1.5
2020-10-22T09:39:22 Stars Found before Filtering: 20397
2020-10-22T09:39:18 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:39:18 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:18 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:39:18 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:38:29 World Coordinate System (WCS) is enabled. CCD rotation must be set either manually in the CCD driver or by solving an image before proceeding to capture any further images, otherwise the WCS information may be invalid.
2020-10-22T09:38:29 Idle.
Attachments:

Please Log in or Create an account to join the conversation.

1 month 1 week ago
Ihoujin
Supernova Explorer
Supernova Explorer
Posts: 1009
Karma: 3
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62036
I removed a couple of the largest unnecessary indexes from my system hoping to get the total low enough for parallel loading into memory. I found it will work once. But after the first solve more ram in consumed and it states there is not enough on the next solve.
Disabling the inParallel option.
2020-10-23T23:33:37 Not enough RAM is available on this system for loading the index files you have in parallel
2020-10-23T23:33:37 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 1.53425 GB, Installed RAM: 3.56387 GB, Free RAM: 1.51933 GB

The thing is, I have 4Gb Pi4. My Conky widget (AstroPi3 script setup, Thanks Rob) is showing only about 650Mb is being used. So I should have enough RAM. Furthermore running a memory check command returns this:
free
            total        used       free          shared    buff/cache  available
Mem:        3736984      610880     1571604       70892     1554500     2912904
Swap:       3736976           0     3736976

It appears the value under free matches closely what Stellarsolver finds. However the memory check states nearly 3Gb is available. Which is closer to what Conky reports.

I hope that is the case and the available RAM can be correctly detected and used.
Thank you.

INDI/KStars on Raspberry Pi 4, 4gb
Raspbian Buster with AstroPi3 script configuration
Skywatcher HEQ5 Pro Mount
Canon 600D Camera
Orion SSAG/ASI120mm @280mm Guide Scope
PHD2
Waveshare Stepper Motor Board - DIY Focuser
Adafruit GPS Module
Generic Bluetooth Joystick.
Startech 7 port powered USB Hub.

Please Log in or Create an account to join the conversation.

1 month 1 week ago
rlancaste
Supernova Explorer
Supernova Explorer
Posts: 2481
Karma: 24
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62037
So I'm trying to err on the side of caution with loading the indexes in parallel. In the documentation, Astrometry.net just warns you to make sure you have enough RAM to load all the indexes at the same time. Earlier I had StellarSolver detect the total installed RAM and just calculate whether it can load all your indexes based upon that. However, since a user might have many programs open at the same time, that really isn't good enough especially on a PI. If you run out of memory you could get crashes and we were getting reported crashes earlier on PI's, which I think were because of this option. So I changed it to be more conservative about using this option. We will certainly keep doing testing and if it becomes apparent that we can give a little more leeway on whether to load all the indexes and it doesn't cause crashes, we can certainly do that. It is also entirely possible that it was not this option that was causing crashes and it was something else that we since fixed, but I would still lean on the side of caution and aim for stability rather than speed.

Please Log in or Create an account to join the conversation.

1 month 1 week ago
rlancaste
Supernova Explorer
Supernova Explorer
Posts: 2481
Karma: 24
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62038
Hi Greg,

Thank you for doing some testing!

I am confused by your first issue: "BTW - when testing with the simulators I left the Solver Action as "Nothing" but the slew still occurred". I think this is a miscommunication in the interface. If you clicked the "Load and Slew" button then it is always supposed to slew to the target. The selections for the Solver action are only applicable if you click "Capture and Solve"

" after the unwanted slew, the Load and Slew button was now greyed out." So was this after it arrived at its target after the slew you mentioned? The buttons should be disabled while it is slewing, but after it gets there they should re-enable. If not, that could be a KStars bug.

" image has some problems including mystery stars". Do you mean that it detected noise as stars? This is certainly possible depending on your selected options profile. We are trying to develop better options, but they aren't quite perfected yet. Did you only try the one profile or did you try any others?

"However, this one of the same field does not solve. The main differences I can determine is the median signal of this frame is higher than the other two and the image is rotated 180deg from the others." I downloaded this file and it solved quickly. Which profile are you using to solve?


" Interesting that it says it needs 19.7GB of RAM. Is that true?" So the way astrometry's "load all indexes in parallel" option is supposed to work is that it loads all your index files into memory simultaneously, at least according to their documentation. This of course will make it much much faster than loading each index file one after the next. But if you have bigger index files than available RAM, this could be serious problem. I am not sure that it always will use that much RAM and if you have SWAP files or ZRAM, that could certainly prevent any crashes that could occur from taking up all the RAM, but I don't want to risk crashes, so in the code right now I calculate how much RAM there is and how big the indexes are and report that and disable the option if it is too much. We can play around with that to see if it really does take that much, but for now, I'm being cautious.

Thanks again for testing and let me know if you have any further questions,

Rob

Please Log in or Create an account to join the conversation.

1 month 1 week ago
xsnrg
Administrator
Administrator
Posts: 428
Karma: 4
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62058

rlancaste wrote: Hi Jim, we will check it out. Thanks!


Just a friendly bump, the latest builds are still failing. download.copr.fedorainfracloud.org/resul.../builder-live.log.gz
make[2]: Leaving directory '/builddir/build/BUILD/stellarsolver-1.4.git'
In file included from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/extract.h:26,
                 from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.h:23,
                 from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:25:
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp: In member function 'int SEP::Lutz::lutzalloc(int, int)':
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/sepcore.h:69:23: error: 'malloc' was not declared in this scope
   69 |   {if (!(ptr = (typ *)malloc((size_t)(nel)*sizeof(typ))))  \
      |                       ^~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:63:5: note: in expansion of macro 'QMALLOC'
   63 |     QMALLOC(info, infostruct, stacksize, status);
      |     ^~~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:32:1: note: 'malloc' is defined in header '<cstdlib>'; did you forget to '#include <cstdlib>'?
   31 | #include <cstring>
  +++ |+#include <cstdlib>
   32 |

12" pier with HDX110 using EQMod
ASI 1600 mono/color mains with ASI290MM in off-axis
ASI filter wheel
Moonlite focusers for the sharpening
AT115EDT w/.8x for the light
Fedora Linux, 100% INDI

Please Log in or Create an account to join the conversation.

1 month 1 week ago
seescho
Senior Boarder
Senior Boarder
Posts: 42
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62063

xsnrg wrote:

rlancaste wrote: Hi Jim, we will check it out. Thanks!


Just a friendly bump, the latest builds are still failing. download.copr.fedorainfracloud.org/resul.../builder-live.log.gz

<code>make[2]: Leaving directory '/builddir/build/BUILD/stellarsolver-1.4.git'
In file included from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/extract.h:26,
from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.h:23,
from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:25:
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp: In member function 'int SEP::Lutz::lutzalloc(int, int)':
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/sepcore.h:69:23: error: 'malloc' was not declared in this scope
69 | {if (!(ptr = (typ *)malloc((size_t)(nel)*sizeof(typ)))) \
| ^~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:63:5: note: in expansion of macro 'QMALLOC'
63 | QMALLOC(info, infostruct, stacksize, status);
| ^~~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:32:1: note: 'malloc' is defined in header '<cstdlib>'; did you forget to '#include <cstdlib>'?
31 | #include <cstring>
+++ |+#include <cstdlib>
32 | </code>


Looks like there are two problems which prevent compiling stellarsolver atm.

Adding this into stellarsolver/sep/lutz.cpp
#include <cstdlib>

and this into stellarsolver/sep/extract.cpp
#include <cstdio>

fix the problem for me.

Please Log in or Create an account to join the conversation.

1 month 1 week ago
jerry
Gold Boarder
Gold Boarder
Posts: 227
Karma: 1
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62066
Not sure if this is related, but I think so. The nightly has been crashing for me: after a load & slew align (which works), during focusing. I am attaching the ekosdebugger results. I'm not sure if I was monitoring the proper modules. It's unclear to me what indi process I should have monitored.

File Attachment:

File Name: indi_logs_...6-01.zip
File Size:4 KB

File Attachment:

File Name: kstars_log...3-53.zip
File Size:4 KB

120 MM Skywatcher Esprit on Celestron CGX
ZWO ASI 224MC or QHY-5L-II-M for guiding
Nikon Z7 8256 x 5504, 35.9 x 23.9mm 4.34 um pixel size
Triad Ultra Quad Band Narrow Filter
Kstars installations:
1) Odroid-N2 Ubuntu-mate 20.04
2) StellarMate on RPI4 8G
3) KStars on Astroberry RPI3.
Attachments:

Please Log in or Create an account to join the conversation.

1 month 1 week ago
schwim
Expert Boarder
Expert Boarder
Posts: 150
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62068

rlancaste wrote: Thank you for doing some testing!

You are very welcome. It's the least I can do to help out. If only I knew C++... Another year perhaps.

I am confused by your first issue: "BTW - when testing with the simulators I left the Solver Action as "Nothing" but the slew still occurred". I think this is a miscommunication in the interface. If you clicked the "Load and Slew" button then it is always supposed to slew to the target. The selections for the Solver action are only applicable if you click "Capture and Solve"

Agreed that it is probably an interface communication issue. It would probably be better to have the button as "Capture & Solve", "Load & Solve" with the Solver Action section giving options "Slew To Target", "Solve Only", and maybe a checkbox for "Sync" in cases where people want the slew and sync to happen. Probably outside of the scope of this topic though - I was sharing as an FYI.

" after the unwanted slew, the Load and Slew button was now greyed out." So was this after it arrived at its target after the slew you mentioned? The buttons should be disabled while it is slewing, but after it gets there they should re-enable. If not, that could be a KStars bug.

Correct - The slew occurred and failed (because I'm using sims and don't have images set up I guess). The Load and Slew button stays greyed out - only the Capture button can be activated.

" image has some problems including mystery stars". Do you mean that it detected noise as stars? This is certainly possible depending on your selected options profile. We are trying to develop better options, but they aren't quite perfected yet. Did you only try the one profile or did you try any others?

Nope - that first image has some "mystery stars" in them that are not present in other images of the same object. My guess is this is a residual bulk image problem that came with switching the camera's mode? Not sure though. I guess the point is that the solver performed well in spite of these mystery stars. I was surprised it didn't confuse the solver.

"However, this one of the same field does not solve. The main differences I can determine is the median signal of this frame is higher than the other two and the image is rotated 180deg from the others." I downloaded this file and it solved quickly. Which profile are you using to solve?

I'm using whatever default was selected when I built the software with SS included the first time. Looks to be 4-ParallelSmallScale. I find it interesting that some of the images of the same object with similar characteristics solve, but some do not. Maybe these images are right at the edge of some threshold.

" Interesting that it says it needs 19.7GB of RAM. Is that true?" So the way astrometry's "load all indexes in parallel" option is supposed to work is that it loads all your index files into memory simultaneously, at least according to their documentation. This of course will make it much much faster than loading each index file one after the next. But if you have bigger index files than available RAM, this could be serious problem. I am not sure that it always will use that much RAM and if you have SWAP files or ZRAM, that could certainly prevent any crashes that could occur from taking up all the RAM, but I don't want to risk crashes, so in the code right now I calculate how much RAM there is and how big the indexes are and report that and disable the option if it is too much. We can play around with that to see if it really does take that much, but for now, I'm being cautious.

Gotcha. Happens to be I have 32G in this PC and I had forgotten about that detail. It's working, and it's fast enough that by the time I switch back to the Ekos window after loading an image that it's already solved. I'll have to try this out on lower memory systems soon.

Thanks again for your work on this!

Greg

Please Log in or Create an account to join the conversation.

1 month 1 week ago
Cerro Torre
Senior Boarder
Senior Boarder
Posts: 78
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #62069

rlancaste wrote: in response to my statement:
"Kstars still crashes if I try to solve M65 with downsampling."

--That is what we are trying to um. . . aim away from. :-)


You found the solution!
I am able to solve M65 in 1.36 seconds with Auto DownSample checked -- you have a RC! But take your time, the weather here in Central Europe is really bad. My last imaging session was on September 22nd and the next two weeks will be ugly too.

Powered by

GNU / Linux
KDE neon
KStars | EKOS | INDI

and some cheap hardware

Please Log in or Create an account to join the conversation.

Time to create page: 0.454 seconds