×

INDI Library v1.9.7 Released (29 Jul 2022)

Bi-monthly INDI Library released with new drivers and bug fixes.

KStars memory leak and crash

  • Posts: 19
  • Thank you received: 1
Nope, still seeing memory usage increase with notifications on or off.
Sorry folks.
I'm not sure how to run heaptrack. It's installed, but not a great linux user to do have to either ask for a how-to crash course, sorry folks.
The following user(s) said Thank You: Rafa Barberá
3 months 1 week ago #83582
The topic has been locked.
  • Posts: 183
  • Thank you received: 41

Replied by nou on topic KStars memory leak and crash

What kind of usage memory increase? Because increase of cache/buffers is no indication of memory leak.
3 months 1 week ago #83584
The topic has been locked.
  • Posts: 18
  • Thank you received: 2

Replied by MH on topic KStars memory leak and crash


This was my experience as well - I delved waaaay into the rabbit hole when I was having this problem and found the "turn notifications and sounds off" potential solution (there are a number of threads on this, with this proposed solution), and it didn't seem to matter.

I recall actively being excited like "oh nice an easy solution" and of course it couldn't be that simple haha
The following user(s) said Thank You: Rafa Barberá
Last edit: 3 months 1 week ago by MH. Reason: Additional thought, quotes still sad
3 months 1 week ago #83585
The topic has been locked.
  • Posts: 19
  • Thank you received: 1
@Nou, sorry if my terminology is incorrect. Just assumed memory leak is when memory is used up continuously, until the system has used all resources and causes things to crash. Let's not get bogged down with terminology!
As in my original post , I've had several sessions crash shortly after going to bed, and believe them to be from a lack of memory on the RPi.
I say this as I've documented the memory usage fill up, then the kstars logfile stops writing (kstars crashes) and the available memory increases and remains steady until I return to offload the imaging data several hours later.
Since really delving deeper (all the different memories, and purging the caches, etc), I've been unable to make it crash again, and as the nights are so short imaging isn't worth it just at present.
Again, I have a 'fix' that should work for me, but I'm disappointed that I've not been able to find the reason and solve the problem, but it does seem like there are others still seeing the same, perhaps without crashes...
Last edit: 3 months 1 week ago by Pete Simpson.
3 months 1 week ago #83586
The topic has been locked.
  • Posts: 18
  • Thank you received: 2

Replied by MH on topic KStars memory leak and crash

Something you may be able to do to recreate (and what I used previously) was to just take a lot of dark frames over a long period of time, especially if you need to build a darks library for specific temperatures and haven't had time yet.

Otherwise, can just take a lot of covered photos even if they aren't needed for calibration frames; the bug tended to reproduce itself when doing something like that for me.

There are also some ways you can log RAM usage over time.

Could use something like one of the answers to log RAM, CPU usage of Kstars over time to a local file and cross-reference, such as a one-liner I quickly cobbled together based on that answer:
while true; do ps -p <kstars_PID> -o %cpu,%mem,cmd; sleep 5; done | while read memStats; do echo "$(date): $memStats"; done | tee -a memtest.logs

This gives you timestamp outputs per line, which you can likely cross-reference with the Kstars logs; I strongly suspect that the memory increase will creep up after each iteration of the memory leak, or around the same time.

Edit: Actually, it looks like pmap may be better suited for this task based on this thread ; so to modify the above, it would look like:
while true; do pmap -x <kstars_PID>; sleep 5; done | while read memStats; do echo "$(date): $memStats"; done | tee -a memtest.logs
Last edit: 3 months 1 week ago by MH. Reason: Likely better diagnostic one-liner
3 months 1 week ago #83587
The topic has been locked.
  • Posts: 183
  • Thank you received: 41

Replied by nou on topic KStars memory leak and crash

Ok just tested and running heaptrack is quite easy. Just install "sudo apt install heaptrack heaptrack-gui" and then run "heaptrack kstars" after you close down kstars it will even tell you next command to analyze results. Try run one short session with few frames exposed and then longer one. Then post screenshot of summary tab so we can investigate further.
3 months 1 week ago #83596
The topic has been locked.
  • Posts: 205
  • Thank you received: 44

Replied by Gene N on topic KStars memory leak and crash

So I tried a varying set of sequences, total after all was 3000 images captured using SIM CCD and SIM Telescope.

Started at this
13863 root 20 0 643244 173444 76296 R 69.3 36.8 1:40.63 kstars

Yes, went -down- a lot after 300
13863 root 20 0 643724 101992 30048 R 5.3 21.7 6:04.19 kstars

( I verifed this, after all the below re-ran with pulse and web enabled)
After 300 coldstart
30821 root 20 0 642988 104668 33792 S 13.8 22.2 8:28.06 kstars
)
Intermixed in the sequences I re-enabled pulse-audio but no Notifications
Also, prior to the last 600 images I re-enabled the web services and logged into port 8080
Also had FITS viewer enabled and disabled intermittently

After 3000 images
13863 root 20 0 645872 98996 37704 R 68.3 21.0 12:50.20 kstars
The image count:
astroberry@astroberry:~ $ ls -ltr TEST/Light/*/* | wc
3000

Then after sitting idel for hours

13863 root 20 0 644536 84824 29560 S 17.4 18.0 89:44.97 kstars
The following user(s) said Thank You: Jasem Mutlaq, MH
3 months 1 week ago #83599
The topic has been locked.
  • Posts: 76
  • Thank you received: 5
Hi Rafa

you go into kstars, settings and select configure notifications. That lists all the different events kstars will notify about. Go through unselecting the ‘play a sound’ tick box for each entry - ones with it set have a black arrow on the left side. Set another type of notification if needed. Once done click ok and enjoy the silence (might not notice the change with an astroberry :) )

Regards

Nigel
3 months 1 week ago #83603
The topic has been locked.
  • Posts: 838
  • Thank you received: 80
Hi everyone
I don't think this is astroberry specific. Tested on two separate kubuntu boxes with the same result: kstars crashes leaving indi up and running.

We think the leak is in Stellar Solver; removing alignment calls to StellarSolver by using ASTAP and preventing calls to StellarSolver's Sextract from the guider by using PHD2, there is no crash (tested now for 4 sessions of 5 hours).

Conclusion and workaround: use ASTAP and PHD2, but we'd really like to be able to use the internal guider again. It would be great is this could be fixed.
Thanks for reading and any testing you need, please just shout.

Clear skies,
Steve

Reported on the forum here (also reproduces the syslog usb error each time the zwo guide camera takes a frame)
www.indilib.org/forum/ekos/11784-kstars-...id-socket-error.html

Bug report here:
bugs.kde.org/show_bug.cgi?id=454866
kubuntu 22.04
700d, eq6, 120mm. Yes, the old one.
The following user(s) said Thank You: Jasem Mutlaq
2 months 3 weeks ago #84045
The topic has been locked.
  • Posts: 53
  • Thank you received: 6

Replied by Outta on topic KStars memory leak and crash

Hi,

I am having issues with crashing as well, it is not sometimes, it is literally every run when I have Journal enabled. So if Journal is not scheduled it works perfectly, when I enable Journal, i get a random crash at some point in time, regardless is Journal running at the moment or just sleeping until conditions met. I even switched to Online astrometry, But it does not help, it just crashes randomly with nothing special in logs... I will try with disabling "Guide" in Journal maybe that will help,

Astroberry on Pi4B 1GB, RPICam

Also another thing I noticed since latest update, my KStars does not get updated from mount in regards to location and time. Mount is OnStep. Mind you might be issue with mount as I have to configure it on every startup, but Kstars never picks up any more. But that is not big issue, crashing is making Journal unusable.
2 months 3 weeks ago #84046
The topic has been locked.
  • Posts: 838
  • Thank you received: 80
Hi
What is 'Journal'?
kubuntu 22.04
700d, eq6, 120mm. Yes, the old one.
2 months 3 weeks ago #84047
The topic has been locked.
  • Posts: 53
  • Thank you received: 6

Replied by Outta on topic KStars memory leak and crash

Sorry it is "Scheduler" I have no idea why I named it Journal :)


There are my logs right before a crash:
"
[2022-07-04T23:55:19.701 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] addFITSKeywords() "
[2022-07-04T23:55:19.702 CEST INFO ][           org.kde.kstars.indi] - RPI Camera :  "[WARNING] Telescope focal length is missing. "
[2022-07-04T23:55:19.703 CEST INFO ][           org.kde.kstars.indi] - RPI Camera :  "[WARNING] Telescope aperture is missing. "
[2022-07-04T23:55:19.729 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] Uploading file. Ext: fits, Size: 6168960, sendImage? Yes, saveImage? No "
[2022-07-04T23:55:19.835 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] BLOB transfer took 0.0828278 seconds "
[2022-07-04T23:55:19.836 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] Upload complete "
[2022-07-04T23:55:19.925 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] File added event completed. "
[2022-07-04T23:55:19.926 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Downloading image... Name: (capt0022.cr2) Folder: (/) Delete from SD card? (false) "
[2022-07-04T23:55:20.000 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG]  Downloaded 0x0 (preview 0x0) "
[2022-07-04T23:55:20.048 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Capture target is INTERNAL RAM. "
[2022-07-04T23:55:20.058 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Capture event completed. "
[2022-07-04T23:55:20.059 CEST INFO ][           org.kde.kstars.indi] - Canon DSLR EOS 550D :  "[INFO] Exposure done, downloading image... "
[2022-07-04T23:55:20.491 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] ISNewNumber(RPI Camera, CCD_EXPOSURE, "
[2022-07-04T23:55:20.492 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: startCapture(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/cameracontrol.cpp:54): entered"
[2022-07-04T23:55:20.492 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:20.493 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG]       value:2.000000, name: CCD_EXPOSURE_VALUE, "
[2022-07-04T23:55:20.493 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] ) "
[2022-07-04T23:55:20.493 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] StartExposure(2.000000) "
[2022-07-04T23:55:20.501 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: setExposureParameters(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:102): Camera crop set to 0,0,2028,1520"
[2022-07-04T23:55:20.501 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: "
[2022-07-04T23:55:20.501 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:20.502 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: setExposureParameters(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:127): setting fps range 167/1000 -> 999/1000"
[2022-07-04T23:55:20.502 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: setExposureParameters(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:134): failed to set fps ranges: low range is 10944/65536, high range is 65470/65536"
[2022-07-04T23:55:20.502 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: setExposureParameters(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:144): shutter speed set to 1999994"
[2022-07-04T23:55:20.502 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:20.503 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: setExposureParameters(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:151): gain set to 1048576/65536"
[2022-07-04T23:55:20.503 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: startCapture(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/cameracontrol.cpp:63): shutter speed after enabling camera: 1999994"
[2022-07-04T23:55:20.503 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:20.506 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:20: Driver indi_rpicam: startCapture(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/mmalcamera.cpp:69): starting capture with speed 1999994"
[2022-07-04T23:55:20.507 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:20.507 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "Download complete. "
[2022-07-04T23:55:22.640 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:22: Driver indi_rpicam: signal_data_received(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/cameracontrol.cpp:127): first buffer received after 2.127430 s"
[2022-07-04T23:55:22.644 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:22.798 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:22: Driver indi_rpicam: data_received(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/jpegpipeline.cpp:131): finished jpeg processing"
[2022-07-04T23:55:22.799 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  "2022-07-04T21:55:22: Driver indi_rpicam: data_received(/home/astroberry/Projects/indi-3rdparty/indi-rpicam/broadcompipeline.cpp:62): finished broadcom processing"
[2022-07-04T23:55:22.799 CEST DEBG ][           org.kde.kstars.indi] - INDI Server:  ""
[2022-07-04T23:55:22.979 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] read_libraw: raw_width: 5344 top_margin 51 left_margin 142 first_visible_pixel 272686 "
[2022-07-04T23:55:22.980 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] read_libraw: rawdata.sizes.width: 5202 rawdata.sizes.height 3465 memsize 36049860 bayer_pattern RGGB "
[2022-07-04T23:55:23.020 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] read_libraw: memsize (36049860) naxis (2) w (5202) h (3465) bpp (16) bayer pattern (RGGB) "
[2022-07-04T23:55:23.044 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Subframing... subFrameSize: 3200000 - oneFrameSize: 3200000 - subX: 1700 - subY: 1200 - subW: 1600 - subH: 1000 "
[2022-07-04T23:55:23.068 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Uploading file. Ext: fits, Size: 3208320, sendImage? Yes, saveImage? No "
[2022-07-04T23:55:23.150 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] BLOB transfer took 0.0614142 seconds "
[2022-07-04T23:55:23.151 CEST DEBG ][           org.kde.kstars.indi] - Canon DSLR EOS 550D : "[DEBUG] Upload complete "
[2022-07-04T23:55:23.183 CEST DEBG ][           org.kde.kstars.indi] - Image received. Mode: "Normal" Size: 3208320
[2022-07-04T23:55:23.189 CEST INFO ][           org.kde.kstars.indi] - "FITS" file saved to "/home/astroberry/Public/Light/ngc6818/Light/lg_Light_10_secs_2022-07-04T23-55-23_204.fits"
[2022-07-04T23:55:23.363 CEST INFO ][   org.kde.kstars.ekos.capture] - "Download Time: 0.21 s, New Download Time Estimate: 3.49 s."
[2022-07-04T23:55:23.366 CEST INFO ][   org.kde.kstars.ekos.capture] - "Received image 5 out of 300."
[2022-07-04T23:55:23.368 CEST INFO ][   org.kde.kstars.ekos.capture] - "Captured /home/astroberry/Public/Light/ngc6818/Light/lg_Light_10_secs_2022-07-04T23-55-23_204.fits"
[2022-07-04T23:55:23.586 CEST DEBG ][                       default] - WARNING: Phonon::createPath: Cannot connect  Phonon::MediaObject ( no objectName ) to  Phonon::AudioOutput ( no objectName ).
[2022-07-04T23:55:23.594 CEST DEBG ][           org.kde.kstars.indi] - Requested exposure 10 closes match is 10.3
[2022-07-04T23:55:23.595 CEST INFO ][   org.kde.kstars.ekos.capture] - "Capturing 10.000-second  image..."
[2022-07-04T23:55:23.604 CEST DEBG ][           org.kde.kstars.indi] - RPI Camera : "[DEBUG] capture_complete "
[2022-07-04T23:55:23.605 CEST DEBG ][ org.kde.kstars.ekos.scheduler] - Capture State "Image Received"
2 months 3 weeks ago #84060
The topic has been locked.
Time to create page: 2.639 seconds