×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Memory Leak

  • Posts: 407
  • Thank you received: 74

Replied by Clive Stachon on topic Memory Leak


Have I missed something - but last i looked there is no official way to "Boot from USB" only boot from a SD card and use root on a SSD.

Have you some new news ?????????

love to be able to boot with SSD card ON RPI4b :-)
RPI3 Ubuntu 16.04 / AMD desktop Kstars under Ubuntu 16.04 Mounts :azeq6 ,SWAZGoTo

RPI3 Fedora testing out on AMD desktop Fedpra 28 - running kstars 2.9.4 , Indilib 1.7.4 ?????
Last edit: 4 years 1 month ago by Clive Stachon.
4 years 1 month ago #48950

Please Log in or Create an account to join the conversation.

  • Posts: 1119
  • Thank you received: 182

Replied by Jose Corazon on topic Memory Leak

Understandable, but mounting computer, SSD, hub, autofocus controller and power supplies on the tripod/pier is still the preferable solution, I think. All you need after that is 3 cables, bundled into a single conduit, running up the rig: Cable to autofocus motor, USB3 to the ZWOASI (which has an integrated USB hub) and 12V power for the cooler. Dew heater controller I have separately mounted on the side of the mount (or with my smaller mount, I incorporated it into the single conduit. This is what mine looks like:

www.indilib.org/forum/general/6075-astro....html?start=12#46351
4 years 1 month ago #48951

Please Log in or Create an account to join the conversation.

  • Posts: 1119
  • Thank you received: 182

Replied by Jose Corazon on topic Memory Leak


Correct, you use the SD card only to start up the Pi4, and point to the SSD, but the entire system files are on the SSD, so any system file swapping and file downloads now goes through USB3 and the SSD, not through the SD card bottleneck. MUCH faster, plus you can now also use a swap file on the SSD. I need that on occasion when using FITS viewer at high magnification during the run to inspect the quality of my guiding. That can max out your memory and crash the machine otherwise.

Try it out, it requires a trivial change in the cmdline.txt file on the boot portion of the SD card to point rootfs now to the SSD. To do that, clone the partition containing the system files and kstars, etc to the SSD first, then boot up the Pi4 from the SD card, determine which drive letter is assigned to the SSD (probably either /dev/sda1 or /dev/sdb1), then point root to that device, save the cmdline.txt file and restart the Pi4. It should now switch to the SSD.
The following user(s) said Thank You: Clive Stachon
4 years 1 month ago #48952

Please Log in or Create an account to join the conversation.

  • Posts: 1208
  • Thank you received: 559

Replied by Hy Murveit on topic Memory Leak

Last night we fixed a bug in the KStars code that writes fits files, which might explain some of the crashes described above. See phabricator.kde.org/D27115
If you git pull it, and compile, let us know if it corrects your issues.

As I understand , there's a different change in the nightly KStars that is dependent on using the nightly Indi, so if you update KStars to the nightly, you might also need a nightly-updated Indi.
The following user(s) said Thank You: Radek Kaczorek, Jose Corazon, Doug S
4 years 1 month ago #48968

Please Log in or Create an account to join the conversation.

  • Posts: 1119
  • Thank you received: 182

Replied by Jose Corazon on topic Memory Leak


Do I need to install the nightly indi-full version or is it sufficient to just install/update the drivers I need for the cameras? I did a more restricted update of Indi using only the drivers I need for my cameras (ASI, QHY and SSAG and required libraries).

BTW, I had no problems with the FITS files from the observation session 2 nights ago. Can I detect whether that specific problem is present in my installation in the files that have been written?
4 years 1 month ago #48971

Please Log in or Create an account to join the conversation.

  • Posts: 1208
  • Thank you received: 559

Replied by Hy Murveit on topic Memory Leak

I'm not sure about the indi thing, it's something I heard from Jasem (and that I experienced when debugging that fix on my mac) but I suspect it's in the indi-main, not the drivers.

WRT the bug fix, if you are compiling from source files, then you can tell by looking at the code (e.g. the changed lines in kstars/kstars/indi/indiccd.cpp should look the way they are on the right side of the link: phabricator.kde.org/D27115).
If you just have binaries, I don't think you can easily tell, but the change was submitted about 11pm PST last night, so anything older for sure doesn't have it.
That change fixes a bug that might cause a crash when saving fits files.
4 years 1 month ago #48979

Please Log in or Create an account to join the conversation.

  • Posts: 983
  • Thank you received: 375

Replied by Radek Kaczorek on topic Memory Leak

Yesterday during my session I was testing for the memory leak. I was running Astroberry 2.0.1 (Raspbian Buster) on Raspberry 3B+ (1GB RAM). I used Atik 460EX CCD, which renders 12 096 kB file size for BIN1 and 3 026 kB for BIN2.
I created a script which recorded amount of available memory to a file. The script was called from Ekos CCD module after each image was captured and saved.
The starting amount of available memory was always around 480MB. It dropped by the same amount every image. As the result after every 30 images KStars must have been restarted to release the memory. Otherwise the Kstars was going down on its own due to lack of memory.

30 x 10s BIN1
average memory drop: 12 025 kB
standard deviation: 1 785 kB

30 x 60s BIN1
average memory drop: 12 101 kB
standard deviation: 2 330 kB

30 x 300s BIN1
average memory drop: 12 415 kB
standard deviation: 2 161 kB

36 x 120s BIN2
average memory drop: 3 436 kB
standard deviation: 1 714 kB

Conclusion
The memory leak is confirmed. The amount of memory that leaks is directly related to the file size grabbed from CCD. There could be positive correlation between frame's number and/or exposure time and memory leaked i.e. more memory leaks for each of subsequent frame and/or if exposure is longer.

Question
Can anybody show directions how can we identify the actual memory leak root cause? Any specific tests and debugs or specific procedures and tools? Jasem aka @knro? anybody else?
The following user(s) said Thank You: Jasem Mutlaq
Last edit: 4 years 1 month ago by Radek Kaczorek.
4 years 1 month ago #49906

Please Log in or Create an account to join the conversation.

  • Posts: 85
  • Thank you received: 9

Replied by Steve Crossman on topic Memory Leak


This explains my crashes as I have been doing the same thing, inspecting the subs by zooming in and often it would stall for up to one minute and recover or Kstars simply crashes.
Last edit: 4 years 1 month ago by Steve Crossman.
4 years 1 month ago #49907

Please Log in or Create an account to join the conversation.

Replied by Jasem Mutlaq on topic Memory Leak

I couldn't locate the source of the leak. So you're sure it's KStars and not the INDI driver?
4 years 1 month ago #49908

Please Log in or Create an account to join the conversation.

  • Posts: 1119
  • Thank you received: 182

Replied by Jose Corazon on topic Memory Leak


This is not a consistent occurrence. A few nights ago, Kstars crashed after ~100 frames (32MB each) and I was greeted with the Desktop telling me that there was not enough memory to send a report.

Last night I had a totally uneventful unattended session, I collected ~15GB of images until the early morning using the scheduler without memory use having increased beyond 2 GB.

I have no idea what triggers these irregular memory leaks/overruns that result in Kstars crashing or how to provoke them reliably.

I am running Indi and Kstars build of 02-13-2020 on Ubuntu MATE 18.04.
4 years 1 month ago #49909

Please Log in or Create an account to join the conversation.

  • Posts: 983
  • Thank you received: 375

Replied by Radek Kaczorek on topic Memory Leak


This is very good point!
The leak is directly related to image files and their size. So there are two components to blame: CCD driver or EKOS.
I used indi_atik_ccd but the test should be run with other CCD drivers to narrow the search.

Please copy this simple script to /usr/local/bin/test_memleak
#!/bin/bash
cat /proc/meminfo | grep MemAvailable | cut -d: -f2 | xargs | cut -d" " -f1 >> /tmp/memstat

Then set the access rights:
sudo chmod 755 /usr/local/bin/test_memleak

Then add the script to EKOS CCD module when defining your session. Capture around 30 images and check the content of /tmp/memstat file , which contains memory available after each capture. Calculate average and compare it to a single FITS file (bin1 and bin2)
4 years 1 month ago #49910
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 983
  • Thank you received: 375

Replied by Radek Kaczorek on topic Memory Leak

I have just tested it with ZWO ASI 120MM (file sizes: 2465 kB for BIN1 and 622 kB for BIN2)

30 x 1s BIN1
average memory drop: 2796 kB
standard deviation: 1101 kB

30 x 1s BIN2
average memory drop: 858 kB
standard deviation: 1126 kB

Apparently the memory leak is still related to file size and does not depend on CCD driver used.
4 years 1 month ago #49913

Please Log in or Create an account to join the conversation.

Time to create page: 2.990 seconds