Hi Paul,

when you write about tests, you mean the original, unmodified ASI library? right?
If I finish some minor fixes on high-frame speed recording, I'll move on to fixing long exposure.
Apart from the hiccups when starting high-speed recording, it seems to me that the situation in my library is under control.

I am also planning to buy a bigger camera from ASI and I think I will get the situation under control as well.
What kind of camera would you buy? Maybe I'll buy the same one.

I put the 8/16-bit version on branch master, which works fine for me. I added the ability to set the size of buffers using the function and using the configuration file. Everything well described in the README and header file.
My next step is to fix timeouts and run capturing smoothly.

Maybe tonight, Polish time, I will manage to find some time and get it over with ;)
After completing the library and Indi, I'll take a look at other programs. FireCapture added to the list.

Read More...

Hello Paul!

The library will be prepared so as to it could be easily replaced. As described in README.md.
Replacement is not an efficient solution because the interface of the original library is prepared to copy data.
However, if you can modify the code I have provided function to get frames without copying.

In short, you will be able to swap the library - it will work better than the original library.
Indi Library (I'll add some fixes) will use the additional features implemented in libAsiCamera2Boost - it will work even better.

I am thinking about adding the possibility to configure the library from the configuration file. This makes it easy to adjust the parameters in the case of a simple replacement.
I bought my first telescope and I'm more in code than under the sky ;)
When I find out what programs you are using, I will try to suggest improvements to this programs.

Read More...

I meant the confusion about the original ZWO library.
I didn't mean why you need 16-bit mode ;)
Above I wrote my thoughts on USB and the library.
They divide each transfer into 1MB. For the ASI178 camera, 7 bulk transfers are created for the 8-bit mode and 14 for the 16-bit mode.
This is what I suggested when writing the wrapper.
Now I have modified it so that the transfer is not split into parts in the application layer.

My goal is to fix the ZWO library so that it is usable on Linux. Because that's where the main problem lies.

Looking at your attachment I can see that everything is fine! right?
When receiving the frames, none got lost and you achieved fps above the manufacturer's declaration.

PDB wrote: Not really want to do this on an RPI...

I do not understand.
You don't want to run the camera on Linux on RPi?
Are you running it on W10 on RPi? How do you use the camera?
How long are your videos? How can I help you?

PDB wrote: Looks logic that you need to keep buffers smaller to have it run on the smaller Rpi's.

I don't understand what you're referring to.
The last modification was based on a smaller number of larger buffers. And in fact, on their distribution of addresses for transfers. I operate on megabytes here, I am far from hardware limitations.

In my project I want to use RPi for wireless transmission of high-speed cameras. After optimizing the code, I think it is real. That's why I started working.
Writing directly to the disk should not be a problem either. Additionally, by introducing a simple lossless compression, it is possible to reduce the size by half.

You can describe how you currently use the hardware/software and how you would like to use it?

After some tweaks, I think libASICamera2Boost will be ready. I still have to fix the Indi Library so that no frame gets lost.

Read More...

I'm confused.
The USB (libusb) works like this, you send a request for how much data you want to receive. If device has data to send, a reply is given.
The amount of data received may be equal to or less than in the request. In an ideal world, a request should be sent with the length of the entire frame. Ideally, several such requests, so that there is always one ready for new data.
Of course, on the lower layer, the request can be split into smaller ones, but at the application layer, you shouldn't be interested in that.
After all, in the library I split the packages into a maximum size of 1MB. I made a suggestion about this size when decompiling the original library.

In the "16-bit" branch I increased this limit to 256MB, and the number of frame queries to 4. Of course, the limit is a limit, if the frame size is, for example, 10MB, a 10MB transfer is initiated (4 times).

You can modify these values ​​in the lib/src/cameraboost.h file.

    enum // constants
    {
        Transfers = 4, // TODO limit to maximum possible transfers
        InitialBuffers = Transfers + 1,
        MaximumTransferChunkSize = 256 * 1024 * 1024 //
    };

At the moment, in the "16-bit" branch, I don't see any problems with the 8/16-bit mode.

Read More...

PDB wrote: Thank you pawel. I quickly compiled it on my desktop intel, and tested without problems. (in highspeed mode just a bit faster than firecaptur) Will do the test on an RPI4 later today.

Just one question: is there a possibility to set the camera to 16 bit? That is on my desktop machine I see the biggest problems.

Paul


The library supports 16-bit resolution. I added the "Format" parameter to the programs.
On RPi in 16-bit mode, frames are lost a bit.
I need to do some size adjustments for buffers and transfers to get reliable communication.
I prepared a "16-bit" branch.
Branch "16-bit" is not good for 8-bit mode.

As for me, it looks better:
./example/simply/AsiCameraSimply Exposure 10000 HighSpeedMode 1 Gain 100 Format 16bit
[ASIOpenCamera]: grab CameraID 0
[CameraBoost]: created
[__wrap_libusb_open]: grab libusb_device_handle 0x1f9d338
[__wrap__ZN11CCameraBase12InitVariableEv]: grab CCameraBase 0x1f9f670
Set 'Exposure' to 10000
Set 'HighSpeedMode' to 1
Set 'Gain' to 100
Set 'Format' to 16bit
...
timeframe:  1000ms, status: 11
timeframe:   148ms, status: 0
timeframe:    51ms, status: 0
...
timeframe:    33ms, status: 0
timeframe:    34ms, status: 0
timeframe:    33ms, status: 0
timeframe:    33ms, status: 0
timeframe:    33ms, status: 0
timeframe:    34ms, status: 0
...
[releaseAsyncXfer]: catched

For the ASI178MM camera, the maximum frame rate is reached.
|---------------------------------------------------------------------------------------------------------------------------------|
| BW [%] |                                            Frame duration [ms]                                                         |
|---------------------------------------------------------------------------------------------------------------------------------|
|  100%  |   16    27    33    33    33    34    33    33    33    34    33    33    33    33    33    33    33    34    33    33 |
|---------------------------------------------------------------------------------------------------------------------------------|

Best bandwidth is 100%, it has 30.4 FPS

Is it the same for you?

Read More...

Hi!

I uploaded the first version on github.
The library still requires a lot of cosmetic work, but it can already be pre-tested.

github.com/pawel-soja/AsiCamera

Currently I only have one camera and it's hard for me to test everything.
I am asking you for tests on your cameras. Share the information you have cameras by attaching information from the AsiCameraPerformance program.

Below is a detailed description of how I tested the camera.

  • download Raspberry Pi OS Lite (Release date: December 2nd 2020, Kernel version: 5.4)
  • enable ssh by placing an empty "ssh" file in the /boot/ directory
  • run Raspberry PI 4
  • connect ethernet cable and login via ssh
  • install pre-requisites from "Install Pre-requisites" section
  • build project from "Get the code and build" section
  • copy udev rule:
    sudo cp ~/Projects/AsiCamera/libasicamera2/lib/asi.rules /etc/udev/rules.d/
  • connect camera to USB 3.0
  • run AsiCameraPerformance
    ./example/performance/AsiCameraPerformance Exposure 10000 HighSpeedMode 1 Gain 100

Of course you can test it on different hardware or operating system and skip a few steps if you know what you are doing.

If something went wrong, run:
./example/simply/AsiCameraSimply Exposure 10000 HighSpeedMode 1 Gain 100
And then post the program result as a text attachment on the forum.

If someone has problems getting through the testing procedure, I can prepare a simple script to do everything for you.

I expect problems for cameras with resolutions above 10MPixel and color cameras. Please confirm.

Read More...

Currently, the library wrapper is intended to verify proof of concept on the ARMV8 architecture and ASI178M camera.
Now I'm trying to unify the library for each camera and architecture. Additionally, I have to get rid of one more redundant copy in the GetImage method.

Without library sources, I have a lot of work to do decompiling and creating new code to be easily injected.

Main issues of the library from ZWO:

  • synchronous USB communication.
  • incorrectly handled communication errors - one error causes an avalanche of errors.
  • too much data copying (3x per frame).
  • I suspect the inability to run several cameras on the same instance of the library (common static variable)

On Raspberry PI, the biggest problem is data copying.
The copying speed is ~ 1400 - 2000 MB / s.
The original library does 3 copies per frame + a copy in libusb from USB.
$ make && ./asitest Exposure 10000 HighSpeedMode 1 BandWidth 100 Gain 100
g++ -o asitest main.cpp -I./libasi/include -I./ -lpthread -lusb-1.0 ./libasi/lib/armv8/libASICamera2.a -Wl,--wrap=memcpy -O2
Set 'Exposure' is 10000
Set 'HighSpeedMode' is 1
Set 'BandWidth' is 100
Set 'Gain' is 100
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
timeframe:   280ms                                 <--- issue
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
timeframe:   546ms                                 <--- issue
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
[!!!] Big copy detected: 6439680 Bytes
timeframe:   546ms                                 <--- issue

For a 6MB frame, at 60fps this is 6MB * 4 * 60 = 1440MB/s

Speed ​​test for Raspberry PI 4 (64bit userland)
$ ./ramspeed 6
std::vector, size: 6 MB, copy: 0.0043 s, speed: 1396.592 MBps
std::vector, size: 6 MB, copy: 0.0042 s, speed: 1416.900 MBps
memcpy, size: 6 MB, copy: 0.0039 s, speed: 1544.185 MBps
memcpy, size: 6 MB, copy: 0.0039 s, speed: 1546.861 MBps

$ ./ramspeed 6
std :: vector, size: 6 MB, copy: 0.0025 s, speed: 2403.668 MBps
std :: vector, size: 6 MB, copy: 0.0025 s, speed: 2409.675 MBps
memcpy, size: 6 MB, copy: 0.0025 s, speed: 2374.098 MBps
memcpy, size: 6 MB, copy: 0.0025 s, speed: 2378.751 MBps

A large number of copies causes the loss of data from the USB.
Each frame in my case consists of 7 transfers (bulk). If one of them is "damaged" the library cannot handle it in an elegant way.
Apart from the fact that there is no copy space in other programs or on the system.

I think in 2 weeks, I will be able to publish my work.
I still had to eliminate one copy per frame and a lot of tests.
./asitest-boost Exposure 10000 HighSpeedMode 1 BandWidth 100 Gain 100
[ASIOpenCamera]: grab CameraID 0
[libusb_open]: grab libusb_device_handle 0x559d731cd0
[CCameraBase::InitVariable]: grab CCameraBase 0x559d73c9b0
[CirBuf::CirBuf]: link CirBuf 0x559d73afe0 and CCameraBase 0x559d73c9b0
[CirBuf::CirBuf]: link CCameraBase 0x559d73c9b0 and libusb_device_handle 0x559d731cd0
[CirBuf::CirBuf]: link iCameraID 0 and CCameraBase 0x559d73c9b0
Set 'Exposure' to 10000
Set 'HighSpeedMode' to 1
Set 'BandWidth' to 100
Set 'Gain' to 100
[CameraBoost::CameraBoost]: created
[CCameraFX3::initAsyncXfer]: init CameraBoost device 0x559d731cd0, endpoint 0x81, buffer size 6439680
[CirBuf::ReadBuff]: 1 1 0x7f7e85f010
[!!!] Big copy detected: 6439680 Bytes
timeframe:   261ms                       <---- warming up
[CirBuf::ReadBuff]: 2 2 0x7f7c15a010
[!!!] Big copy detected: 6439680 Bytes
timeframe:    16ms                        <---- yeah!
[CirBuf::ReadBuff]: 3 3 0x7f778ff010
[!!!] Big copy detected: 6439680 Bytes
timeframe:    17ms                        <---- yeah!
[CirBuf::ReadBuff]: 4 4 0x7f771fe010
[!!!] Big copy detected: 6439680 Bytes
timeframe:    17ms                        <---- yeah!
[CirBuf::ReadBuff]: 5 5 0x7f7c15a010
[!!!] Big copy detected: 6439680 Bytes
timeframe:    16ms                        <---- yeah!
[CirBuf::ReadBuff]: 6 6 0x7f778ff010
[!!!] Big copy detected: 6439680 Bytes
timeframe:    16ms                        <---- yeah! 60 fps
....

By queuing buffers and passing pointers, there will also be some work with the Indi library to take advantage of the new possibilities.
I will also add the ability to restore the original behavior of the library, without the need to compile or replace files.

Read More...