Sparsification gpu

[kif]

3 kernels will be needed:

• one to record a slab
• one to perform the cumsum
• one to copy the data at the proper position.
• Python class implementing the OpenCL version with the same API as the Numpy version.

Beside this, there are modification for the associated tests:

• test compaction on small synthetic dataset to make testing faster
• test in a similar way the OpenCL and the Numpy version.

[pierrepaleo]

Interesting, but is it really worth implementing compaction on GPU, as we have to transfer the whole raw (and mostly zeroes) images ?
If relevant, there is some 2D CSR implementation in silx using pyopencl GenericScanKernel, not sure it's applicable here though.

[kif]

Well, some of your tests droped from 40s to 1s runtime, without implementing the cumsum nor the copy of the comacted data in OpenCL. Sending 3000 frames to the GPU and launching this trivial kernel takes half a second and the cumsum+copy in numpy (unchanged) the other half.

[kif]

   ...: dataset = XPCSDataset("andor_1024_3k")
   ...: from dynamix.correlator.event import EventCorrelator, FramesCompressor, OpenclCompressor
   ...: numpy_compressor=FramesCompressor(
   ...:                 dataset.dataset_desc.frame_shape,
   ...:                 dataset.dataset_desc.nframes,
   ...:                 330,
   ...:                 dtype=dataset.dataset_desc.dtype)
   ...: opencl_compressor=OpenclCompressor(
   ...:                 dataset.dataset_desc.frame_shape,
   ...:                 dataset.dataset_desc.nframes,
   ...:                 330,
   ...:                 dtype=dataset.dataset_desc.dtype, profile=True)
   ...: 
/usr/lib/python3/dist-packages/skcuda/cublas.py:284: UserWarning: creating CUBLAS context to get version number
  warnings.warn('creating CUBLAS context to get version number')
/tmp/dynamix_testdata_kieffer/andor_1024_3k/wcf_Fe79_ramp1_240C_down_1_3000_raw.npz

In [2]: %time res_np=numpy_compressor.compress_all_stack(dataset.data)
CPU times: user 40.1 s, sys: 657 ms, total: 40.8 s
Wall time: 40.8 s

In [3]: %time res_ocl = opencl_compressor.compress_all_stack(dataset.data)
/users/kieffer/workspace-400/dynamix/dynamix/correlator/event.py:353: UserWarning: Setting array from one with different strides/storage order. This will cease to work in 2013.x.
  self.cl_mem["slab"].set(frames)
CPU times: user 924 ms, sys: 236 ms, total: 1.16 s
Wall time: 1.16 s

In [4]: print("\n".join(opencl_compressor.log_profile(stats=True)))

OpenCL kernel profiling statistics in milliseconds for: OpenclCompressor
                                       Kernel name (count):      min   median      max     mean      std
                                        compactor1 (   30):    3.961    4.090    4.162    4.075    0.062
________________________________________________________________________________
                       Total OpenCL execution time        : 122.255ms

[kif]

The cumsum approach implies to have the full stack in GPU memory, or that you slab in the stack in another dimension. It is not really compatible with an acquisition frame per frame or, with Eiger detectors, thousand frame per thousand frames.

[kif]

Yuriy's code does not pass the pylint test ... I find it really anoying working without CI !

[kif]

The code is probably "fast enough" for the moment. Implementing the cumsum & copy on the GPU can bring at best a extra factor 2 in speed.

[pierrepaleo]

If the OpenCL compressor is compatible (both in terms of results and API/usage) with the "numpy compressor", while being faster, then it's worthy.

I haven't looked at the whole code for now, as it's difficult to isolate the OpenCL compressor code from all the files linting - perhaps a dedicated PR for linting would be best.
But I agree that it's not realistic to assume that the whole raw dataset will fit in GPU memory - that's why we use this compaction scheme and "event correlator" in the first place. So we might implement the GPU-cumsum approach later (in another PR) if really needed.

It would also be interesting to see how this implementation performs on CPU. The reference numpy implementation is "naive" and sub-optimal, so I'm not surprised by the 40x speed-up.

[kif]

I merged the orthe PR which contains contains lint-ing ... it should make this PR much simpler.

The code is not directly the same since the numpy class has a staticmethod which is not (cannot) be static in the OpenCL case.
but that's the only difference.