The main objectives of RbCUDA are:

• Map all of CUDA into Ruby
• Ready-made on-GPU linear algebra, reduction, scan using cuBLAS, cuMath, cuSolver libraries.
• Random Numer generator using cuRand
• Near-zero wrapping overhead.
• CUDA profiler for Ruby.

This post explains the architecture of RbCUDA and how you can install it on your machine.

Installation

Install CUDA on your machine.

Building RbCUDA from source.

git clone https://github.com/prasunanand/rbcuda
cd rbcuda
bundle install
rake compile

Installing the gem

gem build rbcuda.gemspec
gem install rbcuda-0.0.0.gem

To check if installation was successful, run pry.

$ rake pry
pry -r './lib/rbcuda.rb'
[1] pry(main)> RbCUDA::CUDA.cuInit(0);
[2] pry(main)> device = RbCUDA::CUDA.cuDeviceGet(0);
[3] pry(main)> puts device
#<RbCUDA::RbCuDevice:0x00000001a9a2d0>
=> nil
[4] pry(main)> puts RbCUDA::CUDA.cuDeviceGetName(100, device);
GeForce GTX 750 Ti

If you are successfully able to retrive the name of GPU card, you are all set.

Code organisation

extconf.rb that helps in building the shared object file can be found here.

rbcuda.h defines all the Ruby structs that correspond to CUDA types. In the following code CUfunction type can be represented as function_ptr.

typedef struct FUNCTION_PTR
{
  CUfunction function;
}function_ptr;

typedef struct DEVICE_PTR
{
  CUdevice device;
}device_ptr;

The struct fuction_ptr is then wrapped by a Ruby object called RbCuFunction in the file ruby_rbcuda.c.

RbCUDA = rb_define_module("RbCUDA");

VALUE RbCuDevice = Qnil;
VALUE RbCuFunction = Qnil;

RbCuDevice    = rb_define_class_under(RbCUDA, "RbCuDevice",    rb_cObject);
RbCuFunction  = rb_define_class_under(RbCUDA, "RbCuFunction",  rb_cObject);

Dev Array

An array in RbCUDA is handled using Dev_Array class. Implementation is as follows:

typedef struct DEV_PTR
{
  double* carray;
}dev_ptr;

Dev_Array = rb_define_class_under(RbCUDA, "Dev_Array", rb_cObject);

A Dev Array stores the pointer to the array data stored on the GPU. The usage will be explained in the next blog.

Functionalities

RbCUDA has the following modules:

1. CUDA : It consists of low-level APIs called the CUDA driver APIs.
2. Runtime : It consists of higher-level APIs called the CUDA runtime APIs that are implemented on top of the CUDA driver APIs.
3. CuBLAS : It consists of BLAS APIs provided by cuBLAS library.
4. CuSolver : It consists of APIs provided by cuSolver library.
5. CuRand : It consists APIs provided by cuRand library.
6. Profiler : It consists of APIs for profiling CUDA code.

Conclusion

I have explained how the underlying architecture looks like.

We have got RbCUDA successfully installed on our system. In the next blog I will talk about implementing Runtime APIs.

In this project, I want to make it possible to combine the beauty of Ruby with transparent GPU processing with minimal overheads, so that software developers can easily use that power when available, and farm out computations transparently to GPU and CPU.

Nvidia has been developing a lot of GPU accelerated libraries for linear algebra(CUDA, cuBLAS, cuSolver), Random Number Generation(cuRand), fast-fourier transform(cuFFT), Parallel Primitives and Data Structures(Thrust) and image processing (NVIDIA Performance Primitives Library). Many Python/Julia/C++ libraries are being built on top of these libraries. RbCUDA would fill this gap in the Ruby ecosystem making it a highly-preferable option for GPU computing. It will work on Nvidia hardware running on client computers and on servers that make use of TESLA’s.

Recently, I was at Nvidia Developer Connect in my city. I was completely amazed by the new hardware launched by Nvidia this year. It makes me really confident about the great impact this project will create in the Ruby ecosystem.

Previous Efforts

sgc-ruby-cuda is an existing alternative for CUDA Ruby bindings. However, this library just helps in executing custom kernels. It also doesn’t have support cuBLAS, and other cuMath libraries.

Objectives

The main objectives of RbCUDA are:

1. Map all of CUDA into Ruby
2. Ready-made on-GPU linear algebra, reduction, scan using cuBLAS, cuMath, cuSolver libraries.
3. Random Numer generator using cuRand
4. Near-zero wrapping overhead.
5. CUDA profiler for Ruby.

In the near future:

1. fast-fourier transform(cuFFT)
2. Parallel Primitives and Data Structures(Thrust)
3. Image processing (NVIDIA Performance Primitives Library).
4. Use CuDNN to power a Deep Learning Library written in Ruby.

Codebase

https://github.com/prasunanand/rbcuda

Contributions are welcome!

Example

Most of the times, when we want to use CUDA for accelerating our code, we end up optimizing matrix-multiplication. I wrote an example that showcases how RbCUDA is the most efficient Ruby library to be used for Number crunching.

The example copies two matrices from CPU to GPU, runs matrix multiplication over it. Then the resulting matrix is copied back to the CPU from GPU.

I benchmarked my code and compare it with other existing Ruby libraries.

(Note: The above benchmarks have been done on an AMD FX 8350 octacore processor and Nvidia GTX 750Ti GPU. CUDA backend of ArrayFire was used with double floating points.)

RbCUDA is the fastest among all. Yay!

RbCUDA is the fastest of all the Ruby libraries. The time taken for matrix multiplication is 0.000017s on my machine. The plain C code takes me 0.000013s for this calculation.

RbCUDA is 24x faster than ArrayFire for matrix multiplication, Most of the speed gain going straight to CUDA is probably from removing an interaction layer (and buffers) as well as how the data is organized and fed to the underlying architecture.

Hence, an overhead of 0.000004s over plain C code makes, it highly efficient Maths library in Ruby.

The great thing about this example is that the code is very Rubyish <3.

The Code Implementation of matrix multiplication will be discussed in my upcoming blog posts.

Profiling matrix multiplication.

I have also created a profiler for RbCUDA . The profiling example can be found here.

Output of the code.

# CUDA_PROFILE_LOG_VERSION 2.0
# CUDA_DEVICE 0 GeForce GTX 750 Ti
# CUDA_CONTEXT 1
# TIMESTAMPFACTOR 14e96c24317d8324
method,gputime,cputime,occupancy
method=[ memcpyHtoD ] gputime=[ 16.544 ] cputime=[ 18.705 ]
method=[ memcpyHtoD ] gputime=[ 16.288 ] cputime=[ 22.089 ]
method=[ memcpyHtoD ] gputime=[ 1.216 ] cputime=[ 6.923 ]
method=[ _Z19gemm_kernel2x2_coreIdLb0ELb0ELb0ELb0ELb0EEvPT_PKS0_S3_iiiiiiS1_S1_S0_S0_i ] gputime=[ 76.736 ] cputime=[ 11.967 ] occupancy=[ 0.469 ]
method=[ _Z19gemm_kernel2x2_coreIdLb0ELb0ELb0ELb0ELb0EEvPT_PKS0_S3_iiiiiiS1_S1_S0_S0_i ] gputime=[ 73.024 ] cputime=[ 6.485 ] occupancy=[ 0.469 ]
method=[ _Z19gemm_kernel2x2_coreIdLb0ELb0ELb0ELb0ELb0EEvPT_PKS0_S3_iiiiiiS1_S1_S0_S0_i ] gputime=[ 72.928 ] cputime=[ 5.739 ] occupancy=[ 0.469 ]
method=[ _Z19gemm_kernel2x2_coreIdLb0ELb0ELb0ELb0ELb0EEvPT_PKS0_S3_iiiiiiS1_S1_S0_S0_i ] gputime=[ 72.928 ] cputime=[ 5.725 ] occupancy=[ 0.469 ]
method=[ _Z19gemm_kernel2x2_coreIdLb0ELb0ELb0ELb0ELb0EEvPT_PKS0_S3_iiiiiiS1_S1_S0_S0_i ] gputime=[ 72.896 ] cputime=[ 5.701 ] occupancy=[ 0.469 ]

(Note: The unit of time is microsecond).

Ruby Grant 2017

This project is funded by Ruby Association. I am very thankful to them for supporting this project. Link.

Contributions are welcome!

I request all the Rubyists if you have a Ruby code that you want to be accelerated by GPU, please try RbCUDA / ArrayFire. If you are into Deep learning, let me know if you want to use RbCUDA to power your neural nets for GPU acceleration.

Feel free to open an issue!

Stay tuned for more posts about GPU computing with Ruby.

Few people realise it, but even modest computers today, including mobile phones, have powerful GPUs. And these GPUs can be used serially and in parallel to CPUs, potentially delivering awesome performance.

In this project I wanted to make it possible to combine the beauty of Ruby with transparent GPU processing so that software developers can easily use that power when available, and farm out computations transparently to GPU and CPU. This will work both on client computers and on servers that make use of TESLA’s and Intel Xeon Phi solutions.

Application

The GSoC 2017 application can be found here.

Code

ArrayFire-rb: The pull request is undergoing a review.

ArrayFire-rb Benchmarks: Codebase can be found here.

Bio::FasterLmmD : Codebase can be found here

Goals

ArrayFire-rb now supports linear algebra on GPU and CPU. Currently only double dtype has been implemented. It supports dense and sparse matrices. It has multiple backends namely, CUDA, OpenCL and CPU.

The work on creating the bindings have been explained in last nine blog posts:

1. Ruby C extensions for complex projects
2. Installation
3. Af_Array(see performance)
4. Test-suite and Algorithm class
5. BLAS and LAPACK routines(see performance)
6. Statistics and Random Engine routines
7. Device and Util
8. Multiple Backends: CUDA, OpenCL and CPU
9. ArrayFire-NMatrix Interface

The performance of ArrayFire-rb is outstanding as expected.

I took a side-track working on Bio::FasterLmmD . This work is not complete and still in progress. It is an effort to call D from Ruby. The work has been explained in a previous blog post.

The work on ArrayFire-rb - JRuby has been postponed for now as I wanted to concentrate on MRI for the best results.

Future Work

The future work involves improving the ArrayFire-rb code and writing tutorials. ArrayFire is not limited to linear algebra so I will create bindings for Signal Processing, Computer Vision, etc. I will also add support for data types other than double.

The work on ArrayFire-rb - JRuby will begin as soon as ArrayFire gem is published.

FOSS

This has been my second GSoC with SciRuby. It has been more than an year contibuting extensively to FOSS. It has made me a 1000X better programmer that I used to be.

I really appreciate the effort by Google Open Source Committee for conducting GSoC every year. It is the best platform for the aspiring programmers improve their skill and give back to society by developing free and open source software.

Last year’s GSoC work helped me to present a talk at FOSDEM 2017 and Ruby Conf India 2017. I got active in the Indian Ruby Community. Recently, I have been invited as a speaker to Ruby World Conference 2017, Matsue, Japan to talk on “GPU computing with Ruby”.

I plan to continue contributing to open source, strive for improving my skills, and help new programmers contribute to FOSS. I would be glad if I could mentor students for upcoming GSoCs.

Acknowledgements

I would like to express my sincere gratitude to my mentor Pjotr Prins, for his guidance, patience and support. I have learn a lot from him since my last GSoC and still learning. I couldn’t have hoped for a better mentor.

I am grateful to Google and the Ruby Science Foundation for this golden opportunity.

I am very thankful to John Woods, Sameer Deshmukh, Alexej Gossmann, Gaurav Tamba and Pradeep Garigipati who mentored me through the project.

The CPU backend requires Intel MKL libraries be installed. To install Intel MKL library, go to the official download link. Next extract the file and run sudo install.sh from the project root directory.

For an AMD CPU, you need to specify the location of libmkl_rt.so.

To solve this on a debian machine:

$ cd /etc/ld.so.conf.d/
$ sudo nano mylibs.conf

Paste line:

/opt/intel/compilers_and_libraries_2017/linux/mkl/lib/intel64_lin/

Next, run

$ sudo ldconfig

Now download arrayfire-rb from link

$ git clone https://github.com/prasunanand/arrayfire-rb
$ rake compile
$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> ArrayFire::Backend.get_backend_count
=> 3
[2] pry(main)> ArrayFire::Backend.get_active_backend
=> "AF_BACKEND_CUDA"
[3] pry(main)> ArrayFire::Backend.set_backend(:AF_BACKEND_CPU)
=> nil
[4] pry(main)> ArrayFire::Backend.get_active_backend
=> "AF_BACKEND_CPU"

Voila! You can run ArrayFire on CPU.

Lets look into the implementation and what else can we do.

Backend

The class structure of Backend class is as follows:

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Backend = rb_define_class_under(ArrayFire, "Backend", rb_cObject);
  rb_define_singleton_method(Backend, "get_backend_count", (METHOD)arf_get_backend_count, 0);
  rb_define_singleton_method(Backend, "get_available_backends", (METHOD)arf_get_available_backends, 0);
  rb_define_singleton_method(Backend, "get_backend_id", (METHOD)arf_get_backend_id, 1);
  rb_define_singleton_method(Backend, "get_active_backend", (METHOD)arf_get_active_backend, 0);
  rb_define_singleton_method(Backend, "get_device_id", (METHOD)arf_get_backend_device_id, 1);
  rb_define_singleton_method(Backend, "set_backend", (METHOD)arf_set_backend, 1);
}

The C bindings were implemented as:

static VALUE arf_get_backend_count(VALUE self){
  uint num_backends;
  af_get_backend_count(&num_backends);
  return UINT2NUM(num_backends);
}

static VALUE arf_get_available_backends(VALUE self){
  int backends;
  af_get_available_backends(&backends);
  return INT2NUM(backends);
}

static VALUE arf_get_backend_id(VALUE self, VALUE array_val){
  afstruct* input;
  Data_Get_Struct(array_val, afstruct, input);
  af_backend backend;
  af_get_backend_id (&backend, input->carray);

  const char* backend_name = get_backend_name(backend);
  return rb_str_new_cstr(backend_name);
}

static VALUE arf_get_active_backend(VALUE self){
  af_backend backend;
  af_get_active_backend(&backend);
  const char* backend_name = get_backend_name(backend);
  return rb_str_new_cstr(backend_name);
}

static VALUE arf_set_backend(VALUE self, VALUE backend_val){
  af_backend backend = arf_backend_type_from_rbsymbol(backend_val);
  af_set_backend(backend);
  return Qnil;
}

An af_backend symbol can be :AF_BACKEND_DEFAULT, :AF_BACKEND_CPU, :AF_BACKEND_CUDA, or :AF_BACKEND_OPENCL. The implemetation is given below.

std::map<char*, size_t> BACKEND_TYPES = {
  {"AF_BACKEND_DEFAULT" , 0},                        ///< Default backend order: OpenCL -> CUDA -> CPU
  {"AF_BACKEND_CPU"     , 1},                        ///< CPU a.k.a sequential algorithms
  {"AF_BACKEND_CUDA"    , 2},                        ///< CUDA Compute Backend
  {"AF_BACKEND_OPENCL"  , 4}                         ///< OpenCL Compute Backend
};

af_backend arf_backend_type_from_rbsymbol(VALUE sym) {
  ID sym_id = SYM2ID(sym);

  for(std::map<char*, size_t>::value_type& entry : BACKEND_TYPES) {
    if (sym_id == rb_intern(entry.first)) {
      return static_cast<af_backend>(entry.second);
    }
  }

  VALUE str = rb_any_to_s(sym);
  rb_raise(rb_eArgError, "invalid backend type symbol (:%s) specified", RSTRING_PTR(str));
}

ArrayFire::Backend.get_available_backends lets you know the backends available. This is the table that helps you decipher the backends available.

Using rake pry to investigate the backend and device available for computing:

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> backends = ArrayFire::Backend.get_available_backends
=> 7
[2] pry(main)> ArrayFire::Backend.get_backend_count
=> 3
[3] pry(main)> ArrayFire::Backend.get_active_backend
=> "AF_BACKEND_CUDA"
[4] pry(main)> ArrayFire::Device.info
ArrayFire v3.4.0 (CUDA, 64-bit Linux, build 75cad40)
Platform: CUDA Toolkit 7.5, Driver: 375.66
[0] GeForce GTX 750 Ti, 4042 MB, CUDA Compute 5.0
=> nil
[5] pry(main)> ArrayFire::Backend.set_backend(:AF_BACKEND_CPU)
=> nil
[6] pry(main)> ArrayFire::Device.info
ArrayFire v3.4.0 (CPU, 64-bit Linux, build 75cad40)
[0] AMD: AMD FX(tm)-8350 Eight-Core Processor           , 16077 MB, Max threads(8)
=> nil
[7] pry(main)> ArrayFire::Backend.set_backend(:AF_BACKEND_OPENCL)
=> nil
[8] pry(main)> ArrayFire::Device.info
ArrayFire v3.4.0 (OpenCL, 64-bit Linux, build 75cad40)
[0] NVIDIA  : GeForce GTX 750 Ti, 4041 MB

ArrayFire also helps in interacting custom OpenCL and CUDA kernels. Since, Ruby currently doesn’t have rbCUDA and rbOpenCL gems (Ruby bindings for CUDA and OpenCL respectively), we can’t do much with ArrayFire-rb.

Conclusion

ArrayFire-rb can be used as a standalone linear algebra library not restricted to GPUs.

A user can change the backend on fly making it easier to leverage the power of CUDA and OpenCL. The order of preference of default backend to harness maximum computing power helps him not worry too much about performance tuning in the early stages of development, hence supporting the philosophy of preferring convention over configuration.

In the next blog post, I will discuss about interfacing ArrayFire-rb to NMatrix.

This post explains how NMatrix and ArrayFire-rb interface has been created. It starts with adding nmatrix as a dependency to arrayfire.

Gemspec

  gem.add_development_dependency 'nmatrix', '~> 0.2.1'

extconf.rb

Next add nmatrix.h header file allowing us to call nmatrix’s C/C++ methods from arrayfire.

nmatrix_path = Gem::Specification.find_all_by_name('nmatrix').compact
abort "Cannot locate NMatrix installation" unless nmatrix_path
nmatrix_header_dir = File.join(nmatrix_path[0].require_path)

HEADER_DIRS = [
  '/opt/local/include',
  '/usr/local/include',
  INCLUDEDIR,
  '/usr/include',
  nmatrix_header_dir
]

LIB_DIRS = [
  '/opt/local/lib',
  '/usr/local/lib',
  LIBDIR,
  '/usr/lib',
  nmatrix_header_dir
]

dir_config(extension_name, HEADER_DIRS, LIB_DIRS)

have_library('af')

NMatrix C++ API

An NMatrix Ruby C object is denoted by cNMatrix. Since, cNMatrix has been initially defined in nmatrix.h file we represent it as extern VALUE and C++ has been specified as we are using the C++interface.

Next, we create the Ruby bindings.

extern "C++" {
  VALUE cNMatrix;
}

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Af_Array = rb_define_class_under(ArrayFire, "Af_Array", rb_cObject);
  rb_define_method(Af_Array, "to_nmatrix", (METHOD)arf_af_array_to_nmatrix, 0);

  cNMatrix = rb_define_class("NMatrix", rb_cObject);
  rb_define_method(cNMatrix, "to_af_array", (METHOD)arf_nmatrix_to_af_array_method, 0);
}

The C functions have been implemented as the following code snippets.

nm namespace is used to store NMatrix types and structs.

Af_Array to NMatrix

An Af_Array#to_nmatrix is called to convert an Af_Array object to NMatrix object. The #to_nmatrix method calls arf_af_array_to_nmatrix method which is responsible for calling NMatrix C++ APIs. The rb_nmatrix_dense_create() is NMatrix API to create a dense NMatrix object. nm::FLOAT64 is the dtype of NMatrix.

// ext/mri/interface/nmatrix.c

static VALUE arf_af_array_to_nmatrix(VALUE self) {
  afstruct* input;
  Data_Get_Struct(self, afstruct, input);
  dim_t count;
  uint ndims;
  af_get_numdims(&ndims, input->carray);

  dim_t* dims = (dim_t*)malloc(ndims * sizeof(dim_t));

  af_get_dims(&dims[0], &dims[1], &dims[2], &dims[3], input->carray);

  size_t* shape = (size_t*)malloc(ndims * sizeof(size_t));;
  for (dim_t index = 0; index < ndims; index++){
    shape[index] = (size_t)(dims[index]);
  }

  af_get_elements(&count, input->carray);

  double* elements = (double*)malloc(count * sizeof(double));
  af_get_data_ptr(elements, input->carray);

  return rb_nmatrix_dense_create(nm::FLOAT64, shape, ndims, elements, (int)count);
}

NMatrix to Af_Array

An NMatrix#to_af_array is called to convert an NMatrix object to Af_Array object. The #to_af_array method calls arf_nmatrix_to_af_array_method method which is responsible for calling NMatrix C++ APIs. NM_DTYPE() is used to check the dtype of an NMatrix object. Currentlt, only nm::FLOAT64 is supported.

The arf_nmatrix_to_af_array is called by arf_nmatrix_to_af_array_method method and an Af_Array object.

// ext/mri/interface/nmatrix.c

extern VALUE arf_nmatrix_to_af_array_method(VALUE nmatrix) {
  if (NM_DIM(nmatrix) > 4) {
    rb_raise(rb_eStandardError,
      "NMatrix must not have greater than 4 dimensions.");
  }

  if (NM_DTYPE(nmatrix) == nm::FLOAT64) {
    return Data_Wrap_Struct(Af_Array, NULL, arf_free, arf_nmatrix_to_af_array(nmatrix));
  }
  else {
    rb_raise(rb_eStandardError,
      "NMatrix should be either :complex64, :complex128, :int32 or :float64 type.");
  }
  return Qnil;
}


afstruct* arf_nmatrix_to_af_array(VALUE nm) {
  DENSE_STORAGE* nmat = NM_STORAGE_DENSE(nm);
  afstruct* output = ALLOC(afstruct);

  if (nmat->dtype != nm::FLOAT64) {
    rb_raise(rb_eStandardError, "requires dtype of :float64 to convert to an Af_Array");
  }

  dim_t* shape = (dim_t*)malloc(nmat->dim * sizeof(dim_t));;
  for (size_t index = 0; index < nmat->dim; index++){
    shape[index] = (size_t)(nmat->shape[index]);
  }

  af_create_array(&output->carray, nmat->elements, nmat->dim, shape, f64);

  return output;
}

Let’s pry it out.

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> require 'nmatrix'
=> true
[2] pry(main)> a = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0,  -2, 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> #<ArrayFire::Af_Array:0x000000019bd6f0>
[3] pry(main)> b = a.to_nmatrix
=>
[
  [ 1.0,  2.0, 2.0, 0.0]
  [-2.0,  2.0, 1.0, 3.0]
  [ 1.0,  4.0, 3.0, 1.0]
  [ 0.0, -3.0, 2.0, 9.0]
]

[4] pry(main)> c = b.to_af_array
=> #<ArrayFire::Af_Array:0x00000001a4d340>
[5] pry(main)> c.elements
=> [1.0, 2.0, 2.0, 0.0, -2.0, 2.0, 1.0, 3.0, 1.0, 4.0, 3.0, 1.0, 0.0, -3.0, 2.0, 9.0]
[6] pry(main)> ArrayFire::Util.print_array(c)
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> true
[7] pry(main)> ArrayFire::Util.print_array(c)
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> true
[8] pry(main)> c = b.to_af_array
=> #<ArrayFire::Af_Array:0x00000001436088>

It works!.

Hence, NMatrix and ArrayFire can be easily interfaced to each other.

Benchmarks

Now, we can see how using ArrayFire can beneficial over using NMatrix. I benchmarked the matrix multiplication for two matrices with same elements using code.

shapeArray.each do |shape|
  elements1 = Array.new(shape[0]*shape[1]) { rand(1...999999) }
  elements2 = Array.new(shape[0]*shape[1]) { rand(1...999999) }
  cpu_matrix1 = NMatrix.new(shape, elements1, dtype: :float64)
  cpu_matrix2 = NMatrix.new(shape, elements2, dtype: :float64)

  gpu_matrix1 = cpu_matrix1.to_af_array
  gpu_matrix2 = cpu_matrix2.to_af_array

  iters.times do
    ArrayFire::BLAS.matmul(gpu_matrix1, gpu_matrix2, :AF_MAT_NONE, :AF_MAT_NONE) # warmup
  end

  result[:mat_mult_cpu] << [ shape[0]*shape[1], Benchmark.measure{cpu_matrix1.dot(cpu_matrix2)}.to_s.tr('()', '').split(" ")[3].to_f ]
  result[:mat_mult_gpu] << [ shape[0]*shape[1], Benchmark.measure{ArrayFire::BLAS.matmul(gpu_matrix1, gpu_matrix2, :AF_MAT_NONE, :AF_MAT_NONE)}.to_s.tr('()', '').split(" ")[3].to_f ]

end

(Note: The above benchmarks have been done on an AMD FX 8350 octacore processor and Nvidia GTX 750Ti GPU. CUDA backend of ArrayFire was used with double floating points.)

As the matrix size increases, we can see the difference is huge.

Using ArrayFire can speed up the calculation by 7 e +5 times.

There may be overheads in copying data from CPU to GPU and vice-versa. But overall, for large matrices(Big Data), we can gain massive speedups when we use the right optimization. I would be discussing more about such optimizations in another blog post.

However, ArrayFire being significantly faster than NMatrix can easily help Rubyists by adding small chunk of code and speeding up the critical/slower steps.

Power to Ruby!

Device

Device class contains of singleton methods like info, device_gc, lock_device_ptr that can be used to get info about the GPU devices used for calculation, explicitly call the garbage collector and locking the pointers on GPU device.

The implementation is as follows:

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Device = rb_define_class_under(ArrayFire, "Device", rb_cObject);
  rb_define_singleton_method(Device, "info", (METHOD)arf_info, 0);
  rb_define_singleton_method(Device, "init", (METHOD)arf_init2, 0);
  rb_define_singleton_method(Device, "info_string", (METHOD)arf_info_string, 0);
  rb_define_singleton_method(Device, "device_info", (METHOD)arf_device_info, 0);
  rb_define_singleton_method(Device, "get_device_count", (METHOD)arf_get_device_count, 0);
  rb_define_singleton_method(Device, "get_dbl_support", (METHOD)arf_get_dbl_support, 1);
  rb_define_singleton_method(Device, "set_device", (METHOD)arf_set_device, 1);
  rb_define_singleton_method(Device, "get_device", (METHOD)arf_get_device, 0);
  rb_define_singleton_method(Device, "sync", (METHOD)arf_sync, 1);
  rb_define_singleton_method(Device, "device_mem_info", (METHOD)arf_device_mem_info, 0);
  rb_define_singleton_method(Device, "print_mem_info", (METHOD)arf_print_mem_info, 2);
  rb_define_singleton_method(Device, "device_gc", (METHOD)arf_device_gc, 0);
  rb_define_singleton_method(Device, "set_mem_step_size", (METHOD)arf_set_mem_step_size, 1);
  rb_define_singleton_method(Device, "get_mem_step_size", (METHOD)arf_get_mem_step_size, 0);
  rb_define_singleton_method(Device, "lock_device_ptr", (METHOD)arf_lock_device_ptr, 1);
  rb_define_singleton_method(Device, "unlock_device_ptr", (METHOD)arf_unlock_device_ptr, 1);
  rb_define_singleton_method(Device, "lock_array", (METHOD)arf_lock_array, 1);
  rb_define_singleton_method(Device, "unlock_array", (METHOD)arf_unlock_array, 1);
  rb_define_singleton_method(Device, "is_locked_array", (METHOD)arf_is_locked_array, 1);
  rb_define_singleton_method(Device, "get_device_ptr", (METHOD)arf_get_device_ptr, 0);
}

The implementation of device_info is interesting as it has 4 return values in the form of strings. ArrayFire::Device.device_info passes the variables to be returned to ArrayFire::Device.device_info_func which modifies the contents of the variables passed.

module ArrayFire
  class Device
    def self.device_info
      d_name     = ""
      d_platform = ""
      d_toolkit  = ""
      d_compute  = ""
      ArrayFire::Device.device_info_func(d_name, d_platform, d_toolkit, d_compute);
      return d_name, d_platform, d_toolkit, d_compute
    end
  end
end

The point to note here is that ArrayFire::Device.device_info uses rb_str_cat2() to concatenate the char* to the VALUE containing empty string.

static VALUE arf_device_info(VALUE self, VALUE name_val, VALUE platform_val, VALUE toolkit_val, VALUE compute_val){
  char* d_name     = (char*)malloc(sizeof(char) * 64);
  char* d_platform = (char*)malloc(sizeof(char) * 10);
  char* d_toolkit  = (char*)malloc(sizeof(char) * 64);
  char* d_compute  = (char*)malloc(sizeof(char) * 10);

  af_device_info(d_name, d_platform, d_toolkit, d_compute);

  rb_str_cat2(name_val,     d_name);
  rb_str_cat2(platform_val, d_platform);
  rb_str_cat2(toolkit_val,  d_toolkit);
  rb_str_cat2(compute_val,  d_compute);

  return Qnil;
}

static VALUE arf_device_mem_info(VALUE self){
  size_t alloc_bytes, alloc_buffers, lock_bytes, lock_buffers;
  af_device_mem_info( &alloc_bytes, &alloc_buffers, &lock_bytes, &lock_buffers);
  printf("Allocated Bytes: %d\nAllocated buffers: %d\nLock Bytes: %d\nLock Buffers: %d\n",
          alloc_bytes, alloc_buffers, lock_bytes, lock_buffers);
  return Qnil;
}

Running pry to check the bindings:

rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> ArrayFire::Device.info
ArrayFire v3.4.0 (CUDA, 64-bit Linux, build 75cad40)
Platform: CUDA Toolkit 7.5, Driver: 375.66
[0] GeForce GTX 750 Ti, 4042 MB, CUDA Compute 5.0
=> nil
[2] pry(main)> ArrayFire::Device.device_info
=> ["GeForce_GTX_750_Ti", "CUDA", "v7.5", "5.0"]
[3] pry(main)> ArrayFire::Device.get_device_count
=> 1
[4] pry(main)> ArrayFire::Device.get_dbl_support(0)
=> true
[5] pry(main)> ArrayFire::Device.device_mem_info
Allocated Bytes: 0
Allocated buffers: 0
Lock Bytes: 0
Lock Buffers: 0
=> nil
[6] pry(main)> arr = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0, -2, 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> #<ArrayFire::Af_Array:0x000000023800c0>
[7] pry(main)> ArrayFire::Device.print_mem_info("mem info", 0)
mem info
---------------------------------------------------------
|     POINTER      |    SIZE    |  AF LOCK  | USER LOCK |
---------------------------------------------------------
|     0x702f80000  |       1 KB |       Yes |        No |
|     0x702f80400  |       1 KB |        No |        No |
---------------------------------------------------------
=> nil
[8] pry(main)> ArrayFire::Device.device_mem_info
Allocated Bytes: 2048
Allocated buffers: 2
Lock Bytes: 1024
Lock Buffers: 1
=> nil
[9] pry(main)> ArrayFire::Device.get_mem_step_size
=> 1024
[10] pry(main)> ArrayFire::Device.is_locked_array(arr)
=> false
[11] pry(main)> ArrayFire::Device.lock_array(arr)
=> true
[12] pry(main)> ArrayFire::Device.is_locked_array(arr)
=> true

Util

Util class contains of methods responsible for printing an Af_Array, saving it in a file or converting it to string and other utility methods.

The implementation is as follows:

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Util = rb_define_class_under(ArrayFire, "Util", rb_cObject);
  rb_define_singleton_method(Util, "print_array", (METHOD)arf_print_array, 1);
  rb_define_singleton_method(Util, "print_array_gen", (METHOD)arf_print_array_gen, 0);
  rb_define_singleton_method(Util, "save_array", (METHOD)arf_save_array, 4);
  rb_define_singleton_method(Util, "read_array_index", (METHOD)arf_read_array_index, 0);
  rb_define_singleton_method(Util, "read_array_key", (METHOD)arf_read_array_key, 0);
  rb_define_singleton_method(Util, "read_array_key_check", (METHOD)arf_read_array_key_check, 0);
  rb_define_singleton_method(Util, "array_to_string", (METHOD)arf_array_to_string, 4);
  rb_define_singleton_method(Util, "get_version", (METHOD)arf_get_version, 0);
  rb_define_singleton_method(Util, "get_revision", (METHOD)arf_get_revision, 0);
  rb_define_singleton_method(Util, "get_size_of", (METHOD)arf_get_size_of, 1);
}

af_array_to_string API has also been used to implement ArrayFire::Af_Array#to_s.

The implementation of arf_get_version is interesting as it returns a hash value that represents the version of ArrayFire C library installed using the keys as major, minor and patch.

rb_hash_new and rb_hash_aset has been used to create a hash.

static VALUE arf_print_array(VALUE self, VALUE input_val){
  afstruct* input;

  Data_Get_Struct(input_val, afstruct, input);

  af_print_array(input->carray);
  return Qtrue;
}

static VALUE arf_save_array(VALUE self, VALUE key_val, VALUE array_val, VALUE fn_val, VALUE append){
  afstruct* input;

  Data_Get_Struct(array_val, afstruct, input);
  const char* key = StringValueCStr(key_val);
  const char* filename = StringValueCStr(fn_val);
  int index;
  af_save_array (&index, key, input->carray, filename, RTEST(append));
  return INT2NUM(index);
}

static VALUE arf_array_to_string(VALUE self, VALUE exp_val, VALUE array_val, VALUE precision, VALUE transpose){
  char* output;
  afstruct* input;

  Data_Get_Struct(array_val, afstruct, input);
  const char* exp = StringValueCStr(exp_val);
  af_array_to_string(&output, exp, input->carray, NUM2INT(precision), RTEST(transpose));

  return rb_str_new_cstr(output);
}


static VALUE arf_get_version(VALUE self){
  int major, minor, patch;
  af_get_version(&major, &minor, &patch);

  VALUE hash = rb_hash_new();
  rb_hash_aset(hash, rb_str_new_cstr("major"), INT2NUM(major));
  rb_hash_aset(hash, rb_str_new_cstr("minor"), INT2NUM(minor));
  rb_hash_aset(hash, rb_str_new_cstr("patch"), INT2NUM(patch));
  return hash;
}

Lets use pry to check the bindings:

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> arr = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0,  -2, 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> #<ArrayFire::Af_Array:0x000000026bde18>
[2] pry(main)> ArrayFire::Util.print_array(arr)
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> true
[3] pry(main)> x = ArrayFire::Util.array_to_string("GPU Array", arr, 5, false)
=> "GPU Array\n[4 4 1 1]\n    1.00000     2.00000     2.00000     0.00000 \n   -2.00000     2.00000     1.00000     3.00000 \n    1.00000     4.00000     3.00000     1.00000 \n    0.00000    -3.00000     2.00000     9.00000 \n\n"
[4] pry(main)> puts x
GPU Array
[4 4 1 1]
    1.00000     2.00000     2.00000     0.00000
   -2.00000     2.00000     1.00000     3.00000
    1.00000     4.00000     3.00000     1.00000
    0.00000    -3.00000     2.00000     9.00000

=> nil
[5] pry(main)> ArrayFire::Util.get_version
=> {"major"=>3, "minor"=>4, "patch"=>0}
[6] pry(main)> ArrayFire::Util.get_revision
=> "75cad40"
[7] pry(main)> ArrayFire::Util.get_size_of(:f64)
=> 8

The bindings work!.

Conclusion

We can easily manage the available GPU devices and manage the memory. Also, we can save the Af_Array output to a file among other utilities.

In the next blog post, I will explain about using multiple backends using ArrayFire , i.e. OpenCL, CUDA and even use CPUs.

Statistics Class

The Statistics class contains of singleton methods like mean, var, median, stddev, etc. Let us take a look at the implementation.

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Statistics = rb_define_class_under(ArrayFire, "Statistics", rb_cObject);
  rb_define_singleton_method(Statistics, "mean", (METHOD)arf_mean, 2);
  rb_define_singleton_method(Statistics, "mean_weighted", (METHOD)arf_mean_weighted, 3);
  rb_define_singleton_method(Statistics, "var", (METHOD)arf_var, 3);
  rb_define_singleton_method(Statistics, "var_weighted", (METHOD)arf_var_weighted, 3);
  rb_define_singleton_method(Statistics, "stdev", (METHOD)arf_stdev, 2);
  rb_define_singleton_method(Statistics, "cov", (METHOD)arf_cov, 3);
  rb_define_singleton_method(Statistics, "median", (METHOD)arf_median, 2);
  rb_define_singleton_method(Statistics, "mean_all", (METHOD)arf_mean_all, 1);
  rb_define_singleton_method(Statistics, "mean_all_weighted", (METHOD)arf_mean_all_weighted, 2);
  rb_define_singleton_method(Statistics, "var_all", (METHOD)arf_var_all, 2);
  rb_define_singleton_method(Statistics, "var_all_weighted", (METHOD)arf_var_all_weighted, 2);
  rb_define_singleton_method(Statistics, "stdev_all", (METHOD)arf_stdev_all, 1);
  rb_define_singleton_method(Statistics, "median_all", (METHOD)arf_median_all, 1);
  rb_define_singleton_method(Statistics, "corrcoef", (METHOD)arf_corrcoef, 2);

}

ArrayFire provides four different methods to calculate the means namely, mean, mean_weighted, mean_all, mean_all_weighted. The implementation is given below.

mean and mean_weighted calculate the mean and weighted mean along the dimnension specified. However, mean_all and mean_weighted_all calculate the mean and weighted mean over all the elements in an array.

static VALUE arf_mean(VALUE self, VALUE array_val, VALUE dim_val){
  afstruct* input;
  afstruct* output = ALLOC(afstruct);

  Data_Get_Struct(array_val, afstruct, input);

  af_mean(&output->carray, input->carray, NUM2UINT(dim_val));
  af_print_array(output->carray);

  return Data_Wrap_Struct(Af_Array, NULL, arf_free, output);
}

static VALUE arf_mean_weighted(VALUE self, VALUE array_val, VALUE weighted_array_val, VALUE dim_val){
  afstruct* input;
  afstruct* weighted_array;
  afstruct* output = ALLOC(afstruct);

  Data_Get_Struct(array_val, afstruct, input);
  Data_Get_Struct(weighted_array_val, afstruct, weighted_array);

  af_mean_weighted(&output->carray, input->carray, weighted_array->carray, NUM2UINT(dim_val));
  af_print_array(output->carray);

  return Data_Wrap_Struct(Af_Array, NULL, arf_free, output);
}

static VALUE arf_mean_all(VALUE self, VALUE array_val){
  afstruct* input;
  double real_part, imag_part;

  Data_Get_Struct(array_val, afstruct, input);

  af_mean_all(&real_part, &imag_part, input->carray);

  return DBL2NUM(real_part);
}

static VALUE arf_mean_all_weighted(VALUE self, VALUE array_val, VALUE weighted_array_val){
  afstruct* input;
  afstruct* weighted_array;
  double real_part, imag_part;

  Data_Get_Struct(array_val, afstruct, input);
  Data_Get_Struct(weighted_array_val, afstruct, weighted_array);

  af_mean_all_weighted(&real_part, &imag_part, input->carray, weighted_array->carray);

  return DBL2NUM(real_part);
}

I check the bindings using pry.

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> arr = ArrayFire::Random.randu(2, [4, 4])=> #<ArrayFire::Af_Array:0x0000000308fb08>
[2] pry(main)> ArrayFire::Util.print_array(arr)No Name Array
[4 4 1 1]
    0.3990     0.7353     0.9455     0.7089
    0.6720     0.5160     0.1587     0.9434
    0.5339     0.3932     0.8831     0.1227
    0.1386     0.2706     0.0621     0.9107

=> true
[3] pry(main)> weighted_array = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0,  -2, 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]
No Name Array
[4 4 1 1]
    1.0000    -2.0000     1.0000     0.0000
    2.0000     2.0000     4.0000    -3.0000
    2.0000     1.0000     3.0000     2.0000
    0.0000     3.0000     1.0000     9.0000

=> #<ArrayFire::Af_Array:0x0000000316c008>
[4] pry(main)> mean = ArrayFire::Statistics.mean(arr, 1)
No Name Array
[4 1 1 1]
    0.6972
    0.5725
    0.4832
    0.3455

=> #<ArrayFire::Af_Array:0x00000003212138>
[5] pry(main)> mean_weighted = ArrayFire::Statistics.mean_weighted(arr, weighted_array, 2)
No Name Array
[4 4 1 1]
    0.3990     0.7353     0.9455        nan
    0.6720     0.5160     0.1587     0.9434
    0.5339     0.3932     0.8831     0.1227
       nan     0.2706     0.0621     0.9107

=> #<ArrayFire::Af_Array:0x000000031d2588>
[6] pry(main)> mean_all = ArrayFire::Statistics.mean_all(arr)=> 0.5246008634567261
[7] pry(main)> mean_all_weighted = ArrayFire::Statistics.mean_all_weighted(arr, weighted_array)
=> 0.5184718370437622

Random Class

ArrayFire supports three types of random engines: :AF_RANDOM_ENGINE_PHILOX_4X32_10, :AF_RANDOM_ENGINE_THREEFRY_2X32_16 and :AF_RANDOM_ENGINE_MERSENNE_GP11213 which can be passed as a type.

The default randon engine is :AF_RANDOM_ENGINE_PHILOX_4X32_10. Random class helps in creating an engine by specifying the type and seed. A programmer can set the seed on the fly and also create arrays with randomly generated values.

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Random = rb_define_class_under(ArrayFire, "Random", rb_cObject);
  rb_define_alloc_func(Random, arf_engine_alloc);
  rb_define_singleton_method(Random, "create_random_engine", (METHOD)arf_create_random_engine, 2);
  rb_define_singleton_method(Random, "retain_random_engine", (METHOD)arf_retain_random_engine, 1);
  rb_define_singleton_method(Random, "random_engine_set_type", (METHOD)arf_random_engine_set_type, 0);
  rb_define_singleton_method(Random, "random_engine_get_type", (METHOD)arf_random_engine_get_type, 0);
  rb_define_singleton_method(Random, "random_uniform", (METHOD)arf_random_uniform, 3);
  rb_define_singleton_method(Random, "random_normal", (METHOD)arf_random_normal, 3);
  rb_define_singleton_method(Random, "random_engine_set_seed", (METHOD)arf_random_engine_set_seed, 2);
  rb_define_singleton_method(Random, "get_default_random_engine", (METHOD)arf_get_default_random_engine, 0);
  rb_define_singleton_method(Random, "set_default_random_engine_type", (METHOD)arf_set_default_random_engine_type, 0);
  rb_define_singleton_method(Random, "random_engine_get_seed", (METHOD)arf_random_engine_get_seed, 0);
  rb_define_singleton_method(Random, "release_random_engine", (METHOD)arf_release_random_engine, 0);
  rb_define_singleton_method(Random, "randu", (METHOD)arf_randu, 2);
  rb_define_singleton_method(Random, "randn", (METHOD)arf_randn, 2);
  rb_define_singleton_method(Random, "set_seed", (METHOD)arf_set_seed, 1);
  rb_define_singleton_method(Random, "get_seed", (METHOD)arf_get_seed, 0);
}

A Random enigne must be created so I write the ruby bindings to alloc memory and dealloc memory to a af_random_engine using arf_engine_alloc and arf_engine_free respectively. The rest is similar to how I created bindings in previous blog posts.

typedef struct RANDOM_ENGINE_STRUCT
{
  af_random_engine cengine;
}afrandomenginestruct;

static VALUE arf_engine_alloc(VALUE klass)
{
  /* allocate */
  afrandomenginestruct* afrandomengine = ALLOC(afrandomenginestruct);
  /* wrap */
  return Data_Wrap_Struct(klass, NULL, arf_engine_free, afrandomengine);
}

static void arf_engine_free(afrandomenginestruct* afrandomengine)
{
  free(afrandomengine);
}

static VALUE arf_create_random_engine(VALUE self, VALUE type_val, VALUE seed_val){
  afrandomenginestruct* output = ALLOC(afrandomenginestruct);
  af_random_engine_type rtype = arf_randome_engine_type_from_rbsymbol(type_val);

  af_create_random_engine(&output->cengine, AF_RANDOM_ENGINE_DEFAULT, NUM2ULL(seed_val) ) ;

  return Data_Wrap_Struct(Random, NULL, arf_engine_free, output);
}

static VALUE arf_randu(VALUE self, VALUE ndims_val, VALUE dim_val){
  afstruct* out_array = ALLOC(afstruct);

  dim_t ndims = (dim_t)FIX2LONG(ndims_val);
  dim_t* dimensions = (dim_t*)malloc(ndims * sizeof(dim_t));
  dim_t count = 1;
  for (dim_t index = 0; index < ndims; index++) {
    dimensions[index] = (dim_t)FIX2LONG(RARRAY_AREF(dim_val, index));
    count *= dimensions[index];
  }
  af_randu(&out_array->carray, ndims, dimensions,f64);
  return Data_Wrap_Struct(Af_Array, NULL, arf_free, out_array);
}

Now, we have the bindings ready, we can check it using pry.

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> engine = ArrayFire::Random.create_random_engine(:AF_RANDOM_ENGINE_PHILOX_4X32_10, 100)
=> #<ArrayFire::Random:0x00000002a41350>
[2] pry(main)> ArrayFire::Random.random_engine_get_seed(engine)
=> 100
[3] pry(main)> ArrayFire::Random.random_engine_set_seed(engine, 123)
=> #<ArrayFire::Random:0x00000002cfbd98>
[4] pry(main)> ArrayFire::Random.random_engine_get_seed(engine)
=> 123
[5] pry(main)> ArrayFire::Random.random_engine_get_type engine
=> "AF_RANDOM_ENGINE_PHILOX_4X32_10"
[6] pry(main)> arr = ArrayFire::Random.randu(2, [4, 4])
=> #<ArrayFire::Af_Array:0x00000002b185f8>
[7] pry(main)> ArrayFire::Util.print_array(arr)
No Name Array
[4 4 1 1]
    0.3990     0.7353     0.9455     0.7089
    0.6720     0.5160     0.1587     0.9434
    0.5339     0.3932     0.8831     0.1227
    0.1386     0.2706     0.0621     0.9107

=> true
[8] pry(main)> arr2 = ArrayFire::Random.randn(2, [4, 4])
=> #<ArrayFire::Af_Array:0x000000029fd4c0>
[9] pry(main)> ArrayFire::Util.print_array(arr2)
No Name Array
[4 4 1 1]
    0.2985    -0.8873    -1.0309     0.5312
   -2.7126    -0.3550    -1.4627    -1.7783
    0.4584     1.9841     0.0075    -0.5459
    1.5579    -0.9308    -1.0512     0.4640

=> true

we have the Random support ready for ArrayFire-rb

Hence, Ruby bindings for Statistics and Random methods have been successfully implemented.

Conclusion

We can now use ArrayFire-rb for statistical analysis on data. Since, the calculations would be on GPU, we can feed it large amount of data from real world.

Random engine can help in creating large arrays with randomly generated values in seconds.

In the next blog, I will explain about using the Device and Util class to manage GPU devices and device memory and other utilities like saving an Af_Array to file.

The performance benchmarks for ArrayFire against NMatrix can be represented by the following figures. The code used for benchmarking and generating the plots can be found here and be used to reproduce similar plots.

Performance metrics

(Note: The above benchmarks have been done on an AMD FX 8350 octacore processor and Nvidia GTX 750Ti GPU. CUDA backend of ArrayFire was used with double floating points.)

The figure shows that ArrayFire takes the least computation time of all. ArrayFire is 3 e +6 times faster than NMatrix for JRuby and NMatrix for Ruby(not BLAS) whereas 7 e +5 times faster than NMatrix for Ruby(using BLAS).

For LAPACK routines, like calculating determinant and lower-upper factorization, ArrayFire is 100 times faster than NMatrix for JRuby whereas 6 times faster than NMatrix for Ruby(using LAPACKE).

Lets take a look at the implementation.

BLAS Routines

All the BLAS methods are singleton_methods.

void Init_arrayfire() {
  ArrayFire = rb_define_module("ArrayFire");

  Blas = rb_define_class_under(ArrayFire, "BLAS", rb_cObject);
  rb_define_singleton_method(Blas, "matmul", (METHOD)arf_matmul, 4);
  rb_define_singleton_method(Blas, "dot", (METHOD)arf_dot, 4);
  rb_define_singleton_method(Blas, "transpose", (METHOD)arf_transpose, 1);

}

arf_matmul and arf_transpose call af_matmul and af_transpose respectively.

static VALUE arf_matmul(VALUE self, VALUE left_val, VALUE right_val, VALUE left_prop_val, VALUE right_prop_val){

  afstruct* left;
  afstruct* right;
  afstruct* result = ALLOC(afstruct);

  Data_Get_Struct(left_val, afstruct, left);
  Data_Get_Struct(right_val, afstruct, right);

  af_mat_prop left_mat_prop = arf_mat_type_from_rbsymbol(left_prop_val);
  af_mat_prop right_mat_prop = arf_mat_type_from_rbsymbol(right_prop_val);

  af_matmul(&result->carray, left->carray, right->carray, left_mat_prop, right_mat_prop);

  af_print_array(result->carray);

  return Data_Wrap_Struct(CLASS_OF(left_val), NULL, arf_free, result);
}

static VALUE arf_transpose(VALUE self, VALUE input){
  afstruct* obj;
  afstruct* result = ALLOC(afstruct);

  Data_Get_Struct(input, afstruct, obj);

  af_transpose(&result->carray, obj->carray, false);

  af_print_array(result->carray);

  return Data_Wrap_Struct(CLASS_OF(input), NULL, arf_free, result);
}

af_matmul expects the property of matrices passed to it. For example, af_mat_prop AF_MAT_TRANS signifies that the matrix is transposed. So, to pass the symbols to Ruby C function arf_matmul, I have used the code given below. I have creared a list of values MAT_PROPERTIES and a function arf_mat_type_from_symbol which is responsible for figuring out af_mat_prop from the symbol.

std::map<char*, size_t> MAT_PROPERTIES = {
  {"AF_MAT_NONE", 0},
  {"AF_MAT_TRANS", 1},
  {"AF_MAT_CTRANS", 2},
  {"AF_MAT_CONJ", 4},
  {"AF_MAT_UPPER", 32},
  {"AF_MAT_LOWER", 64},
  {"AF_MAT_DIAG_UNIT", 128},
  {"AF_MAT_SYM", 512},
  {"AF_MAT_POSDEF", 1024},
  {"AF_MAT_ORTHOG", 2048},
  {"AF_MAT_TRI_DIAG", 4096},
  {"AF_MAT_BLOCK_DIAG", 8192}
};


af_mat_prop arf_mat_type_from_rbsymbol(VALUE sym) {
  ID sym_id = SYM2ID(sym);

  for(std::map<char*, size_t>::value_type& entry : MAT_PROPERTIES) {
    if (sym_id == rb_intern(entry.first)) {
      return static_cast<af_mat_prop>(entry.second);
    }
  }

  VALUE str = rb_any_to_s(sym);
  rb_raise(rb_eArgError, "invalid matrix type symbol (:%s) specified",
   RSTRING_PTR(str));
}

So, now I can check if the bindings work successfully.

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> left = ArrayFire::Af_Array.new 2 , [3,3] , [1, 4, 6, 4, 11 , 2 ,-5, 8, 10]
No Name Array
[3 3 1 1]
    1.0000     4.0000    -5.0000
    4.0000    11.0000     8.0000
    6.0000     2.0000    10.0000

=> #<ArrayFire::Af_Array:0x000000014e56c8>
[2] pry(main)> right = ArrayFire::Af_Array.new 2 , [3,2] , [1, 0, 8, 10, -11, 8]
No Name Array
[3 2 1 1]
    1.0000    10.0000
    0.0000   -11.0000
    8.0000     8.0000

=> #<ArrayFire::Af_Array:0x00000001591db0>
[3] pry(main)> result = ArrayFire::BLAS.matmul(left, right, :AF_MAT_NONE, :AF_MAT_NONE)
No Name Array
[3 2 1 1]
  -39.0000   -74.0000
   68.0000   -17.0000
   86.0000   118.0000

=> #<ArrayFire::Af_Array:0x000000016136f8>
[4] pry(main)> transposed = ArrayFire::BLAS.transpose(left)
No Name Array
[3 3 1 1]
    1.0000     4.0000     6.0000
    4.0000    11.0000     2.0000
   -5.0000     8.0000    10.0000

=> #<ArrayFire::Af_Array:0x00000001762f68>

It works!.

LAPACK Routines

LAPACK routines similar to BLAS ones are singleton methods.

Lapack = rb_define_class_under(ArrayFire, "LAPACK", rb_cObject);
rb_define_singleton_method(Lapack, "svd_func", (METHOD)arf_svd_func, 4);
rb_define_singleton_method(Lapack, "svd_inplace_func", (METHOD)arf_svd_inplace_func, 1);
rb_define_singleton_method(Lapack, "lu_func", (METHOD)arf_lu_func, 4);
rb_define_singleton_method(Lapack, "lu_inplace_func", (METHOD)arf_lu_inplace_func, 1);
rb_define_singleton_method(Lapack, "qr_func", (METHOD)arf_qr_func, 4);
rb_define_singleton_method(Lapack, "qr_inplace_func", (METHOD)arf_qr_inplace_func, 1);
rb_define_singleton_method(Lapack, "cholesky_func", (METHOD)arf_cholesky_func, 3);
rb_define_singleton_method(Lapack, "cholesky_inplace_func", (METHOD)arf_cholesky_inplace_func, 1);
rb_define_singleton_method(Lapack, "solve_func", (METHOD)arf_solve_func, 0);
rb_define_singleton_method(Lapack, "solve_lu_func", (METHOD)arf_solve_lu_func, 0);
rb_define_singleton_method(Lapack, "inverse", (METHOD)arf_inverse, 1);
rb_define_singleton_method(Lapack, "rank", (METHOD)arf_rank, 1);
rb_define_singleton_method(Lapack, "det", (METHOD)arf_det, 1);
rb_define_singleton_method(Lapack, "norm", (METHOD)arf_norm, 1);
rb_define_singleton_method(Lapack, "is_lapack_available", (METHOD)arf_is_lapack_available, 0);

LAPACK.svd must return multiple objects and hence calls LAPACK.arf_svd_func. The implementation is as follows.

static VALUE arf_svd_func(VALUE self, VALUE u_val, VALUE s_val, VALUE vt_val, VALUE val){
  afstruct* input;
  afstruct* u;
  afstruct* s;
  afstruct* vt;

  Data_Get_Struct(val, afstruct, input);
  Data_Get_Struct(u_val, afstruct, u);
  Data_Get_Struct(s_val, afstruct, s);
  Data_Get_Struct(vt_val, afstruct, vt);

  af_svd(&u->carray, &s->carray, &vt->carray, input->carray);

  return Qtrue;
}

module ArrayFire
  class LAPACK

    def self.svd(af_array)
      u =  ArrayFire::Af_Array.new
      s =  ArrayFire::Af_Array.new
      vt = ArrayFire::Af_Array.new
      ArrayFire::LAPACK.svd_func(u, s, vt, af_array)
      return u, s, vt
    end
  end
end

LAPACK.det calls af_det to calculate the determinant. Currently, I haven’t created support for complex dtype. So, I have ignored the imaginary part of the result if it is not a real number.

static VALUE arf_det(VALUE self, VALUE val){
  afstruct* matrix;
  double det_real, det_imag;

  Data_Get_Struct(val, afstruct, matrix);

  af_det(&det_real, &det_imag, matrix->carray);
  return DBL2NUM(det_real);
}

Now, we can check the elementwise operations using pry.

$ rake pry
pry -r './lib/arrayfire.rb'
[1] pry(main)> matrix = ArrayFire::Af_Array.new 2, [4,2], [1,3,5,7,2,4,6,8]
No Name Array
[4 2 1 1]
    1.0000     2.0000
    3.0000     4.0000
    5.0000     6.0000
    7.0000     8.0000

=> #<ArrayFire::Af_Array:0x0000000265cbe0>
[2] pry(main)> u , s, vt = ArrayFire::LAPACK.svd(matrix)
=> [#<ArrayFire::Af_Array:0x000000026ff070>, #<ArrayFire::Af_Array:0x000000026ff048>, #<ArrayFire::Af_Array:0x000000026ff020>]
[3] pry(main)> ArrayFire::Util.print_array(u)
No Name Array
[4 4 1 1]
   -0.1525    -0.8226    -0.3945    -0.3800
   -0.3499    -0.4214     0.2428     0.8007
   -0.5474    -0.0201     0.6979    -0.4614
   -0.7448     0.3812    -0.5462     0.0407

=> true
[4] pry(main)> ArrayFire::Util.print_array(s)
No Name Array
[2 1 1 1]
   14.2691
    0.6268

=> true
[5] pry(main)> ArrayFire::Util.print_array(vt)
No Name Array
[2 2 1 1]
   -0.6414    -0.7672
    0.7672    -0.6414

=> true
[6] pry(main)> left = ArrayFire::Af_Array.new 2 , [3,3] , [1, 4, 6, 4, 11 , 2 ,-5, 8, 10]
No Name Array
[3 3 1 1]
    1.0000     4.0000    -5.0000
    4.0000    11.0000     8.0000
    6.0000     2.0000    10.0000

=> #<ArrayFire::Af_Array:0x00000002984c78>
[7] pry(main)> det = ArrayFire::LAPACK.det(left)
=> 415.9999694824219

It works!

Conclusion

We can now use ArrayFire-rb for linear algebra since the BLAS and LAPACK routines are in place. Also, the performance is outstanding.

In the next blog, I will explain about using the Random Engine generator and Statistics methods for ArrayFire.

As ArrayFire-rb, a GPGPU libray has started taking shape, the next goal is to interface it with NMatrix library, so we can run Linear Mixed Models(mixed_models).

I have also written an LMM(Linear Mixed Models) solver in D for Genome Wide Association studies called faster_lmm_d which has two GPU backends:

1. CUDA backend which helps it directly interact with CUBLAS libraries and runs only on Nvidia Hardware. For CUDA backend, Faster_LMM_D uses cuda_d(The D bindings I wrote for CUDA libraries).

2. ArrayFire backend which helps it run on all major GPU vendors(Nvidia, Intel, AMD) by calling CUDA, CuBLAS, OpenCL, clBLAS libraries using the ArrayFire library. For ArrayFire backend, Faster_LMM_D uses arrayfire-d(The D bindings I wrote for ArrayFire library).

So, I found that it would be really great if we could port faster_lmm_d to Ruby by calling D from Ruby. faster_lmm_d uses some cool tricks to efficiently use GPUs for computing like minimizing the warmup time, minimal copying of CPU memory to GPU memory. It logs CPU memory and GPU memory before and after every significant computation. So, it would be an ideal exercise for GPU computing on Ruby by calling D functions under the hood.

Although, this is still a work in progress, I would like to share about my experiences.

Previous Attempts

There has been previous attempts at Calling D from Ruby. In my search for the existing attempts at “Calling D from Ruby”, I stumbled upon these resources.

1. FFI: Use the Ruby-FFI gem to call a D function from C.
2. RuDy: An effort / library / gem to enable and ease writing Ruby native extensions in D programming language. It depends on creatind D bindings for the API calls in ruby.h header file.
3. Ruby-Dlang: Similar approach as Rudy.
4. Orbit: Similar approach as Rudy.

I have been intereseted in the first two methods, i.e. FFI and Rudy.

Exploring FFI

The idea here is to create a shared object .so and call it with Ruby by loading the shared object using FFI gem.

Dlang has three compilers namely DMD, GDC and LDC. The post on Dlang wiki makes use of DMD, I could easily run the example on my machine using DMD, however I could not run it with LDC. I need to compile the d files with LDC because faster_lmm_d makes use of LDC compiler switches and it would take a lot of effort to be able to run it with DMD.

The error I encountered is:

1. Error with ld :

   $ ldc2 -shared -m64 -relocation-model=pic i.d
   /usr/bin/ld: /home/prasun/ldclatest/ldc-1.1.0-pk9rkm4zvdp6pglam7s2/lib/libdruntime-ldc.a(errno.c.o): relocation R_X86_64_PC32 against undefined symbol `__errno_location@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
   /usr/bin/ld: final link failed: Bad value
   collect2: error: ld returned 1 exit status
   Error: /usr/lib/nvidia-cuda-toolkit/bin/gcc failed with status: 1
   

I have also tried some hit and trial and yet I was unable to properly compile the file.

Exploring RuDy

I find Rudy as a more suitable way to create Ruby native extensions.

When creating a Ruby native extension in C, we have a set of APIs provided by ruby.h that helps in creating a Module or a Class and the methods are nested to it.

For example, rb_define_module helps in creating a module and rb_define_class_under is used to create a class nested in a module. rb_define_method and rb_define_singleton_method are used to create methods. The variables are used as VALUE which are passed around between Ruby frontend and C backend.

A simple code snippet.

// calculater.c
void Init_extension() {
  Calculator = rb_define_module("Calculator");              // Calculator module
  IO = rb_define_class_under(Calculator, "IO", rb_cObject); // Calculator::IO class
  rb_define_alloc_func(IO, io_alloc);                       // memory allocator for C structs
  rb_define_method(IO, "initialize", io_init, -1);          // Calculator::IO#new constructor
  rb_define_method(IO, "get_stream",io_get_stream,1 );      // Calculator::IO#get_stream method
}

static VALUE io_init(int argc, VALUE* argv, VALUE self){
  // code...
  return self;
}

static VALUE io_get_stream(VALUE self, VALUE some_val){
  return Qtrue;
}

After the C code with the module, classes and methods are in place, we compile it to get a shared object extension.so. Then a Ruby file calculater.rb can just load that extension.so as:

# calculater.rb
require 'calculater.so'

This seems simple but as the size of application grows, the extension becomes tough to manage.

I believe D to be a superior C and provides modern programming paradigm and the LDC compiler is very promising. So, it would be great if we could write the native extension in D and get a shared library. Also, LDC has an incredible garbage collecter that would be very helpful for scientific computing Ruby libraries.

To write the extension in D, the ruby.h bindings could be created in D using dstep or bcd.

RuDy doesn’t have complete support for LDC and I am still getting my head around using RuDy to create Ruby native extensions. Here is a nice blog post about RuDy project.

Conclusion

Calling D from Ruby would for sure prove to be an efficient way of writing Ruby native extensions and use D as a better choice over C. In the meanwhile, porting faster-lmm-d is a work in progress and I will share further updates when it is ready.

The progress of Ruby port of faster-lmm-d can be tracked at Bio-faster_lmm_d and would become a part of BioRuby project. The latest progress can be tracked here.

Test suite

I have used minitest for writing the tests for ArrayFire-rb.

To assert if two arrays are equal, I implemented Af_Array#== method. I used af_eq api for implementing ==. However af_eq method checks for exact matches and thus doesn’t work if the values are correct upto a certain number of decimal places. So, I can’t use == to test the output of Trigonometric methods , for-example, Af_Array#sin. I implemented Af_Array#approx_equal that checks if the values are correct upto three decimal places.

This is how arith_test.rb looks like:

class ArrayFire::ArithTest < Minitest::Test

  def setup
    @a = ArrayFire::Af_Array.new 2, [2,2],[1,2,3,4]
    @b = ArrayFire::Af_Array.new 2, [2,2],[2,4,6,8]
    @elements = [10, -11, 48, 21, 65, 0, 1, -7, 112]
    @af_array =  ArrayFire::Af_Array.new 2, [3,3], @elements
  end

  def test_addition
    c = ArrayFire::Af_Array.new 2, [2,2],[3,6,9,12]
    assert_equal c, @a + @b
  end

  def test_subtraction
    assert_equal @a, @b - @a
  end

  def test_multiplication
    c = ArrayFire::Af_Array.new 2, [2,2],[2,8,18,32]
    assert_equal c, @b * @a
  end

  def test_division
    c = ArrayFire::Af_Array.new 2, [2,2],[2,2,2,2]
    assert_equal c, @b / @a
  end

  [:sin, :cos, :tan, :sinh, :cosh, :tanh].each do |method|
    define_method("test_#{method}") do
      x = @elements.map{ |e| Math.send(method, e) }
      res_arr =  ArrayFire::Af_Array.new 2, [3,3], x
      assert res_arr.approx_equal @af_array.send method
    end
  end
end

The implementation of == and approx_equal are as follows:

rb_define_method(Af_Array, "==",(METHOD)arf_eqeq,1);
rb_define_method(Af_Array, "approx_equal",(METHOD)arf_eqeq_approx,1);

static VALUE arf_eqeq(VALUE left_val, VALUE right_val) {
  afstruct* left;
  afstruct* right;
  afstruct* result = ALLOC(afstruct);
  Data_Get_Struct(left_val, afstruct, left);
  Data_Get_Struct(right_val, afstruct, right);
  af_eq(&result->carray,  left->carray, right->carray, true);

  dim_t count;
  af_get_elements(&count, result->carray);
  bool* data = (bool*)malloc(count * sizeof(bool));
  af_get_data_ptr(data, result->carray);

  for (dim_t index = 0; index < count; index++){
    if(!data[index]){
      return Qfalse;
    }
  }
  return Qtrue;
}


static VALUE arf_eqeq_approx(VALUE left_val, VALUE right_val) {

  afstruct* left;
  afstruct* right;

  dim_t left_count;
  dim_t right_count;

  Data_Get_Struct(left_val, afstruct, left);
  Data_Get_Struct(right_val, afstruct, right);

  af_get_elements(&left_count, left->carray);
  af_get_elements(&right_count, right->carray);

  if(left_count != right_count){return Qfalse;}

  float* left_arr = (float*)malloc(left_count * sizeof(float));
  af_get_data_ptr(left_arr, left->carray);

  float* right_arr = (float*)malloc(left_count * sizeof(float));
  af_get_data_ptr(right_arr, right->carray);

  for (dim_t index = 0; index < left_count; index++){
    float diff = left_arr[index] - right_arr[index];
    if(diff < 0){diff *= -1;}
    if(diff > 1e-3){
      return Qfalse;
    }
  }
  return Qtrue;
}