Execution time speed compared to ArrayFire C++

[altaybrusan]

Hi,
I have developed a Arrayfire image processing algorithm in C++. The execution time was around 78-80 ms. Then, I re-implemented the algorithm via the .NET wrapper. Now the execution time is around 120 ms. Is it normal?

PS: I also noticed the .net wrapper leverage 50 of the GPU however the original one used 100 percents
One more thing, the .net wrapped version has no "warm up" time however the original C++ version needs about 1 min to execute for the first time. I am not sure yet if its normal or not!

[shehzan10]

I don't think any of us have benchmarked the performance of the .NET wrapper compared to C++. But I would certainly believe the C++ Wrapper would be faster.

One of the main reasons for this is that the .NET wrapper is using the unified backend to load DLLs, where as the native libraries can be called directly and do not spend time in loading DLLs (not sure if you are counting this towards your time).

Ideally, to benchmark, you should run the main part of your code in a loop and then take the average time.

Once the wrapper is feature complete, we can look into optimizing it further.

[pavanky]

@altaybrusan can you show us the output of af::info() from C++ and the equivalent from .net ?

[altaybrusan]

First, the project is winform and I call the filter function AFTER I call setBackend. The out put of Wrapper is:
Quadro_600
CUDA
v7.5
2.1

same as the out put of the C++.
[0] Quadro 600, 1024 MB, CUDA Compute 2.1

Do you need any further details or something else?
At the end thanks Arrayfire team
especially you Pavanky

[altaybrusan]

Indeed the problem that I have is "How to send a bitmap object to arrayfire" There is a loadMem function but I coud not find a way to send images into.
Do you have any idea about how I can directly send a bitmap object to native C++ functions?
@shehzan10 I had tested it in that way, however still the same results

[altaybrusan]

I also monitor the NVidiea control panel>Manage GPU Utilization> GPU Utilization Graph.
The original C++ functions utilize GPU completely, however the wrapper reaches just the 50%.
Do you have any similar experiences or suggestions?

[royalstream]

@altaybrusan is the image loading process part of the loop? What code is inside the timed loop and what code is outside? I just want to make sure we're all on the same page.
PS: I don't know if this helps but I can also add the loadImage function to the wrapper tomorrow.

[altaybrusan]

@pavanky
Q- is the image loading process part of the loop?

No the image loading is done outside
Q-What code is inside the timed loop and what code is outside?

Here is the code *_## inside *_the filter function

// The logic of the stack is as following:
// Incoming  images are stored in a image stack with fixed depth , the stack depth (i.e. stackDepth) is a // class-level parameter.
// Then the stack content is convolved with a kernel i.e. smoothingKernel, the output is a smooth image
// The smooth image then convolved with laplacian kernel to get sharp image
// Sharp image is then trimmed for improve contrast

 public ArrayFire.Array FluoroFilter(ArrayFire.Array image)
        {
            try
            {

                // STEP ONE: Find the location of the new image in stack and put it in 
                imageIndext = imageIndext % stackDepth;
                imageStack[ArrayFire.Util.Span, ArrayFire.Util.Span, imageIndext] = image;

               // To check the performance start a timer
                Stopwatch sw = new Stopwatch();
                sw.Start();

                // STEP TWO: convolve with smoothingKernel to get smooth image
                smoothImage = ArrayFire.SignalProcessing.convolve3(imageStack, smoothingKernel);
                // STEP THREE: convolve with laplacian to get sharp image.
                sharpImage = ArrayFire.SignalProcessing.convolve(smoothImage, laplacian)[Util.Span, Util.Span, 3];

                // STEP FOUR: improve the contrast
                result= ArrayFire.Arith.Pow(sharpImage/255,gam)*Cgam;                

                // STEP Five: shift he index one more
                imageIndext++;

                 // Report the total elapsed time    
                ArrayFire.Device.Sync();
                sw.Stop();
                Debug.WriteLine(sw.ElapsedMilliseconds.ToString());
                sw.Reset();
                // FINALLY: rescale the image from [0,1] to [0,255] to turn into a bitmap    
                result = ArrayFire.Arith.Floor(result * 256);
                return result;
            }
            catch (Exception e)
            {
                Debug.WriteLine(e);
                throw;
            }
        }

        #endregion
    }

Here is the code *_## Outside *_the filter function

// Make an object from filter class and set its parameters
  FluoroscopeFilter filter = new FluoroscopeFilter();
   const int depth = 4;
   const int wLen = 5;
   const int camLen = 1024; 
filter.SetFilterParameters(wLen, depth, 1024);

// At the moment the images are saved on disk and just loaded.
// However, at the final release the images would come from a camera!!!
// Steps to get images loaded from disk to a arrayfire array object are:
// 1- make a buffer.
// 2- load image to a byte[] 
// 3- turn byte array to int[]
// 4- make an arrya outof int[]
// I think this approach is so niive and I was thinking if there is any other better way?

 int[,] imageBuffer;

// There are 100 images, the file path  to them are loaded in List<string> imageFilesPath
// For each image fill do the filtering operations
 for (int idx = 0; idx < imagesFiles.Count; idx++)
     {
            // Load File gets a string to an image file location then turn it to a 2D int[]
            imageBuffer= FileOperations.LoadFile(imagesFiles[idx]);

              // create array from 2D int[,]
             ArrayFire.Array arr = Data.CreateArray(imageBuffer);

             //Apply filter over array object
             arr = filter.FluoroFilter(arr);

             // the output of the filter function is of type double before making an image turn it to int-type             
             ArrayFire.Array arr2 = Data.Cast<int>(arr);
             int[,] imageBuffer2 = Data.GetData2D<int>(arr2);

   // Images are black and white at the final step turn them int[,] (2D int array) to byte[] (1D byte array)
             byte[] buffer = new byte[imageBuffer2 .GetLength(0) * imageBuffer2 .GetLength(1)];

             for (int index = 0; index < imageBuffer2 .GetLength(0) * imageBuffer2 .GetLength(1); index ++) 
                {

                    int a = (int)(index /  imageBuffer2 .GetLength(0));
                    int b = index %  imageBuffer2 .GetLength(1);

                    buffer[index ] = (byte) bimg[a, b];
                }

                // At the final step make bitmap!
                var columns = 1024;
                var rows = 1024;
                var stride = columns;                                                
                var im = new Bitmap(columns, rows, stride,
                         PixelFormat.Format8bppIndexed,
                         Marshal.UnsafeAddrOfPinnedArrayElement(buffer, 0));                

                // Set the palette for gray shades
                ColorPalette pal = im.Palette;
                for (int i = 0; i < pal.Entries.Length; i++)
                    pal.Entries[i] = Color.FromArgb(i, i, i);
                im.Palette = pal;

                // Display image on the screen!
                pictureBox.Image = im;
                pictureBox.Refresh();

[altaybrusan]

@pavanky As you see the process of sending an image to arrayfire from c# is really cumbersome!
To send:
Bitmap-> int[,]
To get:
array-> double[,] ->int[,] -> byte[,]-> bitmap!

it would be good to short this whole process

[altaybrusan]

@pavanky
In the case you may want to know how do I load bitmap objects
` public static int[,] LoadFile(string path)
{
try
{
image = new Bitmap(path);
bmpData = image.LockBits(new Rectangle(new Point(), image.Size),
ImageLockMode.ReadOnly,
PixelFormat.Format8bppIndexed);

            // initiate temporary variables.
            int width = bmpData.Stride;
            int hight = bmpData.Height;                
            buffer = new byte[width * hight];

            //copy bitmap data into buffer.
            Marshal.Copy(bmpData.Scan0, buffer, 0, width * hight);
            image.UnlockBits(bmpData);

            // convert byte array to integer array.
            ibuffer = buffer.Select((x) => (int)x).ToArray();


            // convert one dimension array to two dimension.     
            ibuffer2D = new int[width, hight];                

            for (int i = 0; i < width; i++)
            {
                for (int j = 0; j < hight; j++)
                {
                    ibuffer2D[i, j] = buffer[i * width + j];
                }
            }

            return ibuffer2D;

        }
        catch(Exception ex)
        {
            throw new Exception("Invalid argument", ex);
        }

    }`

[pavanky]

@altaybrusan I am not really familiar with windows or .Net. I was just trying to make sure the versions being picked are consistent with each other.

I'll let @royalstream (who is doing a great job developing this project) and @shehzan10 (who's helping him out) take care of this.

[shehzan10]

@altaybrusan is there any reason why you are going from C++ -> .NET? Most people would start with the wrapper (because they are most comfortable with a wrapper language) and then move to C++ to get to higher performance.

[altaybrusan]

@royalstream @shehzan10 The main body of the project is in C# (major packages, components). Indeed, this project is going to be a real application of ArrayFire in medical image processing (Fluoroscopy). I think instead of returning back to C++, If I find a way to process the video/image stream using the wrapper would be a good evidence that the ArrayFire in general and its .NET wrapper specifically is applicable in other image/video processing.
As I said the bottleneck is streaming between managed .Net and unmanaged C++ environment.
Do you have any idea/ suggestion on how can I negotiate to solve this bottleneck? This would be a proof of the C++ ArrayFire/ .NET arrayFire for video/image processing.
(One solution that I m wondering is to put the filter in C++ ArrayFire. When a new frame is received save it as a bitmap file on HDD then send the file path to the filter. There Filter load the image internally using load command process it !!! )

[royalstream]

I noticed the code inside the loop has many operations, some of them involving slicing, some of them involving convolutions, etc. In theory all the .Net to C++ marshalling shouldn't be heavy because all we're passing around are pointers to af_array objects and small objects like af_seq.
Maybe you can take the milliseconds at different positions in the loop and do the same on C++, hopefully one of the operations is to blame for most of the additional delay. It could be the slicing (all those temporary af_seq objects that I need to create in .Net) but it could be something else.
I would also make sure the Release build has all the possible optimizations enabled.

[pavanky]

@royalstream

ArrayFire performs copies on assignment if the LHS array of the assignment operation has more than one reference. Because of RAII in C++ this happens fairly rarely and copies are performed only occasionally.

For garbage collected languages such as C#, any temporary references are not cleared until the garbage collector is called. As far as arrayfire knows, those references are still in play. So it performs copies every time you do slice + assignment operations.

One hack around this would be to call the garbage collector in the wrapper right before any time af_assign_* functions are called. This might slow down other parts of C# program, but it would speed up the arrayfire parts..

[pavanky]

@unbornchikken brought up this issue a few days ago when talking about the arrayfire-js wrapper.