[GPU] fix matrix_nms_ref kernel GPU memory allocation issue

[yuanxion]

Details

fixed matrix_nms_ref stage 0 kernel GPU memory allocation issue

Description of the issue

Symptom

pp_yolo model will be failed to run inference, and CL_OUT_OF_RESOUCE will be prompted when creating stage 0 matrix_nms_kernel.

Root cause

• It will try to allocate 1 * 80 * 22743 * 22742/2 * 4 = 82.7GB GPU memory (for batch 1 * classes 80 = 80 GPU work items) in the matrix_nms_ref stage 0 kernel, which exceeds the GPU's total memory size and will be failed to create matrix_nms_kernel.
• Also, the matrix_nms_kernel has large size (22743) "for loop" which takes much GPU resources.

How to fix it

• Use global memory buffer allocated by host instead of private memory.
• Use chunking for the "for loop".

The code and line that caused this issue

• big memory allocation
  

  openvino/src/plugins/intel_gpu/src/kernel_selector/cl_kernels/matrix_nms_ref.cl

  Line 192 in bbed74e

  const int matrix_size = MAX_BOXES_PER_CLASS < 3 ? 1 : (MAX_BOXES_PER_CLASS * (MAX_BOXES_PER_CLASS - 1)) >> 1;

• big size (22743) for loop
  

  openvino/src/plugins/intel_gpu/src/kernel_selector/cl_kernels/matrix_nms_ref.cl

  Line 182 in bbed74e

  for (int i = 0; i < NUM_BOXES; i++) {

Reproduction step and snapshot

• benchmark_app
  benchmark_app -inference_only false -b 1 -d GPU.0 -hint none -infer_precision f32 -m FP32/1/ov/pp-yolo.xml

Problematic graph

• N/A

Checklist

• Is it a proper fix? (not a workaround)
• Did you include test case for this fix, if necessary? Yes, testcase matrix_nms_test_inputs.get_matrix_nms_large_value_of_max_boxes_per_class is added.
• Did you review existing test that can be extended to cover this scenario? Which test did you review? No existing test can cover the issue.

Tickets:

• CVS-141140

[wilson-seok]

@yuanxion Please add func or unit test for this case.

[yuanxion]

@yuanxion Please add func or unit test for this case.

Thanks, added.

[wilson-seok]

LGTM