G-API Alumni Forum

[dmatveev]

Moving our bi-weekly notes to Github Discussions

Old notes are available here.

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions

[dmatveev]

G-API Alumni Forum -- 2023 May 23

Meeting minutes

Attendees

• @dmatveev
• Ruslan
• @TolyaTalamanov (synced offline)

Agenda

OpenCV 4.8 Release

@dmatveev - selected PRs to merge:

• Anna’s old PRs: #21685, #21797
• ONNX & Python-related changes: #23597, #20579
• OpenVINO 2.0 backend (partial): #23595
• <Possible fix in IE/OV1.0 backend - Anatoliy TBD>
• <Fixes to existing reported problems (MKL, etc) - Dmitry TBD>
• Optional (if time permits to complete): Multi-stream #23334

[dmatveev]

G-API Alumni Forum -- 2023 Jun 06

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

1. @dmatveev - INTC
2. @TolyaTalamanov - INTC

Agenda

• Discussed OpenCV 4.8 release status, all we wanted is merged
• Discussed OpenVINO 2.0 backend status. The foundational changes are merged. What's remaining: infer-list, infer-2. infer-roi may not be required.

OpenCV 4.8 Release

• Anna’s old PRs: GAPI Fluid: SIMD SSE41 for Resize F32C1. opencv/opencv#21685, GAPI Fluid SIMD:Add support of new several types for the Merge3. opencv/opencv#21797
• ONNX & Python-related changes: G-API: Integration branch for ONNX & Python-related changes opencv/opencv#23597, [G-API] Draft: Wrap GComputation::compile into python opencv/opencv#20579
• OpenVINO 2.0 backend (partial): [G-API] Implement OpenVINO 2.0 backend opencv/opencv#23595
• <Possible fix in IE/OV1.0 backend - Anatoliy TBD>
• <Fixes to existing reported problems (MKL, etc) - Dmitry TBD>
• Optional (if time permits to complete): Multi-stream WIP: G-API: Multi-stream pipeline execution opencv/opencv#23334

Last-minute stuff to add:

• infer-list in OV2.0 backend
• fix for OV1.0 backend w.r.t. input images (continuation of WIP: [G-API] IE Backend: Update the condition for applying the resize preprocessing opencv/opencv#23668)

GSoC - 2023

• @TolyaTalamanov - discussing the roadmap with the contributor.
• @dmatveev - meeting with contributor later today, will check for updates. So far repo is created at https://github.com/g-api-org/tutorials-py

Personal updates

Future agenda

• GSoC
• Plugins for heavy dependencies
• ONNX RT with OV UEP and Tensor RT - device (EP) selection.
• DNN backend - e.g. https://github.com/opencv/opencv_zoo/tree/master/models/handpose_estimation_mediapipe
• Tracking[?]

[dmatveev]

G-API Alumni Forum -- 2023 Aug 15

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• Dmitry - INTC
• Anatoliy - INTC
• Ruslan - HWEI

Agenda

• Updated 5.0 roadmap (added more items)
• Discussed moving preprocessing out of inference backends. May be not that trivial now. A lot of corner cases. Probably a new OCV DNN backend could be written "as is" now to avoid blocking with the implementation
• TBB executor - complex and never complete, may be replaced with something smaller but what actually works.

Personal updates

n/a

Future agenda

Dmitry to turn 5.0 roadmap into Discussions.

[dmatveev]

G-API Alumni Forum -- 2023 Oct 10

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• Dmitry - INTC

Agenda

Personal updates

Dmitry

• Fixing ADE build (once again): Don't override CMAKE_CXX_STANDARD set by parent projects opencv/ade#40 + related PRs
• Introduced CV over OV (early stage PoC but looks promising): G-API: Introduce some classic CV ops via OV backend opencv/opencv#24379
• Updated 5.0 roadmap sections
  • To fix: https://github.com/opencv/opencv/wiki/G-API-Ideas-towards-OpenCV-5.0#what-wed-like-to-fix
  • To remove: https://github.com/opencv/opencv/wiki/G-API-Ideas-towards-OpenCV-5.0#what-wed-like-to-remove

Future agenda

TBC

[dmatveev]

G-API Alumni Forum -- 2023 Nov 7

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev , @TolyaTalamanov

Agenda

PR Status for OpenCV 4.9

Confident PRs:

• Ported VAS OT into OpenCV G-API opencv/opencv#24224

  • WIP, PR is red (needs test update)

• G-API: Advanced device selection for ONNX DirectML Execution Provider opencv/opencv#24060

  • Stuck mainly with CMake-level changes.

RIsky PRs:

• G-API: Support CoreML Execution Providers for ONNXRT Backend opencv/opencv#24068

  • Changes are done but unable to test. No suitable machine available. ⚠️Merge conflicts in place

• WIP: G-API: Multi-stream pipeline execution opencv/opencv#23334

  • API + tests only, implementation isn't there yet. Dmitry will implement as best effort during 4.9 timeframe (hopefully)

• [G-API] Draft: Wrap GComputation::compile into python opencv/opencv#20579

  • Stuck in merge 2 years ago w/o any major issues, needs to be cleaned up and merged. Added into 4.9

PRs that needed but don't exist

• Expose missing imgproc operations to G-API python opencv/opencv#22736
• Plugin Inference Engine G-API Backend opencv/opencv#23578

On plugins

• Approach via C++ / dlopen

  • pyopencv.so is loaded in Python, G-API makes lookup for libraries like OV, ONNX, VPL, etc... to dispatch calls into it
  • Individual G-API components try to load external dependencies dynamically and fail if they couldn't. The API contract is a question here.

• Approach via Python

  • Implement new backends via Python. G-API won't know about the external dependencies, they will come from the Python side as callbacks
  • Pros: any new API can be externally linked into G-API
  • Cons: performance + need to expose the API internals to allow implementing backends. Graph structure, to name a few.

Future agenda

Continue w/4.9 & 5.0 plans.

[TolyaTalamanov]

G-API Alumni Forum -- 2023 Nov 21

Meeting protocol

Meeting minutes

Attendees

• @rgarnov, @TolyaTalamanov

Agenda

PR Status for OpenCV 4.9

• G-API: Advanced device selection for ONNX DirectML Execution Provider opencv/opencv#24060

  • Merged

• G-API: Support CoreML Execution Providers for ONNXRT Backend opencv/opencv#24068

  • @rgarnov volunteered to validate changes on his machine.
  • AR: @TolyaTalamanov resolve merge conflicts and provide instructions to @rgarnov

• [G-API] Draft: Wrap GComputation::compile into python opencv/opencv#20579

  • No updates so far....

[dmatveev]

G-API Alumni Forum -- 2023 Dec 5

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev, @TolyaTalamanov

Agenda

Personal updates

• @TolyaTalamanov - trying to bump ONNX RT version to 0.16, but it requires C++14. OpenCV forces C++11: Fix path to ONNX Runtime include folder opencv/opencv#24601

  • Check if ENABLE_CXX14 (or ENABLE_C++14) option exist at CMake level.

• @smirnov-alexey fixed several issues in the IE 1.0 backend to fix OMZ demos, but we need to update our demos to the OV backend (2.0):

  • G-API: Soften the argument check in infer2 opencv/opencv#24615
  • Open: when do we deprecate the IE 1.0 API?

• Vadim contributed 0d/1d data types for DNN so several G-API tests were failed and so disabled: attempt to add 0d/1d mat support to OpenCV opencv/opencv#23473

  • @TolyaTalamanov to come up with test cases which we've worked on - and check if it affects our tooling.

New bugs:

• Several G-API sources files use old OpenVINO headers unconditionally  opencv/opencv#24431
  • @TolyaTalamanov - to follow-up

Future agenda

[TolyaTalamanov]

G-API Alumni Forum -- 2023 Dec 19

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @rgarnov, @TolyaTalamanov

Agenda

Personal updates

• @TolyaTalamanov - ONNXRT version bump is merged

Future agenda

[dmatveev]

G-API Alumni Forum -- 2024 Jan 16

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev , @TolyaTalamanov
• @rgarnov

Agenda

OpenCV 4.9 is out!

• https://github.com/opencv/opencv/wiki/ChangeLog#version490
• Includes OT - closes one of our 5.0 plans! 🎉🎉🎉

Personal updates

• @TolyaTalamanov

  • A new executor G-API: Implement concurrent executor opencv/opencv#24845

• @rgarnov

  • Mentioned https://github.com/dmlc/dlpack as a new DL data structure for all frameworks

Future agenda

• @TolyaTalamanov - share drafts of infer manifesto here: https://github.com/orgs/g-api-org/discussions/14
• @TolyaTalamanov - align on the threaded executor implementation https://github.com/orgs/g-api-org/discussions/13

[TolyaTalamanov]

G-API Alumni Forum -- 2024 Jan 30

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev , @TolyaTalamanov

Agenda

Personal updates

• @TolyaTalamanov
  • A new executor G-API: Implement concurrent executor opencv/opencv#24845

Future agenda

• @TolyaTalamanov - share drafts of infer manifesto here: https://github.com/orgs/g-api-org/discussions/14
• @TolyaTalamanov - align on the threaded executor implementation https://github.com/orgs/g-api-org/discussions/13
• @TolyaTalamanov - Revive python bindings for QueueSource

[dmatveev]

G-API Alumni Forum -- 2024 Feb 13

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev , @TolyaTalamanov

Agenda

• GSoC 2024: https://github.com/opencv/opencv/wiki/GSoC_2024 - call for project ideas

• OpenCV 5.0 development started

  • OpenCV 5.0 (view)
  • Some breaking changes, Vadim's PR Extended several core functions to support new types opencv/opencv#24962

Personal updates

• @dmatveev - High-order constructs discussion is started:
  • https://github.com/orgs/g-api-org/discussions/16
  • Discussed past (previously unpublished) MTCNN experience: high-order constructs like collect<> and map<> were introduced there, and run nested GComputations as part of the larger graph (were opaque to the parent graph's executors, just like user kernels).

Future agenda

• @TolyaTalamanov
  • understand why Python's GInferInputs is not a dict.
  • propose how to update/pass-around GInferInputs in transformer-like scenarios.

[dmatveev]

G-API Alumni Forum -- 2024 Mar 12

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev, @TolyaTalamanov

Agenda

Personal updates

• @dmatveev - Merged a couple of PRs for 5.0

  • Makings tests lighter: G-API: Make test execution lighter (first attempt) opencv/opencv#25060
  • Value-initialized GMat G-API: A quick value-initialization support GMat opencv/opencv#25055 - a base prerequisite for:
    • In-graph weights (for DNN-like engines)
    • Accumulated execution (T-API-like, but for expressions)

• @TolyaTalamanov - no updates so far

  • G-API: Lower supported IE backend version opencv/opencv#25054 - Fixed test hang by lowering the IE version
    • Alexander reports hangs again - like here: https://github.com/opencv/opencv/actions/runs/8153782389/job/22285849425?pr=25149
    • Likely it is a VideoCapture problem, not G-API's - as investigated earlier by @dmatveev

• Required updates:

  • GAPI Fluid Resize:AVX2 implementation. opencv/opencv#22469 - Resize in AVX - opened by @anna-khakimova back in 2022
  • [G-API] Draft: Wrap GComputation::compile into python opencv/opencv#20579 - compile() in Python - Got conflicts, need rebase(?)

Future agenda

• 5.0 plans continued?
• DNN module for infer - @dmatveev - WIP
• Preprocessing migration - @dmatveev - WIP
• HAL prototype - @dmatveev - WIP

[dmatveev]

G-API Alumni Forum -- 2024 Apr 09

Meeting protocol

• Participation is voluntary.
• Respect your company's NDA! Don't share any roadmaps, plans, and IP.
• Discuss only OpenCV and G-API public efforts and questions.

Meeting minutes

Attendees

• @dmatveev, @TolyaTalamanov
• @rgarnov

Agenda

Incoming issue reports from OpenCV

• InferROI.TestStreamingInfer test hangs on CI in OpenVINO configuration opencv/opencv#25053 - hang in streaming-related tests. Need to prepare a VideoCapture-based reproducer (issue was reopened due to recent occurences)
• Skip InferList.TestStreamingInfer opencv/opencv#25321 - related but now drafted (need follow-up from us)
• Cross-build pipeline for Windows on ARM opencv/ci-gha-workflow#155 - cross-build issue

Personal updates

• @dmatveev - no major updates, just back from vacation
  • WIP: G-API: Multi-stream pipeline execution opencv/opencv#23334 - likely needs to be revived soon
  • A DNN-based infer - skeleton done, need to complete the implementation, no PR yet
  • G-API: Introduce some classic CV ops via OV backend opencv/opencv#24379 - OVCV may get attention soon
  • GSOC 2024 #18 - we've got a GSoC candidate, sadly we don't participate this year (timelines missed for project proposals? https://developers.google.com/open-source/gsoc/timeline)

@rgarnov - no updates, can follow-up on Anna's PR? opencv/opencv#22469

@TolyaTalamanov - no updates.

Future agenda

• @TolyaTalamanov - Proposal on loops? Continue discussion.
• @dmatveev - Concern: our Fluid kernels may require setting CPU_BASELINE manually. Currently the default one is SSE3.1 (?) and we don't do dynamic dispatch in our Fluid kernels. How to make it work out of the box?