DeformConv operation from Onnx to TFLite

[jlamperez]

Issue Type

Feature Request

OS

Linux

onnx2tf version number

1.16.2

onnx version number

1.14.1

onnxruntime version number

1.15.1

onnxsim (onnx_simplifier) version number

0.4.33

tensorflow version number

2.13.0

Download URL for ONNX

dcnv2.zip

Parameter Replacement JSON

N/A

Description

1. Personal
2. What

user@80ee9d66194e:/workdir$ onnx2tf -i dcnv2.onnx   

Model optimizing started ============================================================
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃            ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Constant   │ 4              │ 4                │
│ DeformConv │ 1              │ 1                │
│ Model Size │ 876.1KiB       │ 872.3KiB         │
└────────────┴────────────────┴──────────────────┘

Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃            ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Constant   │ 4              │ 4                │
│ DeformConv │ 1              │ 1                │
│ Model Size │ 872.3KiB       │ 872.3KiB         │
└────────────┴────────────────┴──────────────────┘

Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃            ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Constant   │ 4              │ 4                │
│ DeformConv │ 1              │ 1                │
│ Model Size │ 872.3KiB       │ 872.3KiB         │
└────────────┴────────────────┴──────────────────┘

Model optimizing complete!

Automatic generation of each OP name started ========================================
Automatic generation of each OP name complete!

Model loaded ========================================================================

Model conversion started ============================================================
INFO: input_op_name: input.1 shape: [1, 3, 64, 64] dtype: float32
WARNING: The optimization process for shape estimation is skipped because it contains OPs that cannot be inferred by the standard onnxruntime.
WARNING: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for DeformConv(19) node with name '/DeformConv'
ERROR: DeformConv OP is not yet implemented.

4. I would like to have DeformConv conversion from ONNX to TFLite.
5. Resources:

• https://github.com/onnx/onnx/blob/main/onnx/reference/ops/op_deform_conv.py
• Deformable Convolutional Op Support tensorflow/addons#179
• Fix Deformable Convolutional Op Support #179 tensorflow/addons#1129
• Add Modulated Deformable Convolution layer tensorflow/addons#2196

[PINTO0309]

I have been considering implementing this for half a year, but it is hard to verify that the output of the converted DCNv2 OP is correct because onnxruntime does not yet support DeformConv. Since onnx2tf is a personal tool developed almost exclusively by myself, I will try to implement it appropriately. However, I think it will take very long time to verify it.

https://github.com/onnx/onnx/blob/main/docs/Changelog.md#deformconv-19

https://github.com/masamitsu-murase/deform_conv2d_onnx_exporter

https://github.com/PeymanTahghighi/DeformConvLSTM/blob/master/DeformableConv2D.py

https://arxiv.org/pdf/1811.11168.pdf

dcnv2_19.onnx.zip

sit4onnx -if dcnv2_19.onnx -oep cpu

onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented:
[ONNXRuntimeError] : 9 : NOT_IMPLEMENTED :
Could not find an implementation for DeformConv(19) node with name '/DeformConv'

Ref: microsoft/onnxruntime#20810