Errors when trying to pruning a YOLOv5 model, and use ModelSpeedup.speedup_model() to export to one model pt. #3790

ichejun · 2021-04-20T06:12:58Z

ichejun
Apr 20, 2021

Environment:

NNI version:2.1
NNI mode (local|remote|pai):local
Client OS:
Server OS (for remote mode only):
Python version:3.7.5
PyTorch/TensorFlow version:PyTorch 1.7.1
Is conda/virtualenv/venv used?:no
Is running in Docker?:no

When I tried to use prunner do the yolov5 model compression (FPGMPruner used), the mask.pt and model.pt were generated.
Then, trying to use the ModelSpeedup.speedup_model() to export the two pt( mask.pt and model.pt ) in one model pt.

Q1. error log shows that the 'Concat' node not support?

log:

  File "detect_nni.py", line 447, in <module>
    detect()
  File "detect_nni.py", line 279, in detect
    m_speedup.speedup_model()
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 183, in speedup_model
    self.infer_modules_masks()
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 140, in infer_modules_masks
    self.infer_module_mask(module_name, None, mask=mask)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 92, in infer_module_mask
    .format(m_type, module_name))
RuntimeError: Has not supported infering output shape from input shape for module/function: `Concat`, model.16

Q2. I have tried to modify the source code in nni, to deal with the 'Concat' as 'atten::cat', then errorlog showed that 'Upsample' node not support, which I tried deal as 'MaxPool2d' node. At last, some error showed in the 'view' node fuction ('view_inshape'), seems that it only support output shape size as 2.(assert len(shape['out_shape']) == 2) But in the 'Detect' fuction in yolov5, the tensor were viewed as 3,5 or 6 dimentions.(https://github.com/ultralytics/yolov5/blob/238583b7d5c19029920d56c417c406c829569c75/models/yolo.py#L24).I am wondering is there any better way to fix this?

Above all, I am trying to find a solution to export the two pt( mask.pt and model.pt ) in one model pt, is there any better way?
Many thanks for your answering～

Answered by ping-Huang

Apr 22, 2021

@ichejun
Q1:
In yolov5, you should add Concat, upsample and SiLU to infer_from_inshape and replace_module.

In Cancat node, you should modify the graph_utils.py and compressor.py

Q2:
I recommend that you pruning the yolov5 without "Detect layer".
After you producing the speedup model, you should add Detect layer after the speedup model.

View full answer

ping-Huang · 2021-04-22T01:46:39Z

ping-Huang
Apr 22, 2021

@ichejun
Q1:
In yolov5, you should add Concat, upsample and SiLU to infer_from_inshape and replace_module.

In Cancat node, you should modify the graph_utils.py and compressor.py

Q2:
I recommend that you pruning the yolov5 without "Detect layer".
After you producing the speedup model, you should add Detect layer after the speedup model.

0 replies

ichejun · 2021-04-22T07:03:48Z

ichejun
Apr 22, 2021
Author

@ichejun
Q1:
In yolov5, you should add Concat, upsample and SiLU to infer_from_inshape and replace_module.

In Cancat node, you should modify the graph_utils.py and compressor.py

Q2:
I recommend that you pruning the yolov5 without "Detect layer".
After you producing the speedup model, you should add Detect layer after the speedup model.

Thanks for your reply. I am going to modify the files as soon as possible.
As for the Q2, I am wondering if there is any recomended way to ‘pruning the yolov5 without "Detect layer"’？Could you please provide some suggestions?
Thank you very much!

0 replies

ichejun · 2021-04-25T06:52:36Z

ichejun
Apr 25, 2021
Author

Hi, @ping-Huang
After I completed the modification of 6 files and adjusted the prune configuration of "Detect layer", I was able to quantify the yolov5 model successfully.Thanks!
For related modifications, please refer to https://github.com/ichejun/nni/commits/master. The 6 commits on "Commits on Apr 25, 2021" are adjustments to the corresponding 6 files.

However, after the adjustment, I encountered 3 questions and troubles:
Q1. After completing the adjustment and modification of the above 6 files, although the yolov5 model can be successfully compressed and speed_up. But when I try to run the test/ut/sdk/test_pruners.py unit test from nni, I get an error, the error log is as follows,

/usr/local/lib/python3.7/site-packages/scipy/fft/__init__.py:97: DeprecationWarning: The module numpy.dual is deprecated.  Instead of using dual, use the functions directly from numpy or scipy.
  from numpy.dual import register_func
/usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
/usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
/usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
/usr/local/lib/python3.7/site-packages/scipy/special/orthogonal.py:81: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int,
/usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
..EE
======================================================================
ERROR: test_pruners (__main__.PrunerTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_pruners.py", line 294, in test_pruners
    pruners_test(bias=True)
  File "test_pruners.py", line 235, in pruners_test
    pruner.compress()
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/auto_compress_pruner.py", line 225, in compress
    m_speedup = ModelSpeedup(self._model_to_prune, self._dummy_input, masks_file, device)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 38, in __init__
    self.torch_graph = build_module_graph(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 24, in build_module_graph
    return TorchModuleGraph(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 271, in __init__
    super().__init__(model, dummy_input, traced_model)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 66, in __init__
    self._trace(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 75, in _trace
    print(model.model[-1].export)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'Model' object has no attribute 'model'

======================================================================
ERROR: test_pruners_no_bias (__main__.PrunerTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_pruners.py", line 297, in test_pruners_no_bias
    pruners_test(bias=False)
  File "test_pruners.py", line 235, in pruners_test
    pruner.compress()
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/auto_compress_pruner.py", line 225, in compress
    m_speedup = ModelSpeedup(self._model_to_prune, self._dummy_input, masks_file, device)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/speedup/compressor.py", line 38, in __init__
    self.torch_graph = build_module_graph(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 24, in build_module_graph
    return TorchModuleGraph(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 271, in __init__
    super().__init__(model, dummy_input, traced_model)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 66, in __init__
    self._trace(model, dummy_input)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/common/graph_utils.py", line 75, in _trace
    print(model.model[-1].export)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'Model' object has no attribute 'model'

----------------------------------------------------------------------
Ran 4 tests in 439.166s

FAILED (errors=2)

I think some bug has been introduced by my modification(), could you please take a look at my modifications. Is there anything wrong?
The modified logs were:
ichejun@82a2226
ichejun@b07c76f
ichejun@761f6b3
ichejun@6e8b031
ichejun@4420f35
ichejun@3fc20a1

Q2.About pruning the yolov5 without "Detect layer". My current approach is to put the convolution starting from the 3 detection heads as the exclude parameter in the prune configuration parameters:

    config_list = [{
        'sparsity': 0.03,
        'op_types': ['Conv2d']
    }, {
        'op_names':['model.24.m.0','model.24.m.1','model.24.m.2'],
        'exclude': True
    }]

It works! But I am wondering if there is a more general method or suggestion, otherwise, I have to manually find the relevant convolution of the detection head every time, and manually write the configuration to try?

Q3. I encountered a problem when I tried another pruner algorithm, namely AGPPruner.
The default configuration can run through, but when I try to add the exclude configuration, like Q2. There is a problem with configuration parsing.
The code:

    config_list = [{
        'initial_sparsity': 0.,
        'final_sparsity': 0.05,
        'start_epoch': 0,
        'end_epoch': 10,
        'frequency': 1,
        'op_types': ['Conv2d']
    }, {
        'op_names':['model.24.m.0','model.24.m.1','model.24.m.2'],
        'exclude': True
    }]
    pruner = AGPPruner(model, config_list, optimizer, pruning_algorithm='fpgm')
    pruner.compress()

The error log:

Traceback (most recent call last):
  File "train_nni_up_agp.py", line 980, in <module>
    train(hyp, opt, device, tb_writer)
  File "train_nni_up_agp.py", line 537, in train
    pruner = AGPPruner(model, config_list, optimizer, pruning_algorithm='fpgm')
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/agp.py", line 44, in __init__
    super().__init__(model, config_list, optimizer)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/compressor.py", line 322, in __init__
    super().__init__(model, config_list, optimizer)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/compressor.py", line 44, in __init__
    self.validate_config(model, config_list)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/agp.py", line 70, in validate_config
    schema.validate(config_list)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/utils/config_validation.py", line 53, in validate
    self.compressor_schema.validate(data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 357, in validate
    return type(data)(o.validate(d) for d in data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 357, in <genexpr>
    return type(data)(o.validate(d) for d in data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 167, in validate
    [self._error.format(data) if self._error else None] + errors,
schema.SchemaError: Or(And({'initial_sparsity': And(<class 'float'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74b90>), 'final_sparsity': And(<class 'float'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74c20>), 'start_epoch': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74cb0>), 'end_epoch': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74d40>), 'frequency': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74dd0>), Optional('op_types'): And([<class 'str'>], <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74e60>), Optional('op_names'): And([<class 'str'>], <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74ef0>)}, <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74f80>)) did not validate {'op_names': ['model.24.m.0', 'model.24.m.1', 'model.24.m.2'], 'exclude': True}
Missing keys: 'end_epoch', 'final_sparsity', 'frequency', 'initial_sparsity', 'start_epoch'

Besides the parsing problem, can AGPPruner with pruning_algorithm, like 'fpgm', work in dependency_aware mode? It seems that there is no corresponding api interface?
Thanks a lot!

1 reply

mvpzhangqiu Aug 29, 2022

hello, can you show me how to quantify your yolov5 model? thanks!

ping-Huang · 2021-04-25T09:05:45Z

ping-Huang
Apr 25, 2021

Hi, @ichejun

Q1: The second parameter of ModelSpeedup() should be the original model with training weights.
model_speedup.py

Q2: If you don't skip the Detect layer or exclude the exclude the parameters, you may encounter two problems.

First, the filter size of last three layer may be the (class_number+5)*anchor_number. If you don't exclude these layers, the pruning model output may not fit the class number of original model.

Second, pruning yolov5 with Detect layer will cause the node difference between pruning model and original model in ModelSpeedup() step.

Q3: In our previous experiment, some pruners do not implement the dependency_aware.
We only successfully speedup model of yolov5 using L1Filter Pruner.

0 replies

ichejun · 2021-04-25T10:18:35Z

ichejun
Apr 25, 2021
Author

@ping-Huang Thank you very much for your prompt reply, it helps a lot. As for questions 1 and 2, I will refer to your suggestions and try as soon as possible. For the question 3 about agp pruner, I am going to raise an issue to follow up. Thanks~

0 replies

sharoseali · 2021-05-11T19:55:37Z

sharoseali
May 11, 2021

@ping-Huang Thank you very much for your prompt reply, it helps a lot. As for questions 1 and 2, I will refer to your suggestions and try as soon as possible. For the question 3 about agp pruner, I am going to raise an issue to follow up. Thanks~

Hi @ichejun i want to do same work ( speed up model with nni ) .. did u get success.. Can u share some early steps to put me step in . In my case i m working on yolov3 spp.. and will soon start on v5 .. any suggestions..

0 replies

ichejun · 2021-05-17T01:37:20Z

ichejun
May 17, 2021
Author

@ping-Huang Thank you very much for your prompt reply, it helps a lot. As for questions 1 and 2, I will refer to your suggestions and try as soon as possible. For the question 3 about agp pruner, I am going to raise an issue to follow up. Thanks~

Hi @ichejun i want to do same work ( speed up model with nni ) .. did u get success.. Can u share some early steps to put me step in . In my case i m working on yolov3 spp.. and will soon start on v5 .. any suggestions..

Hi @sharoseali , as for yolo v5, you may follow #3548 (comment). The comments for Q1 and Q2 may help in some one-shot pruner. I have not tried yolov3 spp yet...:)

1 reply

hygxy Jul 8, 2022

Hi, I am also interested in the pruning of yolov5. Would you mind if you can open source the scripts you´ve used to prune yolov5 except the "Detect" module?

sharoseali · 2021-05-23T18:09:07Z

sharoseali
May 23, 2021

@ping-Huang Thank you very much for your prompt reply, it helps a lot. As for questions 1 and 2, I will refer to your suggestions and try as soon as possible. For the question 3 about agp pruner, I am going to raise an issue to follow up. Thanks~

Hi @ichejun i want to do same work ( speed up model with nni ) .. did u get success.. Can u share some early steps to put me step in . In my case i m working on yolov3 spp.. and will soon start on v5 .. any suggestions..

Hi @sharoseali , as for yolo v5, you may follow #3548 (comment). The comments for Q1 and Q2 may help in some one-shot pruner. I have not tried yolov3 spp yet...:)

@ichejun For Q2 where i have to make changes , that you wrote and for pruning what exactely parameters and py files I have to run

Q2.About pruning the yolov5 without "Detect layer". My current approach is to put the convolution starting from the 3 detection heads as the exclude parameter in the prune configuration parameters: these ones below ?

config_list = [{ 'sparsity': 0.03, 'op_types': ['Conv2d'] }, { 'op_names':['model.24.m.0','model.24.m.1','model.24.m.2'], 'exclude': True }]

0 replies

woshituobaye · 2021-10-20T11:56:20Z

woshituobaye
Oct 20, 2021

hi 您好！请问您跑通了吗？是的话，可以开源代码吗？我最近也在用NNI对yolov5剪枝，也报了很多错。

0 replies

Errors when trying to pruning a YOLOv5 model, and use ModelSpeedup.speedup_model() to export to one model pt. #3790

Uh oh!

Replies: 9 comments · 2 replies

Uh oh!

Uh oh!

Uh oh!

ichejun Apr 22, 2021 Author

Uh oh!

ichejun Apr 25, 2021 Author

Uh oh!

Uh oh!

Uh oh!

ichejun Apr 25, 2021 Author

Uh oh!

Uh oh!

Uh oh!

ichejun May 17, 2021 Author

Uh oh!

Uh oh!

Uh oh!

Replies: 9 comments 2 replies

ichejun
Apr 22, 2021
Author

ichejun
Apr 25, 2021
Author

ichejun
Apr 25, 2021
Author

ichejun
May 17, 2021
Author