Skip to content

Commit 08553e1

Browse files
rwightmanrwightman
committed
Merge branch 'master' into attention
2 parents fa38f24 + f098fda commit 08553e1

File tree

4 files changed

+226
-23
lines changed

4 files changed

+226
-23
lines changed

‎README.md

Lines changed: 44 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22

33
## What's New
44

5+
### Feb 12, 2020
6+
* Add EfficientNet-L2 and B0-B7 NoisyStudent weights ported from [Tensorflow TPU](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet)
7+
58
### Feb 6, 2020
69
* Add RandAugment trained EfficientNet-ES (EdgeTPU-Small) weights with 78.1 top-1. Trained by [Andrew Lavin](https://github.com/andravin) (see Training section for hparams)
710

@@ -98,15 +101,16 @@ Included models:
98101
* DPN (from [myself](https://github.com/rwightman/pytorch-dpn-pretrained))
99102
* DPN-68, DPN-68b, DPN-92, DPN-98, DPN-131, DPN-107
100103
* EfficientNet (from my standalone [GenEfficientNet](https://github.com/rwightman/gen-efficientnet-pytorch)) - A generic model that implements many of the efficient models that utilize similar DepthwiseSeparable and InvertedResidual blocks
101-
* EfficientNet AdvProp (B0-B8) (https://arxiv.org/abs/1911.09665) -- TF weights ported
102-
* EfficientNet (B0-B7) (https://arxiv.org/abs/1905.11946) -- TF weights ported, B0-B2 finetuned PyTorch
103-
* EfficientNet-EdgeTPU (S, M, L) (https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html) --TF weights ported
104-
* MixNet (https://arxiv.org/abs/1907.09595) -- TF weights ported, PyTorch finetuned (S, M, L) or trained models (XL)
105-
* MNASNet B1, A1 (Squeeze-Excite), and Small (https://arxiv.org/abs/1807.11626) -- trained in PyTorch
104+
* EfficientNet NoisyStudent (B0-B7, L2) (https://arxiv.org/abs/1911.04252)
105+
* EfficientNet AdvProp (B0-B8) (https://arxiv.org/abs/1911.09665)
106+
* EfficientNet (B0-B7) (https://arxiv.org/abs/1905.11946)
107+
* EfficientNet-EdgeTPU (S, M, L) (https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html)
108+
* MixNet (https://arxiv.org/abs/1907.09595)
109+
* MNASNet B1, A1 (Squeeze-Excite), and Small (https://arxiv.org/abs/1807.11626)
106110
* MobileNet-V2 (https://arxiv.org/abs/1801.04381)
107-
* FBNet-C (https://arxiv.org/abs/1812.03443) -- trained in PyTorch
108-
* Single-Path NAS (https://arxiv.org/abs/1904.02877) -- pixel1 variant
109-
* MobileNet-V3 (https://arxiv.org/abs/1905.02244) -- pretrained PyTorch model, official TF weights ported
111+
* FBNet-C (https://arxiv.org/abs/1812.03443)
112+
* Single-Path NAS (https://arxiv.org/abs/1904.02877)
113+
* MobileNet-V3 (https://arxiv.org/abs/1905.02244)
110114
* HRNet
111115
* code from https://github.com/HRNet/HRNet-Image-Classification, paper https://arxiv.org/abs/1908.07919
112116
* SelecSLS
@@ -178,30 +182,48 @@ For the models below, the model code and weight porting from Tensorflow or MXNet
178182

179183
| Model | Prec@1 (Err) | Prec@5 (Err) | Param # | Image Scaling | Image Size |
180184
|---|---|---|---|---|---|
185+
| tf_efficientnet_l2_ns *tfp | 88.352 (11.648) | 98.652 (1.348) | 480 | bicubic | 800 |
186+
| tf_efficientnet_l2_ns | TBD | TBD | 480 | bicubic | 800 |
187+
| tf_efficientnet_l2_ns_475 | 88.234 (11.766) | 98.546 (1.454)f | 480 | bicubic | 475 |
188+
| tf_efficientnet_l2_ns_475 *tfp | 88.172 (11.828) | 98.566 (1.434) | 480 | bicubic | 475 |
189+
| tf_efficientnet_b7_ns *tfp | 86.844 (13.156) | 98.084 (1.916) | 66.35 | bicubic | 600 |
190+
| tf_efficientnet_b7_ns | 86.840 (13.160) | 98.094 (1.906) | 66.35 | bicubic | 600 |
191+
| tf_efficientnet_b6_ns | 86.452 (13.548) | 97.882 (2.118) | 43.04 | bicubic | 528 |
192+
| tf_efficientnet_b6_ns *tfp | 86.444 (13.556) | 97.880 (2.120) | 43.04 | bicubic | 528 |
193+
| tf_efficientnet_b5_ns *tfp | 86.064 (13.936) | 97.746 (2.254) | 30.39 | bicubic | 456 |
194+
| tf_efficientnet_b5_ns | 86.088 (13.912) | 97.752 (2.248) | 30.39 | bicubic | 456 |
181195
| tf_efficientnet_b8_ap *tfp | 85.436 (14.564) | 97.272 (2.728) | 87.4 | bicubic | 672 |
182-
| tf_efficientnet_b8 *tfp | 85.384 (14.616) | 97.394 (2.606) | 87.4 | bicubic | 672 |
183-
| tf_efficientnet_b8 | 85.37 (14.63) | 97.39 (2.61) | 87.4 | bicubic | 672 |
196+
| tf_efficientnet_b8 *tfp | 85.384 (14.616) | 97.394 (2.606) | 87.4 | bicubic | 672 |
197+
| tf_efficientnet_b8 | 85.370 (14.630) | 97.390 (2.610) | 87.4 | bicubic | 672 |
184198
| tf_efficientnet_b8_ap | 85.368 (14.632) | 97.294 (2.706) | 87.4 | bicubic | 672 |
199+
| tf_efficientnet_b4_ns *tfp | 85.298 (14.702) | 97.504 (2.496) | 19.34 | bicubic | 380 |
200+
| tf_efficientnet_b4_ns | 85.162 (14.838) | 97.470 (2.530) | 19.34 | bicubic | 380 |
185201
| tf_efficientnet_b7_ap *tfp | 85.154 (14.846) | 97.244 (2.756) | 66.35 | bicubic | 600 |
186202
| tf_efficientnet_b7_ap | 85.118 (14.882) | 97.252 (2.748) | 66.35 | bicubic | 600 |
187-
| tf_efficientnet_b7 *tfp | 84.940 (15.060) | 97.214 (2.786) | 66.35 | bicubic | 600 |
188-
| tf_efficientnet_b7 | 84.932 (15.068) | 97.208 (2.792) | 66.35 | bicubic | 600 |
203+
| tf_efficientnet_b7 *tfp | 84.940 (15.060) | 97.214 (2.786) | 66.35 | bicubic | 600 |
204+
| tf_efficientnet_b7 | 84.932 (15.068) | 97.208 (2.792) | 66.35 | bicubic | 600 |
189205
| tf_efficientnet_b6_ap | 84.786 (15.214) | 97.138 (2.862) | 43.04 | bicubic | 528 |
190206
| tf_efficientnet_b6_ap *tfp | 84.760 (15.240) | 97.124 (2.876) | 43.04 | bicubic | 528 |
191207
| tf_efficientnet_b5_ap *tfp | 84.276 (15.724) | 96.932 (3.068) | 30.39 | bicubic | 456 |
192208
| tf_efficientnet_b5_ap | 84.254 (15.746) | 96.976 (3.024) | 30.39 | bicubic | 456 |
193-
| tf_efficientnet_b6 *tfp | 84.140 (15.860) | 96.852 (3.148) | 43.04 | bicubic | 528 |
194-
| tf_efficientnet_b6 | 84.110 (15.890) | 96.886 (3.114) | 43.04 | bicubic | 528 |
195-
| tf_efficientnet_b5 *tfp | 83.822 (16.178) | 96.756 (3.244) | 30.39 | bicubic | 456 |
196-
| tf_efficientnet_b5 | 83.812 (16.188) | 96.748 (3.252) | 30.39 | bicubic | 456 |
209+
| tf_efficientnet_b6 *tfp | 84.140 (15.860) | 96.852 (3.148) | 43.04 | bicubic | 528 |
210+
| tf_efficientnet_b6 | 84.110 (15.890) | 96.886 (3.114) | 43.04 | bicubic | 528 |
211+
| tf_efficientnet_b3_ns *tfp | 84.054 (15.946) | 96.918 (3.082) | 12.23 | bicubic | 300 |
212+
| tf_efficientnet_b3_ns | 84.048 (15.952) | 96.910 (3.090) | 12.23 | bicubic | 300 |
213+
| tf_efficientnet_b5 *tfp | 83.822 (16.178) | 96.756 (3.244) | 30.39 | bicubic | 456 |
214+
| tf_efficientnet_b5 | 83.812 (16.188) | 96.748 (3.252) | 30.39 | bicubic | 456 |
197215
| tf_efficientnet_b4_ap *tfp | 83.278 (16.722) | 96.376 (3.624) | 19.34 | bicubic | 380 |
198216
| tf_efficientnet_b4_ap | 83.248 (16.752) | 96.388 (3.612) | 19.34 | bicubic | 380 |
199-
| tf_efficientnet_b4 | 83.022 (16.978) | 96.300 (3.700) | 19.34 | bicubic | 380 |
200-
| tf_efficientnet_b4 *tfp | 82.948 (17.052) | 96.308 (3.692) | 19.34 | bicubic | 380 |
217+
| tf_efficientnet_b4 | 83.022 (16.978) | 96.300 (3.700) | 19.34 | bicubic | 380 |
218+
| tf_efficientnet_b4 *tfp | 82.948 (17.052) | 96.308 (3.692) | 19.34 | bicubic | 380 |
219+
| tf_efficientnet_b2_ns *tfp | 82.436 (17.564) | 96.268 (3.732) | 9.11 | bicubic | 260 |
220+
| tf_efficientnet_b2_ns | 82.380 (17.620) | 96.248 (3.752) | 9.11 | bicubic | 260 |
201221
| tf_efficientnet_b3_ap *tfp | 81.882 (18.118) | 95.662 (4.338) | 12.23 | bicubic | 300 |
202222
| tf_efficientnet_b3_ap | 81.828 (18.172) | 95.624 (4.376) | 12.23 | bicubic | 300 |
203-
| tf_efficientnet_b3 | 81.636 (18.364) | 95.718 (4.282) | 12.23 | bicubic | 300 |
204-
| tf_efficientnet_b3 *tfp | 81.576 (18.424) | 95.662 (4.338) | 12.23 | bicubic | 300 |
223+
| tf_efficientnet_b3 | 81.636 (18.364) | 95.718 (4.282) | 12.23 | bicubic | 300 |
224+
| tf_efficientnet_b3 *tfp | 81.576 (18.424) | 95.662 (4.338) | 12.23 | bicubic | 300 |
225+
| tf_efficientnet_b1_ns *tfp | 81.514 (18.486) | 95.776 (4.224) | 7.79 | bicubic | 240 |
226+
| tf_efficientnet_b1_ns | 81.388 (18.612) | 95.738 (4.262) | 7.79 | bicubic | 240 |
205227
| gluon_senet154 | 81.224 (18.776) | 95.356 (4.644) | 115.09 | bicubic | 224 |
206228
| gluon_resnet152_v1s | 81.012 (18.988) | 95.416 (4.584) | 60.32 | bicubic | 224 |
207229
| gluon_seresnext101_32x4d | 80.902 (19.098) | 95.294 (4.706) | 48.96 | bicubic | 224 |
@@ -233,10 +255,12 @@ For the models below, the model code and weight porting from Tensorflow or MXNet
233255
| tf_efficientnet_em *tfp | 78.958 (21.042) | 94.458 (5.542) | 6.90 | bicubic | 240 |
234256
| tf_mixnet_l *tfp | 78.846 (21.154) | 94.212 (5.788) | 7.33 | bilinear | 224 |
235257
| tf_efficientnet_b1 | 78.826 (21.174) | 94.198 (5.802) | 7.79 | bicubic | 240 |
258+
| tf_efficientnet_b0_ns *tfp | 78.806 (21.194) | 94.496 (5.504) | 5.29 | bicubic | 224 |
236259
| gluon_inception_v3 | 78.804 (21.196) | 94.380 (5.620) | 27.16M | bicubic | 299 |
237260
| tf_mixnet_l | 78.770 (21.230) | 94.004 (5.996) | 7.33 | bicubic | 224 |
238261
| tf_efficientnet_em | 78.742 (21.258) | 94.332 (5.668) | 6.90 | bicubic | 240 |
239262
| gluon_resnet50_v1s | 78.712 (21.288) | 94.242 (5.758) | 25.68 | bicubic | 224 |
263+
| tf_efficientnet_b0_ns | 78.658 (21.342) | 94.376 (5.624) | 5.29 | bicubic | 224 |
240264
| tf_efficientnet_cc_b0_8e *tfp | 78.314 (21.686) | 93.790 (6.210) | 24.0 | bicubic | 224 |
241265
| gluon_resnet50_v1c | 78.010 (21.990) | 93.988 (6.012) | 25.58 | bicubic | 224 |
242266
| tf_efficientnet_cc_b0_8e | 77.908 (22.092) | 93.656 (6.344) | 24.0 | bicubic | 224 |

‎sotabench.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ def _entry(model_name, paper_model_name, paper_arxiv_id, batch_size=BATCH_SIZE,
116116
model_desc='Block cfg of SE-ResNeXt-34 w/ Bottleneck, deep tiered narrow stem, and avg-pool in downsample layers.'),
117117
_entry('spnasnet_100', 'Single-Path NAS', '1904.02877',
118118
model_desc='Trained in PyTorch with SGD, cosine LR decay'),
119+
119120
_entry('tf_efficientnet_b0', 'EfficientNet-B0 (AutoAugment)', '1905.11946',
120121
model_desc='Ported from official Google AI Tensorflow weights'),
121122
_entry('tf_efficientnet_b1', 'EfficientNet-B1 (AutoAugment)', '1905.11946',
@@ -134,6 +135,7 @@ def _entry(model_name, paper_model_name, paper_arxiv_id, batch_size=BATCH_SIZE,
134135
model_desc='Ported from official Google AI Tensorflow weights'),
135136
_entry('tf_efficientnet_b8', 'EfficientNet-B8 (RandAugment)', '1905.11946', batch_size=BATCH_SIZE // 8,
136137
model_desc='Ported from official Google AI Tensorflow weights'),
138+
137139
_entry('tf_efficientnet_b0_ap', 'EfficientNet-B0 (AdvProp)', '1911.09665',
138140
model_desc='Ported from official Google AI Tensorflow weights'),
139141
_entry('tf_efficientnet_b1_ap', 'EfficientNet-B1 (AdvProp)', '1911.09665',
@@ -152,6 +154,28 @@ def _entry(model_name, paper_model_name, paper_arxiv_id, batch_size=BATCH_SIZE,
152154
model_desc='Ported from official Google AI Tensorflow weights'),
153155
_entry('tf_efficientnet_b8_ap', 'EfficientNet-B8 (AdvProp)', '1911.09665', batch_size=BATCH_SIZE // 8,
154156
model_desc='Ported from official Google AI Tensorflow weights'),
157+
158+
_entry('tf_efficientnet_b0_ns', 'EfficientNet-B0 (NoisyStudent)', '1911.04252',
159+
model_desc='Ported from official Google AI Tensorflow weights'),
160+
_entry('tf_efficientnet_b1_ns', 'EfficientNet-B1 (NoisyStudent)', '1911.04252',
161+
model_desc='Ported from official Google AI Tensorflow weights'),
162+
_entry('tf_efficientnet_b2_ns', 'EfficientNet-B2 (NoisyStudent)', '1911.04252',
163+
model_desc='Ported from official Google AI Tensorflow weights'),
164+
_entry('tf_efficientnet_b3_ns', 'EfficientNet-B3 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 2,
165+
model_desc='Ported from official Google AI Tensorflow weights'),
166+
_entry('tf_efficientnet_b4_ns', 'EfficientNet-B4 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 2,
167+
model_desc='Ported from official Google AI Tensorflow weights'),
168+
_entry('tf_efficientnet_b5_ns', 'EfficientNet-B5 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 4,
169+
model_desc='Ported from official Google AI Tensorflow weights'),
170+
_entry('tf_efficientnet_b6_ns', 'EfficientNet-B6 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 8,
171+
model_desc='Ported from official Google AI Tensorflow weights'),
172+
_entry('tf_efficientnet_b7_ns', 'EfficientNet-B7 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 8,
173+
model_desc='Ported from official Google AI Tensorflow weights'),
174+
_entry('tf_efficientnet_l2_ns_475', 'EfficientNet-L2 475 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 16,
175+
model_desc='Ported from official Google AI Tensorflow weights'),
176+
_entry('tf_efficientnet_l2_ns', 'EfficientNet-L2 (NoisyStudent)', '1911.04252', batch_size=BATCH_SIZE // 64,
177+
model_desc='Ported from official Google AI Tensorflow weights'),
178+
155179
_entry('tf_efficientnet_cc_b0_4e', 'EfficientNet-CondConv-B0 4 experts', '1904.04971',
156180
model_desc='Ported from official Google AI Tensorflow weights'),
157181
_entry('tf_efficientnet_cc_b0_8e', 'EfficientNet-CondConv-B0 8 experts', '1904.04971',

0 commit comments

Comments
 (0)