Deep Compression/Acceleration(模型压缩加速论文汇总)

论文速递 Admin ⋅ 于 1个月前 ⋅ 1151 阅读

写在前面:一直以来,极市得到了许许多多开发者的关注和支持,极市的成长离不开各位开发者的见证,为此我们特开设极市开发者祝愿帖,期待听到您真实的心声和建议~φ (> ω<*) :

极市开发者祝愿活动 (有奖回馈)

深度学习(Deep Learning)因其计算复杂度或参数冗余,在一些场景和设备上限制了相应的模型部署,需要借助模型压缩、优化加速、异构计算等方法突破瓶颈。其中模型压缩算法能够有效降低参数冗余,从而减少存储占用、通信带宽和计算复杂度,有助于深度学习的应用部署,本文汇总了近几年模型压缩方面的相关研究paper,更新中,欢迎关注~
作者:Mars_WH
原文:https://blog.csdn.net/hw5226349/article/details/84888416

结构structure

Searching for MobileNetV3
arxiv:https://arxiv.org/abs/1905.02244v1
中文解读:重磅!MobileNetV3 来了!

[BMVC2018] IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
arxiv:https://arxiv.org/abs/1806.00178
github:https://github.com/homles11/IGCV3


[CVPR2018] IGCV2: Interleaved Structured Sparse Convolutional Neural Networks

arxiv:https://arxiv.org/abs/1804.06202


[CVPR2018] MobileNetV2: Inverted Residuals and Linear Bottlenecks

arxiv:https://arxiv.org/abs/1801.04381
github:https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet


[ECCV2018] ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design

arxiv:https://arxiv.org/abs/1807.11164

量化quantization

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
intro:二值网络
arxiv:https://arxiv.org/abs/1602.02830
github: https://github.com/MatthieuCourbariaux/BinaryNet
https://github.com/itayhubara/BinaryNet


[ACM2017] FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
intro:二值网络
pdf:http://www.idi.ntnu.no/~yamanu/2017-fpga-finn-preprint.pdf
github:https://github.com/Xilinx/FINN


[CVPR2016] DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
intro:低bit位
arxiv:https://arxiv.org/abs/1606.06160
github:https://github.com/tensorpack/tensorpack/tree/master/examples/DoReFa-Net


[CVPR2016] XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
intro:darknet团队出品
arxiv:https://arxiv.org/abs/1603.05279
github:https://github.com/allenai/XNOR-Net


[CVPR2016] Ternary Weight Networks

arxiv:https://arxiv.org/abs/1605.04711
github:https://github.com/fengfu-chris/caffe-twns


Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Google出品
arxiv:https://arxiv.org/abs/1712.05877
github:https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize


[ACM2017] Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
intro:QNNs
arxiv:https://arxiv.org/abs/1609.07061
github:https://github.com/peisuke/qnn


Two-Step Quantization for Low-bit Neural Networks

paper:http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Two-Step_Quantization_for_CVPR_2018_paper.pdf

剪枝pruning

通道裁剪channel pruning

[NIPS2018] Discrimination-aware Channel Pruning for Deep Neural Networks

arxiv:https://arxiv.org/abs/1810.11809
github:https://github.com/Tencent/PocketFlow支持DisChnPrunedLearner


[ICCV2017] Channel Pruning for Accelerating Very Deep Neural Networks
intro:Lasso回归
arxiv:https://arxiv.org/abs/1707.06168
github:https://github.com/yihui-he/channel-pruning


[ECCV2018] AMC: AutoML for Model Compression and Acceleration on Mobile Devices
intro:自动学习优化
arxiv:https://arxiv.org/abs/1802.03494
https://www.jiqizhixin.com/articles/AutoML-for-Model-Compression-and-Acceleration-on-Mobile-Devices论文翻译
github:https://github.com/Tencent/PocketFlow


[ICCV2017] Learning Efficient Convolutional Networks through Network Slimming
intro:Zhuang Liu
arxiv:https://arxiv.org/abs/1708.06519
github:https://github.com/Eric-mingjie/network-slimming
https://github.com/foolwood/pytorch-slimming


[ICLR2018] Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers

arxiv:https://arxiv.org/abs/1802.00124
github:[PyTorch]https://github.com/jack-willturner/batchnorm-pruning
[TensorFlow]https://github.com/bobye/batchnorm_prune


[CVPR2017] NISP: Pruning Networks using Neuron Importance Score Propagation

arxiv:https://arxiv.org/abs/1711.05908


[ICCV2017] ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression

web:http://lamda.nju.edu.cn/luojh/project/ThiNet_ICCV17/ThiNet_ICCV17_CN.html
github:https://github.com/Roll920/ThiNet
https://github.com/Roll920/ThiNet_Code

稀疏sparsity

SBNet: Sparse Blocks Network for Fast Inference
intro: Uber
arxiv:https://arxiv.org/abs/1801.02108
github:https://github.com/uber/sbnet


To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression
intro:稀疏
arxiv:https://arxiv.org/abs/1710.01878
github:https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/model_pruning


Submanifold Sparse Convolutional Networks
intro:Facebook
arxiv:https://arxiv.org/abs/1706.01307
github:https://github.com/facebookresearch/SparseConvNet

融合fusion


蒸馏distillation

[NIPS2014] Distilling the Knowledge in a Neural Network
intro:Hinton出品
arxiv:https://arxiv.org/abs/1503.02531
github:https://github.com/peterliht/knowledge-distillation-pytorch

综合comprehensive

[ICLR2016] Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
intro:开创先河
arxiv:https://arxiv.org/abs/1510.00149
github:https://github.com/songhan


Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification
intro:实验比较多,适合工程化
arxiv:https://arxiv.org/abs/1709.02929





ps.本周四(3月28日)晚,极市邀请了澳大利亚阿德莱德大学博士生王鑫龙,为我们分享联合点云分割中的实例和语义(CVPR2019),欢迎参与直播,与嘉宾一起互动交流~
详情请关注:CVPR2019 专题直播 | 王鑫龙:联合分割点云中的实例和语义

回复数量: 0
暂无回复~
您需要登陆以后才能留下评论!