Neural Network Compression

Since 2017, the international standardization group of ISO/IEC MPEG focusses on efficient compression methods for neural networks. For this, a new Call for Proposals on Neural Network Compression and Representation (NNR) was issued in 2018, which finally led to a new standard in 2021. This NNR Standard (ISO/IEC 15938-17) provides neural networks to be compressed to less than 5% of their original size, without degrading inference capabilities, i.e. maintaining accuracies of the original uncompressed nets. NNR includes a set of preprocessing methods, i.e. Pruning, Sparsification or Low-Rank Decomposition for neural nets. The main coding engine includes efficient quantization methods and DeepCABAC as arithmetic coding tool.

For this standard, our Efficient Deep Learning Group jointly cooperates with the Video Coding Technologies Group on the international standardization of neural network compression and representation (NNR) within the current ISO/IEC MPEG Video Standardization Group (ISO/IEC JTC1/SC29/WG04). For this we contributed significant parts of the core coding engine, including the arithmetic coding DeepCABAC and a number of quantization tools, like uniform nearest-neighbor and dependent quantization. Further tools include arithmetic coding optimization, batch norm folding and local scaling. Also we developed and contributed a number of high-level syntax methods, like tensor-wise decoding methods, random access, improved codebook signaling and parallel processing with enhanced CABAC optimization, and finally specific syntax for bitstream definition with external frameworks, namely TensorFlow, PyTorch, ONNX and NNEF.

Scientific Publications

  1. P. Haase, H. Schwarz, H. Kirchhoffer, S. Wiedemann, T. Marinc, A. Marban, K. Müller, W. Samek, D. Marpe, T. Wiegand. “Dependent Scalar Quantization for Neural Network Compression.” International Conference on Image Processing 2020.
  2. S. Wiedemann et al., "DeepCABAC: A universal compression algorithm for deep neural networks," in IEEE Journal of Selected Topics in Signal Processing, doi: 10.1109/JSTSP.2020.2969554.
  3. S. Wiedemann, H. Kirchhoffer, S. Matlage, P. Haase, A. Marban, T. Marin, D. Neumann, A. Osman, D. Marpe, H. Schwarz, T. Wiegand, and W. Samek, “DeepCABAC: context-adaptive binary arithmetic coding for deep neural network compression,” in Proceedings of the 36th International Conference on Machine Learning (ICML). Long Beach, California, US, June 2019.

Standard Contributions

  1. H. Kirchhoffer, K. Müller, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] Harmonization of codebook quantization and NNR_PT_BLOCK compressed data payload type”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55575, Online (Rennes, FR), Oct. 2020.
  2. B. Choi, W. Wang, H. Kirchhoffer, K. Müller, E. Aksu, H. Tavakoli, F. Cricri, “[NNR] Compact design of NNR unit header”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55540, Online (Rennes, FR), Oct. 2020.
  3. P. Haase, D. Becking, H. Kirchhoffer, K. Müller, W. Samek, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] CE4 method 19: Results on QP optimizations”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55073, Online (Rennes, FR), Oct. 2020.
  4. H. Kirchhoffer, K. Müller, W. Samek, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] Committee draft cleanups, improvements, and bug fixes”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55068, Online (Rennes, FR), Oct. 2020.
  5. D. Becking, H. Kirchhoffer, K. Müller, W. Samek, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] HLS for additional framework support”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55067, Online (Rennes, FR), Oct. 2020.
  6. H. Kirchhoffer, P. Haase, K. Müller, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] HLS for parallel CABAC decoding including parameter optimizations”, ISO/IEC JTC1/SC29/WG04, MPEG Video Coding m55066, Online (Rennes, FR), Oct. 2020.
  7. H. Kirchhoffer, K. Müller, H. Schwarz, D. Marpe, T. Wiegand, “[NNR]: Low-rank decomposition syntax for NNR compressed payload type NNR_PT_BLOCK_LS on top of m54395”, ISO/IEC JTC1/SC29/WG11, MPEG20/m54806, Online (Geneva, CH), June/July 2020.
  8. P. Haase, H. Kirchhoffer, K. Müller, H. Schwarz, D. Marpe, T. Wiegand, “[NNR]: HLS adaptation for integer codebook representation”, ISO/IEC JTC1/SC29/WG11, MPEG20/m54397, Online (Geneva, CH), June/July 2020.
  9. H. Kirchhoffer, P. Haase, S. Wiedemann, T. Marinc, K. Müller, W. Samek, H. Schwarz, D. Marpe, T. Wiegand, “[NNR] CE4: Results on parameter optimization for DeepCABAC (method 18) and local scaling adaptation (method 19)”, ISO/IEC JTC1/SC29/WG11, MPEG20/m54395, Online (Geneva, CH), June/July 2020.
  10. K. Müller, H. Kirchhoffer, T. Marinc, S. Wiedemann, H. Schwarz, W. Samek, D. Marpe, T. Wiegand, “[NNR] Additional HLS and decoding process specification for Neural Network Compression (ISO/IEC 15938-17)”, ISO/IEC JTC1/SC29/WG11, MPEG20/m53518, Online (Alpbach, AT), April 2020.
  11. S. Wiedemann, P. Haase, H. Kirchhoffer, T. Marinc, K. Müller, W. Samek, H. Schwarz, D. Marpe, T. Wiegand, “[NNR] CE2-CE3-related: Local parameter scaling”, ISO/IEC JTC1/SC29/WG11, MPEG20/m53517, Online (Alpbach, AT), April 2020.
  12. S. Wiedemann, P. Haase, H. Kirchhoffer, T. Marinc, K. Müller, W. Samek, H. Schwarz, D. Marpe, T. Wiegand, “[NNR] CE2: Results on importance-weighted quantization”, ISO/IEC JTC1/SC29/WG11, MPEG20/m53516, Online (Alpbach, AT), April 2020.
  13. P. Haase, H. Kirchhoffer, S. Wiedemann, T. Marinc, K. Müller, W. Samek, H. Schwarz, D. Marpe, T. Wiegand, “[NNR] CE3-related: Parameter-Optimization for DeepCABAC”, ISO/IEC JTC1/SC29/WG11, MPEG20/m53515, Online (Alpbach, AT), April 2020.
  14. P. Haase, H. Kirchhoffer, S. Wiedemann, T. Marinc, K. Müller, W. Samek, H. Schwarz, D. Marpe, T. Wiegand, “[NNR] CE2-CE3: Results on dependent scalar quantization”, ISO/IEC JTC1/SC29/WG11, MPEG20/m53514, Online (Alpbach, AT), April 2020.
  15. F. Sattler, D. Neumann, S. Wiedemann, K. Mueller, W. Samek, D. Marpe, T. Wiegand, “[NNR] CE2-related: Dependent scalar quantization for neural network parameter approximation”, ISO/IEC JTC1/SC29/WG11, MPEG20/m52375, Brussels, BE, January 2020.
  16. P. Haase, K. Schwarz, H. Kirchhoffer, S. Wiedemann, S. Matlage, T. Marinc, A. Marban, K. Müller, W. Samek, D. Marpe, T. Wiegand, “[NNR] CE2-related: Dependent scalar quantization for neural network parameter approximation”, ISO/IEC JTC1/SC29/WG11, MPEG20/m52358, Brussels, BE, January 2020.
  17. K. Müller, R. Skupin, Y. Sanchez, S. Wiedemann, H. Kirchhoffer, H. Schwarz, W. Samek, D. Marpe, T. Wiegand, “[NNR] Basic High-Level Syntax for Neural Network Compression (ISO/IEC 15938-17, i.e. MPEG-7 part 17)”, ISO/IEC JTC1/SC29/WG11, MPEG20/m52352, Brussels, BE, January 2020.
  18. S. Wiedemann, H. Kirchhoffer, W. Samek, K. Müller, D. Marpe, H. Schwarz, T. Wiegand, “[NNR] Proposal of python interfaces for an NNR test model”, ISO/IEC JTC1/SC29/WG11, MPEG19/m49867, Gothenburg, SE, July 2019.