Feature Extraction in DNN

This repo contains suppoting demos for [1]. The features learned by DNNs are compared with theoretical results.

1. exp1.py: feature extraction for network with ideal expressive power
2. exp2.py: feature extraction for network with restricted expressive power
3. exp3.py: demo for H-score

Environments: keras 2.3.1

A more detailed illustration and Pytorch implementations can be found in this series.

Reference

[1] Xu, Xiangxiang, Shao-Lun Huang, Lizhong Zheng, and Gregory W. Wornell. 2022. "An Information Theoretic Interpretation to Deep Neural Networks" Entropy 24, no. 1: 135. https://doi.org/10.3390/e24010135