The raw datasets used in the paper can be downloaded via:
20NEWS:
http://qwone.com/~jason/20Newsgroups/
Reuters:
https://www.nltk.org/book/ch02.html
Wikitext-103:
https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
We use the same preprocessing steps as described in Miao et al. (2016), Wu et al. (2020), Nan et al. (2019) to obtain the vocabulary of 20NEWS, Reuters and Wikitext-103 respectively.
The model can be trained on Reuters by running:
python HNTM.py
The best hyperparameter values on the validation set are as follows:
decay_rate = 0.03
discrete_rate = 0.1
balance_rate = 0.01
manifold_rate = 0.3
learning_rate = 5e-4
batch_size = 64
n_epoch = 100
hidden_size = 256
tensorflow==1.12.0
numpy==1.14.5