Merging cooccurrence files: processed 0 lines. Unable to open file overflow_1021.bin.

[summerZXH]

$ build/cooccur -memory 4.0 -vocab-file vocab.txt -verbose 2 -window-size 15 < ./train_data/data.txt > cooccurrence.bin COUNTING COOCCURRENCES window size: 15 context: symmetric max product: 13752509 overflow length: 38028356 Reading vocab from file "vocab.txt"...loaded 7459765 words. Building lookup table...table contains 215584655 elements. Processed 7713292825 tokens. Writing cooccurrences to disk............1141 files in total. Merging cooccurrence files: processed 0 lines.Unable to open file overflow_1021.bin.

when I train glove with big dataset (46G) , I met the problem, anyone knows why?

[ousou]

This seems to be the same error as mentioned in pull request #138 . The issue is probably that there are too many files open at the same time, and can be fixed by increasing the amount of allowed files, for instance: ulimit -n 2048. See the PR for more info.

[jairajrouth]

Hello Guys,
I am bit new in this GloVe topic. I am interested to see how this project works. I am using an online editor to run this project. After running the demo.sh script i can see the build folder is created and text8 and vocab.txt files are been created. I was next interested to see the cooccur tool. I ran below command and get this message, I was expecting a file with cooccurance of words from the vocab.txt which is passed. Am i doing anything wrong with the command here ? Please let me know.
All suggestions are welcome.

@ousou & @summerZXH As you guys were also working on it do you have any idea what i am doing wrong ?

Thanks and Regards.
Jai

./build/cooccur -vocab-file vocab.txt <------- Command
COUNTING COOCCURRENCES <------- Message
window size: 15
context: symmetric
max product: 10485784
overflow length: 28521267
Reading vocab from file "vocab.txt"...loaded 71290 words.
Building lookup table...table contains 75253375 elements.
Processing token: 0