Collecting data of kanji usage frequencies from Twitter Streaming API.

See POST statuses/filter for details.

Usage

1. git clone https://github.com/THIS/REPO.git
2. npm install
3. cp tokens.json.dist tokens.json
4. Fill in your tokens in tokens.json (create a new app)
5. node collect-data.js - a new stream is started right away, but data is not being saved automatically!
6. Enter help to see a list of available commands
7. Enter save to store data in a file (kanji.json in root directory)

Status message

{ state: 'RUNNING',
  percentFull: 0,
  uniqueKanji: 3666,
  totalKanji: 2300350 }

• state - if 'PAUSED', data is not being collected
• percentFull - how full is a stream queue, see description of stall_warnings parameter
• uniqueKanji - how many distinct kanji were seen so far
• totalKanji - how many kanji were seen in total, including repetitions