Export clips.tsv with user_id column

[kdavis-mozilla]

The current clips.tsv does not have a "user_id" column. However, for various reasons, one is needed for the final export. The directory of the wav can not be used as the value of a "user_id" column as it's not properly hashed.

[kdavis-mozilla]

@Gregoor Have you had a chance to create an export with a properly hashed "user_id" column?

[Gregoor]

I've added a user column to the tsv, it's not hashed though. Could we maybe do this as part of the postprocessing? We would also need to throw out the clip paths, once we got them from S3 (as they also contain the user id).

For hashing we could do this:

import os, hashlib, binascii
salt = os.urandom(23)
hashed = binascii.hexlify(hashlib.pbkdf2_hmac('sha256', b'<client_id>', salt, 100000))
print(hashed)

[kdavis-mozilla]

Two points come to mind:

• First a nit, could the column be called user_id ?
• Second, what are we shipping for alpha users? clips.tsv or the output of this CLI?

[Gregoor]

1. Will change for next export. Was just going with the naming we've been using in CV
2. My thinking would've been the output. And maybe an example tsv (with random user-ids) if they want to work on the CorporaCreator

[kdavis-mozilla]

I'll make the code use client_id.

[kdavis-mozilla]

It's best if the users, without asking for audio, get the entire clips.tsv.