Add support for all datasets of the GLUE benchmark

[vcm2114]

🚀 Feature

Add support for all 8 remaining datasets (SST-2 is already supported) of the GLUE benchmark: CoLA, MRPC, QQP, STS-B, MNLI, QNLI, RTE, WNLI.

Motivation

In itself adding support for all GLUE datasets has a lot of value: GLUE is one of the most widely used benchmark in the NLP community. Furthermore, our planned effort to develop a cohesive API for Multi-Task Learning requires us to enhance our suite our datasets, starting with GLUE.

Additional context

We have already created a streamlined dataset API, with a consistent use of DataPipes for dataset download and load operations, as well as a testing methodology relying on mock data (see #1493). This feature will only require to add support for these 8 datasets following that methodology.

Datasets

• CoLA Add support for CoLA dataset with unit tests #1711
• MRPC Add support for MRPC dataset with unit tests #1712
• QQP Add support for QQP dataset with unit tests #1713
• STS-B Add support for STS-B dataset with unit tests #1714
• MNLI Add support for MNLI dataset with unit tests #1715
• QNLI Add support for QNLI dataset with unit tests #1717
• RTE Add support for RTE dataset with unit tests #1721
• WNLI Add support for WNLI dataset with unit tests #1724

[parmeet]

Thanks @VirgileHlav for creating this issue! Just a general comment regarding dataset implementation: let's avoid usage of lambda functions, please refer issue #1716 for details.

[vcm2114]

Thanks @parmeet!

let's avoid usage of lambda functions

Acknowledged, I will make updates to the PRs.

[Nayef211]

Thanks for creating this issue to keep track of the new datasets we are adding! One suggestion from my end would be to update our documentation for each new dataset we add here.

[parmeet]

Let's also make sure we add shuffle and sharding filters: Reference Issue: #1727. Reference PR #1729

[vcm2114]

Solved lint issues and added documentation + shuffle/sharding on all PRs.

[parmeet]

closing issue, since all the datasets have been added :)