Dict feature is non-nullable while nested dict feature is

[polinaeterna]

When i try to create a Dataset object with None values inside a dict column, like this:

from datasets import Dataset, Features, Value

Dataset.from_dict(
    {
        "dict": [{"a": 0, "b": 0}, None],
    }, features=Features(
        {"dict": {"a": Value("int16"), "b": Value("int16")}}
    )
)

i get ValueError: Got None but expected a dictionary instead.

At the same time, having None in nested dict feature works, for example, this doesn't throw any errors:

from datasets import Dataset, Features, Value, Sequence

dataset = Dataset.from_dict(
    {
        "list_dict": [[{"a": 0, "b": 0}], None],
        "sequence_dict": [[{"a": 0, "b": 0}], None],
    }, features=Features({
        "list_dict": [{"a": Value("int16"), "b": Value("int16")}],
        "sequence_dict": Sequence({"a": Value("int16"), "b": Value("int16")}),
    })
)

Other types of features also seem to be nullable (but I haven't checked all of them).

Version of datasets is the latest atm (2.18.0)

Is this an expected behavior or a bug?

[lhoestq]

It looks like a bug, by default every feature should be nullable.

[mariosasko]

I've linked a PR with a fix :)

[polinaeterna]

@mariosasko awesome thank you!