Feat(schema): add method to check if column exists

[georgesittas]

Thought something like this might be an improvement to the API

[barakalon]

I'm assuming this is intended to be used somewhere in sqlmesh?

If this is intended to be used anywhere within the optimizer, it needs to be on the abstract Schema class.

[georgesittas]

I'm assuming this is intended to be used somewhere in sqlmesh?

If this is intended to be used anywhere within the optimizer, it needs to be on the abstract Schema class.

It started out that way, but I'm not sure if I'm going to use it there after all. I just made the PR regardless because it seemed like a useful utility, since the only alternative I think would be to use get_column_type which doesn't provide a clear signal about the column missing:

>>> from sqlglot.schema import MappingSchema
>>> s = MappingSchema({"x": {"y": "int"}})
>>> import sqlglot
>>> s.get_column_type("x", sqlglot.exp.column("y"))
(DATATYPE this: Type.INT, nested: False, prefix: False)
>>> s.get_column_type("x", sqlglot.exp.column("f"))
(DATATYPE this: Type.UNKNOWN)

[barakalon]

I guess we can't just do "y" in s.column_names("x") because of normalization?

And still, if you think this will be a useful utility, we probably want it on the Schema interface. Unless you enforce a MappingSchema is used in sqlmesh?

[georgesittas]

I guess we can't just do "y" in s.column_names("x") because of normalization?

I think so, because the schema may or may not contain normalized names (not necessarily related to internal schema normalize attribute), which is why I started off with this interface instead of __contains__, i.e. to allow for extra flexibility.

And still, if you think this will be a useful utility, we probably want it on the Schema interface. Unless you enforce a MappingSchema is used in sqlmesh?

Happy to add this to the Schema interface, it seems like a general-purpose schema utility anyway. The only reason I didn't do it from the start was to avoid breaking custom schema implementations in client code, since no one's currently using this interface anyway.

[barakalon]

Maybe we should ditch ABC in favor of regular ol' raise NotImplementedError. Or perhaps we can just add a simple, default implementation that doesn't normalize and uses column_names.

[tobymao]

what about @barakalon comment on schema vs mapping schema?

[georgesittas]

what about @barakalon comment on schema vs mapping schema?

Just added a simple implementation in the base class here c92017c, @barakalon is this what you had in mind?

[georgesittas]

The previous type hint was more restrictive than needed, we already supported strings in the implementation, e.g. as shown in this test: https://github.com/tobymao/sqlglot/blob/main/tests/test_schema.py#L167

[barakalon]

perfect 👍