acryl-sqlglot fork - plans to contribute/switch to upstream?

[heavelock]

Hey!

I just noticed that datahub is not using original release of sqlglot but rather an own fork.
The fork is currently 1 commit ahead and 18 commits behind the main branch of original library.

tobymao/sqlglot@main...hsheth2:sqlglot:main#diff-7857fedd1d1451b1b9a5b8efaa1cc292c02e7ee4f0d04d7e2f9d5bfb9565802cR38

Looking briefly at the code, it would be quite straightforward to contribute those changes to upstream, obviously apart of that one import. Is that somewhere on your roadmap? Are you planning to bring this code upstream?

I'm asking because we were strongly considering contributing to sqlglot to better support Clickhouse sql dialect. Those contributions were reasoned mostly due to DH's data lineage feature. But if there is one additional hoop to jump and no guarantee that we will be able to benefit from our sqlglot improvements in DH, the contributions I am trying to argue for, could be questioned.

[hsheth2]

@heavelock I am going to keep the acryl-sqlglot fork up to date with the upstream sqlglot library. Because our changes are pretty small, it is pretty easy to keep up to date. I usually sync them every other week or so, and would be happy to sync them on demand if there's commits you're looking to make use of.

We already try to report issues/fixes to the upstream instead of fixing them locally in our branch. We've already created a handful of PRs and bug reports. I also intend to continue contributing back, and do want to move more of our changes upstream as they mature.

[heavelock]

Hey @hsheth2!
Thanks for the insights!

[usmanovbf]

Hi @hsheth2 ! Is there any news about switching upstream? As well as @heavelock I want to contribute and it would be easier to do while working with the upstream.

[hsheth2]

You can directly contribute to the upstream. When your PR is merged, let me know and I can pull those changes into my fork.