Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query parameters support #854

Merged
merged 6 commits into from
Dec 23, 2022
Merged

Query parameters support #854

merged 6 commits into from
Dec 23, 2022

Conversation

jkaflik
Copy link
Contributor

@jkaflik jkaflik commented Dec 16, 2022

Add a basic query parameters support as implemented in ClickHouse/ClickHouse#39906
This implementation is basic and does not support type conversion. The user is forced to pass parameter values as a string compatible with CH syntax literals.

The native protocol implementation is not documented (yet). For an idea of how it works you can check:
https://clickhouse.com/docs/en/interfaces/http/#cli-queries-with-parameters
https://clickhouse.com/docs/en/interfaces/cli/#cli-queries-with-parameters

ClickHouse supports the following parameter syntax:

{<name>:<data type>}

Example SQL:

SELECT {str:String}, {arr:Array(String)}

There are two ways to provide parameters. With query context:

chCtx := clickhouse.Context(ctx, clickhouse.WithParameters(clickhouse.Parameters{
	"num":   "42",
	"str":   "hello",
	"array": "['a', 'b', 'c']",
}))

row := client.QueryRow(chCtx, "SELECT {num:UInt64} v, {str:String} s, {array:Array(String)} a")

or as compatible with named param:

row := client.QueryRow(
	ctx,
	"SELECT {num:UInt64}, {str:String}",
	clickhouse.Named("num", "42"),
	clickhouse.Named("str", "hello"),
)

resolves #625

@jkaflik jkaflik marked this pull request as draft December 16, 2022 22:54
@jkaflik jkaflik changed the title first implementation Query parameters support Dec 16, 2022
@jkaflik jkaflik added this to the 2.5.0 milestone Dec 20, 2022
@jkaflik jkaflik marked this pull request as ready for review December 20, 2022 20:53
conn_query.go Outdated Show resolved Hide resolved
@jkaflik jkaflik merged commit a81b180 into main Dec 23, 2022
@jkaflik jkaflik deleted the 625-Rewrite-parameter-substitution branch May 5, 2023 16:18
et added a commit to highlight/highlight that referenced this pull request Jul 27, 2023
## Summary

<!--
Ideally, there is an attached GitHub issue that will describe the "why".

If relevant, use this section to call out any additional information
you'd like to _highlight_ to the reviewer.
-->

Clickhouse tests are currently failing on main
([slack](https://highlightcorp.slack.com/archives/C02CJANPHQS/p1690479345204819)):
```
    cursor_test.go:196: 
        	Error Trace:	/home/runner/work/highlight/highlight/backend/clickhouse/cursor_test.go:196
        	Error:      	Received unexpected error:
        	            	clickhouse [Decode]:  custom serialization for column Body. not supported
        	Test:       	TestClickhouseDecode
--- FAIL: TestClickhouseDecode (0.04s)
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0
````

I can't tell what is exactly going, but something with the [latest
tag](https://hub.docker.com/layers/clickhouse/clickhouse-server/23.7.1/images/sha256-4ee2d642df3481db1809ca632bd3f9d67b1245a75813fb2d618293d7aabed414?context=explore)
in docker hits this codepath:

https://github.com/ClickHouse/clickhouse-go/blame/main/lib/proto/block.go#L222-L230

Per the [PR](ClickHouse/clickhouse-go#854) that
implemented this, perhaps we need to to be explicit on the column types
when we query? I've filed #6141 to figure that out.

This PR hardcodes the clickhouse version we fetch in docker. The version
is pulled from the closet tag that matches clickhouse cloud (`SELECT
version()` => `23.5.1.34446`).

## How did you test this change?

<!--
Frontend - Leave a screencast or a screenshot to visually describe the
changes.
-->

Reset go test cache: `go clean -testcache`
Rerun any clickhouse test and confirm it passes.

CI should also pass as well.


## Are there any deployment considerations?

<!--
 Backend - Do we need to consider migrations or backfilling data?
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rewrite parameter substitution
2 participants