[pull] main from datafuselabs:main #46
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
* improve show streams performance * fix test * check invalid in parallel
* refactor(query): add query id for query log * refactor(query): add query id for query log * refactor(query): add query id for query log * refactor(query): add query id for query log * refactor(query): add query id for query log
* feat(executor): record errors info into profiling log * feat(executor): record errors info into profiling log
…r 1, http 1 (#15442) * Save current work Signed-off-by: Xuanwo <[email protected]> * Refactor layer Signed-off-by: Xuanwo <[email protected]> * Save work Signed-off-by: Xuanwo <[email protected]> * Save current work Signed-off-by: Xuanwo <[email protected]> * Build pass Signed-off-by: Xuanwo <[email protected]> * cargo fix Signed-off-by: Xuanwo <[email protected]> * cargo check pass Signed-off-by: Xuanwo <[email protected]> * Cleanup deps Signed-off-by: Xuanwo <[email protected]> * Format files Signed-off-by: Xuanwo <[email protected]> * Fix bytes reader use too small range Signed-off-by: Xuanwo <[email protected]> * Split read offset and consume offset Signed-off-by: Xuanwo <[email protected]> * Fix input pipeline Signed-off-by: Xuanwo <[email protected]> * leave a todo here Signed-off-by: Xuanwo <[email protected]> * Fix eof not calculated correctly Signed-off-by: Xuanwo <[email protected]> * Fix offset check Signed-off-by: Xuanwo <[email protected]> * Try concurrent load Signed-off-by: Xuanwo <[email protected]> * format toml Signed-off-by: Xuanwo <[email protected]> * Let's try use opendal's async read Signed-off-by: Xuanwo <[email protected]> * reduce to 2 concurrent Signed-off-by: Xuanwo <[email protected]> * Also fix support for input pipeline Signed-off-by: Xuanwo <[email protected]> * try 4 concurrent Signed-off-by: Xuanwo <[email protected]> * Remove an extra head Signed-off-by: Xuanwo <[email protected]> --------- Signed-off-by: Xuanwo <[email protected]>
* chore: fix copy/spill log message * fix query plan output, to avoid breakline * add line by line log for physical log
Signed-off-by: Xuanwo <[email protected]>
* feat: optimize vacuum drop table, execute in parallel * feat: optimize vacuum drop table, execute in parallel * feat: optimize vacuum drop table, execute in parallel * feat: optimize vacuum drop table, execute in parallel * feat: optimize vacuum drop table, execute in parallel * Update src/query/ee/src/storages/fuse/operations/vacuum_drop_tables.rs --------- Co-authored-by: dantengsky <[email protected]>
|
At least one test kind must be checked in the PR description. |
|
This pull request's title is not fulfill the requirements. @pull[bot] please update it 🙏. Valid format: Valid types:
|
|
Pull request description must contain CLA like the following: |
|
At least one type of change must be checked in the PR description. |
* refactor: improve code readability for DATABEND_COMMIT_VERSION. * feat: http handler add endpoint `/v1/login/`.
* chore: Bump poem to 3.0 Signed-off-by: Xuanwo <[email protected]> * remove not used crates Signed-off-by: Xuanwo <[email protected]> * remove the crate who have problem Signed-off-by: Xuanwo <[email protected]> * Fix test Signed-off-by: Xuanwo <[email protected]> --------- Signed-off-by: Xuanwo <[email protected]>
Refactor: `SessionContext` doesn't require `Arc` Embrace idiomatic Rust practices by reducing unnecessary sharing. In this case, since `SessionContext` is not shared among threads, there is no need to wrap it in an `Arc`. This change streamlines the code and enhances performance by eliminating unneeded atomic reference counting.
* feat(functions): add map_contains_key scalar function * feat(functions): update function_list.txt and add more test_map case * feat(functions): add sqllogictests in map_contains_key * feat(functions): fix function_list.txt * feat(functions): update logic tests for map_contains_key
* chore: remove project operator * fix * remove project
Signed-off-by: Xuanwo <[email protected]>
…untime (#15504) * refactor: `SessionContext::current_tenant` should not be changed at runtime Due to the design principle that `SessionContext::current_tenant` should not be modified at runtime, this commit transitions `SessionContext::current_tenant` from a shared `RwLock<Option<Tenant>>` to a non-shared `Option<Tenant>`. Additionally, the process of constructing a `Session` has been divided into two distinct steps: 1. Initially build an instance and set up initial values, including `current_tenant`. 2. Convert the `Session` instance into a readonly `Arc<Session>` to facilitate sharing across `QueryContext` and other components. Other Changes: - Methods of `Session` are now bound to `&self` instead of `&Arc<Self>`. - The method `SessionCtx::set_current_tenant()` has been made private within its module. - The function `SessionManager::register_session()` has been extracted from `SessionManager::create_with_settings()`. Because `Session` creation and registration must be two distinct steps. * chore: return Err instead of using `unwrap()`
* chore: Upgrade minitrace related crates to 0.6.5 * Update cargo.lock
Prior to this commit, the snapshot writer would attempt to commit the snapshot upon closure of the input channel, operating under the assumption that the snapshot was fully written. This assumption is flawed as the closure of the channel might be due to a process shutdown, not completion of the snapshot. To address this issue, this commit introduces a change where items sent to the snapshot writer are encapsulated within a `WriteEntry`, which can be either `Data(T)` or `Commit`. The snapshot is only considered complete and ready for commit when a `Commit` variant is received. This adjustment ensures that premature snapshot commits are avoided in cases of early channel closures.
* fix(query): view query use default dialect * fix
* fix(query): convert to arrow column remove ignore inside nullable * fix * fix * fix * fix * tmp * fix * fix
…ry log (#15531) * feat(query): add query query_hash and query_parameterized_hash to query log * fix fmt check * fix test * add query_hash test * use md5 generate hash * fix check * only hash select statement; In query log, the query_text maybe like this: ``` query_text: INSERT INTO products (id, name, description) VALUES (?,?,?) ``` It not a normal query, can not be format. * refactor: attach hash value into query ctx * extract method;
* feat: add meta key_num metrics * feat: add meta key_num metrics
* chore: leave all un-committed data to purge/vacuum * chore: log update_table_meta failure
* refactor(query): refactor some commits * fix(query): fix qualify display * fix(query): fix qualify display * fix(query): fix qualify display
…UDFs (#15529) I made some enhancements to the arrow-udf-js code to improve Databend support. JS UDFs now support Variant, Timestamp, Date and Decimal types in arguments and return values. This version also enables all intrinsics in rquickjs for better JS compatibility. Co-authored-by: Bohu <[email protected]>
* return mismatched tid * handle multi stmt txn * resolve conflict when multi table insert commit * fix lost update * use megt, reuse table meta, return all tables with update failures at once * enhances the code safety and add comment
* chore: mask the config sensitive field for display * fix configs table basic test
* feat(query): udf script support load from stage location * feat(query): udf script support load from stage location
* refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions * refactor(cluster): refactor flight service actions
* feat: select support consume stream * add test * fix * add test * chore: more logic test - stream can be consumed individually - stream should not be consumed if query goes wrong * add defensive check of catalog names --------- Co-authored-by: dantengsky <[email protected]>
* feat(query): show users support display roles * show roles display inherited_roles_names * fix test
* chore: Remove tokio-console since we don't need it Signed-off-by: Xuanwo <[email protected]> * Remove not used deps Signed-off-by: Xuanwo <[email protected]> --------- Signed-off-by: Xuanwo <[email protected]>
Signed-off-by: Xuanwo <[email protected]>
Signed-off-by: Xuanwo <[email protected]>
* chore: bump opensrv to 0.7.0 Signed-off-by: Chojan Shang <[email protected]> * chore: minor update Signed-off-by: Chojan Shang <[email protected]> * chore: minor update Signed-off-by: Chojan Shang <[email protected]> --------- Signed-off-by: Chojan Shang <[email protected]>
* feat: remove XML input format. * fix
…ng statistics (#15555) chore(executor): set false plain_statistics for spill time profiling statistics
* chore: support aborting table navigation * refact: introduce AbortChecker * add unit test * cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )