Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] main from datafuselabs:main #44

Merged
merged 1,429 commits into from
Apr 28, 2024
Merged

[pull] main from datafuselabs:main #44

merged 1,429 commits into from
Apr 28, 2024

Conversation

pull[bot]
Copy link

@pull pull bot commented Feb 11, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

Copy link

Pull request description must contain CLA like the following:

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

## Summary

Summary about this PR

- Close #issue

Copy link

github-actions bot commented Feb 11, 2024

This pull request's title is not fulfill the requirements. @pull[bot] please update it 🙏.

Valid format:

fix(query): fix group by string bug
  ^         ^---------------------^
  |         |
  |         +-> Summary in present tense.
  |
  +-------> Type: rfc, feat, fix, refactor, ci, docs, chore

Valid types:

  • rfc: this PR proposes a new RFC
  • feat: this PR introduces a new feature to the codebase
  • fix: this PR patches a bug in codebase
  • refactor: this PR changes the code base without new features or bugfix
  • ci: this PR changes build/testing/ci steps
  • docs: this PR changes the documents or websites
  • chore: this PR only has small changes that no need to record

@pull pull bot added the ⤵️ pull label Feb 11, 2024
drmingdrmer and others added 27 commits March 19, 2024 11:07
* refactor: Use CrudMgr to impl ConnectionMgr

* chore: use customized error message for UnknownConnection
* fix agg spill

* fix agg spill

---------

Co-authored-by: jw <[email protected]>
* reactor(query): add copy into location physical plan

* reactor(query): add copy into location physical plan

* reactor(query): add copy into location physical plan
* adjust purge batch size

* add explicit timing logs for list and purge
* csv reader support prefetch.

* fix num of processor in CSV read pipeline.

* Update src/query/pipeline/sources/src/prefetch_async_source.rs

Co-authored-by: Winter Zhang <[email protected]>

* Update prefetch_async_source.rs

* Update src/query/pipeline/sources/src/prefetch_async_source.rs

---------

Co-authored-by: Winter Zhang <[email protected]>
* add log about presign elapsed

* update
* feat(query): support domain contains in string type

* feat(query): support domain contains in string type

* add random tests
…view (#14986)

* feat: check view's inner table privilege when create or alter view

Note:

Select view no need to check the inner table privilege.

Because the user only can query view when the admin user grant select on
the view to the user.

So the admin user needs to know why the user need access this view.

This pr is avoid that:

User only has create or alter on db. And then the user can query table
as create or alter view. Like this:

```sql
--root
grant create on default.* to a;
--user a
create view v_t1 as select * from t1; -- If it can be success, the user
a can query table t1 according to query view v_t1. It maybe dangerous.
```

* create view only need check the user has select privilege on inner tables

* add test: create view as select view
* refactor codes

* add truncate pipeline

* fix test

* fix review comment
meta-app is a crate defines application level meta-service keys and
values.
…m stage (#15016)

* refactor(query): use scan table physical plan for copy into stage

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

* refactor(query): use scan table physical plan for copy into satge

---------

Co-authored-by: Bohu <[email protected]>
* refactor: remove innecessary serde

* refactor: remove innecessary serde
* chore(query): add spill profile for join

* chore(query): add spill profile for join

---------

Co-authored-by: Bohu <[email protected]>
andylokandy and others added 29 commits April 23, 2024 22:43
* chore: tune auto-compact

* tweak comments

* verify row number during commit for compaction

* refactor, introduce CompactionLimits

* refact: remove operation name, just use mutation kind

* revert default query node config

* fmt code

* fix: compaction_num_block_hint should be acquired inside the on_finish hook

* refact: tweak CompactionLimits construction

* chore: tweak code comments

* Update src/query/catalog/src/table.rs

Co-authored-by: zhya <[email protected]>

---------

Co-authored-by: zhya <[email protected]>
* fix: incorrect overlap checking (replace into)

should return false if column being checked has no range index.

* revert default query node config

* add logic test

* chore: tweak code comments
* chore(planner): fix merge into statistics

* chore: fix sqllogictest

* chore: fix sqllogictest
* chore: add disable_merge_into_join_reorder settings

* chore: add sqllogictest

* chore: update merge into explain sqllogictest
* Feature: top k syntax support

n

The maximum number of rows to return in the result set.

```sql
select TOP 4 c1 from testable ORDER BY c1;

-- similar to

select c1 from testable order by c1 limit 4;

```

* fix ut err

* add test: keyword top as column name

* improve err hints

* improve error hint

* fix

---------

Co-authored-by: Andy Lok <[email protected]>
chore: fix RulePushDownLimitWindow id
…sh (#15325)

* impl st_asewkb/st_aswkb/st_asewkt/st_aswkt/st_geohash

Signed-off-by: Fan Yang <[email protected]>

* impl st_asewkb/st_aswkb/st_asewkt/st_aswkt/st_geohash

Signed-off-by: Fan Yang <[email protected]>

---------

Signed-off-by: Fan Yang <[email protected]>
* chore(query): add merge ratio in sort spill

* chore(query): fix tests
This update includes a fix for excessive log scanning during startup to
locate the membership configuration.

See: databendlabs/openraft@14d42e4
* feat(query): support with clause in copy into

* feat(query): support with clause in copy into

* feat(query): support with clause in copy into
* add err to capture new agg ht hang

* fix

* fix err info

---------

Co-authored-by: jw <[email protected]>
Move the IO into a separate thread and use `std::io` instead of `tokio::io`.
* feat(query): support compute quota feature

* feature(query): support compute quota feature

* feature(query): support compute quota feature
* fix(executor): schedule points calc problem

Signed-off-by: Liuqing Yue <[email protected]>

* chore: make clippy happy

Signed-off-by: Liuqing Yue <[email protected]>

* chore: make clippy happy

Signed-off-by: Liuqing Yue <[email protected]>

* disable queries executor

Signed-off-by: Liuqing Yue <[email protected]>

---------

Signed-off-by: Liuqing Yue <[email protected]>
Co-authored-by: Winter Zhang <[email protected]>
@TCeason TCeason merged commit d8ddbec into TCeason:main Apr 28, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.