succ ver #1
Open
succ ver #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
abc982627271
force-pushed
the
alter_lake_table_drop_partition
branch
from
caa94ea
to
8fc8f95
Compare
Add blacklist for insert stmt:
admin set frontend config ("enable_sql_blacklist" = "true")
ADD SQLBLACKLIST "insert into .+ values.+";
then, execute
insert into action333 values(333, 'ddd', '2020-02-25 17:33:22');
ERROR 1064 (HY000): Access denied; This sql is in blacklist, please contact your admin
…lock interface (StarRocks#9318) Many logs upon start is non-sense and the pull request remove these logs. The AlterTask is already not used and drop_tablet_unlocked can be unified.
After StarRocks#8578 fix, the chunk and arrow batch columns order is already base on output exprs. Spark uses selected columns as rdd schema, so generate selected columns from output exprs.
…StarRocks#9137) Add a new alias annotation method to keep forward compatibility in ConfigBase class. Rename and rearrange some configurations in Config.java.
* [Doc]delete content that prepared for 2.4 * modifications * Update REVOKE.md
…hange (column change) operation-related statements (StarRocks#7941) (StarRocks#9123)
1. FE creates DeletePredicatePB which is used in BE tablet metadata directly. 2. No longer use the old push task api.
Related to StarRocks#8881 When there are unsupported types in an Iceberg table, exception will not be thrown if users don't select columns with such types. Like the followings: ```sql hive> select * from t.m; OK 1 {"a":"b"} MySQL [(none)]> select * from ice.t.m; ERROR 1064 (HY000): External Table Column [m] convert failed, and column type is known! MySQL [(none)]> select i from ice.t.m; +------+ | i | +------+ | 1 | +------+ 1 row in set (0.08 sec) ```
Signed-off-by: xyz <[email protected]>
…has expression (StarRocks#9438) When correlated subquery sql like ``` select t0.v1 from t0 where (t0.v2 in (select t1.v4 from t1 where t0.v3 + t1.v5 = 1)) is NULL ``` The QuantifiedApply2OuterJoinRule would transfrom plan like ``` /* * In: * before: select t0.v1, t0.v2 in (select t1.v2 from t1 where t0.v3 = t1.v3) from t0; * after: with xx as (select t1.v2, t1.v3 from t1) * select t0.v1, * case * // t1 empty table and without where clause `t0.v3 = t1.v3`, then t1Rows may be 0 * // t1 empty table, if t1Rows is null, means the result of join correlation predicate must be false * // `any_or_null in (empty) -> false` * when t1Rows = 0 or t1Rows is null then false * // `null in (any_or_null...) -> null` * when t0.v2 is null then null * // `a in (a, [any_or_null...]) -> true` * when t1d.v2 is not null then true * // `a in (null, [not_a_or_null...]) -> null` * when v2NotNulls < t1Rows then null * // `a in (not_a...) -> false`, not_a cannot be null * else false * end * from t0 * left outer join (select xx.v2, xx.v3 from xx group by xx.v2, xx.v3) as t1d * on t0.v2= t1d.v2 and t0.v3 = t1d.v3 * left outer join (select v3, count(*) as t1Rows, count(xx.v2) as v2NotNulls from xx group by xx.v3) as t1c * on t1c.v3 = t0.v3 * * CTEAnchor * / \ * CTEProduce Project * Apply(t0.v2 in t1.v2) / \ * / \ ===> Project left/cross join * t0 t1 / / \ * Filter Left join Agg(count) * / / \ \ * t1 t0 Agg(distinct) CTEConsume * \ * CTEConsume * */ ``` The agg(distinct) and agg(count) should add the v5 column to group by which will used in left join/cross join predicate
No need use write lock for base tablet
wyb
reviewed
Aug 3, 2022
fe/fe-core/src/main/java/com/starrocks/persist/PartitionPersistInfoV2.java
Outdated
Show resolved
Hide resolved
| import java.io.IOException; | ||
| import java.util.List; | ||
|
|
||
| public class AddPartitionsInfoV2 implements Writable { |
wyb
reviewed
Aug 3, 2022
fe/fe-core/src/main/java/com/starrocks/catalog/CatalogUtils.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/com/starrocks/server/LocalMetastore.java
Outdated
Show resolved
Hide resolved
…ble syntax bug（2/2) (StarRocks#9718)
…tarRocks#9520) Support load json type from csv format through stream load by adding a json_converter
…tionary optimization (StarRocks#9632) we will add a new dictionary column for MAX/MIN aggregate function in dictionary optimization. so when we have a string function for a old dictionary column, we also need apply it to the new dictionary column.
abc982627271
force-pushed
the
alter_lake_table_add_partition
branch
from
caea7a8
to
08c0136
Compare
abc982627271
force-pushed
the
alter_lake_table_add_partition
branch
from
4ee982c
to
6d109a6
Compare
abc982627271
pushed a commit
that referenced
this pull request
May 8, 2023
Sub expr of LambdaFunction is not closed;
```
Direct leak of 120 byte(s) in 1 object(s) allocated from:
#0 0x9596567 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99
#1 0x1252065c in starrocks::LikePredicate::like_prepare(starrocks::FunctionContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/like_predicate.cpp:107
#2 0xe64d8b4 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:90
#3 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#4 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#5 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#6 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#7 0xe64d085 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:71
StarRocks#8 0xd61b983 in starrocks::ExprContext::open(starrocks::RuntimeState*) /root/starrocks/be/src/exprs/expr_context.cpp:86
StarRocks#9 0xd627b0d in starrocks::Expr::open(std::vector<starrocks::ExprContext*, std::allocator<starrocks::ExprContext*> > const&, starrocks::RuntimeState*) /root/starrocks/be/src/exprs/expr.cpp:445
StarRocks#10 0xa95e33a in starrocks::pipeline::ProjectOperatorFactory::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/project_operator.cpp:99
StarRocks#11 0x12279e75 in starrocks::pipeline::Pipeline::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/pipeline.h:76
StarRocks#12 0x1227bf24 in starrocks::pipeline::FragmentContext::prepare_all_pipelines() /root/starrocks/be/src/exec/pipeline/fragment_context.h:97
StarRocks#13 0x1226c76b in starrocks::pipeline::FragmentExecutor::_prepare_pipeline_driver(starrocks::ExecEnv*, starrocks::pipeline::UnifiedExecPlanFragmentParams const&) /root/starrocks/be/src/exec/pipeline/fragment_executor.cpp:551
StarRocks#14 0x1227099c in starrocks::pipeline::FragmentExecutor::prepare(starrocks::ExecEnv*, starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&) /root/starrocks/be/src/exec/pipeline/fragment_executor.cpp:673
StarRocks#15 0x12b78a15 in starrocks::PInternalServiceImplBase<doris::PBackendService>::_exec_plan_fragment_by_pipeline(starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&) /root/starrocks/be/src/service/internal_service.cpp:412
StarRocks#16 0x12b73b0d in starrocks::PInternalServiceImplBase<doris::PBackendService>::_exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*) /root/starrocks/be/src/service/internal_service.cpp:342
StarRocks#17 0x12b82795 in starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}::operator()() const /root/starrocks/be/src/service/internal_service.cpp:308
StarRocks#18 0x12b8f869 in void std::__invoke_impl<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}&>(std::__invoke_other, starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
```
abc982627271
pushed a commit
that referenced
this pull request
May 23, 2023
```
==3247495==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040029dd090 at pc 0x00000e1296bf bp 0x7f5ef72f60c0 sp 0x7f5ef72f60b0
READ of size 8 at 0x6040029dd090 thread T415
#0 0xe1296be in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::UpdateCounter() /root/starrocks/be/src/util/runtime_profile.h:658
#1 0xe125dfd in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::~ScopedTimer() /root/starrocks/be/src/util/runtime_profile.h:665
#2 0xe7156a6 in starrocks::HashJoinBuilder::append_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/hash_join_components.cpp:86
#3 0xecc1382 in starrocks::pipeline::SpillableHashJoinProbeOperator::_load_partition_build_side(starrocks::RuntimeState*, std::shared_ptr<starrocks::spill::SpillerReader> const&, unsigned long) /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:258
StarRocks#4 0xecc2982 in operator() /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:281
StarRocks#5 0xeccb32f in __invoke_impl<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:61
StarRocks#6 0xeccacc2 in __invoke_r<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:111
```
https://stackoverflow.com/questions/25292580/why-am-i-getting-an-access-violation-when-locking-a-weak-ptr
weak_ptr.lock() should capture the resource
```
#include <unistd.h>
#include <iostream>
#include <memory>
#include <thread>
struct LongDesc : public std::enable_shared_from_this<LongDesc> {
LongDesc() = default;
~LongDesc() { std::cout << "desc finish" << std::endl; }
private:
long epoh{};
};
int main() {
auto desc = std::make_shared<LongDesc>();
std::weak_ptr<LongDesc> weak = desc;
std::thread t([&]() {
std::cout << "start thread" << std::endl;
if (weak.lock()) {
std::cout << "hold resource" << std::endl;
sleep(5);
std::cout << "call in thread" << std::endl;
}
});
sleep(1);
std::cout << "reset" << std::endl;
desc.reset();
t.join();
return 0;
}
```
output:
```
start thread
hold resource
reset
desc finish
call in thread
```
Signed-off-by: stdpain <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
May 29, 2023
```
==3247495==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040029dd090 at pc 0x00000e1296bf bp 0x7f5ef72f60c0 sp 0x7f5ef72f60b0
READ of size 8 at 0x6040029dd090 thread T415
#0 0xe1296be in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::UpdateCounter() /root/starrocks/be/src/util/runtime_profile.h:658
#1 0xe125dfd in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::~ScopedTimer() /root/starrocks/be/src/util/runtime_profile.h:665
#2 0xe7156a6 in starrocks::HashJoinBuilder::append_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/hash_join_components.cpp:86
#3 0xecc1382 in starrocks::pipeline::SpillableHashJoinProbeOperator::_load_partition_build_side(starrocks::RuntimeState*, std::shared_ptr<starrocks::spill::SpillerReader> const&, unsigned long) /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:258
StarRocks#4 0xecc2982 in operator() /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:281
StarRocks#5 0xeccb32f in __invoke_impl<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:61
StarRocks#6 0xeccacc2 in __invoke_r<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:111
```
https://stackoverflow.com/questions/25292580/why-am-i-getting-an-access-violation-when-locking-a-weak-ptr
weak_ptr.lock() should capture the resource
```
#include <unistd.h>
#include <iostream>
#include <memory>
#include <thread>
struct LongDesc : public std::enable_shared_from_this<LongDesc> {
LongDesc() = default;
~LongDesc() { std::cout << "desc finish" << std::endl; }
private:
long epoh{};
};
int main() {
auto desc = std::make_shared<LongDesc>();
std::weak_ptr<LongDesc> weak = desc;
std::thread t([&]() {
std::cout << "start thread" << std::endl;
if (weak.lock()) {
std::cout << "hold resource" << std::endl;
sleep(5);
std::cout << "call in thread" << std::endl;
}
});
sleep(1);
std::cout << "reset" << std::endl;
desc.reset();
t.join();
return 0;
}
```
output:
```
start thread
hold resource
reset
desc finish
call in thread
```
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit 7ad62ba)
abc982627271
pushed a commit
that referenced
this pull request
Jun 2, 2023
Fixes #issue
```
mysql> SELECT regexp_replace('a b c', " ", "-");
+-----------------------------------+
| regexp_replace('a b c', ' ', '-') |
+-----------------------------------+
| a-b-c |
+-----------------------------------+
1 row in set (0.01 sec)
```
```
Direct leak of 2319 byte(s) in 1 object(s) allocated from:
#0 0x974c2af in __interceptor_malloc ../../.././libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x14bdb529 in alloc_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:121
#2 0x14bdbb3f in hs_clone_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:399
#3 0x12d4f791 in regexp_replace_use_hyperscan /root/starrocks/be/src/exprs/string_functions.cpp:2846
StarRocks#4 0x12d5044a in starrocks::StringFunctions::regexp_replace(starrocks::FunctionContext*, std::vector<std::shared_ptr<starrocks::Column>, std::allocator<std::shared_ptr<starrocks::Column> > > const&) /root/starrocks/be/src/exprs/string_functions.cpp:2913
StarRocks#5 0xec8af37 in starrocks::VectorizedFunctionCallExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) /root/starrocks/be/src/exprs/function_call_expr.cpp:159
StarRocks#6 0xd924e9b in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:176
StarRocks#7 0xd9246dc in starrocks::ExprContext::evaluate(starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:160
StarRocks#8 0xab4f521 in starrocks::pipeline::ProjectOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/pipeline/project_operator.cpp:55
StarRocks#9 0x9c9ec7c in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:320
StarRocks#10 0x12a59131 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:154
StarRocks#11 0x12a57871 in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:69
StarRocks#12 0x12a61555 in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#13 0x12a60a5a in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#14 0x12a5fd64 in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#15 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#16 0x1172d057 in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58
StarRocks#17 0x11729e0c in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553
StarRocks#18 0x11745a65 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73
StarRocks#19 0x117453be in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.
3.0/bits/invoke.h:95
StarRocks#20 0x117447b5 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416
StarRocks#21 0x11743117 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499
StarRocks#22 0x1174017b in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#23 0x1173dadf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadP
ool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#24 0x11739b48 in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#25 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#26 0x11711bc8 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364
StarRocks#27 0x7fc585af0ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
```
Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jun 5, 2023
Sub expr of LambdaFunction is not closed;
```
Direct leak of 120 byte(s) in 1 object(s) allocated from:
#0 0x9596567 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99
#1 0x1252065c in starrocks::LikePredicate::like_prepare(starrocks::FunctionContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/like_predicate.cpp:107
#2 0xe64d8b4 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:90
#3 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#4 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#5 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#6 0xd62805c in starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/expr.cpp:453
StarRocks#7 0xe64d085 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:71
StarRocks#8 0xd61b983 in starrocks::ExprContext::open(starrocks::RuntimeState*) /root/starrocks/be/src/exprs/expr_context.cpp:86
StarRocks#9 0xd627b0d in starrocks::Expr::open(std::vector<starrocks::ExprContext*, std::allocator<starrocks::ExprContext*> > const&, starrocks::RuntimeState*) /root/starrocks/be/src/exprs/expr.cpp:445
StarRocks#10 0xa95e33a in starrocks::pipeline::ProjectOperatorFactory::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/project_operator.cpp:99
StarRocks#11 0x12279e75 in starrocks::pipeline::Pipeline::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/pipeline.h:76
StarRocks#12 0x1227bf24 in starrocks::pipeline::FragmentContext::prepare_all_pipelines() /root/starrocks/be/src/exec/pipeline/fragment_context.h:97
StarRocks#13 0x1226c76b in starrocks::pipeline::FragmentExecutor::_prepare_pipeline_driver(starrocks::ExecEnv*, starrocks::pipeline::UnifiedExecPlanFragmentParams const&) /root/starrocks/be/src/exec/pipeline/fragment_executor.cpp:551
StarRocks#14 0x1227099c in starrocks::pipeline::FragmentExecutor::prepare(starrocks::ExecEnv*, starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&) /root/starrocks/be/src/exec/pipeline/fragment_executor.cpp:673
StarRocks#15 0x12b78a15 in starrocks::PInternalServiceImplBase<doris::PBackendService>::_exec_plan_fragment_by_pipeline(starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&) /root/starrocks/be/src/service/internal_service.cpp:412
StarRocks#16 0x12b73b0d in starrocks::PInternalServiceImplBase<doris::PBackendService>::_exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*) /root/starrocks/be/src/service/internal_service.cpp:342
StarRocks#17 0x12b82795 in starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}::operator()() const /root/starrocks/be/src/service/internal_service.cpp:308
StarRocks#18 0x12b8f869 in void std::__invoke_impl<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}&>(std::__invoke_other, starrocks::PInternalServiceImplBase<doris::PBackendService>::exec_batch_plan_fragments(google::protobuf::RpcController*, starrocks::PExecBatchPlanFragmentsRequest const*, starrocks::PExecBatchPlanFragmentsResult*, google::protobuf::Closure*)::{lambda()#1}&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
```
abc982627271
pushed a commit
that referenced
this pull request
Jun 5, 2023
```
==3247495==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040029dd090 at pc 0x00000e1296bf bp 0x7f5ef72f60c0 sp 0x7f5ef72f60b0
READ of size 8 at 0x6040029dd090 thread T415
#0 0xe1296be in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::UpdateCounter() /root/starrocks/be/src/util/runtime_profile.h:658
#1 0xe125dfd in starrocks::ScopedTimer<starrocks::MonotonicStopWatch>::~ScopedTimer() /root/starrocks/be/src/util/runtime_profile.h:665
#2 0xe7156a6 in starrocks::HashJoinBuilder::append_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/hash_join_components.cpp:86
#3 0xecc1382 in starrocks::pipeline::SpillableHashJoinProbeOperator::_load_partition_build_side(starrocks::RuntimeState*, std::shared_ptr<starrocks::spill::SpillerReader> const&, unsigned long) /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:258
StarRocks#4 0xecc2982 in operator() /root/starrocks/be/src/exec/pipeline/hashjoin/spillable_hash_join_probe_operator.cpp:281
StarRocks#5 0xeccb32f in __invoke_impl<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:61
StarRocks#6 0xeccacc2 in __invoke_r<void, starrocks::pipeline::SpillableHashJoinProbeOperator::_load_all_partition_build_side(starrocks::RuntimeState*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:111
```
https://stackoverflow.com/questions/25292580/why-am-i-getting-an-access-violation-when-locking-a-weak-ptr
weak_ptr.lock() should capture the resource
```
#include <unistd.h>
#include <iostream>
#include <memory>
#include <thread>
struct LongDesc : public std::enable_shared_from_this<LongDesc> {
LongDesc() = default;
~LongDesc() { std::cout << "desc finish" << std::endl; }
private:
long epoh{};
};
int main() {
auto desc = std::make_shared<LongDesc>();
std::weak_ptr<LongDesc> weak = desc;
std::thread t([&]() {
std::cout << "start thread" << std::endl;
if (weak.lock()) {
std::cout << "hold resource" << std::endl;
sleep(5);
std::cout << "call in thread" << std::endl;
}
});
sleep(1);
std::cout << "reset" << std::endl;
desc.reset();
t.join();
return 0;
}
```
output:
```
start thread
hold resource
reset
desc finish
call in thread
```
Signed-off-by: stdpain <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jun 5, 2023
Fixes #issue
```
mysql> SELECT regexp_replace('a b c', " ", "-");
+-----------------------------------+
| regexp_replace('a b c', ' ', '-') |
+-----------------------------------+
| a-b-c |
+-----------------------------------+
1 row in set (0.01 sec)
```
```
Direct leak of 2319 byte(s) in 1 object(s) allocated from:
#0 0x974c2af in __interceptor_malloc ../../.././libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x14bdb529 in alloc_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:121
#2 0x14bdbb3f in hs_clone_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:399
#3 0x12d4f791 in regexp_replace_use_hyperscan /root/starrocks/be/src/exprs/string_functions.cpp:2846
StarRocks#4 0x12d5044a in starrocks::StringFunctions::regexp_replace(starrocks::FunctionContext*, std::vector<std::shared_ptr<starrocks::Column>, std::allocator<std::shared_ptr<starrocks::Column> > > const&) /root/starrocks/be/src/exprs/string_functions.cpp:2913
StarRocks#5 0xec8af37 in starrocks::VectorizedFunctionCallExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) /root/starrocks/be/src/exprs/function_call_expr.cpp:159
StarRocks#6 0xd924e9b in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:176
StarRocks#7 0xd9246dc in starrocks::ExprContext::evaluate(starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:160
StarRocks#8 0xab4f521 in starrocks::pipeline::ProjectOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/pipeline/project_operator.cpp:55
StarRocks#9 0x9c9ec7c in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:320
StarRocks#10 0x12a59131 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:154
StarRocks#11 0x12a57871 in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:69
StarRocks#12 0x12a61555 in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#13 0x12a60a5a in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#14 0x12a5fd64 in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#15 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#16 0x1172d057 in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58
StarRocks#17 0x11729e0c in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553
StarRocks#18 0x11745a65 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73
StarRocks#19 0x117453be in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.
3.0/bits/invoke.h:95
StarRocks#20 0x117447b5 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416
StarRocks#21 0x11743117 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499
StarRocks#22 0x1174017b in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#23 0x1173dadf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadP
ool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#24 0x11739b48 in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#25 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#26 0x11711bc8 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364
StarRocks#27 0x7fc585af0ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
```
Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jun 12, 2023
) Fixes bug introduced by StarRocks#22892, Morsel's internal rowsets references to local variables in pick_morsels. In ASAN mode, tests on multiversion query cache would crash. ``` ==13613==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f92031bca78 at pc 0x00000a06a8cd bp 0x7f92031bb830 sp 0x7f92031bb828 READ of size 8 at 0x7f92031bca78 thread T211 (pip_wg_executor) #0 0xa06a8cc in std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > >::size() const /opt/gcc/usr/include/c++/10.3.0/bits/stl_vector.h:919 #1 0xa0672e9 in std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > >::vector(std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/stl_vector.h:555 #2 0xfc210d5 in starrocks::TabletReader::TabletReader(std::shared_ptr<starrocks::Tablet>, starrocks::Version const&, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /root/starrocks/be/src/storage/tablet_reader.cpp:53 #3 0xbbb63e4 in void __gnu_cxx::new_allocator<starrocks::TabletReader>::construct<starrocks::TabletReader, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(starrocks::TabletReader*, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/ext/new_allocator.h:150 StarRocks#4 0xbbb6134 in void std::allocator_traits<std::allocator<starrocks::TabletReader> >::construct<starrocks::TabletReader, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::allocator<starrocks::TabletReader>&, starrocks::TabletReader*, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/alloc_traits.h:512 StarRocks#5 0xbbb5d43 in std::_Sp_counted_ptr_inplace<starrocks::TabletReader, std::allocator<starrocks::TabletReader>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::allocator<starrocks::TabletReader>, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr_base.h:551 StarRocks#6 0xbbb5564 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<starrocks::TabletReader, std::allocator<starrocks::TabletReader>, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(starrocks::TabletReader*&, std::_Sp_alloc_shared_tag<std::allocator<starrocks::TabletReader> >, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr_base.h:682 StarRocks#7 0xbbb4bff in std::__shared_ptr<starrocks::TabletReader, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<starrocks::TabletReader>, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::_Sp_alloc_shared_tag<std::allocator<starrocks::TabletReader> >, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr_base.h:1371 StarRocks#8 0xbbb3ee0 in std::shared_ptr<starrocks::TabletReader>::shared_ptr<std::allocator<starrocks::TabletReader>, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::_Sp_alloc_shared_tag<std::allocator<starrocks::TabletReader> >, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr.h:408 StarRocks#9 0xbbb2fd3 in std::shared_ptr<starrocks::TabletReader> std::allocate_shared<starrocks::TabletReader, std::allocator<starrocks::TabletReader>, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::allocator<starrocks::TabletReader> const&, std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr.h:860 StarRocks#10 0xbbb1d80 in std::shared_ptr<starrocks::TabletReader> std::make_shared<starrocks::TabletReader, std::shared_ptr<starrocks::Tablet>&, starrocks::Version, starrocks::Schema, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&>(std::shared_ptr<starrocks::Tablet>&, starrocks::Version&&, starrocks::Schema&&, std::vector<std::shared_ptr<starrocks::Rowset>, std::allocator<std::shared_ptr<starrocks::Rowset> > > const&) /opt/gcc/usr/include/c++/10.3.0/bits/shared_ptr.h:876 StarRocks#11 0xbba8ac7 in starrocks::pipeline::OlapChunkSource::_init_olap_reader(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:298 StarRocks#12 0xbb9d97e in starrocks::pipeline::OlapChunkSource::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:79 StarRocks#13 0xa0b95d0 in starrocks::pipeline::ScanOperator::_pickup_morsel(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:485 StarRocks#14 0xa0b5088 in starrocks::pipeline::ScanOperator::_try_to_trigger_next_scan(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:297 StarRocks#15 0xa0b3eec in starrocks::pipeline::ScanOperator::pull_chunk(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:229 StarRocks#16 0x9f8ccfa in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:297 StarRocks#17 0x12de6363 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:154 StarRocks#18 0x12de4aa3 in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:69 StarRocks#19 0x12dee787 in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#20 0x12dedc8c in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#21 0x12decf96 in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#22 0x9bd122f in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#23 0x11a92f75 in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58 StarRocks#24 0x11a8fd2a in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#25 0x11aab983 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#26 0x11aab2dc in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:95 StarRocks#27 0x11aaa6d3 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#28 0x11aa9035 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#29 0x11aa6099 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#30 0x11aa39fd in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#31 0x11a9fa66 in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#32 0x9bd122f in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#33 0x11a77ae6 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364 StarRocks#34 0x7f926a69cea4 in start_thread (/lib64/libpthread.so.0+0x7ea4) StarRocks#35 0x7f9269cb7b0c in clone (/lib64/libc.so.6+0xfeb0c) ``` ---- Signed-off-by: satanson <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jun 20, 2023
When load channel not open yet, the `_row_desc` will be null, avoid dereference it when `add_chunk`. This will happen when rpc packets arrive not in order. crash stack: ``` (gdb) bt #0 0x00000000057cd3e2 in starrocks::serde::build_protobuf_chunk_meta(starrocks::RowDescriptor const&, starrocks::ChunkPB const&) () #1 0x0000000004d1ee07 in starrocks::LoadChannel::_build_chunk_meta(starrocks::ChunkPB const&) () #2 0x0000000004d21816 in starrocks::LoadChannel::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () #3 0x0000000004d1abb2 in starrocks::LoadChannelMgr::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () StarRocks#4 0x0000000004dd0835 in starrocks::BackendInternalServiceImpldoris::PBackendService::tablet_writer_add_chunk(google::protobuf::RpcController*, starrocks::PTabletWriterAddChunkRequest const*, starrocks::PTabletWriterAddBatchResult*, google::protobuf::Closure*) () ```
abc982627271
pushed a commit
that referenced
this pull request
Jul 4, 2023
Fixes #issue ``` Thread 912 (Thread 0x7f50197ff700 (LWP 3944)): #0 0x00007f501ad7854d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f501ad73e9b in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f501ad73d68 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000000006059a27 in pthread_mutex_lock_impl (mutex=<optimized out>) at src/bthread/mutex.cpp:564 StarRocks#4 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:818 StarRocks#5 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:817 StarRocks#6 0x000000000697819f in malloc_mutex_lock_final (mutex=0x7f5019a03268) at include/jemalloc/internal/mutex.h:155 StarRocks#7 je_malloc_mutex_lock_slow (mutex=mutex@entry=0x7f5019a03268) at src/mutex.c:85 StarRocks#8 0x000000000696f8c8 in malloc_mutex_lock (mutex=0x7f5019a03268, tsdn=0x7f50197f9040) at include/jemalloc/internal/mutex.h:221 StarRocks#9 je_extents_evict (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, r_extent_hooks=r_extent_hooks@entry=0x7f50197f84d8, extents=extents@entry=0x7f5019a03268, npages_min=npages_min@entry=18533) at src/extent.c:572 StarRocks#10 0x0000000006936004 in arena_stash_decayed (decay_extents=<synthetic pointer>, npages_decay_max=492, npages_limit=<optimized out>, extents=0x7f5019a03268, r_extent_hooks=0x7f50197f84d8, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:833 StarRocks#11 arena_decay_to_limit (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, decay=decay@entry=0x7f5019a06ca0, extents=extents@entry=0x7f5019a03268, all=all@entry=false, npages_limit=<optimized out>, npages_decay_max=<optimized out>, is_background_thread=<optimized out>) at src/arena.c:934 StarRocks#12 0x00000000069363fb in arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:722 StarRocks#13 arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:611 StarRocks#14 arena_maybe_decay (is_background_thread=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:762 StarRocks#15 arena_maybe_decay (tsdn=<optimized out>, arena=0x7f5019a00980, decay=0x7f5019a06ca0, extents=0x7f5019a03268, is_background_thread=<optimized out>) at src/arena.c:714 StarRocks#16 0x00000000069389ab in arena_decay_impl (all=false, is_background_thread=true, extents=0x7f5019a03268, decay=0x7f5019a06ca0, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:964 StarRocks#17 arena_decay_dirty (all=false, is_background_thread=true, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:985 StarRocks#18 je_arena_decay (tsdn=0x7f50197f9040, arena=0x7f5019a00980, is_background_thread=true, all=<optimized out>) at src/arena.c:998 StarRocks#19 0x00000000069403e9 in background_work_sleep_once (ind=0, info=<optimized out>, tsdn=<optimized out>) at src/background_thread.c:295 StarRocks#20 background_thread0_work (tsd=<optimized out>) at src/background_thread.c:452 StarRocks#21 background_work (ind=<optimized out>, tsd=<optimized out>) at src/background_thread.c:490 StarRocks#22 background_thread_entry () at src/background_thread.c:522 StarRocks#23 0x00007f501ad71ea5 in start_thread () from /lib64/libpthread.so.0 StarRocks#24 0x00007f501a38cb0d in clone () from /lib64/libc.so.6 ``` upgrade jemalloc from 5.2.1 to 5.3.0 to fix the deadlock: jemalloc/jemalloc#2019 Redis also encountered this problem redis/redis#12115 Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jul 4, 2023
Fixes #issue
```
mysql> SELECT regexp_replace('a b c', " ", "-");
+-----------------------------------+
| regexp_replace('a b c', ' ', '-') |
+-----------------------------------+
| a-b-c |
+-----------------------------------+
1 row in set (0.01 sec)
```
```
Direct leak of 2319 byte(s) in 1 object(s) allocated from:
#0 0x974c2af in __interceptor_malloc ../../.././libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x14bdb529 in alloc_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:121
#2 0x14bdbb3f in hs_clone_scratch /root/starrocks/thirdparty/src/hyperscan-5.4.0/src/scratch.c:399
#3 0x12d4f791 in regexp_replace_use_hyperscan /root/starrocks/be/src/exprs/string_functions.cpp:2846
StarRocks#4 0x12d5044a in starrocks::StringFunctions::regexp_replace(starrocks::FunctionContext*, std::vector<std::shared_ptr<starrocks::Column>, std::allocator<std::shared_ptr<starrocks::Column> > > const&) /root/starrocks/be/src/exprs/string_functions.cpp:2913
StarRocks#5 0xec8af37 in starrocks::VectorizedFunctionCallExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) /root/starrocks/be/src/exprs/function_call_expr.cpp:159
StarRocks#6 0xd924e9b in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:176
StarRocks#7 0xd9246dc in starrocks::ExprContext::evaluate(starrocks::Chunk*, unsigned char*) /root/starrocks/be/src/exprs/expr_context.cpp:160
StarRocks#8 0xab4f521 in starrocks::pipeline::ProjectOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) /root/starrocks/be/src/exec/pipeline/project_operator.cpp:55
StarRocks#9 0x9c9ec7c in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:320
StarRocks#10 0x12a59131 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:154
StarRocks#11 0x12a57871 in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:69
StarRocks#12 0x12a61555 in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#13 0x12a60a5a in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#14 0x12a5fd64 in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#15 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#16 0x1172d057 in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58
StarRocks#17 0x11729e0c in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553
StarRocks#18 0x11745a65 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73
StarRocks#19 0x117453be in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.
3.0/bits/invoke.h:95
StarRocks#20 0x117447b5 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416
StarRocks#21 0x11743117 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499
StarRocks#22 0x1174017b in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60
StarRocks#23 0x1173dadf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadP
ool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110
StarRocks#24 0x11739b48 in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291
StarRocks#25 0x98e2d87 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622
StarRocks#26 0x11711bc8 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364
StarRocks#27 0x7fc585af0ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
```
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit ee8c12f)
abc982627271
pushed a commit
that referenced
this pull request
Jul 4, 2023
When load channel not open yet, the `_row_desc` will be null, avoid dereference it when `add_chunk`. This will happen when rpc packets arrive not in order. crash stack: ``` (gdb) bt #0 0x00000000057cd3e2 in starrocks::serde::build_protobuf_chunk_meta(starrocks::RowDescriptor const&, starrocks::ChunkPB const&) () #1 0x0000000004d1ee07 in starrocks::LoadChannel::_build_chunk_meta(starrocks::ChunkPB const&) () #2 0x0000000004d21816 in starrocks::LoadChannel::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () #3 0x0000000004d1abb2 in starrocks::LoadChannelMgr::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () StarRocks#4 0x0000000004dd0835 in starrocks::BackendInternalServiceImpldoris::PBackendService::tablet_writer_add_chunk(google::protobuf::RpcController*, starrocks::PTabletWriterAddChunkRequest const*, starrocks::PTabletWriterAddBatchResult*, google::protobuf::Closure*) () ``` (cherry picked from commit d5b5eae)
abc982627271
pushed a commit
that referenced
this pull request
Jul 4, 2023
Fixes #issue ``` Thread 912 (Thread 0x7f50197ff700 (LWP 3944)): #0 0x00007f501ad7854d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f501ad73e9b in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f501ad73d68 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000000006059a27 in pthread_mutex_lock_impl (mutex=<optimized out>) at src/bthread/mutex.cpp:564 StarRocks#4 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:818 StarRocks#5 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:817 StarRocks#6 0x000000000697819f in malloc_mutex_lock_final (mutex=0x7f5019a03268) at include/jemalloc/internal/mutex.h:155 StarRocks#7 je_malloc_mutex_lock_slow (mutex=mutex@entry=0x7f5019a03268) at src/mutex.c:85 StarRocks#8 0x000000000696f8c8 in malloc_mutex_lock (mutex=0x7f5019a03268, tsdn=0x7f50197f9040) at include/jemalloc/internal/mutex.h:221 StarRocks#9 je_extents_evict (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, r_extent_hooks=r_extent_hooks@entry=0x7f50197f84d8, extents=extents@entry=0x7f5019a03268, npages_min=npages_min@entry=18533) at src/extent.c:572 StarRocks#10 0x0000000006936004 in arena_stash_decayed (decay_extents=<synthetic pointer>, npages_decay_max=492, npages_limit=<optimized out>, extents=0x7f5019a03268, r_extent_hooks=0x7f50197f84d8, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:833 StarRocks#11 arena_decay_to_limit (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, decay=decay@entry=0x7f5019a06ca0, extents=extents@entry=0x7f5019a03268, all=all@entry=false, npages_limit=<optimized out>, npages_decay_max=<optimized out>, is_background_thread=<optimized out>) at src/arena.c:934 StarRocks#12 0x00000000069363fb in arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:722 StarRocks#13 arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:611 StarRocks#14 arena_maybe_decay (is_background_thread=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:762 StarRocks#15 arena_maybe_decay (tsdn=<optimized out>, arena=0x7f5019a00980, decay=0x7f5019a06ca0, extents=0x7f5019a03268, is_background_thread=<optimized out>) at src/arena.c:714 StarRocks#16 0x00000000069389ab in arena_decay_impl (all=false, is_background_thread=true, extents=0x7f5019a03268, decay=0x7f5019a06ca0, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:964 StarRocks#17 arena_decay_dirty (all=false, is_background_thread=true, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:985 StarRocks#18 je_arena_decay (tsdn=0x7f50197f9040, arena=0x7f5019a00980, is_background_thread=true, all=<optimized out>) at src/arena.c:998 StarRocks#19 0x00000000069403e9 in background_work_sleep_once (ind=0, info=<optimized out>, tsdn=<optimized out>) at src/background_thread.c:295 StarRocks#20 background_thread0_work (tsd=<optimized out>) at src/background_thread.c:452 StarRocks#21 background_work (ind=<optimized out>, tsd=<optimized out>) at src/background_thread.c:490 StarRocks#22 background_thread_entry () at src/background_thread.c:522 StarRocks#23 0x00007f501ad71ea5 in start_thread () from /lib64/libpthread.so.0 StarRocks#24 0x00007f501a38cb0d in clone () from /lib64/libc.so.6 ``` upgrade jemalloc from 5.2.1 to 5.3.0 to fix the deadlock: jemalloc/jemalloc#2019 Redis also encountered this problem redis/redis#12115 Signed-off-by: trueeyu <[email protected]> (cherry picked from commit 33b1342)
abc982627271
pushed a commit
that referenced
this pull request
Jul 10, 2023
…StarRocks#26631) VExtLiteral is not deleted. ``` ==76869==ERROR: LeakSanitizer: detected memory leaks Direct leak of 40 byte(s) in 1 object(s) allocated from: #0 0x9793a37 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99 #1 0x129dd44e in operator()<int const> /root/starrocks/be/src/exec/es/es_predicate.cpp:416 #2 0x129dd817 in build_inpred_values<(starrocks::LogicalType)5, starrocks::EsPredicate::_build_in_predicate(const starrocks::Expr*, bool*)::<lambda(auto:29&)> > /root/starrocks/be/src/exec/es/es_predicate.cpp:366 #3 0x129d7824 in starrocks::EsPredicate::_build_in_predicate(starrocks::Expr const*, bool*) /root/starrocks/be/src/exec/es/es_predicate.cpp:416 StarRocks#4 0x129d2f2d in starrocks::EsPredicate::_vec_build_disjuncts_list(starrocks::Expr const*) /root/starrocks/be/src/exec/es/es_predicate.cpp:185 StarRocks#5 0x129d2a53 in starrocks::EsPredicate::build_disjuncts_list() /root/starrocks/be/src/exec/es/es_predicate.cpp:148 StarRocks#6 0x128ab50d in starrocks::connector::ESDataSource::_build_conjuncts() /root/starrocks/be/src/connector/es_connector.cpp:123 StarRocks#7 0x128aa7c5 in starrocks::connector::ESDataSource::open(starrocks::RuntimeState*) /root/starrocks/be/src/connector/es_connector.cpp:89 StarRocks#8 0x9e33847 in starrocks::pipeline::ConnectorChunkSource::_open_data_source(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/connector_scan_operator.cpp:504 StarRocks#9 0x9e33ed0 in starrocks::pipeline::ConnectorChunkSource::_read_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk>*) /root/starrocks/be/src/exec/pipeline/scan/connector_scan_operator.cpp:533 StarRocks#10 0xabc8b65 in starrocks::pipeline::ChunkSource::buffer_next_batch_chunks_blocking(starrocks::RuntimeState*, unsigned long, starrocks::workgroup::WorkGroup const*) /root/starrocks/be/src/exec/pipeline/scan/chunk_source.cpp:67 StarRocks#11 0x9e168c7 in operator() /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:396 StarRocks#12 0x9e1c157 in __invoke_impl<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#13 0x9e1c005 in __invoke_r<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#14 0x9e1be7a in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#15 0x992a271 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#16 0x9e519fa in starrocks::workgroup::ScanExecutor::worker_thread() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:67 StarRocks#17 0x9e5121f in operator() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:31 StarRocks#18 0x9e52c5d in __invoke_impl<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#19 0x9e52925 in __invoke_r<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#20 0x9e5249a in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#21 0x992a271 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#22 0x1181f23d in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58 StarRocks#23 0x1181bff2 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#24 0x11837c4b in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#25 0x118375a4 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10. 3.0/bits/invoke.h:95 StarRocks#26 0x1183699b in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#27 0x118352fd in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#28 0x11832361 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#29 0x1182fcc5 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 ``` Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Jul 12, 2023
Fixes #issue ``` Thread 912 (Thread 0x7f50197ff700 (LWP 3944)): #0 0x00007f501ad7854d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f501ad73e9b in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f501ad73d68 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000000006059a27 in pthread_mutex_lock_impl (mutex=<optimized out>) at src/bthread/mutex.cpp:564 StarRocks#4 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:818 StarRocks#5 pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:817 StarRocks#6 0x000000000697819f in malloc_mutex_lock_final (mutex=0x7f5019a03268) at include/jemalloc/internal/mutex.h:155 StarRocks#7 je_malloc_mutex_lock_slow (mutex=mutex@entry=0x7f5019a03268) at src/mutex.c:85 StarRocks#8 0x000000000696f8c8 in malloc_mutex_lock (mutex=0x7f5019a03268, tsdn=0x7f50197f9040) at include/jemalloc/internal/mutex.h:221 StarRocks#9 je_extents_evict (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, r_extent_hooks=r_extent_hooks@entry=0x7f50197f84d8, extents=extents@entry=0x7f5019a03268, npages_min=npages_min@entry=18533) at src/extent.c:572 StarRocks#10 0x0000000006936004 in arena_stash_decayed (decay_extents=<synthetic pointer>, npages_decay_max=492, npages_limit=<optimized out>, extents=0x7f5019a03268, r_extent_hooks=0x7f50197f84d8, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:833 StarRocks#11 arena_decay_to_limit (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, decay=decay@entry=0x7f5019a06ca0, extents=extents@entry=0x7f5019a03268, all=all@entry=false, npages_limit=<optimized out>, npages_decay_max=<optimized out>, is_background_thread=<optimized out>) at src/arena.c:934 StarRocks#12 0x00000000069363fb in arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:722 StarRocks#13 arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:611 StarRocks#14 arena_maybe_decay (is_background_thread=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:762 StarRocks#15 arena_maybe_decay (tsdn=<optimized out>, arena=0x7f5019a00980, decay=0x7f5019a06ca0, extents=0x7f5019a03268, is_background_thread=<optimized out>) at src/arena.c:714 StarRocks#16 0x00000000069389ab in arena_decay_impl (all=false, is_background_thread=true, extents=0x7f5019a03268, decay=0x7f5019a06ca0, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:964 StarRocks#17 arena_decay_dirty (all=false, is_background_thread=true, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:985 StarRocks#18 je_arena_decay (tsdn=0x7f50197f9040, arena=0x7f5019a00980, is_background_thread=true, all=<optimized out>) at src/arena.c:998 StarRocks#19 0x00000000069403e9 in background_work_sleep_once (ind=0, info=<optimized out>, tsdn=<optimized out>) at src/background_thread.c:295 StarRocks#20 background_thread0_work (tsd=<optimized out>) at src/background_thread.c:452 StarRocks#21 background_work (ind=<optimized out>, tsd=<optimized out>) at src/background_thread.c:490 StarRocks#22 background_thread_entry () at src/background_thread.c:522 StarRocks#23 0x00007f501ad71ea5 in start_thread () from /lib64/libpthread.so.0 StarRocks#24 0x00007f501a38cb0d in clone () from /lib64/libc.so.6 ``` upgrade jemalloc from 5.2.1 to 5.3.0 to fix the deadlock: jemalloc/jemalloc#2019 Redis also encountered this problem redis/redis#12115 Signed-off-by: trueeyu <[email protected]> (cherry picked from commit 33b1342)
abc982627271
pushed a commit
that referenced
this pull request
Jul 12, 2023
When load channel not open yet, the `_row_desc` will be null, avoid dereference it when `add_chunk`. This will happen when rpc packets arrive not in order. crash stack: ``` (gdb) bt #0 0x00000000057cd3e2 in starrocks::serde::build_protobuf_chunk_meta(starrocks::RowDescriptor const&, starrocks::ChunkPB const&) () #1 0x0000000004d1ee07 in starrocks::LoadChannel::_build_chunk_meta(starrocks::ChunkPB const&) () #2 0x0000000004d21816 in starrocks::LoadChannel::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () #3 0x0000000004d1abb2 in starrocks::LoadChannelMgr::add_chunk(starrocks::PTabletWriterAddChunkRequest const&, starrocks::PTabletWriterAddBatchResult*) () StarRocks#4 0x0000000004dd0835 in starrocks::BackendInternalServiceImpldoris::PBackendService::tablet_writer_add_chunk(google::protobuf::RpcController*, starrocks::PTabletWriterAddChunkRequest const*, starrocks::PTabletWriterAddBatchResult*, google::protobuf::Closure*) () ``` (cherry picked from commit d5b5eae)
abc982627271
pushed a commit
that referenced
this pull request
Jul 12, 2023
…StarRocks#26631) VExtLiteral is not deleted. ``` ==76869==ERROR: LeakSanitizer: detected memory leaks Direct leak of 40 byte(s) in 1 object(s) allocated from: #0 0x9793a37 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99 #1 0x129dd44e in operator()<int const> /root/starrocks/be/src/exec/es/es_predicate.cpp:416 #2 0x129dd817 in build_inpred_values<(starrocks::LogicalType)5, starrocks::EsPredicate::_build_in_predicate(const starrocks::Expr*, bool*)::<lambda(auto:29&)> > /root/starrocks/be/src/exec/es/es_predicate.cpp:366 #3 0x129d7824 in starrocks::EsPredicate::_build_in_predicate(starrocks::Expr const*, bool*) /root/starrocks/be/src/exec/es/es_predicate.cpp:416 StarRocks#4 0x129d2f2d in starrocks::EsPredicate::_vec_build_disjuncts_list(starrocks::Expr const*) /root/starrocks/be/src/exec/es/es_predicate.cpp:185 StarRocks#5 0x129d2a53 in starrocks::EsPredicate::build_disjuncts_list() /root/starrocks/be/src/exec/es/es_predicate.cpp:148 StarRocks#6 0x128ab50d in starrocks::connector::ESDataSource::_build_conjuncts() /root/starrocks/be/src/connector/es_connector.cpp:123 StarRocks#7 0x128aa7c5 in starrocks::connector::ESDataSource::open(starrocks::RuntimeState*) /root/starrocks/be/src/connector/es_connector.cpp:89 StarRocks#8 0x9e33847 in starrocks::pipeline::ConnectorChunkSource::_open_data_source(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/connector_scan_operator.cpp:504 StarRocks#9 0x9e33ed0 in starrocks::pipeline::ConnectorChunkSource::_read_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk>*) /root/starrocks/be/src/exec/pipeline/scan/connector_scan_operator.cpp:533 StarRocks#10 0xabc8b65 in starrocks::pipeline::ChunkSource::buffer_next_batch_chunks_blocking(starrocks::RuntimeState*, unsigned long, starrocks::workgroup::WorkGroup const*) /root/starrocks/be/src/exec/pipeline/scan/chunk_source.cpp:67 StarRocks#11 0x9e168c7 in operator() /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:396 StarRocks#12 0x9e1c157 in __invoke_impl<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#13 0x9e1c005 in __invoke_r<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#14 0x9e1be7a in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#15 0x992a271 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#16 0x9e519fa in starrocks::workgroup::ScanExecutor::worker_thread() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:67 StarRocks#17 0x9e5121f in operator() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:31 StarRocks#18 0x9e52c5d in __invoke_impl<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#19 0x9e52925 in __invoke_r<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#20 0x9e5249a in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#21 0x992a271 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#22 0x1181f23d in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58 StarRocks#23 0x1181bff2 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#24 0x11837c4b in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#25 0x118375a4 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10. 3.0/bits/invoke.h:95 StarRocks#26 0x1183699b in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#27 0x118352fd in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#28 0x11832361 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#29 0x1182fcc5 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 ``` Signed-off-by: trueeyu <[email protected]> (cherry picked from commit ff551fb)
abc982627271
pushed a commit
that referenced
this pull request
Jul 31, 2023
Fixes [StarRocks#3000](StarRocks/StarRocksTest#3000) ``` ==2390==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f3c65102a14 at pc 0x00000a0c5def bp 0x7f3c65101c40 sp 0x7f3c65101c38 READ of size 4 at 0x7f3c65102a14 thread T197 (load_rpc) #0 0xa0c5dee in starrocks::RuntimeState::chunk_size() const /root/starrocks/be/src/runtime/runtime_state.h:123 #1 0xd0a9cdb in starrocks::ORCScanner::open() /root/starrocks/be/src/exec/orc_scanner.cpp:84 #2 0x12b6dcff in starrocks::PInternalServiceImplBase<doris::PBackendService>::_get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*) /root/starrocks/be/src/service/internal_service.cpp:818 #3 0x12b7a02f in starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}::operator()() const /root/starrocks/be/src/service/internal_service.cpp:757 StarRocks#4 0x12b89b89 in void std::__invoke_impl<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>(std::__invoke_other, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#5 0x12b87158 in std::enable_if<is_invocable_r_v<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>, std::enable_if>::type std::__invoke_r<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>(void&&, (starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&)...) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#6 0x12b83377 in std::_Function_handler<void (), starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#7 0x99f88bd in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#8 0xa436e73 in starrocks::FunctionRunnable::run() (/home/disk1/sr/be/lib/starrocks_be+0xa436e73) StarRocks#9 0xa4339c8 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#10 0xa450765 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#11 0xa450250 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:95 StarRocks#12 0xa44f199 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#13 0xa44e107 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#14 0xa44bc61 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xa44bc61) StarRocks#15 0xa4485cf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#16 0xa4448ba in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#17 0x99f88bd in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#18 0xa41b326 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364 StarRocks#19 0x7f3cc51b5ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4) StarRocks#20 0x7f3cc47d0b0c in clone (/lib64/libc.so.6+0xfeb0c) ``` Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Aug 15, 2023
…ks#28929) Fixes [#issue](StarRocks/StarRocksTest#3087) ss.str() is one local var, already released before execute `fill_column_with_slot`. ``` crash log: ==2761==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000ca1280 at pc 0x000009bdfe27 bp 0x7f566078ad80 sp 0x7f566078ad78 READ of size 1 at 0x60b000ca1280 thread T243 (pip_wg_scan_io) #0 0x9bdfe26 in decltype (::new ((void*)(0)) unsigned char((declval<char&>)())) std::construct_at<unsigned char, char&>(unsigned char*, char&) /opt/gcc/usr/include/c++/10.3.0/bits/stl_construct.h:97 #1 0x9bdfeaa in void std::allocator_traits<std::allocator<unsigned char> >::construct<unsigned char, char&>(std::allocator<unsigned char>&, unsigned char*, char&) /opt/gcc/usr/include/c++/10.3.0/bits/alloc_traits.h:514 #2 0x9bdf441 in void starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> >::construct<unsigned char, char&>(unsigned char*, char&) /root/starrocks/be/src/util/raw_container.h:69 #3 0x9bdca44 in std::enable_if<std::__and_<std::allocator_traits<starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::__construct_helper<unsigned char, char&>::type>::value, void>::type std::allocator_traits<starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::_S_construct<unsigned char, char&>(starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> >&, unsigned char*, char&) /opt/gcc/usr/include/c++/10.3.0/bits/alloc_traits.h:247 StarRocks#4 0x9bd9f73 in decltype (_S_construct({parm#1}, {parm#2}, (forward<char&>)({parm#3}))) std::allocator_traits<starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::construct<unsigned char, char&>(starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> >&, unsigned char*, char&) /opt/gcc/usr/include/c++/10.3.0/bits/alloc_traits.h:360 StarRocks#5 0x9bd4921 in unsigned char* std::__uninitialized_copy_a<char*, unsigned char*, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >(char*, char*, unsigned char*, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> >&) /opt/gcc/usr/include/c++/10.3.0/bits/stl_uninitialized.h:311 StarRocks#6 0x9bc9ad8 in void std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::_M_range_insert<char*>(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > > >, char*, char*, std::forward_iterator_tag) /opt/gcc/usr/include/c++/10.3.0/bits/vector.tcc:778 StarRocks#7 0x9bbac6e in void std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::_M_insert_dispatch<char*>(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > > >, char*, char*, std::__false_type) /opt/gcc/usr/include/c++/10.3.0/bits/stl_vector.h:1665 StarRocks#8 0x9ba647c in __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > > > std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > >::insert<char*, void>(__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, starrocks::raw::RawAllocator<unsigned char, 16ul, std::allocator<unsigned char> > > >, char*, char*) /opt/gcc/usr/include/c++/10.3.0/bits/stl_vector.h:1383 StarRocks#9 0x9b8358f in starrocks::BinaryColumnBase<unsigned int>::append(starrocks::Slice const&) /root/starrocks/be/src/column/binary_column.h:167 StarRocks#10 0xd24e6d1 in void starrocks::fill_data_column_with_slot<(starrocks::LogicalType)17>(starrocks::Column*, void*) /root/starrocks/be/src/exec/schema_scanner/schema_helper.h:116 StarRocks#11 0xd24e23d in void starrocks::fill_column_with_slot<(starrocks::LogicalType)17>(starrocks::Column*, void*) /root/starrocks/be/src/exec/schema_scanner/schema_helper.h:127 StarRocks#12 0xd2808d3 in starrocks::SchemaLoadTrackingLogsScanner::fill_chunk(std::shared_ptr<starrocks::Chunk>*) /root/starrocks/be/src/exec/schema_scanner/schema_load_tracking_logs_scanner.cpp:112 StarRocks#13 0xd281dca in starrocks::SchemaLoadTrackingLogsScanner::get_next(std::shared_ptr<starrocks::Chunk>*, bool*) /root/starrocks/be/src/exec/schema_scanner/schema_load_tracking_logs_scanner.cpp:160 StarRocks#14 0xd3f5dd5 in starrocks::pipeline::OlapSchemaChunkSource::_read_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk>*) /root/starrocks/be/src/exec/pipeline/scan/olap_schema_chunk_source.cpp:132 StarRocks#15 0xd3d0b33 in starrocks::pipeline::ChunkSource::buffer_next_batch_chunks_blocking(starrocks::RuntimeState*, unsigned long, starrocks::workgroup::WorkGroup const*) /root/starrocks/be/src/exec/pipeline/scan/chunk_source.cpp:67 StarRocks#16 0xc3deac7 in operator() /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:400 StarRocks#17 0xc3e4345 in __invoke_impl<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#18 0xc3e41f3 in __invoke_r<void, starrocks::pipeline::ScanOperator::_trigger_next_scan(starrocks::RuntimeState*, int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#19 0xc3e4068 in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#20 0x9ae6d83 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#21 0xc916608 in starrocks::workgroup::ScanExecutor::worker_thread() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:70 StarRocks#22 0xc915e2d in operator() /root/starrocks/be/src/exec/workgroup/scan_executor.cpp:34 StarRocks#23 0xc917de3 in __invoke_impl<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#24 0xc9179b2 in __invoke_r<void, starrocks::workgroup::ScanExecutor::initialize(int)::<lambda()>&> /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#25 0xc91734f in _M_invoke /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#26 0x9ae6d83 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#27 0xa533efd in starrocks::FunctionRunnable::run() /root/starrocks/be/src/util/threadpool.cpp:58 StarRocks#28 0xa5309bc in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#29 0xa54d7ef in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#30 0xa54d2da in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:95 StarRocks#31 0xa54c223 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#32 0xa54b191 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#33 0xa548ceb in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xa548ceb) StarRocks#34 0xa545659 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#35 0xa541944 in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#36 0x9ae6d83 in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#37 0xa518286 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364 StarRocks#38 0x7f56ec0b0ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4) StarRocks#39 0x7f56eb6cbb0c in clone (/lib64/libc.so.6+0xfeb0c) ``` Signed-off-by: trueeyu <[email protected]>
abc982627271
pushed a commit
that referenced
this pull request
Oct 7, 2023
Fixes [StarRocks#3000](StarRocks/StarRocksTest#3000) ``` ==2390==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f3c65102a14 at pc 0x00000a0c5def bp 0x7f3c65101c40 sp 0x7f3c65101c38 READ of size 4 at 0x7f3c65102a14 thread T197 (load_rpc) #0 0xa0c5dee in starrocks::RuntimeState::chunk_size() const /root/starrocks/be/src/runtime/runtime_state.h:123 #1 0xd0a9cdb in starrocks::ORCScanner::open() /root/starrocks/be/src/exec/orc_scanner.cpp:84 #2 0x12b6dcff in starrocks::PInternalServiceImplBase<doris::PBackendService>::_get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*) /root/starrocks/be/src/service/internal_service.cpp:818 #3 0x12b7a02f in starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}::operator()() const /root/starrocks/be/src/service/internal_service.cpp:757 StarRocks#4 0x12b89b89 in void std::__invoke_impl<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>(std::__invoke_other, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:60 StarRocks#5 0x12b87158 in std::enable_if<is_invocable_r_v<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>, std::enable_if>::type std::__invoke_r<void, starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&>(void&&, (starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}&)...) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#6 0x12b83377 in std::_Function_handler<void (), starrocks::PInternalServiceImplBase<doris::PBackendService>::get_file_schema(google::protobuf::RpcController*, starrocks::PGetFileSchemaRequest const*, starrocks::PGetFileSchemaResult*, google::protobuf::Closure*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#7 0x99f88bd in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#8 0xa436e73 in starrocks::FunctionRunnable::run() (/home/disk1/sr/be/lib/starrocks_be+0xa436e73) StarRocks#9 0xa4339c8 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:553 StarRocks#10 0xa450765 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:73 StarRocks#11 0xa450250 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:95 StarRocks#12 0xa44f199 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/gcc/usr/include/c++/10.3.0/functional:416 StarRocks#13 0xa44e107 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/gcc/usr/include/c++/10.3.0/functional:499 StarRocks#14 0xa44bc61 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xa44bc61) StarRocks#15 0xa4485cf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/gcc/usr/include/c++/10.3.0/bits/invoke.h:110 StarRocks#16 0xa4448ba in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:291 StarRocks#17 0x99f88bd in std::function<void ()>::operator()() const /opt/gcc/usr/include/c++/10.3.0/bits/std_function.h:622 StarRocks#18 0xa41b326 in starrocks::Thread::supervise_thread(void*) /root/starrocks/be/src/util/thread.cpp:364 StarRocks#19 0x7f3cc51b5ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4) StarRocks#20 0x7f3cc47d0b0c in clone (/lib64/libc.so.6+0xfeb0c) ``` Signed-off-by: trueeyu <[email protected]> (cherry picked from commit 9635e6e)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this：
Which issues of this PR fixes ：
Fixes #
Problem Summary(Required) ：