Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: batch query with limit: internal error: channel closed #4132

Closed
fuyufjh opened this issue Jul 25, 2022 · 4 comments · Fixed by #4187 or #4371
Closed

bug: batch query with limit: internal error: channel closed #4132

fuyufjh opened this issue Jul 25, 2022 · 4 comments · Fixed by #4187 or #4371
Assignees
@liurenjie1024 @BowenXiao1999
Labels
component/batch Batch related related issue. type/bug Something isn't working

Comments

@fuyufjh
Copy link
Member

fuyufjh commented Jul 25, 2022

Describe the bug

On executing

select * from orders limit 1;

The results looks good but an error is printed in log.

2022-07-25T05:34:06.359856Z ERROR risingwave_batch::task::task_execution: Execution failed [TaskId { task_id: 0, stage_id: 1, query_id: "c10083d3-efee-48bb-8257-9078273d4f92" }]: internal error: FifoSender::send: channel closed
   0: std::backtrace_rs::backtrace::libunwind::trace
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2: std::backtrace::Backtrace::create
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/backtrace.rs:328:13
   3: <risingwave_common::error::RwError as core::convert::From<risingwave_common::error::ErrorCode>>::from
             at ./src/common/src/error.rs:231:33
   4: <T as core::convert::Into<U>>::into
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/convert/mod.rs:550:9
   5: <core::result::Result<T,E> as risingwave_common::error::ToRwResult<T,E>>::to_rw_result_with::{{closure}}
             at ./src/common/src/error.rs:425:13
   6: core::result::Result<T,E>::map_err
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/result.rs:855:27
   7: <core::result::Result<T,E> as risingwave_common::error::ToRwResult<T,E>>::to_rw_result_with
             at ./src/common/src/error.rs:424:9
   8: <risingwave_batch::task::fifo_channel::FifoSender as risingwave_batch::task::channel::ChanSender>::send::{{closure}}
             at ./src/batch/src/task/fifo_channel.rs:39:13
   9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  10: risingwave_batch::task::channel::ChanSenderImpl::send::{{closure}}
             at ./src/batch/src/task/channel.rs:49:53
  11: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  12: risingwave_batch::task::task_execution::BatchTaskExecution<C>::try_execute::{{closure}}
             at ./src/batch/src/task/task_execution.rs:301:59
  13: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  14: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:345:35
  15: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:231:19
  16: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:342:13
  17: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-futures-0.2.5/src/lib.rs:283:9
  18: risingwave_batch::task::task_execution::BatchTaskExecution<C>::async_execute::{{closure}}::{{closure}}::{{closure}}
             at ./src/batch/src/task/task_execution.rs:265:23
  19: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  20: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/future.rs:124:9
  21: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.34/src/instrument.rs:272:9
  22: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:165:17
  23: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/loom/std/unsafe_cell.rs:14:9
  24: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:155:13
  25: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:480:19
  26: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  27: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  28: ___rust_try
  29: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  30: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  31: tokio::runtime::task::harness::poll_future
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:468:18
  32: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:104:27
  33: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:57:15
  34: tokio::runtime::task::raw::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:144:5
  35: tokio::runtime::task::raw::RawTask::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:84:18
  36: tokio::runtime::task::LocalNotified<S>::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/mod.rs:381:9
  37: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:458:21
  38: tokio::coop::with_budget::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:102:9
  39: std::thread::local::LocalKey<T>::try_with
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/local.rs:445:16
  40: std::thread::local::LocalKey<T>::with
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/local.rs:421:9
  41: tokio::coop::with_budget
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:95:5
  42: tokio::coop::budget
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:72:5
  43: tokio::runtime::thread_pool::worker::Context::run_task
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:434:9
  44: tokio::runtime::thread_pool::worker::Context::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:401:24
  45: tokio::runtime::thread_pool::worker::run::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:386:17
  46: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/macros/scoped_tls.rs:61:9
  47: tokio::runtime::thread_pool::worker::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:383:5
  48: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:362:45
  49: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/task.rs:42:21
  50: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.34/src/instrument.rs:272:9
  51: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:165:17
  52: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/loom/std/unsafe_cell.rs:14:9
  53: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:155:13
  54: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:480:19
  55: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  56: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  57: ___rust_try
  58: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  59: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  60: tokio::runtime::task::harness::poll_future
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:468:18
  61: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:104:27
  62: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:57:15
  63: tokio::runtime::task::raw::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:144:5
  64: tokio::runtime::task::raw::RawTask::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:84:18
  65: tokio::runtime::task::UnownedTask<S>::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/mod.rs:418:9
  66: tokio::runtime::blocking::pool::Task::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:91:9
  67: tokio::runtime::blocking::pool::Inner::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:325:17
  68: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:300:13
  69: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/sys_common/backtrace.rs:122:18
  70: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/mod.rs:505:17
  71: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  72: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  73: ___rust_try
  74: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  75: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  76: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/mod.rs:504:30
  77: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/ops/function.rs:248:5
  78: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/alloc/src/boxed.rs:1935:9
  79: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/alloc/src/boxed.rs:1935:9
  80: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/sys/unix/thread.rs:108:17
  81: __pthread_deallocate

2022-07-25T05:34:06.359904Z ERROR risingwave_batch::task::task_execution: Execution failed [TaskId { task_id: 1, stage_id: 1, query_id: "c10083d3-efee-48bb-8257-9078273d4f92" }]: internal error: FifoSender::send: channel closed
   0: std::backtrace_rs::backtrace::libunwind::trace
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2: std::backtrace::Backtrace::create
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/backtrace.rs:328:13
   3: <risingwave_common::error::RwError as core::convert::From<risingwave_common::error::ErrorCode>>::from
             at ./src/common/src/error.rs:231:33
   4: <T as core::convert::Into<U>>::into
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/convert/mod.rs:550:9
   5: <core::result::Result<T,E> as risingwave_common::error::ToRwResult<T,E>>::to_rw_result_with::{{closure}}
             at ./src/common/src/error.rs:425:13
   6: core::result::Result<T,E>::map_err
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/result.rs:855:27
   7: <core::result::Result<T,E> as risingwave_common::error::ToRwResult<T,E>>::to_rw_result_with
             at ./src/common/src/error.rs:424:9
   8: <risingwave_batch::task::fifo_channel::FifoSender as risingwave_batch::task::channel::ChanSender>::send::{{closure}}
             at ./src/batch/src/task/fifo_channel.rs:39:13
   9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  10: risingwave_batch::task::channel::ChanSenderImpl::send::{{closure}}
             at ./src/batch/src/task/channel.rs:49:53
  11: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  12: risingwave_batch::task::task_execution::BatchTaskExecution<C>::try_execute::{{closure}}
             at ./src/batch/src/task/task_execution.rs:301:59
  13: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  14: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:345:35
  15: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:231:19
  16: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/task/task_local.rs:342:13
  17: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-futures-0.2.5/src/lib.rs:283:9
  18: risingwave_batch::task::task_execution::BatchTaskExecution<C>::async_execute::{{closure}}::{{closure}}::{{closure}}
             at ./src/batch/src/task/task_execution.rs:265:23
  19: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/mod.rs:91:19
  20: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/future/future.rs:124:9
  21: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.34/src/instrument.rs:272:9
  22: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:165:17
  23: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/loom/std/unsafe_cell.rs:14:9
  24: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:155:13
  25: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:480:19
  26: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  27: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  28: ___rust_try
  29: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  30: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  31: tokio::runtime::task::harness::poll_future
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:468:18
  32: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:104:27
  33: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:57:15
  34: tokio::runtime::task::raw::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:144:5
  35: tokio::runtime::task::raw::RawTask::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:84:18
  36: tokio::runtime::task::LocalNotified<S>::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/mod.rs:381:9
  37: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:458:21
  38: tokio::coop::with_budget::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:102:9
  39: std::thread::local::LocalKey<T>::try_with
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/local.rs:445:16
  40: std::thread::local::LocalKey<T>::with
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/local.rs:421:9
  41: tokio::coop::with_budget
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:95:5
  42: tokio::coop::budget
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/coop.rs:72:5
  43: tokio::runtime::thread_pool::worker::Context::run_task
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:434:9
  44: tokio::runtime::thread_pool::worker::Context::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:408:24
  45: tokio::runtime::thread_pool::worker::run::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:386:17
  46: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/macros/scoped_tls.rs:61:9
  47: tokio::runtime::thread_pool::worker::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:383:5
  48: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/thread_pool/worker.rs:362:45
  49: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/task.rs:42:21
  50: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.34/src/instrument.rs:272:9
  51: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:165:17
  52: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/loom/std/unsafe_cell.rs:14:9
  53: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/core.rs:155:13
  54: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:480:19
  55: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  56: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  57: ___rust_try
  58: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  59: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  60: tokio::runtime::task::harness::poll_future
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:468:18
  61: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:104:27
  62: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/harness.rs:57:15
  63: tokio::runtime::task::raw::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:144:5
  64: tokio::runtime::task::raw::RawTask::poll
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/raw.rs:84:18
  65: tokio::runtime::task::UnownedTask<S>::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/task/mod.rs:418:9
  66: tokio::runtime::blocking::pool::Task::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:91:9
  67: tokio::runtime::blocking::pool::Inner::run
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:325:17
  68: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.0/src/runtime/blocking/pool.rs:300:13
  69: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/sys_common/backtrace.rs:122:18
  70: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/mod.rs:505:17
  71: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/panic/unwind_safe.rs:271:9
  72: std::panicking::try::do_call
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:492:40
  73: ___rust_try
  74: std::panicking::try
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panicking.rs:456:19
  75: std::panic::catch_unwind
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/panic.rs:137:14
  76: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/thread/mod.rs:504:30
  77: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/core/src/ops/function.rs:248:5
  78: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/alloc/src/boxed.rs:1935:9
  79: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/alloc/src/boxed.rs:1935:9
  80: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/263edd43c5255084292329423c61a9d69715ebfa/library/std/src/sys/unix/thread.rs:108:17
  81: __pthread_deallocate

To Reproduce

select * from orders limit 1;

Expected behavior

No error logs should be produced.

Additional context

None
@fuyufjh fuyufjh added type/bug Something isn't working component/batch Batch related related issue. labels Jul 25, 2022
@liurenjie1024
Copy link
Contributor

Seems difficult to reproduce.

@fuyufjh
Copy link
Member Author

fuyufjh commented Jul 26, 2022

I can reproduce it stably with local playground (./risedev p)

@liurenjie1024
Copy link
Contributor

Yes, I'm working on this. It happens on ./risedev p, but not on distributed environment.

@liurenjie1024 liurenjie1024 mentioned this issue Jul 26, 2022
Merged
3 tasks
@mergify mergify bot closed this as completed in #4187 Jul 26, 2022
@BowenXiao1999
Copy link
Contributor

Seems like this case is not only for limit.

Cuz #4187 now ignore the send error and only warn! it, we can still observe the warn message WARN risingwave_batch::task::task_execution: Task receiver closed!.

Reproduce:

  1. ./risedev p
  2. ./risedev slt -p 4566 -u root -d dev './e2e_test/batch/basic/basic.slt.part'

We can still observe the warn! message

@BowenXiao1999 BowenXiao1999 reopened this Jul 27, 2022
@liurenjie1024 liurenjie1024 mentioned this issue Aug 2, 2022
Merged
3 tasks
@mergify mergify bot closed this as completed in #4371 Aug 2, 2022
@BowenXiao1999 BowenXiao1999 mentioned this issue Aug 2, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@liurenjie1024 liurenjie1024

@BowenXiao1999 BowenXiao1999

Labels
component/batch Batch related related issue. type/bug Something isn't working
Projects
None yet
Milestone
No milestone
3 participants
@liurenjie1024 @fuyufjh @BowenXiao1999