About continues_on, when_all, then and issues with Domains

[DavidElesTheFox]

Motivation

I was playing around different schedulers and creating senders. I was ending up in a situation where I wanted to join two senders where one was a kind of background task(coroutine). I ended up basically with a simple when_all expression:

auto final_sender = ex::when_all(std::move(do_something), std::move(background_task))
    | ex::then(join_results_fn); 
auto [result] = ex::sync_wait(std::move(final_sender))

Devil lives in the details

How the background task looks like is a bit tricky. I mean, I need to schedule there a new sender. So, I tried to embed there the task:

auto bg_task = ex::let_value(
            ex::read_env(ex::get_scheduler),
                        [p_dependency = std::move(dependency)]( auto p_scheduler) mutable
            {
                return ex::starts_on(p_scheduler, std::move(p_dependency));
            }
        );

My other sender was a simple just call

auto start_point = ex::just(3);

So, my intention was to read the scheduler from the environment and start there the background sender (called dependency).

And it worked well when the dependency was a a simple coroutine or another sender. Here is a working example

The interesting things

I've started to play around different schedulers and I find that sometimes it works and sometimes it doesn't. And of course it was not clear the why:

starts_on

auto start_point = ex::starts_on(pool.get_scheduler(), ex::just(3));

First, I thought I'm using starts_on when the first sender is creating. It worked fine, no issues.
(see here)

continues_on

Then I tried to say not start on but using the continues_on API call. So, simulating that let's run the source where it was scheduled, then continue the things on 'my' scheduler.

auto start_point = continues_on(ex::just(3), pool.get_scheduler());

And it blown up during an async_wait call. The error message was about not common domains. (see the example here

The why

After finding this documentation about customizing senders and browsing the implementations, I think I understand the issue.
I'm just pasting here the code, to explain my current understanding (having some baseline to the conversation :))

    auto start_point = ex::continues_on(ex::just(3), pool.get_scheduler());
    auto join_fn =  [](int x, int y) {return x + y;};
    auto bg_task = ex::let_value(
            ex::read_env(ex::get_scheduler),
                        [p_dependency = std::move(dependency)]( auto p_scheduler) mutable
            {
                return ex::starts_on(p_scheduler, std::move(p_dependency));
            }
        );
    auto final_sender = ex::when_all(std::move(bg_task), std::move(start_point)) 
        | ex::then(std::move(join_fn));

    auto [result] = ex::sync_wait(std::move(final_sender)).value();

But first some terminology sync:

• Environment: 'Early' customization goes here. This is the environment basically where the sender defined.
• Domain: 'Late' customization goes here. So this is the domain where the sender will run (i.e.: where the receiver receives it, i,e: where the scheduler schedules it)

And back to the code:

• start_point environment is a so called dependent environment. While it is not clear yet. Its domain is the local-pool.
• bg_task: uses let_value. It preserve the original domain(scheduler), so its domain is coming from the sync_wait.

And this is an issue, while these domains are really not common.

Comparing this with starts_on:

• start_point: it uses also a let_value so it also preserves the domain when the when_all is called.
• bg_task is the same.

And it works, because the two senders domain are the same.

The discussion

Maybe it is all about learning and when the documentations/examples gonna be more in the final state it won't be an issue anymore. But right now from first look it was really surprising and the error message looked not related to the change and I think it's not gonna be change by documentation.

I was thinking a lot what's the source of the confusion and what I realized:
When you read the code you can see continues_on(a_sender, scheduler). Because one does not see 'What?' is gonna continues on the given scheduler, it is easy to interpret it as a starts_on. I think it is because all the other cases the Object is the parameter of the function (starts_on(where, what), when_all(what, what), then(what). But here the Object is not the parameter, because what is continuing is coming in the next operation (continues_on(who, where) | then(what)).

about the origin: transfer

I've seen that the function was recently renamed from transfer to continues_on. Without the intention to rename it again to transfer, It is a fact that in case of transfer the Object was the function parameter and I think it fits better to the other API functions. (But I believe that there were several discussions about it already).

About transfer_when_all

I've also seen the function transfer_when_all and it brings me to the other idea, what is the situation when you would like to continue to do something on somewhere without knowing, what is the something? So, what if there is no transfer or continues_on function, but only transfer_when_all (I.e.: Finish your things and bring me to that context) and transfer_then (I want to continue on this scheduler and execute that) functions?

Leave it as it is

And of course there is the option to leave it as it is and the documentation will make it clear. But my thoughts on it that people are reading code first and that should be clear as far as it is possible and then comes the documentation. But, I know naming is the hardest thing and maybe the current one is the less confusing from another perspective.

What do you think?

[ericniebler]

thank you for raising these issues. it has brought my attention to the fact that stdexec's handling of the common domain requirement is not in conformance with the c++26 working draft.

the following PR should help: #1451

after i fix the bugs we can properly discuss whether the current behavior makes sense or not.

[DavidElesTheFox]

I like the fix, but I have the feeling that it's gonna fix only the issue when the scheduler is the static_thread_pool. I think if the static thread pool would use a common pool base - what we were discussing earlier and did not forget to finish the implementation :) - would make the situation much better.

But I was also thinking about a potential improvement. I.e.: The restriction of the when_all about having a common domain is fair, otherwise there is no domain where to run the when_all. But actually transfer_when_all doesn't really need this restriction, it can use the scheduler/domain what is given as parameter.

I've made a small branch. Here I made a test first:

      auto snd = ex::transfer_when_all(scheduler_2{},
        ex::transfer_just(scheduler_1{}, 3),
        ex::transfer_just(scheduler_3{}, 0.1415)
      );

Currently, transfer_when_all first makes a when_all and then a continues_on. But actually it could make first many continues_on and then a when_all. So, basically it can make a common domain. (the branch contains this change as well, on the latest commit)

A side effect for sure: that when_all will run on the scheduler that was given. Before, it was running on the common_domain.

What do you think? I'm happy to make a PR from the branch.

[ericniebler]

it's best to ignore that the transform_xxx algorithms exist. they were removed from std::execution many moons ago. i should probably deprecate them in stdexec.

to the extent that it matters at all, transform_when_all(sched, sndrs...) should behave exactly like when_all(sndrs...) | continues_on(sched).

[ericniebler]

And it blown up during an async_wait call. The error message was about not common domains. (see the example here)

The code in the linked godbolt compiles now. Do you have other examples that you think should work that currently don't?

[DavidElesTheFox]

Hey Eric,

Sorry, going offline during the New Year holiday season went too well and I totally missed this message. Thanks for the support and the fix. And no I don't have any other particular issues.