[BUG] Nested @task invocations should not supported

[eapolinario]

Describe the bug

The documentation defines tasks as the units of execution. This doesn't explicitly disallow nested tasks, in other words, it's reasonable to assume that some users will extrapolate the assertion to cover nested tasks as well (i.e. separate tasks running in separate environments/containers).

For example, in this code:

@task
def nested_task() -> int:
  return 42

@task
def t1() -> int:
  return nested_task()

@workflow
def wf() -> int:
  return t1()

nested_task runs in the same process as task, as though it was a local execution of that task.

Expected behavior

Users should receive some indication that nested tasks are not allowed. A log message should suffice.

Maybe worth calling out in the documentation as well.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[eapolinario]

The documentation might call this case out explicitly and offer dynamic tasks as an alternative.

[cosmicBboy]

offer dynamic tasks as an alternative

ahem dynamic workflows

[oliverhu]

Need some help on this task...it seems there is no existing variable in FlyteContext to tell whether a task is executed inside a @task or @workflow or invoked directly, so the approach seems to be adding a flag for that in the context, that seems like a non-trivial change. Is that the expected way?

[oliverhu]

@eapolinario @pingsutw please check flyteorg/flytekit#1727 if that is the right way to address this issue! Not sure if this is a good first issue, but it is a good exploration issue for the code base...

[eapolinario]

flyteorg/flytekit#1727 fixes this and will go out in 1.9.0.