Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unset tracing variables after finalizing databases. #1137

Merged
merged 1 commit into from
Jul 12, 2022

Conversation

criemen
Copy link
Contributor

@criemen criemen commented Jul 12, 2022

The tracer is very good at preserving itself, so unsetting the tracing-specific
variables from within a process will not end tracing for children of
that process.
The way the actions process model works means that we're running inside
a process for the entire build step that was launched with the tracer
variables set, so we'll have the tracer injected into the entire build
step and its children.
If we unset the variables in end-tracing, we will get into an intermediate
state: Not all variables in there are preserved by the tracer,
but the tracer is still active.
Usually, that wouldn't be a problem, but the autobuilders called from
the finalize step will suddenly run under a half-configured tracer.

Particularly, this half-configured tracer is unable to execute the dotnet
CLI without hangs, as the environment variable that prevents hangs for
dotnet on MacOS has been unset, but the tracer is still active.

This is an issue for the the go autobuilder, that invokes
user-provided build scripts in the hope of installing dependencies.
If that build script then invokes dotnet, it will hang.

This is only of concern for the Lua tracer that now implements proper
multi-language tracing: Previously, when encountering the go autobuilder,
the tracer disabled itself entirely, thus side-stepping any hangs.
In the new, multi-language tracing world, the tracer will stay active
as long as there is at least one other language that's been set up
for tracing.
Thus, we also get hangs when invoking the dotnet CLI through the go
autobuilder.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Confirm the readme has been updated if necessary.
  • Confirm the changelog has been updated if necessary.

@criemen criemen requested a review from a team as a code owner July 12, 2022 10:52
The tracer is very good at preserving itself, so unsetting the tracing-specific
variables from within a process will not end tracing for children of
that process.
The way the actions process model works means that we're running inside
a process for the entire build step that was launched with the tracer
variables set, so we'll have the tracer injected into the entire build
step and its children.
If we unset the variables in end-tracing, we will get into an intermediate
state: Not all variables in there are preserved by the tracer,
but the tracer is still active.
Usually, that wouldn't be a problem, but the autobuilders called from
the finalize step will suddenly run under a half-configured tracer.

Particularly, this half-configured tracer is unable to execute the dotnet
CLI without hangs, as the environment variable that prevents hangs for
dotnet on MacOS has been unset, but the tracer is still active.

This is an issue for the the go autobuilder, that invokes
user-provided build scripts in the hope of installing dependencies.
If that build script then invokes dotnet, it will hang.

This is only of concern for the Lua tracer that now implements proper
multi-language tracing: Previously, when encountering the go autobuilder,
the tracer disabled itself entirely, thus side-stepping any hangs.
In the new, multi-language tracing world, the tracer will stay active
as long as there is at least one other language that's been set up
for tracing.
Thus, we also get hangs when invoking the dotnet CLI through the go
autobuilder.
Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the change is to move the end tracing code until after the database is finalized. This allows tracing to continue for other languages that are transitively kicked off during the finalize process?

I think this makes sense but just want to ensure i understand this.

@criemen
Copy link
Contributor Author

criemen commented Jul 12, 2022

Short answer, yes:

  • The finalize process runs the autobuilders for interpreted languages and go (go is unfortunately a bit of a special case, which is tech debt I hope we can address soon)
  • The tracer is present for that anyways because there's no way to currently get rid of the tracer once it's been injected, but if we do unset the variables from end-tracing, we break the tracer in subtle ways, as the tracer doesn't preserve all environment variables on its own, only the ones it deems most critical

Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation.

@criemen criemen merged commit e1ec697 into main Jul 12, 2022
@criemen criemen deleted the criemen/fix-end-tracing branch July 12, 2022 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants