Type resolution from DWARF

[viktormalik]

When tracing userspace programs, this extension automatically resolves types from DWARF (if it is available). This has mostly 2 advantages:

• bpftrace program doesn't need to define the types (or include the headers) used in the traced binary
• uprobe arguments of struct type can be accessed by name

Version 2:
Parsing of structure types from DWARF is done in FieldAnalyser, which directly fills structs in StructManager. ClangParser may override these types if user defined some custom types or included some headers.

Version 1 (abandoned):
Since we're relying on Clang parser to do the actual type resolution, we need to dump types from DWARF into compilable C format. Even though there probably exist libraries that can do this (e.g. libdwarves), I decided to implement this by hand, to have more control over the supported types and to avoid yet another dependency.
We now have two sources of type information - BTF and DWARF. To avoid potential collisions, for now we use DWARF for userspace tracing and BTF for kernel tracing.

I also believe that this resolves #1742.

Checklist

• Language changes are updated in man/adoc/bpftrace.adoc and if needed in docs/reference_guide.md
• User-visible and non-trivial changes updated in CHANGELOG.md
• The new behaviour is covered by tests

[danobi]

What's the rationale for converting DWARF to C and then parsing C to bpftrace-internal representation? I would have thought that going from DWARF -> bpftrace-internal representation ought to be faster (at runtime) and easier.

FWIW, the reason we go BTF -> C -> bpftrace-internal representation is b/c libbpf already had the C dumping API and it was easier to code from bpftrace's side.

[viktormalik]

What's the rationale for converting DWARF to C and then parsing C to bpftrace-internal representation? I would have thought that going from DWARF -> bpftrace-internal representation ought to be faster (at runtime) and easier.

The reason is that the final type resolution is done by Clang parser. So even if we resolved part of the types from DWARF, we still need to dump them to C b/c their defs may be necessary for the parser (e.g. if the user used some of them in his custom types).

[danobi]

(e.g. if the user used some of them in his custom types).

Hmm, is this a likely scenario? What types could the user be defining that are not already available in DWARF? I'm not necessarily against going to C then to clang parser, but I think it should be a conscious tradeoff. If it's significantly less complex to skip C, then we ought to consider that. If it's similar complexity, then going to C has theoretical advantages. What do you think?

[viktormalik]

(e.g. if the user used some of them in his custom types).

Hmm, is this a likely scenario? What types could the user be defining that are not already available in DWARF? I'm not necessarily against going to C then to clang parser, but I think it should be a conscious tradeoff. If it's significantly less complex to skip C, then we ought to consider that. If it's similar complexity, then going to C has theoretical advantages. What do you think?

I agree that this seems to be a very unlikely situation (in fact, I haven't come with a reasonable example so far). What I'd suggest is:

• if there are no user-defined types, skip the ClangParser and resolve all from DWARF (and we can eventually do the same for BTF, too)
• if there are some user-defined types or includes, use ClangParser with DWARF->C or BTF->C dumping

Does it make sense?

[danobi]

Sure, that sounds fine to me

[viktormalik]

After some time, coming with version 2.

The main change is that parsing of structure types from DWARF is now done in FieldAnalyser, which directly fills structs in StructManager. ClangParser is not changed by this PR, so it will run if there is something to parse (i.e. user provided some type definitions or included some headers). Since ClangParser is run after FieldAnalyser, the effect of this is that user may override DWARF-parsed types, if he wishes to.

[danobi]

Not very familiar w/ DWARF but nothing stands out to me. Great work!