Bazel, Runfiles, and C++ TestMate - how to use properly?

[tmaslach]

This is more of a question right now than it is a feature request, though it may be that as well after some discussion.

I am using bazel to build our company's software. I also love C++ Testmate and have used it while developing. We have figured out how to use C++ TestMate with bazel using an execution wrapper. Here is the code for setting that up:

                "testMate.cpp.test.advancedExecutables": [
                    {
                        "pattern": "bazel-bin/**/*.runfiles/**/*{test}",
                        "cwd": "${workspaceFolder}",
                        "executionWrapper": {
                            "path": "${workspaceFolder}/.vscode/run_unit_test.sh",
                            "args": [ "${cmd}", "${argsFlat}" ]
                        }
                    }
                ],

This enables us to discover tests and run tests from C++ TestMate with no issue. It also allows us to use the debugger for some tests, but not all.

The subset of test that do not work require that the *.runfiles folder (listed in pattern above) be the cwd. This seems to be a bazel requirement. Two examples of paths:

bazel-bin/component1/test/component1_test.runfiles/repo/component1/test/component1_test
or
bazel-bin/component2/subcomponent2/test/subcomponent2_test.runfiles/repo/component2/subcomponent2/test/subcomponent2_test

Note in the above two examples the number of directories between the *.runfiles folder and test is variable.

When running via 'bazel test', it automatically sets the cwd properly. But, I can't use that when trying to debug via C++ TestMate (or can I somehow?) I'd much rather use the VSCode IDE, so I'm hopign there is a way to do this.

I think (and I could be wrong here) a good solution would be to make the cwd in advance executables be the *.runfiles folder. However, I don't believe this is possible. Can I call a bash command to tweak the cwd string (assuming I make it the executable folder?).

For just straight running tests and discovery, I did do this logic in the execution wrapper and it just works. Unfortunately, I don't know how to do it for the debugging case.

Any thoughts/help would be appreciated! Thanks!

[tmaslach]

Is there perhaps a way for me to run gdb via a gdbwrapper (kind of like the execution wrapper?). That would give me the flexibility in solving the problem. AFAIK, VSCode Json doesn't support a way of dynamically trimming the cwd path.

[matepek]

could you give me a simple bazel repo which contains some tests and I can compile and run it on macos?
it would be a great help.

[tmaslach]

Hi @matepek, I've finally gotten around to creating an example for you. Hope this works for you, as I haven't tested on a mac.

Unzip this file:
testmatev2.zip

Go into the root of that workspace. From the internal readme:

SETUP:

1. Install Bazelisk
   On Mac, brew install bazelisk
   On Linux (can do Mac this way, too), download from here. File can be placed right in repo folder:
   https://github.com/bazelbuild/bazelisk/releases
2. Make sure you have gcc installed? Different versions will work, I'm using gcc 9.4.0. MacOS may point to clang
   automatically, which I think will work fine. This one is a leap of faith, since I don't know how to test without Mac

RECREATING:

I'm going to assume you have bazelisk in your path.

1. cd into the root of the workspace
2. To build: bazelisk build --config=debug //... - Can skip this as #3 will build, too.
   This may take a minute or two depending on machine speed/cores.
3. To run tests: bazelisk test --config=debug //... . Just to see it work
4. Now, go into testmate - you should see one test
5. Hit play on that test. It should run and pass.
   • This is going through .vscode/run_unit_test.sh, which cds into the runfiles folder
6. Hit debug. It will fail.

It fails for two reasons:

1. The executable name is actually testmate_test_impl, not testmate_test. This is a ros2 thing. testmate_test
   is a setup script. We mix ros2 tests with standard cc_tests, so somtimes we want to debug the _impl, other times
   that file doesn't exist. I might be able to hack this to work somehow, but wanted to make you aware.
   If I could get a hook via script to override the name, I could code up the logic myself, so you aren't having to do
   anything bazel specific.

2. To workaround #1 and see the second issue, change the search pattern from:

   "pattern": "bazel-bin/**/*.runfiles/**/*{test,Test,TEST,tests}",
   

   to:

   "pattern": "bazel-bin/**/*.runfiles/**/*{test,Test,TEST,tests}_impl",
   

   This is a hack as not all tests end with impl, but good enough for now. Make sure to reload tests.
   Now, try debugging the test. It'll fail. This is because rclcpp::init is being called while we are in
   the wrong folder. The runfiles folder needs to be the cwd.

   You can see that it fails with the following. From the root of the workspace after building:

   cd bazel-bin/code/testmate_test.runfiles/testmate_test/code
   ./testmate_test_impl

   The test will fail. HOWEVER, if you go up one folder and run from there, it will work:

   cd .. # or cd bazel-bin/code/testmate_test.runfiles/testmate_test if you didn't run above example
   ./code/testmate_test_impl

   If I could customize the cwd somehow, that would fix this issue as well.

[matepek]

Gonna check, thanks

[matepek]

Hello,

ERROR: Error computing the main repository mapping: no such package '@com_github_mvukov_rules_ros2//repositories': error running 'git fetch origin refs/heads/*:refs/remotes/origin/* refs/tags/*:refs/tags/*' while working with @com_github_mvukov_rules_ros2:
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights

[tmaslach]

testmatev2.zip

Whoops, I had one bad assumption in there that would make it work for me, but not you. Try this one, it should work.
I updated the link in the above post as well.

[matepek]

Le me summarise the problem.
So the main issue that the exec cannot find the dynamic library unless it is under the right path and finding out that wht is the right path is tricky with bazel.

[tmaslach]

Not 100% sure if the issue is not finding a dynamic library or some other configuration file. I haven't looked under the ROS2 hood much

There are two key issues. The one you noted, where finding the right path is tricky. The second is that while the test target name is testmate_test, the executable name is testmate_test_impl. ros2_cpp_test does this because it wants to have the script setup some env vars and such. I believe I can fake this in other ways, if it becomes necessary. For now, it seems to have little impact.

More background information in case it helps, but not related to your question. I'm not sure how much you know about bazel:

• Bazel is all about having hermetic builds. It wants to build the unit test and have it exist in the repo with it's dependecies and nothing else in the repo. It does this by mirroring a subset of the repo structure in the runfiles folder

• If you look in bazel-bin/code/testmate_test.runfiles, you'll see testmate_test. testmate_test.runfiles by the way is named after the test target. The testmate_test inside the runfiles folder is named after the workspace name, found in the root WORKSPACE file. Sorry, I should've named them differently.

• bazel test CLI command always runs from the root of the workspace in runfiles. That is: bazel-bin/code/testmate_test.runfiles/testmate_test

• I will always be searching in runfiles folders for *_test executables. Most of the time, they are standard cc_test. All will work. For ros2_cpp_test, it won't work because it's a script. If I had the right bash script hook, I'd just have a script transform the name if it finds a corresponding impl file. There may be cleaner ways to do this.

• Finding the path to launch from should be easy - I just climb the directory structure until I get to one level below a folder matching the patter *.runfiles. As mentioned above, bazel-bin/code/testmate_test.runfiles/testmate_test

Hope this information is helpful!

[tmaslach]

We have an approach now for debugging through the launch.json.

    "configurations": [
        {
            "name": "(gdb) Debug Bazel Binary",
            "preLaunchTask": "Build Binary Run Script (Debug)",
            "type": "cppdbg",
            "request": "launch",
            "program": "/bin/bash",
            "args": ["/tmp/debug_launch"],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "sourceFileMap": {
                "/proc/self/cwd": "${workspaceFolder}"
            },
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                },
                {
                    "description": "Set Disassembly Flavor to Intel",
                    "text": "-gdb-set disassembly-flavor intel",
                    "ignoreFailures": true
                }
            ]
        }
}

The pre launch task builds the /tmp/debug_launch script via bazel. And, we always debug /bin/bash. The launch script uses exec to run the test executable, so we can transparently jump into our program to debug.

This works great for the "Run and Debug" side bar. But, is there a way to specify these params so that they work through TestMate? I believe this would create a seamless experience for bazel developers if there were a way to specify these.

Thanks!

PS. I don't think this will be a complete solution, I need to think through the scenario some more. But, I think this will help me get one step further.

[matepek]

I'm a bit tired maybe that's why I don't get your comment yet. I will re-read it later..

Meanwhile, have your checked this?

[tmaslach]

Let me dig into that. I didn't realize the configTemplate was possible, but that might be what I need to get things working.