[🚀 Feature]: Implement high level BiDi network commands

[titusfortner]

Feature and motivation

At the Selenium Dev Summit we agreed on this API to be generally applied across the bindings; we'll keep this labeled beta while we make sure that it works for what is needed

We want the methods to be accessible from a network() method available directly from the Driver class (e.g., driver.network.addRequestHandler(), driver.network().addRequestHandler()). We can't do everything just like this in all the languages, because, for example, .NET uses events with a += and -= for adding and removing handler events so we don't went "add" and "remove" methods.

Implementations:

Method	Java	NodeJS	Python	Ruby	.NET
addRequestHandler()
removeRequestHandler()
clearRequestHandlers()
addResponseHandler()
removeResponseHandler()
clearResponseHandlers()
addAuthenticationHandler()	#14334	#14345		#14523
removeAuthenticationHandler()	#14334	#14345		#14523
clearAuthenticationHandlers()	#14334	#14345		#14523

Considerations:

If we can figure out how to get the "add" methods return an id that can be used by the "remove" methods, that would be a lot easier for or users. Might be too complicated to implement.

[titusfortner]

I've been thinking about this, and I think the add methods should return an id. This is going to be much easier for languages that want to pass in lambdas without storing a reference to the location os the object in memory.

This is how moz/addon/install endpoint works and the id gets passed into the payload to remove the addon.

I was looking for common wisdom here and it generally seems to match — https://softwareengineering.stackexchange.com/questions/314066/restful-api-should-i-be-returning-the-object-that-was-created-updated

[p0deje]

I've been thinking about this, and I think the add methods should return an id. This is going to be much easier for languages that want to pass in lambdas without storing a reference to the location os the object in memory.

Totally fine by me!

[pujagani]

Can someone help refresh my memory on the "addRequestHandler()"? - I vaguely remember us discussing the current C# implementation using the matcher and transformer https://github.com/SeleniumHQ/seleniumhq.github.io/blob/c6769ef064d8bc3ffb6343fba3c10ab4ddbad78c/examples/dotnet/SeleniumDocs/BiDi/CDP/NetworkTest.cs#L85. Is that what we want? How would that look like?

[pujagani]

@titusfortner @diemol @p0deje

[p0deje]

Can someone help refresh my memory on the "addRequestHandler()"? - I vaguely remember us discussing the current C# implementation using the matcher and transformer https://github.com/SeleniumHQ/seleniumhq.github.io/blob/c6769ef064d8bc3ffb6343fba3c10ab4ddbad78c/examples/dotnet/SeleniumDocs/BiDi/CDP/NetworkTest.cs#L85. Is that what we want? How would that look like?

I might be wrong, but IIRC the idea was to let addRequestHandler to accept two arguments:

• filter/matcher for request to be handled
• callback that receives an original request and return a handled request

[nvborisenko]

I've been thinking about this, and I think the add methods should return an id. This is going to be much easier for languages that want to pass in lambdas without storing a reference to the location os the object in memory.

I guess id can be any object. In .net I implemented it like:

var subscription = await context.Network.OnBeforeRequestSentAsync(Console.WriteLine);
await context.NavigateAsync("https://selenium.dev");
await subscription.UnsubscribeAsync();

Or network interception:

await using var intercept = await context.Network.AddInterceptedRequestAsync(async args =>
{
  await args.Request.Request.ContinueAsync(new() { Method = "POST" });
});

Subscription is a disposable object. The same for Intercept object. Generally saying it is object, not just magic ID.

[junjuping]

Can I use the INetwork-NetworkResponseReceived feature in Selenium. WebDriver version 4.24.0 in Firefox?

[bogishvili]

Hi I am trying to replace devtools with BiDi in my project. Can I get request and response bodies from BeforeRequestSent and ResponseDetails? any form like json or byte sequence or string would be great.

[pujagani]

@bogishvili I understand your use case here. However, we can support what is available in the WebDriver BiDi spec https://w3c.github.io/webdriver-bidi/#module-network. Currently, there is no way defined to get the request and response bodies. There is an issue to track this though w3c/webdriver-bidi#747. So this can be supported in the future once it is present in the spec and supported by the browsers.