Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chromium NetworkService API Changes #2743

Closed
amaitland opened this issue Apr 24, 2019 · 11 comments · Fixed by #2809
Closed

Chromium NetworkService API Changes #2743

amaitland opened this issue Apr 24, 2019 · 11 comments · Fixed by #2809

Comments

@amaitland
Copy link
Member

amaitland commented Apr 24, 2019

Chromium is moving from an in-process network stack implementation to a separate NetworkService process.

Due to Chromium changes CEF will be implementing some dramatic API changes. Relevant upstream issue is https://bitbucket.org/chromiumembedded/cef/issues/2622/implement-support-for-networkservice

Chromium is moving to a Network Service, this means that network requests will be handled using a separate process, just like rendering, gpu acceleration, etc.

Known behaviour changes:

  • Modifying the newUrl parameter in OnResourceRedirect will no longer result in the method being called an additional time (likely a bug in the old implementation).
  • Modifying the request URL in OnResourceResponse would previously cause a redirect. This behaviour is now deprecated because the NetworkService does not support this functionality when using default network loaders. Temporary support has been added in combination with IResourceHandler usage only. - Other changes to the request object in OnResourceResponse will now cause the request to be restarted. This means that OnBeforeResourceLoad, etc, will be called an additional time with the new request information.
  • IResponse.MimeType will now be empty for non-200 responses.
  • Requests using custom schemes can now be handled via IResourceRequestHandler with the same callback behaviour as builtin schemes.
  • Redirects of custom scheme requests will now be followed as expected.
  • Default handling of builtin schemes can now be disabled by setting disableDefaultHandling to true in GetResourceRequestHandler.
  • Unhandled requests (custom scheme or builtin scheme with default handling disabled) will fail with an Response.Error value of ERR_UNKNOWN_URL_SCHEME.
  • The ISchemeHandlerFactory.Create callback will now include cookie headers.
  • Unsupported chrome hosts no longer redirect to chrome://version.
  • All CachePath values must be relative to a new CefSettings.RootCachePath value.
  • ICookieManager callbacks are now executed on the CEF UI thread (previously the CEF IO thread).
  • Resource-related callbacks have been moved from IRequestHandler to a new IResourceRequestHandler interface which is returned via the IRequestHandler.GetResourceRequestHandler method. If the IRequestHandler declines to handle a resource it can optionally be handled by the IRequestContextHandler, if any, associated with the loading context.
  • The OnProtocolExecution callback has been moved from IRequestHandler to IResourceRequestHandler and will be called if a custom scheme request is unhandled.
  • Cookie send/save permission callbacks have been moved from IRequestHandler and IResourceHandler to IResourceRequestHandler.
  • New methods added to IResourceHandler that better match NetworkService execution sequence expectations. The old methods are now deprecated.
  • New methods added to IRequest and IResponse.
  • All pending and in-progress requests will now be aborted when the CEF context
    or associated browser is destroyed. The OnResourceLoadComplete callback will
    now also be called in this case for in-progress requests that have a handler.
  • The IResourceHandler.Cancel method will now always be called when resource
    handling is complete, irrespective of whether handling completed successfully.
  • Request callbacks that arrive after the OnBeforeClose callback for the
    associated browser (which may happen for in-progress requests that are aborted
    on browser destruction) will now always have a non-nullptr IBrowser
    parameter.
  • Allow empty parameters to IRequest and IResponse methods where it makes
    sense (e.g. resetting default response state, or clearing a referrer value).
  • Modifying the URL in OnBeforeResourceLoad causes an internal redirect response.
  • In cases where the request is cross-origin (containing a non-null "Origin"
    header) the redirect response must include the "Access-Control-Allow-Origin"
    header, otherwise the request will be blocked.
  • For 303 redirects all request methods except HEAD are converted to GET as per
    the latest http draft. For historical reasons the draft also allows POST
    requests to be converted to GETs when following 301/302 redirects. Most major
    browsers do this and so shall we. When a request is converted to GET any POST
    data should also be removed.
  • Use 307 redirects instead if you want the request to be repeated using the same
    method and POST data.

API changes:

  • Add IResourceRequestHandler
  • Add ICookieAccessFilter
  • IRequestHandler.OnBeforeResourceLoad moved to IResourceRequestHandler.OnBeforeResourceLoad
  • IRequestHandler.OnResourceRedirect moved to IResourceRequestHandler.OnResourceRedirect
  • IRequestHandler.OnResourceResponse moved to IResourceRequestHandler.OnResourceResponse
  • IRequestHandler.GetResourceResponseFilter moved to IResourceRequestHandler.GetResourceResponseFilter
  • IRequestHandler.OnResourceLoadComplete moved to IResourceRequestHandler.OnResourceLoadComplete
  • IRequestHandler.OnProtocolExecution moved to IResourceRequestHandler.OnProtocolExecution
  • IRequestHandler.GetAuthCredentials signature change, IFrame frame replaced with string originUrl
  • IDragHandler.OnDraggableRegionsChanged now includes the IFrame param

Cookie Handling

  • Cookies are access is now handled using ICookieAccessFilter which is returned using IResourceRequestHandler.GetCookieAccessFilter
  • Removed IRequestHandler.CanGetCookie
  • Removed IRequestHandler.CanSetCookie
  • Removed IResourceHandler.CanGetCookie
  • Removed IResourceHandler.CanSetCookie
  • Removed ICookieManager.CreateManager
  • Removed ICookieManager.GetBlockingManager
  • Removed ICookieManager.SetStoragePath
  • Removed IRequestContextHandler.GetCookieManager
  • IRequestContext.GetDefaultCookieManager renamed to IRequestContext.GetCookieManager
  • ICookieManager.SetSupportedSchemes has additional bool includeDefaults param.

Request/Scheme Handling

The IRequestHandler interface has been rewritten as the CEF API changed

ResourceHandler Headers

IRequest now has SetHeaderByName and GetHeaderByName which greatly simplifies header get/set.

CefSharp Specific API Changes

  • IResourceHandler.ProcessRequestAsync now returns CefReturnValue instead of bool.
  • IResourceHandler.ProcessRequestAsync is now called in IResourceHandler.Open which is a new method, as the structure has changed.
  • IResourceHandler.GetResponse has been removed
  • 'DefaultRequestHandler' renamed to RequestHandler (DefaultRequestHandler exists as a class for now that's marked as Obsolete)
  • RequestHandler now implements IRequestHandler explicitly an exposes protected virtual methods to override, previously they were public.
  • ISchemeRegistrar.AddCustomScheme now takes a set of SchemeOptions flags to represent the different options instead of a bool for each param type. Brings the interface in line with the CEF API.
  • CefSharp.Legacy.LegacyResourceHandlerFactory has been removed.
@mitchcapper
Copy link
Contributor

From a documentation clarifying point of view even if rootcachepath is set to a real path, CachePath can still be set empty for using in-memory cache.

@amaitland
Copy link
Member Author

From a documentation clarifying point of view even if rootcachepath is set to a real path, CachePath can still be set empty for using in-memory cache.

Thanks 👍 It's probably worth seeing if the CEF documentation can be revised, there are a few inconsistencies/clarifications required.

If non-empty this must be either equal to or a child directory of CefSettings.root_cache_path

Which implies you have to set CefSettings.root_cache_path, which isn't the case, you can leave it blank and it will default to CefSettings.cache_path as per If this value is empty and CefSettings.cache_path is non-empty then this value will default to the CefSettings.cache_path value

I've made some improvements in c0ebbb4 Hopefully it makes sense, feedback welcome.

@amaitland
Copy link
Member Author

amaitland commented Jun 14, 2019

Experiencing a strange issue where CefFrame::GetIdentifier() returns a different value in the browser process than it does in the render process. As a result IJavascriptCallback's are broken currently. I've asked https://magpcss.org/ceforum/viewtopic.php?f=6&t=16830 for clarification and that's not the expected behaviour, will test with cefclient to confirm that it's reproducible.

Experiencing a strange issue where CefFrame::GetIdentifier() returns a different value in the browser process than it does in the render process. As a result IJavascriptCallback's are broken currently. I've asked https://magpcss.org/ceforum/viewtopic.php?f=6&t=16830 for clarification and that's not the expected behaviour, will test with cefclient to confirm that it's reproducible.

Not the expected behaviour, opened issue https://bitbucket.org/chromiumembedded/cef/issues/2687/cefframe-getidentifier-differs-between

To allow IJavascriptCallback in the main frame (and unfortunately only the main frame), I've hacked around the problem see 9d47c2e

This should be reverted when the upstream issue is resolved.

  • Revert hack
  • Confirm issue resolved.

@amaitland
Copy link
Member Author

amaitland commented Jun 16, 2019

List of know CEF issues

Network Service
https://bitbucket.org/chromiumembedded/cef/issues/2703/crash-when-closing-if
https://bitbucket.org/chromiumembedded/cef/issues/2715/onprotocolexecution-page-goes-blank-after

https://bitbucket.org/chromiumembedded/cef/issues/2685/networkservice-custom-scheme-unable-to
https://bitbucket.org/chromiumembedded/cef/issues/2668/multi-threaded-message-loop-crashes-again
https://bitbucket.org/chromiumembedded/cef/issues/2316/crash-during-shutdown-due-to
https://bitbucket.org/chromiumembedded/cef/issues/2718/networkservice-proxy-and-request
https://bitbucket.org/chromiumembedded/cef/issues/2720/networkservice-does-not-honor-the
https://bitbucket.org/chromiumembedded/cef/issues/2699/networkservice-issue-with-cross-origin
https://bitbucket.org/chromiumembedded/cef/issues/2708/windows-networkservice-crash-in
https://bitbucket.org/chromiumembedded/cef/issues/2689/networkservice-unpredictable-behavior-with
https://bitbucket.org/chromiumembedded/cef/issues/2709/windows-networkservice-crash-on-shutdown
https://bitbucket.org/chromiumembedded/cef/issues/2727/networkservice-pdfs-not-loading-with-and
https://bitbucket.org/chromiumembedded/cef/issues/2605/cef-crashes-on-cefcontentrendererclient

Unrelated to network service
https://bitbucket.org/chromiumembedded/cef/issues/2678/cefclient-crashes-with-out-of-memory-when
https://bitbucket.org/chromiumembedded/cef/issues/2276/drag-and-drop-is-broken-in-combination
https://bitbucket.org/chromiumembedded/cef/issues/2662/crash-on-mouse-event-from-devtools-remote

amaitland added a commit to amaitland/CefSharp that referenced this issue Jun 17, 2019
JavascriptCallback's are limited to only working in the main frame
Frame ids aren't the same between processes, so makes it almost impossible to handle callbacks in sub frames

https://bitbucket.org/chromiumembedded/cef/issues/2687/cefframe-getidentifier-differs-between
cefsharp#2743 (comment)

This should be reverted after the problem has been resolved.
@helpr helpr bot removed the pr-available label Jun 25, 2019
amaitland added a commit that referenced this issue Jun 25, 2019
JavascriptCallback's are limited to only working in the main frame
Frame ids aren't the same between processes, so makes it almost impossible to handle callbacks in sub frames

https://bitbucket.org/chromiumembedded/cef/issues/2687/cefframe-getidentifier-differs-between
#2743 (comment)

This should be reverted after the problem has been resolved.
@helpr helpr bot added the pr-merged label Jun 25, 2019
@amaitland amaitland reopened this Jun 25, 2019
@amaitland
Copy link
Member Author

amaitland commented Jul 4, 2019

The BrowserSubProcess has been updated as, the new Network Service process needs to handle custom scheme registration.

amaitland added a commit that referenced this issue Jul 4, 2019
The network services requires that custom schemes be registered, previously we only registered them in render processes.

Add SubProcessApp to handle scheme registration
CefAppUnmanagedWrapper inherits SubProcessApp now

#2743
@amaitland amaitland pinned this issue Jul 25, 2019
@amaitland
Copy link
Member Author

  • New RequestHandler is available to override only the methods you require
  • New ResourceRequestHandler is available to override only the methods you require
  • ICookieAccessFilter is used to control cookie access. There is no default implementation as it currently only has two methods and you likely need to implement both of them. Might add a default implementation later if there is enough need for one.

@amaitland
Copy link
Member Author

Changes are complete, any bugs please open a new issue making sure to use the Bug Report Template

@amaitland
Copy link
Member Author

For reference it's still possible to disable the Network Service, see https://github.com/cefsharp/CefSharp/blob/cefsharp/75/CefSharp.Example/CefExample.cs#L108 for an example for both WPF and WinForms (For OffScreen use the WPF example).

This will only work with version 75, this won't do anything for past or future versions.

amaitland added a commit that referenced this issue Sep 29, 2019
…nges

- MemoryStreamResponseFilter renamed to StreamResponseFilter
- Stream is now passed into StreamResponseFilter constructor
- ExampleResourceRequestHandler updated to reflect that it now represents a single resource request
- ExampleResourceRequestHandler is now used for all requests from the CefSharpSchemeHandlerFactory.SchemeName custom scheme

Issue #2743
amaitland added a commit that referenced this issue Sep 30, 2019
…nges

- MemoryStreamResponseFilter renamed to StreamResponseFilter
- Stream is now passed into StreamResponseFilter constructor
- ExampleResourceRequestHandler updated to reflect that it now represents a single resource request
- ExampleResourceRequestHandler is now used for all requests from the CefSharpSchemeHandlerFactory.SchemeName custom scheme

Issue #2743
@amaitland
Copy link
Member Author

The ResponseFilter examples have been updated to reflect the NetworkService changes, see 195569d for the relevant commit.

ResourceRequestHandler maps to a single Resource Request and as such there is no need for a dictionary to track IResponseFilter implementations like in the previous example. IResponseFilter.Dispose is also called before OnResourceLoadComplete now, the example has been adapted to deal with this.

@joaompneves
Copy link
Contributor

joaompneves commented Nov 5, 2019

For reference it's still possible to disable the Network Service, see https://github.com/cefsharp/CefSharp/blob/cefsharp/75/CefSharp.Example/CefExample.cs#L108 for an example for both WPF and WinForms (For OffScreen use the WPF example).

This will only work with version 75, this won't do anything for past or future versions.

Tried disabling the Network Service, as you mentioned, (due to a severe libcef crash: https://bitbucket.org/chromiumembedded/cef/issues/2795/crash-in-openinputstreamwrapper) but I cannot get custom schemes to work. Navigation aborts right away with an unknown custom schema error code. Probably expected, or not? (Most likely a cef issue, but still asking here)

@joaompneves
Copy link
Contributor

I had to put back the SchemeHandlerFactory when creating the CustomScheme (removed when upgraded to 75) and had to implement my own ResourceHandler because the CefSharp builtin ResourceHandler has deprecated the ReadResponse and ProcessRequest methods, and those seem to be used by the old network layer.

@amaitland amaitland unpinned this issue Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants