Chrome 72 breaks download

[outlineruler]

I am using Chrome Version 72.0.3626.81 (Official Build) (64-bit) and immediately after the upgrade the credentials download stopped working. I attempted to pull logs from the extension console using instructions listed in previous issues of this kind but there is not any output.

[nitrocode]

Is there anything that was removed recently that this depended on?

https://developers.google.com/web/updates/2018/12/chrome-72-deps-rems

[prolane]

I've updated to Chrome 72 myself and I can reproduce the issue.

What I see happening is Chrome is still trying to create a new download (as is supposed to happen), but then the download fails. I've also verified the actual content of the credentials file is still there using this extra debug line:

samltoawsstskeys/background/script.js

Lines 221 to 222 in 215fb5e

	console.log('INFO: Now going to download doc...');
	console.log(docContent)

In other words, the issue is in either of these two lines of code:

samltoawsstskeys/background/script.js

Lines 223 to 225 in 215fb5e

	var doc = URL.createObjectURL( new Blob([docContent], {type: 'application/octet-binary'}) );
	// Triggers download of the generated file
	chrome.downloads.download({ url: doc, filename: FileName, conflictAction: 'overwrite', saveAs: false });

When I have the time, I'll investigate further and fix it. Suggestions on why chrome 72 breaks this one line of code is welcome of course.

[prolane]

Quick update:

samltoawsstskeys/background/script.js

Line 223 in 215fb5e

var doc = URL.createObjectURL( new Blob([docContent], {type: 'application/octet-binary'}) );

This goal for this line of code is to create a local 'downloadable' object. The URL that is generated is in the form of chrome-extension://chjpchfchpknieidncedaigaednecjlg/d87e96e1-d12d-4e27-86e3-d3c549a1ef37.

This is not working anymore. It does seem to generate the object, as it seems to create an uuid for the object (d87e96e1-d12d-4e27-86e3-d3c549a1ef37), but when trying to download from the URL that is generated it will fail. ("File not found" is what Chrome says when I try the URL manually in the address bar).

I guess all in all we'll have to find an alternate way of pushing the credentials into a file.

[nitrocode]

That sounds like a bug with the native createObjectURL or one of the parameters. Do you see a difference its function arguments between chrome 71 vs 72?

[prolane]

I did some more experiments. There is no difference in the createObjectURL function arguments. However, I did confirm there is a difference in how Chrome handles requesting the URL coming out of the createObjectURL function.

First of all, for anyone who is not aware, the idea behind createObjectURL is it will create an URL which points to the user agents local Blob store. The format of the resulting URL is like: blob:https://example.org/9115d58c-bcda-ff47-86e5-083e9a2153041

You can try yourself in the console on any website:

# Create the (local) URL to a new Blob
var doc = URL.createObjectURL( new Blob(['test'], {type: 'text/plain'}) );

# Print the URL
console.log(doc)

You'll see when you paste the blob URL in any Chrome tab and request the URL, Chrome will download the local blob and show you test.

When you do the same from the extensions console window, the blob URL will be in the format of blob:chrome-extension://chjpchfchpknieidncedaigaednecjlg/5d115834-4cc1-46d9-9e27-78b96be6b270. You'll see that Chrome 72 simply ignores the request when you paste and request this URL in a Chrome tab. While when I do the same in an older Chrome version, this does work!

One thing I noticed when testing this is in an ancient Chrome version is the console prints the blob URL as a URL safe string. That means, chrome-extension:// is printed as chrome-extension%3A//. In Chrome 72 I tried if this was the actual issue, but its not, it still fails. Requesting the blob URL in this way does finally print some error in the console though: Not allowed to load local resource: blob:chrome-extension%3A//chjpchfchpknieidncedaigaednecjlg/5d115834-4cc1-46d9-9e27-78b96be6b270

So I'm thinking it seems they made a change in Chrome 72 that forbids using the local Blob URL store from within an extension.

[bob2build]

@prolane With your code as basis, I had built a similar extension. The only catch here is that, it supports multiple roles.
https://github.com/bob2build/idpawssamlinterceptor-chromextension

based on my observation, there is intermittent failure during intercepting saml assertion that is sent to amazonaws.com. I don't think this has anything to do with the download api since my plugin doesn't even receive the assertion.

I haven't debugged the plugin yet. However, I will get to it in the next few hours.

[prolane]

@bob2build That sounds like a different issue to me. For this particular issue I've already confirmed the extension does its job right all the way up to the actual download of the credentials file. I'm now at the point that I'd like to submit a bug report for Chromium or work on an alternate approach.

If you found something else that broke in the extension, please create a new issue.

P.S.
Very nice to see when others take what you've build and make it even greater. Nice job :-)

[prolane]

I've submitted a bug report for Chromium. Lets first see what that will bring us.

https://bugs.chromium.org/p/chromium/issues/detail?id=928307

[bob2build]

@prolane, you are right. I have 2 different issues,
This plugin no longer works with okta. (It used to work previously, but stropped working when a new AWS okta app was added which combined all AWS accounts into single one). Hence I had created my own plugin. I have traced back this issue to permissions. If I add permissions to *.okta.com in manifest alone, Then my plugin properly intercepts the saml requests correctly.

As far as downloading, my download is also broken. Haven't gotten a chance to look at it yet. However, I will follow your bug report for any insights.

[prolane]

Alright folks, the Chromium team has confirmed its a bug. We'll have to wait for the fix now.

Anyone who is looking for a temporary workaround could do the following:

• Go to chrome://flags/#network-service
• Disable the Network Service
• Relaunch Chrome

[prolane]

The Chromium team fixed the bug and its merged with the stable branch. Just a matter of waiting for the next Chrome release now.

[prolane]

I'll keep this issue open until the new Chrome release with the fix is available. Just so its easier for other users of the extension to spot this Github issue.

[Ebram-Tharwat]

@prolane , disabled the Network Service and Relaunching chrome didn't work.
Are there any extra missing steps?

[prolane]

@Ebram-Tharwat No additional steps. Its because the Chrome bug is in the Network Service path. So if you disable the Network Service, you don't face this bug.

If its not working for you, then I'm assuming you're facing a different issue. You could check by following this procedure:

• Go to Extensions
• Turn on 'Developer Mode'
• Click 'Inspect views background/background.html' for this particular extension

This will give you a new window with the console open. Use the extension and see what the error is.

[ashemedai]

Hi @prolane 👋

FWIW: can't reproduce this on my MacOS Mojave 10.14.2 with 72.0.3626.81 (Official Build) (64-bit).

Every time I log into AWS I download the credentials file.

Also, when I do blob:chrome-extension://chjpchfchpknieidncedaigaednecjlg/5d115834-4cc1-46d9-9e27-78b96be6b270 I get:

is blocked Requests to the server have been blocked by an extension.
Try disabling your extensions.
ERR_BLOCKED_BY_CLIENT

[prolane]

Heey @ashemedai !
You mean you don't have the issue or you can't get it to work with the work around?

Yeah, we can forget the test I mentioned with requesting blob:chrome-extension://chjpchfchpknieidncedaigaednecjlg/5d115834-4cc1-46d9-9e27-78b96be6b270. This is not supposed to work. I tried this in an ancient version (I think 49?) where this did work. So I assumed wrongly this was part of the issue.

The Chrome bug is in the chrome.downloads API which can't download blobs from the local URL store anymore.

[prolane]

Google pushed the new minor version 72.0.3626.96. I checked myself and the extension works now again for me, also with the Network Service set back to 'default' again.

@outlineruler @nitrocode Could you please confirm?

[tiesmaster]

@prolane Hi, direct collegae of @Ebram-Tharwat, for us, the issue is not fixed with the new bugfix release of Chrome (72.0.3626.96 (Official Build) (64-bit)). I also tried the workaround you mentioned, but didn't fix for me on .96. I guess that makes sense, since it looks like that it's not only an issue in the network service, but also somewhere else.

I'll dig into the Chromium bug, and their bugfix to see if I can find anything else that related.

[prolane]

Hi @tiesmaster , I didn't hear anything back from @Ebram-Tharwat. Please try the steps I suggested to @Ebram-Tharwat and let me know if the console shows any errors.

[ashemedai]

@prolane Just didn't have the issue at all. Colleagues here all seemed to have managed to trip it (see above) except my system.

[prolane]

Ah I see, @tiesmaster and @Ebram-Tharwat are also colleagues of you @ashemedai.

Quite interesting you didn't have this issue at all as there was clearly a bug in Chromium. Maybe the Network Service was somehow disabled for you as the default? Anyway, good for you :-) Unfortunately not so good for your colleagues. My gut feeling says your colleagues are facing a different issue though.

[outlineruler]

@prolane the update to Chrome 72.0.3626.96 (Official Build) (64-bit) did not resolve the issue for me.

[prolane]

@outlineruler How strange. Did the workaround with disabling the Network Service work for you?

When I upgraded to 72.0.3626.81 the extension also broke for me. What I saw happening is Chrome was still trying to create a new download. I could see that Download bar at the bottom. But then the download just failed. Something with a 'Network Error'.

Is this also what you see?

[outlineruler]

The disabling network-service did not work for me with the previous version or the new update and I do not see the download bar at the bottom of my browser.

[prolane]

Okay thanks @outlineruler
Must indeed be a second issue with Chrome 72 then. I'll push a patched version of the extension with some more debug. Hopefully like this we'll know more about whats going on at your end and for the others.

Quite interesting this issue affects a few people, but not everyone....

[tiesmaster]

@prolane Ok, this is really weird. This afternoon, I tried to put a breakpoint on L221, in order to at least capture the access key, so I could paste it into the credentials file, however, that never hit.

I thought I was doing something wrong in Dev Tools (I'm not a webdev 😜), but this evening I did some more digging, and found out that the registering/deregistering of the listener does break on activation/deactivation of through the popup, however, the onBeforeRequestEvent doesn't hit anymore. I ran Fiddler on the side, and I see that https://signin.aws.amazon.com/saml still is requested (with successful result), so I'm out of ideas, atm :(

Tomorrow, I'll see if I can walk by @ashemedai to see why things are working for him (if he has time for that). Testing here at home, I did with both OSX, and Windows, and both don't work for me, so doesn't seem to be related to platforms.

[prolane]

@tiesmaster Thats interesting. So you're saying these lines of code are probably not working for you anymore?

samltoawsstskeys/background/script.js

Lines 37 to 41 in dafc132

	chrome.webRequest.onBeforeRequest.addListener(
	onBeforeRequestEvent,
	{urls: ["https://signin.aws.amazon.com/saml"]},
	["requestBody"]
	);

This is where the Listener is added for the onBeforeRequest event. When it hits, the onBeforeRequestEvent() function should be called.

[prolane]

For those of you for which the extension isn't working yet, there is a new version of the extension: version 2.6. Once updated to this version, please go to the options panel and enable Debug logs. After enabling, please restart Chrome so we'll see the debug logs from the start of the extension.

Then do the following:

• Go to Extensions
• Turn on 'Developer Mode'
• Click 'Inspect views background/background.html' for this particular extension
• Go the the 'Console' tab (if its not already active)
• Sign In to AWS
• Copy the logs and paste them here in this Github issue (just scramble the sensitive data before pasting it here)

[outlineruler]

Followed the above steps but I am not getting much in the log other than the initialization of the extension. I am using Duo to auth. It seems like the extension is not getting triggered after I pass the auth challenge and am redirected to AWS.

script.js:33 DEBUG: Extension is activated
script.js:42 DEBUG: onBeforeRequest Listener added

[prolane]

Okay, thanks @outlineruler
What the debug logs show is the Listener (which listens for requests to the AWS saml url) is added. However, this event never happens.

For testing purposes, you could enable the extension and then just request the url https://signin.aws.amazon.com/saml in a new tab. I'm pretty sure you'll now see a 3rd debug log line saying: DEBUG: onBeforeRequest event hit!. Followed by a js error because there isn't any SAML payload.

While searching for open Chromium issues on the chrome.webRequest API I came across something interesting in the docs:

Starting from Chrome 72, an extension will be able to intercept a request only if it has host permissions to both the requested URL and the request initiator.

I've done some experiments and it seems to me that request initiator means the value of the origin HTTP header of a request. I.e. the website that redirects you to the AWS saml url. For most people this will be the url of their IDP (e.g. ADFS or Okta). For anyone who is allowed to assume only one AWS role (i.e. just one Role claim in the saml payload), this is a problem! Its a problem because the origin of the request is not equal to https://signin.aws.amazon.com/saml while the extension only asks permission to https://signin.aws.amazon.com/saml.

My hypothesis is people that are allowed to assume more than 1 AWS role can still successfully use the extension. The difference here is that those people will first see an AWS webpage asking them to select the role they want to assume. This page is also hosted on the url https://signin.aws.amazon.com/saml. So therefore, when you then select one of the roles and hit sign in, the origin of the next request is https://signin.aws.amazon.com/saml. Since origin (a.k.a. request initiator) and requested URL are then both https://signin.aws.amazon.com/saml it does work.

I'm pretty convinced about this hypothesis as I can actually reproduce this behaviour. I have multiple roles to assume myself, but I should still see the DEBUG: onBeforeRequest event hit! log line when I hit the AWS saml page for the first time. But I don't see it. However, I do see this log line when I change the extensions permissions to also access the URL of the IDP I use.

As the url of the SAML endpoint (IDP) will be different for every company, it makes no sense anymore to request permission to only https://signin.aws.amazon.com/saml. I'll have to ask permission for any url. I think I'll raise a ticket about this at the Chromium project because in my opinion this make it less secure for users.

I'll change the extension to ask the user permission for accessing any url. Again, I'm not in favor, but I think its the only way.

[prolane]

I've pushed version 2.7 of the extension. This will probably solve the issue @outlineruler @tiesmaster @Ebram-Tharwat

[outlineruler]

@prolane I confirm that I get the 3rd DEBUG line when following the steps. I'll give the new version a try when my browser updates.

[outlineruler]

The new version has solved the issue and the credentials file is downloading now. Thanks!

[prolane]

Awesome. Thanks for reporting the issue.

[tiesmaster]

@prolane I've also validated the fix 🎉 Time for 🍺

Thanks for digging into this in more detail, and the extensive explanation behind this issue 👍

[prolane]

Great! Glad to hear that! 👍

[Ebram-Tharwat]

I also confirm that the updated version of the extension(2.7) for chrome 72.0.3626.96 works correctly