Blazor WASM apps being blocked by firewalls due to .NET assemblies, even after renaming DLLs

[whereisaaron]

Describe the bug

Users are unable to access our Blazor application because their corporate firewalls block downloading the .NET assembly files. We followed the guidelines MS published and mentioned in #5477 (comment) to rename the '.dll' files to '.bin', but the assembly files are still being blocked.

https://docs.microsoft.com/en-us/aspnet/core/blazor/host-and-deploy/webassembly?view=aspnetcore-5.0#change-the-filename-extension-of-dll-files

One of the firewall systems we've identified is TrustWave WebMarshal. Of course such products are configurable, but by default this product appears to block Blazor apps. It appears it may be the content of the .Net assemblies that are being blocked. The firewall security team have made it clear than allowing DLL's is a hard 'no'.

Others report in #21996 and #19552 (comment) that Sophos Antivirus also blocks Blazor application due to the .Net assembly files and that renaming the files does not work there either. Others suggest implementing XOR encoding for transit, but I haven't found a good implementation.

Others report in #23084 that McAfee Web Gateway blocks Blazor .Net assemblies, even when renamed, based on content because they are detected as "application/dotnet-assembly".

Others mention in #21996 (comment) that their (unnamed) firewalls block on content not file extension so renaming does not help and that a fairly complex base64 encoding process to preload the assemblies into the cache and then refresh is a workaround.

Others report in #19552 (comment) that Palo Alto firewall and in #19552 (comment) and #16247 and #5477 (comment) and #5477 (comment) that Avast Antivirus / AVG Antivirus blocks Blazor, but possibly the renaming of DLLs works there? I could find anyone confirm if that actually worked.

Others report in #16247 (comment) that Symantec Web Firewalls blocks Blazor apps, unclear if only by filename or content.

Others report in https://forums.asp.net/t/2161027.aspx that Norton Antivirus blocks Blazor apps by detection of the DLL content.

This appears to be a massive road-block to Blazor adoption. Past security sins mean that no one trusts anything that smell like a dll/exe. I think Blazor needs a native / built-in solution to either encodes the payloads so they they don't look like traditional security risks. Really I think the Blazor application should be compiled to WASM when published, so it is only wasm code that is being downloaded. There are discussion of upcoming 'AOT' compiling feature coming since 2018, which sounds like a proper solution.

@Vectorix states it well:
"If the Blazor model does not play 100% well with existing firewalls it is almost a show stopper. It should flow through all firewalls effortlessly just like Javascript. If I was a firewall administrator, I am not sure if I would want to add a new white list entry for each new languages that has its own IL format."

Questions:

Has any developed a workaround that will enable Blazor use in corporate settings?

Does Microsoft have published advice for firewall vendors how to allow Blazor while blocking traditional DLL attacks?

Does the .Net 6.0 preview 1 fix this in any way?

Should deploying Blazor in the real world be this hard? 😀

@mkArtakMSFT you have often commented on this issue, and often say you hear of this issue rarely. But are you sure there lots of people using Blazor with corporate users and not experiencing this problem? If so many products like WebMarshall, Sophos, McAfee, Norton, maybe Symmantec and other all block Blazor/.Net by default on content not filename, that seems like a huge barrier to adoption? And if all of those plus Palo Alto, Avast/AVG, and presumably others block on filename, should '.dll' even be the filename or encoding used for IL files?

To Reproduce

Publish with:

dotnet publish --output foo --configuration Release

Then rename DLLs with:

for f in wwwroot/_framework/*.dll; do mv "$f" "${f/.dll}".bin; done
sed -i 's/\.dll"/.bin"/g' wwwroot/_framework/blazor.boot.json
sed -i 's/\.dll\$/.bin\$/g' wwwroot/service-worker.published.js

The resulting application works fine when the firewall is not blocking it, but try to access it behind many web proxy firewalls will fail.

Exceptions (if any)

Each .Net assembly is blocked by the firewall and fails with:

Class name: TypeError
Message: Failed to fetch
Tags: UnhandledPromiseRejection

Further technical details

• ASP.NET Core version 5.0.201

• Include the output of dotnet --info

.NET SDK (reflecting any global.json):
Version: 5.0.201
Commit: a09bd5c86c

Runtime Environment:
OS Name: ubuntu
OS Version: 18.04
OS Platform: Linux
RID: ubuntu.18.04-x64
Base Path: /usr/share/dotnet/sdk/5.0.201/

Host (useful for support):
Version: 5.0.4
Commit: f27d337295

.NET SDKs installed:
3.1.404 [/usr/share/dotnet/sdk]
5.0.201 [/usr/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.10 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.4 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.10 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.4 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download

• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

Linux CLI and VS 2019

References

#5477 (comment)
#21996
#21996 (comment)
#23084
#19552 (comment)
#19552 (comment)
#16247
#5477 (comment)
https://forums.asp.net/t/2161027.aspx
Microsoft guidelines for renaming files
Encoding files with service worker idea

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[Andrzej-W]

I would like to add some information about my experience with ESET Internet Security installed on my own laptop. This laptop has Core i7-6500U processor which is optimized for battery life and not performance. When ESET doesn't monitor HTTPS traffic I have to wait about 1.5s to download and run default Blazor WASM ASP.NET Hosted application. When HTTPS traffic monitoring is enabled I have to wait three! times as long.

I have confirmed that browser downloads compressed files (*.br) but before they are delivered to the browser they are decompressed and scanned by ESET. This slows down the loading considerably. After scanning file is returned to the browser in uncompressed form and Content-Encoding header is removed. There is no double decompression. The slowdown is only due to an antivirus scanning.

It seems that the only solution is renaming and encoding but before you implement it you should contact with firewall/antivirus vendors and ask what they are going to do. It is possible that they implement an algorithm to decode all of these files, and we'll be back to where we started.

[rogihee]

Just want to add that I'm also running into this issue: our app was blocked by an internal VPN connection that people had to connect to with WFH restrictions in an enterprise setting. This impacts Blazors potential in enterprise environments and this is a real risk for adoption.

[whereisaaron]

Thanks @rogihee. From our ongoing discussions Blazor is also having a negative impact on WebAssembly in general. Clients are now treating WebAssembly with suspicion because of the 9-alarm alerts the DLL/PE files sets off for firewalls. Until AOT compilation is available, is it possible Blazor could download assemblies but not using the DLL/PE wrapper?

[Schaeri]

During development of our current Balzor WASM application we selected some of our customers to alpha test our new application, using their network and infrastructure. Internally we never had issues starting our Balzor WASM application but about 50% of the customer that are testing the application can not start our Blazor WASM application due to their firewall blocking the application. Especially if the customer works in finances or insurance. In the financial sector it is not an option to advice the customer to change their firewall settings. We are suffering a massive loss of trust due to this issue. This issue is an absolute killer for Blazor WASM for our industry, and we are losing a lot of development effort if this issue will not be fixed…

Of course we tried workarounds: Like renaming the files. But this did not resolve the issue.

Currently we are trying the “Encoding files with service worker idea“ workaround but without using the service worker. The following sample code demonstrates how we try to solve it:

1. During our build process we generate for every dll additionally a base64 text files. sample_code.zip (DllEncoder.cs).

2. We disabled the auto start of the blazor app in our index.html
   <script src"_framework/blazor.webassembly.js" autostart="false"></script>

3. We use the “loadBootResource” extension to fetch the dlls and in case of 403 error we fallback to load the base64 text files. sample_code.zip (sample.js)

Probably this can help other developers with the same issue. But at the end we hope that Microsoft will fix the issue. Since tricking out firewall to get Blazor WASM running is not a good nor acceptable plan for us... This will not strengthen the trust in our software.

[legistek]

Well this is a problem...

A few possible solutions:

• encrypt / obfuscate the DLLs so firewalls can't know what they are
• use a different format for storing assembly metadata. An SQLite database for example?

[Andrzej-W]

@legistek Obfuscation / encryption is not an option in my opinion. "Good viruses" have been using these techniques for years, and antivirus vendors have caught them anyway. This solution will work a few hours, maybe a few days at most.

[eguardiola]

As stated before base64 encoding of DLL files is enough. At least as a workaround.

[legistek]

As stated before base64 encoding of DLL files is enough

I hope so, although as @Andrzej-W points out firewalls are good at adapting.

That said, while I'm sure "encrypted" viruses have a signature, if you encrypted the .NET DLLs with a random key for each session I don't know how an FW would distinguish that from any other encrypted content like a pw-protected ZIP file.

But anyway, we shouldn't have to be talking about this. Surely there is a way to transmit the .NET metadata for reflection (the only reason the DLLs are sent AFAIK) in a format that won't set off alarms. Auto-generate .JS files for example. Surely Microsoft wouldn't have spent all these years developing this platform if they didn't intend to make it usable in real life.

[Andrzej-W]

@legistek Unfortunately DLLs are not for metadata only: #35302 (comment)

[legistek]

@legistek Unfortunately DLLs are not for metadata only: #35302 (comment)

I saw that comment but didn't interpret that as meaning more than reflection. I asked for clarification as I think that's very relevant even beyond this issue. I don't particularly like the idea of my de-compilable .NET code being downloadable.

[Andrzej-W]

@legistek WASM is also decompilable (https://github.com/WebAssembly/wabt). Experience is currently not as good as in .NET IL code, but if you want to keep your code private the only solution is Blazor Server.

[smartprogrammer93]

@legistek WASM is also decompilable (https://github.com/WebAssembly/wabt). Experience is currently not as good as in .NET IL code, but if you want to keep your code private the only solution is Blazor Server.

I mean this is not a problem specific for blazor. I am pretty sure all javascript ui frameworks' code is visible to the user in the dev tools of a browser, is it not?

[legistek]

Yes it's a similar issue and yes .NET code can be obfuscated but I was looking forward to not having to worry about that believing the WASM compilation step would take care of that.

I remain very confused why true compilation (including with reflection) is doable on Mono platforms and .NET native but not WASM. I'll just have to trust that people far smarter than me had good reasons for this.

[tedd]

Same issue in my company, 5000 employees unable to use client side Blazor apps. I have tried many creative solutions previously for downloading data through the firewall. I.e. Renaming files doesn't work, zip with executables doesn't work, password protected zip works. We could make a workaround for our company apps, but I think this is a broader problem for adoption.

Obfuscation should not be necessary, nor any complicated container format. Files are blocked based on two simple criteria: Extension is .dll or file type is DOS/Windows executable (identifiable by MZ-header). First one is solved by renaming files, second one is solved by modifying the header signature.

I did a test for our firewall using these files: https://files.tedd.no/DllTest/

• Plain.dll BLOCKED extension + MZ header
• Plain.bin BLOCKED MZ header
• HeaderFlipped.dll BLOCKED extension
• HeaderFlipped.bin ALLOWED

Code to rewrite header:

using var fs = File.Open("HeaderFlipped.dll", FileMode.Open, FileAccess.ReadWrite);
using var bw = new BinaryWriter(fs);
var bz = Encoding.ASCII.GetBytes("BZ");
bw.Write(bz);

This can easily be ignored or reversed once files are downloaded.

[tedd]

Ok, I tested modifying the header back on client side. It works and is fairly simple to implement. (Except that the published js file is minimized.)

I added the line data[0] = 77; after this line

aspnetcore/src/Components/Web.JS/src/Platform/Mono/MonoPlatform.ts

Line 460 in c1d16e3

const data = new Uint8Array(dataBuffer);

This restores header from BZ to MZ with near zero overhead. (Note that this is a dirty proof of concept hack that assumes all files that goes through this function are supposed to start with M.)

Doing this the WebAssembly loaded successfully.

Attaching the complete code for modifying binaries for test, including updating compressed files and updating sha256. It is just a quick hack ran from LINQPad.

void Main()
{
	var dir = @"C:\path\to\project\bin\Release\net6.0\browser-wasm\publish\wwwroot\_framework";
	var dllFiles = Directory.GetFiles(dir, "*.dll");
	// Modify header on .dll-files
	foreach (var file in dllFiles)
		FlipBz(file);

	// Create new compressed copies
	foreach (var file in dllFiles)
	{
		CompressBr(file);
		CompressGz(file);
	}

	// Update sha256 checksum in blazor.boot.json
	var shas = new Dictionary<string, string>();
	foreach (var file in dllFiles)
	{
		using SHA256 mySHA256 = SHA256.Create();
		using var fs = File.Open(file, FileMode.Open, FileAccess.ReadWrite);
		var sha256 = mySHA256.ComputeHash(fs);
		var base64Sha256 = Convert.ToBase64String(sha256);
		shas.Add(Path.GetFileName(file), base64Sha256);
	}
	var jsonFile = Path.Combine(dir, "blazor.boot.json");
	var json = File.ReadAllText(jsonFile);
	foreach (var kvp in shas)
	{
		// This is what happens when you used to be a Perl programmer
		var r = $"(\"{Regex.Escape(kvp.Key)}\":)[^,]+(,)?$";
		json = Regex.Replace(json, r, $"$1 \"sha256-{kvp.Value}\"$2", RegexOptions.Multiline);
	}
	File.WriteAllText(jsonFile, json);
	CompressBr(jsonFile);
	CompressGz(jsonFile);
	//json.Dump();
}
void CompressBr(string file)
{
	using var input = File.OpenRead(file);
	using var output = File.Create(file + ".br");
	using var compressor = new BrotliStream(output, CompressionMode.Compress);
	input.CopyTo(compressor);
}
void CompressGz(string file)
{
	using var input = File.OpenRead(file);
	using var output = File.Create(file + ".gz");
	using var compressor = new GZipStream(output, CompressionMode.Compress);
	input.CopyTo(compressor);
}
void FlipBz(string fn)
{
	using var fs = File.Open(fn, FileMode.Open, FileAccess.ReadWrite);
	using var bw = new BinaryWriter(fs);
	var bz = Encoding.ASCII.GetBytes("BZ");
	bw.Write(bz);
}

[rmencia-isv]

I don't think this is only an issue for Blazor. WebAssembly is a standard and may more companies are creating WebAssembly libraries and they all should have the same problem, unless they are doing something different...

If JS is not being blocked, maybe WebAssembly should be converted into a JS file that loads the Webassembly from an Base64 encoded string using eval(webassemblyLibrary) or similar.

Anyways, I don't think this is a Blazor problem, but something that would need to be discussed with Google, Amazon, WebBrowsers developers and other big ones that back the WebAssembly standard....

But... there may be a simpler way already described in the Standards and other companies are already using it.

[eguardiola]

@rmencia-isv The problem described here is not with the webassembly files instead with the DLL files Blazor uses.

[rmencia-isv]

But the DLL files Blazor uses are webassembly files, right? So the problem should be the same for all of them. If all the webassembly files followed the same standard approved for WebAssembly file transfers it should work the same as it happens for JS files (if there was such a standard).

Also, if Blazor WebAssembly was being used by MS in Bing, they would have noticed the problem first day. If Bing was being blocked by the FW, they would have found a solution already, because it generates money for them. MS should do more dog-feeding. Now, guys like us need to worry about our applications not working on 50% of our customers. This bug was created in March and now 6 months later there is still no fix. It's been moved from version to version since June. If MS was using it in Bing, it would have been fixed already.

Maybe the only way is to have Blazor to automatically revert to Server side for the blocked scenarios and instead of having Blazor Server and Blazor WebAssembly, if should be the same model that if it can it runs in the client and if it can't runs in the server.

Regardless of the solution, we should not need to be using a drill and an angle grinder and a welder to get it working, like in some solutions posted here. The solution should be out-of-the-box and work just by publishing. Anything else is a fail.

[chrdlx]

But the DLL files Blazor uses are webassembly files, right?
@rmencia-isv

AFAIK No, .DLL files are normal windows DLL... that's why they're blocked... webassembly files have the .wasm extension.
Regards

[javiercn]

Hi folks,

We have an update on this topic here

We added a feature on RC1 that allows people to create their own "packaging format" for blazor applications and have created an experimental package that packages the dlls in a different way as described in the blog post above.
You should be able to use our extension package (or create your own package to reuse and share) that customizes how a Blazor app loads and where you can apply any transformation to the app dlls as part of the build process and undo the transformation afterwards.

[javiercn]

@guardrex can you use the blog post as a base to document how to do this?

[tedd]

Just note that if packaging specifically for these firewalls is an extra step, an opt-in, then most Blazor wasm deployments will be inaccessible from these locations (i.e. large corporations with content inspection in proxy). As a developer you need to be aware of the problem and fix it. This means every developer has to learn this (often the hard way) and figure out how to solve it.

In terms of adoption, having a new tech that sometimes doesn't work for some visitors for mysterious reasons is probably not good.

[rmencia-isv]

Just note that if packaging specifically for these firewalls is an extra step, an opt-in, then most Blazor wasm deployments will be inaccessible from these locations (i.e. large corporations with content inspection in proxy). As a developer you need to be aware of the problem and fix it. This means every developer has to learn this (often the hard way) and figure out how to solve it.

In terms of adoption, having a new tech that sometimes doesn't work for some visitors for mysterious reasons is probably not good.

I completely agree with this. An out-of-the-box solution should work for this without having to do anything.
Then, if something needs to be customised for additional requirements, then allow the mechanisms to make the changes or by project configuration choose the method to build those packages.
Something like: Project options / Web/ Build/ Packaging : Default (Basic and works), Method2, Method3, None, Custom.

But by default make it work when compiling in Release mode.

[Andrzej-W]

There is a new interesting issue: #36978 I think we should vote for it.