repo contains binaries that are hard to verify

[omajid]

This repo contains some binaries used in benchmarks and tests:

$ find -type f -exec file {} \; > file-types
$ grep -Ev ': (empty|ASCII text|Unicode text|XML 1.0|JSON text|HTML document|exported SGML document|CSV text|troff or preprocessor|unified diff|PNG image|SVG Scalable Vector Graphics|SVG XML|PC bitmap|MS Windows icon resource|OpenType|TrueType|Web Open Font|Embedded OpenType|POSIX shell|Bourne-Again shell|DOS batch|C source|C\+\+ source|Algol 68 source)' file-types 
./src/Razor/benchmarks/Microsoft.AspNetCore.Razor.Microbenchmarks/Resources/project.razor.bin: PGP Secret Sub-key -
./src/Razor/test/Microsoft.VisualStudio.Razor.IntegrationTests/TestFiles/BlazorProject.zip: Zip archive data, at least v2.0 to extract, compression method=deflate

Given the recent supply chain attack against xz, would it be possible to remove these binaries from the repo? Or, perhaps to generate them at build time? Or perhaps to have a way to reproduce/re-generate the binaries via some script/code to make the generation more auditable and reproducible?

cc @richlander

[davidwengier]

Just to be very clear to anyone reading, it's worth noting that the "binaries" identified under this issue are not executables or libraries, and do not contain any code. Additionally they do not ship with any product.

I would be very interested to know about any vulnerabilities or attack vectors these files create, but I personally can't think of one. Additionally I think generating these files at build time would obfuscate their contents and make it much harder to reason about when or how they might change.

For example, let's assume a malicious contributor wanted to replace the zip file with a zip bomb, as an attack of the build machines we use. If that file were generated by a script then we would have to validate all changes to the script, and any dependencies, to determine if it had changed in any malicious way. That would expose us to exactly the sort of supply chain attacks mentioned in the issue. As it stands now however, if a contributor opened a PR that modified the zip file in any way, it would be immediately suspicious and something we would definitely notice.

[richlander]

Read this post: https://gynvael.coldwind.pl/?lang=en&id=782