Feature Request: Fully Support AoT Compilation & CIL Stripping for Android #44855
Comments
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Dotnet-GitSync-Bot
added
the
untriaged
|
Tagging subscribers to this area: @CoffeeFlux Issue DetailsProvide AoT compilation and CIL stripping for Android builds, both armeabi-v7a and arm64-v8a targets, and fully support in Visual Studio starting with .NET 6. APKs containing IL are trivial to unpack and decompile (bundled or not), presenting major risks to security and IP. Considering armeabi-v7a market share, the experimental support that only works for arm64-v8a targets in Xamarin.Android is a major deterrent to any serious development with Microsoft's cross-platform app building solution. The current state of affairs for .NET forces us to choose between the armeabi-v7a user base and secure source code suitable for distribution on app stores, which is a no-go for any risk conscious closed-source development effort. See dotnet/android#1218 for some history and additional user feedback. Tagging @akoeplinger as per the docs.
|
CoffeeFlux
added
area-Codegen-meta-mono
and removed
area-VM-meta-mono
untriaged
|
Tagging subscribers to this area: Issue DetailsProvide AoT compilation and CIL stripping for Android builds, both armeabi-v7a and arm64-v8a targets, and fully support in Visual Studio starting with .NET 6. APKs containing IL are trivial to unpack and decompile (bundled or not), presenting major risks to security and IP. Considering armeabi-v7a market share, the experimental support that only works for arm64-v8a targets in Xamarin.Android is a major deterrent to any serious development with Microsoft's cross-platform app building solution. The current state of affairs for .NET forces us to choose between the armeabi-v7a user base and secure source code suitable for distribution on app stores, which is a no-go for any risk conscious closed-source development effort. See dotnet/android#1218 for some history and additional user feedback. Tagging @akoeplinger as per the docs.
|
|
I've made some clarifications in the feature request. @SamMonoRT, @lambdageek, @CoffeeFlux, what is the likelihood of getting a user story added to the 6.0.0 milestone? As MAUI developer targeting Android, I want to be able to strip the CIL from our shared and head projects when building on armeabi-v7a and arm64-v8a architectures, reducing the resulting assemblies to nothing but metadata, so that my app packages can't be decompiled back to their original source code. Of course machine code can be disassembled and then decompiled, but the loss of information that happens when assembling the IL code is very far removed from the original, maintainable C# source that you get back when decompiling from CIL (regardless of obfuscation). |
|
We would like to support AOT mode for Android RIDs but we will probably run out of time to finish the work in .NET6 time frame. |
|
Thanks for your reply @marek-safar. Could you clarify whether the AOT compilation you're working towards for Android is a profiled AOT, or a full AOT where the JIT engine could be dropped from the package? The most dialog I could find on the subject was in /issues/34693, is there a more relevant issue to track? |
jxbrenner
changed the title
|
Please implement this feature. |
|
cc @fanyang-mono @jonathanpeppers Moving this to next milestone, but something we will want to consider in next few months. |
|
I think the two parts here are:
We have not really focused on Hybrid AOT due to its experimental status and lack of demand so far. If we had some app size or performance benefits, that might be a good reason to implement this. |
|
"lack of demand so far" |
|
I think the lack of demand comes from missing features and not the other way around in this case. |
Isn't the intent of setting the AndroidAotMode build property to I've always been confused about the "hybrid" nomenclature and referring to it as this separate kind of AoT compilation. I get that in MonoAndroid it drops Mono.Android as a target for cil-strip (I imagine there was some issue with stripping the Android bindings), so it's technically not "full" AoT - perhaps this is why its called as such? There already seems to be a commitment here, ex. it's been assigned to a milestone? Are mitigating unnecessary risks to intellectual property and security not good reasons to implement and fully support? I think a good two-part approach here could be:
Item 1 completely mitigates the security and IP risks and I assume is easier to accomplish. This is probably naive but perhaps it can be accomplished by fixing whatever broke IL stripping in Xamarin.Android 12.3 (see dotnet/android#7088), applying the fix to .NET SDK, and just moving the CilStrip build task back to Xamarin.Android.Common.targets from Xamarin.Android.Legacy.targets since legacy doesn't apply when you use AndroidNETSdk? |
|
I don't think it is quite as simple as you describe here. If we called the There is also a "hybrid" AOT mode value set at runtime: Since, literally no one has ever tried it (I haven't), I would expect apps would crash (maybe I'm just pessimistic?) -- and we'd need to fix something. Can you give it a try, just using our .NET 6 stable packages? Assuming apps build, if you enable full logging, it should show what happens with AOT mode at runtime: Then look at the log, such as Except it would say |
|
I have no doubt my understanding of the problem is superficial and incomplete. Thanks for taking the time to help me understand better. So hybrid isn't just some arbitrary value name for the AndroidAotMode build property, it's the setting for mono_jit_set_aot_mode that tells the runtime to still include the JIT so it can execute IL stripped assemblies with dynamic features? So is the path forward here:
Additionally, is the goal to solve the issue for all Android ABIs or just a subset? Whatever it is, I'm just very hopeful this issue is resolved before Google Play requires apps published with an SDK greater than whatever is supported by Xamarin.Android 12.2. That's where I'm stuck until either the regression that breaks IL stripping in 12.3 is fixed or I have a path forward to a unified .NET that at least supports IL stripping of my shared and head project assemblies. Preferably a fair bit sooner then that so I have time to rewrite in native or move to a different cross-platform framework instead of angry clients and no revenue. I believe that the cil-strip binary is the same for Android and iOS. I'll try to run a .NET 6 app with hybrid AoT and post the logs. Should I try it with and without IL stripping? I imagine I need to strip the IL from an assembly with some dynamic code to fully test, but I appreciate that just knowing the app doesn't explode with just setting Mono AOT mode to hybrid is helpful as well. |
I don't think you'll actually be able to do the stripping part easily -- so we're just checking if things at runtime work at all. |
|
No trouble with the Android Application template in Visual Studio 17.3.0. Here's the project file: I'm targeting SDK 31 and running it on an ARM 64 device. Did not crash. Here's the runtime saying it's configured for hybrid AOT: |
|
Yeah, my project also uses hybrid AOT and I'm stuck at Xamarin.Android 12.2, whatever broke in 12.3 would be just fine, but not even that I was looking forward for this feature to start migrating in to MAUI |
|
@gabriel-kozma @jonathanpeppers @marek-safar @SamMonoRT |
|
@marek-safar @SamMonoRT @fanyang-mono, just checking in to understand the likelihood of resolving the regressions to AoT and CIL stripping in .NET 8. With Android 13 and iOS 16 being the final platform versions supported by Xamarin and support for Xamarin ending on 2024-05-01, .NET 8 is the last oppotunity for MAUI to live up to its promise of allowing an upgrade of our Xamarin apps to MAUI without a rewrite. Without closing the gap created by .NET Android's lack of AoT complitlation and CIL stripping in .NET 8, we'll see our Xamarin apps dropped by the app stores before .NET 9. Migrating to MAUI without these features is also a non-starter because our apps will no longer meet our customer's security requirements. |
|
Yes, we are on track to deliver this in next couple months, unless we hit any unforeseen issues. |
|
The overall progress of this has been tracked by #80953 |
|
The change for enabling this feature for Android apps is dotnet/android#8172 |
Context: xamarin/monodroid@388bf4b Context: 59ec488 Context: c929289 Context: 88215f9 Context: dotnet/runtime#86722 Context: dotnet/runtime#44855 Once Upon A Time™ we had a brilliant thought: if AOT pre-compiles C# methods, do we need the managed method anymore? Removing the C# method body would allow assemblies to be smaller. ("Even better", iOS does this too! Why Can't Android™?!) While the idea is straightforward, implementation was not: iOS uses ["Full" AOT][0], which AOT's *all* methods into a form that doesn't require a runtime JIT. This allowed iOS to run [`cil-strip`][1], removing all method bodies from all managed types. At the time, Xamarin.Android only supported "normal" AOT, and normal AOT requires a JIT for certain constructs such as generic types and generic methods. This meant that attempting to run `cil-strip` would result in runtime errors if a method body was removed that was actually required at runtime. (This was particularly bad because `cil-strip` could only remove *all* method bodies, not some!) This limitation was relaxed with the introduction of "Hybrid" AOT, which is "Full AOT while supporting a JIT". This meant that *all* methods could be AOT'd without requiring a JIT, which allowed method bodies to be removed; see xamarin/monodroid@388bf4b3. Unfortunately, this wasn't a great long-term solution: 1. Hybrid AOT was restricted to Visual Studio Enterprise customers. 2. Enabling Hybrid AOT would slow down Release configuration builds. 3. Hybrid AOT would result in larger apps. 4. As a consequence of (1), it didn't get as much testing 5. `cil-strip` usage was dropped as part of the .NET 5+ migration (c929289) Re-intoduce IL stripping for .NET 8. Add a new `$(AndroidStripILAfterAOT)` MSBuild property. When true, the `<MonoAOTCompiler/>` task will track which method bodies were actually AOT'd, storing this information into `%(_MonoAOTCompiledAssemblies.MethodTokenFile)`, and the new `<ILStrip/>` task will update the input assemblies, removing all method bodies that can be removed. By default setting `$(AndroidStripILAfterAOT)`=true will *override* the default `$(AndroidEnableProfiledAot)` setting, allowing all trimmable AOT'd methods to be removed. Profiled AOT and IL stripping can be used together by explicitly setting both within the `.csproj`: <PropertyGroup> <AndroidStripILAfterAOT>true</AndroidStripILAfterAOT> <AndroidEnableProfiledAot>true</AndroidEnableProfiledAot> </PropertyGroup> `.apk` size results for a `dotnet new android` app: | `$(AndroidStripILAfterAOT)` | `$(AndroidEnableProfiledAot)` | `.apk` size | | --------------------------- | ----------------------------- | ------------- | | true | true | 7.7MB | | true | false | 8.1MB | | false | true | 7.7MB | | false | false | 8.4MB | Note that `$(AndroidStripILAfterAOT)`=false and `$(AndroidEnableProfiledAot)`=true is the *default* Release configuration environment, for 7.7MB. A project that *only* sets `$(AndroidStripILAfterAOT)`=true implicitly sets `$(AndroidEnableProfiledAot)`=false, resulting in an 8.1MB app. Co-authored-by: Fan Yang <[email protected]> [0]: https://www.mono-project.com/docs/advanced/aot/#full-aot [1]: https://github.com/mono/mono/tree/2020-02/mcs/tools/cil-strip
|
PR completing this work was merged - dotnet/android#8172 /cc @fanyang-mono - please add details when that will be available for public release, and close this issue. |
Context: xamarin/monodroid@388bf4b Context: 59ec488 Context: c929289 Context: 88215f9 Context: dotnet/runtime#86722 Context: dotnet/runtime#44855 Once Upon A Time™ we had a brilliant thought: if AOT pre-compiles C# methods, do we need the managed method anymore? Removing the C# method body would allow assemblies to be smaller. ("Even better", iOS does this too! Why Can't Android™?!) While the idea is straightforward, implementation was not: iOS uses ["Full" AOT][0], which AOT's *all* methods into a form that doesn't require a runtime JIT. This allowed iOS to run [`cil-strip`][1], removing all method bodies from all managed types. At the time, Xamarin.Android only supported "normal" AOT, and normal AOT requires a JIT for certain constructs such as generic types and generic methods. This meant that attempting to run `cil-strip` would result in runtime errors if a method body was removed that was actually required at runtime. (This was particularly bad because `cil-strip` could only remove *all* method bodies, not some!) This limitation was relaxed with the introduction of "Hybrid" AOT, which is "Full AOT while supporting a JIT". This meant that *all* methods could be AOT'd without requiring a JIT, which allowed method bodies to be removed; see xamarin/monodroid@388bf4b3. Unfortunately, this wasn't a great long-term solution: 1. Hybrid AOT was restricted to Visual Studio Enterprise customers. 2. Enabling Hybrid AOT would slow down Release configuration builds. 3. Hybrid AOT would result in larger apps. 4. As a consequence of (1), it didn't get as much testing 5. `cil-strip` usage was dropped as part of the .NET 5+ migration (c929289) Re-intoduce IL stripping for .NET 8. Add a new `$(AndroidStripILAfterAOT)` MSBuild property. When true, the `<MonoAOTCompiler/>` task will track which method bodies were actually AOT'd, storing this information into `%(_MonoAOTCompiledAssemblies.MethodTokenFile)`, and the new `<ILStrip/>` task will update the input assemblies, removing all method bodies that can be removed. By default setting `$(AndroidStripILAfterAOT)`=true will *override* the default `$(AndroidEnableProfiledAot)` setting, allowing all trimmable AOT'd methods to be removed. Profiled AOT and IL stripping can be used together by explicitly setting both within the `.csproj`: <PropertyGroup> <AndroidStripILAfterAOT>true</AndroidStripILAfterAOT> <AndroidEnableProfiledAot>true</AndroidEnableProfiledAot> </PropertyGroup> `.apk` size results for a `dotnet new android` app: | `$(AndroidStripILAfterAOT)` | `$(AndroidEnableProfiledAot)` | `.apk` size | | --------------------------- | ----------------------------- | ------------- | | true | true | 7.7MB | | true | false | 8.1MB | | false | true | 7.7MB | | false | false | 8.4MB | Note that `$(AndroidStripILAfterAOT)`=false and `$(AndroidEnableProfiledAot)`=true is the *default* Release configuration environment, for 7.7MB. A project that *only* sets `$(AndroidStripILAfterAOT)`=true implicitly sets `$(AndroidEnableProfiledAot)`=false, resulting in an 8.1MB app. Co-authored-by: Fan Yang <[email protected]> [0]: https://www.mono-project.com/docs/advanced/aot/#full-aot [1]: https://github.com/mono/mono/tree/2020-02/mcs/tools/cil-strip
|
The new feature should be available in .NET8 RC1, which should be available around mid-September. @jxbrenner Will you be able to try it out and let us know if it meets your requirements? |
|
@fanyang-mono Absolutely, I'll test and report back as soon as RC1 is released. |
|
@fanyang-mono @jonathanpeppers - do we need to update any documentation with this ? also @jxbrenner - I'll mark this as complete and close the issue. If you encounter issues with RC1, please re-open and/or create a new issue with the specific failure and we'll try to backport a fix for 8.0 release. |
|
We usually start with this doc of our public MSBuild properties that is synced to MS docs: It does not go into details on how it works, though. |
ghost
locked as resolved and limited conversation to collaborators
Provide AoT compilation and CIL stripping when targeting Android, and fully support in Visual Studio starting with .NET 6. At a minimum, this request is for static compilation to armeabi-v7a and arm64-v8a architectures and CIL stripping to reduce assemblies to nothing but metadata.
The primary motivation for the request is that APKs containing IL are trivial to unpack and decompile (bundled or not), presenting unnecessary risks to security and IP. Secondary motivation is performance related.
Alternative mobile app development frameworks such as Kotlin and Flutter provide static compilation. This request is an opportunity for Microsoft to keep up with the status-quo and appeal to businesses that wish to use MAUI for risk-conscious, closed-source mobile app development efforts.
The current state of affairs with Xamarin and CIL stripping can be found in dotnet/android#1218. In summary, CIL stripping for armeabi-v7a once worked but has since been broken in regressions, and currently only works for arm64-v8a targets. The feature is undocumented and has only experimental support. As it stands, Xamarin Android forces us to choose between the armeabi-v7a user base and packages from which our original source code cannot be obtained.
Tagging @akoeplinger as per the docs.
The text was updated successfully, but these errors were encountered: