wap packaging: .xbf and .pri generation is not working for us

[DHowett-MSFT]

The WAP packaging project, CascadiaPackage, consumes most of its important dependencies transitively. It depends on WindowsTerminal and Host.EXE.

By default, wapproj puts all files collected from a project into a subdirectory named after that project, even if those files were originally from a nested dependency. This includes the .exes.

We've suppressed that behavior so that WindowsTerminal and conhost can live in the appx package root next to their DLLs.

We must also do this so that the .winmd -> AppX Manifest generator works right -- if we don't, it will insert references like this:

<InProcessServer>
  <Path>Whatever.dll</Path>
</InProcessServer>

when our directory structure looks like this:

WindowsTerminal/Whatever.dll
WindowsTerminal/Whatever.xbf

This will fail validation (and activation)! This cannot be fixed, so we stomp the directory for DLL and EXE files.

terminal/src/cascadia/CascadiaPackage/CascadiaPackage.wapproj

Lines 300 to 310 in eae920e

	<Target Name="OpenConsoleStompSourceProjectForWapProject" BeforeTargets="_ConvertItems">
	<ItemGroup>
	<!-- Stomp all "SourceProject" values for all incoming dependencies to flatten the package. -->
	<_TemporaryFilteredWapProjOutput Include="@(_FilteredNonWapProjProjectOutput)" />
	<_FilteredNonWapProjProjectOutput Remove="@(_TemporaryFilteredWapProjOutput)" />
	<_FilteredNonWapProjProjectOutput Include="@(_TemporaryFilteredWapProjOutput)">
	<!-- Override the filename for OpenConsole.exe (only) -->
	<TargetPath Condition="'%(Filename)' == 'OpenConsole' and '%(Extension)' == '.exe'">conhost.exe</TargetPath>
	<!-- Blank the SourceProject here to vend all files into the root of the package. -->
	<SourceProject></SourceProject>
	</_FilteredNonWapProjProjectOutput>

Well, we're also stomping the directory for XBF files. Therefore, MinMaxCloseControl.xbf and App.xbf end up in the appx root.

This works, because we've overridden their resource locator paths:

terminal/src/cascadia/TerminalApp/MinMaxCloseControl.cpp

Lines 17 to 18 in eae920e

	const winrt::Windows::Foundation::Uri resourceLocator{ L"ms-appx:///MinMaxCloseControl.xaml" };
	winrt::Windows::UI::Xaml::Application::LoadComponent(*this, resourceLocator, winrt::Windows::UI::Xaml::Controls::Primitives::ComponentResourceLocation::Nested);

That works great when deployed from Visual Studio. However, when the project is deployed by MSBuild (like in our CI), those loose XBF files are missing. If you look in one of the PRI files, it says that the path is TerminalApp/MinMaxCloseControl.xbf.

Well, that never existed on disk (see above: wapproj would put it in WindowsTerminal/MinMaxCloseControl.xbf because it was a transitive dependency.)

It gets worse. That path would actually work because the XBF file has been merged into the PRI file(!). However, the PRI files from wapproj transitive dependencies do not get merged into the global resources.pri.

If the transitive dependency PRI files can be merged into resources.pri, we can skip overriding the resource loader and remove the loose XBF files from our package.

/cc @metathinker, who originally reported a crash from the CI builds.

[DHowett-MSFT]

Oh, also, this results in a crash when you use builds from Azure DevOps or MSBuild. Yeah.

[metathinker]

Previous (and occasionally silly) discussion as part of another issue thread: #1364 (comment)

As Dustin notes above, .appx packages and loose-file app package layouts with this defect will cause Terminal to crash on startup; the precise nature of the crash is discussed in the other thread.

[DHowett-MSFT]

I have a terrible idea. I'm going to use my holiday and try to replace the WAP packaging project with a native UWP project. Instead of going from EXE to Centennial (which is apparently very hard), why not go from UWP to Centennial (which will probably expose hundreds of other issues, but solve this set of a hundred issues 😁)

[DHowett-MSFT]

Everything is terrible! 😄

feature	WAP	UAP
transitive dependencies	❌ (this issue)	✔️
PRI merging	❌ (this issue)	✔️
dependency resources	❌ (this issue)	✔️
simpler project structure	❌ (one extra project)	✔️
isn't a weird .NET project type	❌ (this makes debugging annoying)	✔️
partially works today	✔️	❌ (new uncharted unknown)
depends on VCLibs.Desktop	✔️	❌ (this seems hard)
intended for this purpose	✔️	❌ (throws a warning)
can be debugged (with some hoops)	✔️	❌ (application thinks it isn't deployed) Project not selected to build for this solution configuration, even after it builds.

[mdtauk]

WinUI and XAML Desktop apps - may change all this. I hope you are sharing your findings and thoughts with that team.

[DHowett-MSFT]

@mdtauk not only are we sharing our findings and thoughts with that team, we report to the same engineering manager as that team. 😄
Unfortunately, that team doesn't own the packaging rules shipped with Visual Studio.

[mdtauk]

@DHowett-MSFT I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come

[zadjii-msft]

IIRC the wsl distro launchers don't use a wapproj, maybe we can use their structure

…

________________________________ From: Martin Anderson <[email protected]> Sent: Thursday, July 4, 2019 5:08:28 PM To: microsoft/terminal Cc: Subscribed Subject: Re: [microsoft/terminal] wap packaging: .xbf and .pri generation is a house of horrors (#1816) @DHowett-MSFT<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDHowett-MSFT&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105194204&sdata=f8MCJ4c9S4VwRbtg8gQ%2F%2BLL2IEhkzahq2CUSc51KMNU%3D&reserved=0> I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fterminal%2Fissues%2F1816%3Femail_source%3Dnotifications%26email_token%3DAEMBTVR2KAKDEOJZGEKJ67DP5ZYFZA5CNFSM4H5QYKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZIF45I%23issuecomment-508583541&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=o22iE5vxqwP64M2d6wXqvUjTmawoU2BJ4eVRrYfxkRs%3D&reserved=0>, or mute the thread<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMBTVXXP5UH4UJM2JSPWYLP5ZYFZANCNFSM4H5QYKJQ&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=tRktrG6IpsrCFMqbYUu0lVGudegtVzSjW3jQTd1fOok%3D&reserved=0>.

[DHowett-MSFT]

feature	WAP
If you add the dependencies directly to the wap project, they fail with new and interesting linker errors	✔️

Yeah, if you add TerminalConnection as a direct dependency to wapproj, it has to re-link it during deployment, and it refuses to link in ConTypes.lib.

[DHowett-MSFT]

/cc @ocalvo for info

[ocalvo]

/cc @azchohfi for info

[metathinker]

I can't reopen issues after they have been closed, so paging @DHowett-MSFT:

It looks like the issue wasn't fixed or was reintroduced - I built the current master commit 988fe0b and installed the loose-file AppX layout. The crash on startup is back, and occurs at the same place. This time, the resource that failed to load is: ms-appx:///TerminalApp/TerminalPage.xaml

[DHowett-MSFT]

Yuuuup! #2018 to fix :)