Integrate into Git for Windows proper

[dscho]

It would appear that an even better way to provide all that gitsetup provides would be to integrate the enhancements into Git for Windows itself.

[robmen]

If you are interested in that, we could definitely look into that.

It is important to note that this setup takes a different approach than the current Git for Windows setup. It has different "opinions" about how to do an install and might not work well for everyone. It doesn't try to please everyone. :)

How would you best like to proceed?

[dscho]

If you are interested in that, we could definitely look into that.

I am.

It is important to note that this setup takes a different approach than the current Git for Windows setup. It has different "opinions" about how to do an install and might not work well for everyone.

This is interesting. What is this "opinion" thing? How does it work? Who does it target?

How would you best like to proceed?

That depends whether I could interest you to integrate your WiX script into Git for Windows' standard release cycle or not... 😜

[hmemcpy]

That depends whether I could interest you to integrate your WiX script into Git for Windows' standard release cycle or not...

That would be a "win-win" 👍

[robmen]

@dscho are you thinking about adding this as a 4th "installer type" to git-for-windows/build-extra? I'm going to assume, "Yes". Ignore the remainder of this response if you have different ideas. 😄

This is interesting. What is this "opinion" thing? How does it work? Who does it target?

gitsetup.exe is designed for Windows developers that use cmd and/or PowerShell and want to treat git as a typical Windows command-line executable. As a result, the many Unix-isms of git are not exposed.

That depends whether I could interest you to integrate your WiX script into Git for Windows' standard release cycle or not...

I took a look at the build process a couple years ago. I do not envy your build dependencies. 😄

I've effectively zero bash experience (and near zero interest in extending it at this time) so I'm of minimal use adding/enhancing the existing scripts in git-for-windows/build-extra.

However, I know the WiX Toolset and the Windows Installer (.msi) very well. I can explain what the current batch files do and manage the .wxs code (and continue to enhance going forward since there are certainly areas to improve).

So I'm game but the question is anyone available to do the build script integration?

[dscho]

are you thinking about adding this as a 4th "installer type" to git-for-windows/build-extra?

Yes 😄

I know the WiX Toolset and the Windows Installer (.msi) very well. I can explain what the current batch files do and manage the .wxs code (and continue to enhance going forward since there are certainly areas to improve).

Perfect. That would help me a lot because I have no time to read up on WiX. Could you walk me through the process?

For comparison, the Git for Windows installer is generated using a script that performs these steps:

• from an up-to-date MSys2 installation, a file list is generated by starting with a list of required packages, getting the full list of transitive dependencies, then excluding a number of unneeded files (this is unfortunately necessary to bring down the installer size from ~55MB to ~30MB which is still more than I would like)
• with this file list (and a couple of added files, such as the release notes), InnoSetup is called to compile the installer from the install.iss definition.

I imagine that a very similar strategy is used to make the .msi here, correct?

the question is anyone available to do the build script integration?

Yes. Me. 😄

[bleissem]

@robmen
I would like to commit myself as a developer for this issue if any help is needed.
I think I can help / support / do the work for @dscho .

[dscho]

@robmen could you give us a walk-through of the basic concepts how gitsetup uses WiX to generate an .msi file? I would be very interested to learn how this is done.

[robmen]

@dscho, yes the strategy will be identical with different input file formats (.wxs files are XML) and tools being called (WiX toolset breaks the build into two steps: compile then link). I expect generating the .wxs files to be the "hardest" part of the shell files.

Do you have a directory structure in mind for organizing the multiple "installers" now? There are enough files unique to each installer build that we won't want to intermingle them. For example, do you want a top level "installer-innosetup" and "installer-wix" folders or do you want sub-folders under "installer"? WIth that guidance in place, I can start bringing over code from this repo into the correct location in git-for-windows/build-extra.

@bleissem very cool. Just need to get a few structural issues out of the way then should be able to start iterating and making progress.

[dscho]

Do you have a directory structure in mind for organizing the multiple "installers" now?

So far, build-extra sports the following subdirectories:

• installer/: the files to build the official .exe installers
• portable/: the files to build the self-extracting .7z.exe portable Gits
• archive/: the files to build the .bz2 archives
• sdk-installer/: the files to build the self-extracting Git for Windows SDK installer

The last one does not concern us, of course, as its release cycle is completely independent from Git for Windows'.

The other three actually share the call to make-file-list.sh (which is located in the top-level directory for that reason) to generate the list of files to include into the installer/portable Gits.

I imagine that we want to add an msi/ subdirectory, containing a release.sh file to generate the .wxs files and call WiX (installing it if required, I guess) to generate the .msi files.

Sounds good?

[robmen]

@dscho (too funny, ships--or issue comments--passing in the night).

For anyone looking at the repo here (namely harvest.cmd), I'd like to move away from using heat.exe and have a script like make-file-list.sh that does the file list generation.

At some point, it will likely be prudent to refactor the shell scripts to extract the commonalities between InnoSetup based build and WiX based build. But I'm probably stating the obvious there. 😸

[dscho]

At some point, it will likely be prudent to refactor the shell scripts to extract the commonalities between InnoSetup based build and WiX based build.

Which files should I look at to figure out what the WiX-based build does before calling the compiler/linker?

[robmen]

I imagine that we want to add an msi/ subdirectory, containing a release.sh file to generate the .wxs files and call WiX (installing it if required, I guess) to generate the .msi files.

Sounds good. I missed the fact that make-file-list.sh was in the root (clicked straight through to the code) but now I can see that it generates a newline delimited list of files. That should be easy enough to work with.

Which files should I look at to figure out what the WiX-based build does before calling the compiler/linker?

Note: this is what my comment above about not using harvest.cmd going forward. Today, the process is three steps.

1. harvest.cmd to generate the GitComponents.wxs.
2. Update the version numbers in the .wxs scripts. I did this because I wasn't confident in the Git for Windows versioning scheme yet and thus hadn't devised a good way of automatically extracting the version from the portable install. This will be much easier now with the version as part of the release.
3. Call make.bat.

So two things:

1. I would look at the GitComponents.wxs file to see what the output should be. The harvest.cmd will not be helpful going forward.
2. I've been seriously considering moving gitsetup to WiX v4 (pre-release). Doing so would make GitComponents.wxs much easier to generate and I would like to make improvements in the core toolset to improve the installation package based on Git for Windows. That was one of the reasons I created the gitsetup project in the first place.

[robmen]

@dscho any thoughts on moving to WiX v4? Are you for/against/neutral using pre-release WiX software?

With that answered, I should be able to create a pull request to create the /msi folder in build-extras with first draft of all the code. I could also add text files with the steps that need to be converted into shell scripts. That should get us into a place to iterate fairly quickly then.

[dscho]

Are you for/against/neutral using pre-release WiX software?

Neutral, but I trust your judgement more than mine.

Thank you for all your explanations, and I look forward to seeing the progress!

[robmen]

@dscho, if you're neutral then I'll go with WiX 4 since that is the branch under active development and I'll be able to take my "learnings" from Git for Windows back to improving the WiX toolset. It'll also make the code cleaner.

I'll get the pull request together in the next couple days.

PS: I should note I may go dark every couple days as FireGiant work takes priority. I should always respond within a week though.

[dscho]

I'll get the pull request together in the next couple days.

Awesome!

[bleissem]

@robmen
It's an honor to me to have the opportunity to contribute.

[bleissem]

@dscho
Just a question. Are you currently building git for windows using some kind of continuous integration ?
If not, I would like to try to set up a build on appveyor and provide my experience I made with building git for windows.
The preinstalled software on their build workers:
https://www.appveyor.com/docs/installed-software
look like the same that you are currently using for building git for windows:
https://github.com/git-for-windows/build-extra/blob/master/README.md
( regardless of the version )

[dscho]

Are you currently building git for windows using some kind of continuous integration ?

I always wanted to do that, but haven't managed so far (the only VM I could use for that purpose was used by myself to develop actively, and that CI interfered rather badly, and besides, I managed to break my Jenkins instance).

The preinstalled software on their build workers:
https://www.appveyor.com/docs/installed-software
look like the same that you are currently using for building git for windows

Almost, but not quite.

The main difference is that we still use a forked MSys2 runtime (see Alexpux/Cygwin#8 -- which I really need to update, badly). With the default MSys2 runtime, Git's regression test suite will not pass. Besides, we edit a couple of files using a dirty trick, some of which are then bundled into the Git for Windows installer (most notably, /etc/nsswitch.conf).

So much as I would like to, currently I do not think we can use AppVeyor as-is.

The best course of action might be to push into the direction of getting the MSys2-related patches upstream (git-for-windows/git#284).

[dscho]

The preinstalled software on their build workers:
https://www.appveyor.com/docs/installed-software
look like the same that you are currently using for building git for windows

Almost, but not quite.

Oh, and of course we would need a 32-bit MSys2, too. I am sure that AppVeyor would support us, there, but that still leaves the other issues I have to address first.

[dscho]

@robmen thanks for your Pull Request! For the record, I have built on your work, the current progress can be seen here. Have you seen the comments I left after merging?

[dscho]

So the msi branch is now in a state where I can successfully compile an .msi file. Have not had a chance to test it yet, though.

[jchoover]

FYI, I am also interested in a MSI based deployment of Git. I also work with @robmen on WiX and would be willing to contribute some time to getting this done officially.

[robmen]

The was an attempt to integrate an MSI into the official Git for Windows project but there wasn't enough interest to get it over the finish line. I continue to maintain and use this project instead. You are welcome to use it as well.

[dscho]

I am also interested in a MSI based deployment of Git. I also work with @robmen on WiX and would be willing to contribute some time to getting this done officially.

I'd love that! Please do feel free to contribute patches to https://github.com/git-for-windows/build-extra/tree/master/msi/

[goyalyashpal]

umh, what's the conclusion?

• all the comments looked positive "So the msi branch is now in a state where I can successfully compile an .msi file"
• and thereafter, boom, silence for ~3 years with no links or any reference in between,
• then the sad ending "wasn't enough interest"
• the build-extra/msi folder still exists in g-f-w repo

i can't connect the dots, can someone explain?

[robmen]

I think you did a great job connecting the dots, @yashpalgoyal1304.

I continue to maintain this fork for my own enjoyment. It builds on the fantastic work of @dscho while providing me with a very clean and simple installation.

[dscho]

While I appreciate the work that has gone into the msi/ directory of git-for-windows/build-extra, from my side, the maintenance cost of integrating this into the regular release process and provide .msi versions of every Git for Windows version was just too high.

We have added and removed options from Git for Windows' installer in the meantime and I either forgot to port them to the msi/ directory or did not even know how to port them. If there was an enthusiastic and active contributor volunteering to keep the files in msi/ up to date, sure. I would even pay the price of a substantially longer release process (because the .msi needs to be built, too, and uploaded) and of the substantially larger size than the LZMA-compressed installer. But as things stand, I have serious doubts there.