Building binaries on FreeBSD

FreeBSD binaries are built using Poudriere, a FreeBSD-specific tool to build binary packages using a clean-room environment. Poudriere:

• Creates and manages lightweight virtual environments (jails) for i386 and amd64
• Keeps virtual environments up to date
• Manages the building process, logging and packaging. It can expose results via the HTTP interface.

Another tools being used is portshaker which helps to merge different FreeBSD port trees and keep them up to date. A FreeBSD port tree describes how applications should be built into binary packages and installed. This is the standard way of building third-party software on FreeBSD.

#Custom ports tree

Since node-sass and many npm packages are not yet part of the official FreeBSD ports tree, we maintain a custom ports tree which contains the necessary elements. That's why we need portshaker to combine the official tree (containing python, compilers, node) and the custom tree.

We currently have the following custom ports:

Port name	Description
misc/node-gyp-headers	Development headers for various node and iojs versions
www/iojs42	Old version of io.js 1.0.4 to build binaries for NODE_MODULE_VERSION == 42
www/npm	npm port modified to support io.js 1.0.4 as well as to workaround segfault in i386 environment
textproc/libsass	updated libsass FreeBSD port
textproc/sassc	sassc to match the updated libsass
textproc/node-sass	Instructions how to build node-sass (depends on the libsass port)

Because www/npm already exists in the official FreeBSD tree we need to have a higher version number in our custom port, which is easily achieved by bumping up PORTREVISION. If we have an older npm version, portshaker complains:

www/npm: port version going backward (I will not merge this port)!

Challenges to using FreeBSD ports infrastructure

FreeBSD port building is split into certain well-defined phases, like fetch, depend, extract, building and stage and install. Every phase has its own restrictions - for example during extract/build and later we are not supposed to have network access. Ports working directories can be written during extraction phase and later. This is a much more strict model than that one used by the npm, where fetching of dependencies is mixed with the construction of the package. To work this around we try to fetch all dependencies in advance into a download directory that serves also as the npm package cache. NPM_CONFIG_CACHE is set to this directory (usually /usr/ports/distfiles/npm) to achieve this.

Also node development files (headers etc.) that are required by node-gyp are provided beforehand (by installing the misc/node-gyp-headers port and setting HOME enviroment variable is set to /usr/local//share/node-gyp which contains a .node-gyp subdirectory needed to feed node-gyp.

Since poudriere enforces the above restrictions (network access, etc.) during build the building machine fails at the npm install stage when trying to resolve DNS hostname. If all dependencies are provided in advance, npm proceeds with the installation using locally cached files and does not try to access the network.

#Environments available

We can build binaries for FreeBSD 8.4, 9.3 and 10.1. For official binaries we ship only 10.1 binaries because 8.4 and 9.3 ship only gcc 4.2.1 by default and we need newer C++ runtime (stdlibc++ library). The packages for 8.4 and 9.3 are build and tested; but putting them into node-sass binary distribution is currently not possible; see sass/node-sass#733 for more details. Using node-sass is perfectly possible on those platforms if the newer gcc is installed.

FreeBSD version	architecture	Environment name
10.1-RELEASE-p6	amd64	iojs_10_1_amd64
10.1-RELEASE-p6	amd64	nodejs010_10_1_amd64
10.1-RELEASE-p6	amd64	nodejs_10_1_amd64
10.1-RELEASE-p6	i386	iojs_10_1_i386
10.1-RELEASE-p6	i386	nodejs010_10_1_i386
10.1-RELEASE-p6	i386	nodejs_10_1_i386
10.1-RELEASE-p8	amd64	iojs42_10_1_amd64
10.1-RELEASE-p8	i386	iojs42_10_1_i386
8.4-RELEASE-p24	amd64	iojs_8_4_amd64
8.4-RELEASE-p24	amd64	nodejs010_8_4_amd64
8.4-RELEASE-p24	amd64	nodejs_8_4_amd64
8.4-RELEASE-p24	i386	iojs_8_4_i386
8.4-RELEASE-p24	i386	nodejs010_8_4_i386
8.4-RELEASE-p24	i386	nodejs_8_4_i386
8.4-RELEASE-p26	amd64	iojs42_8_4_amd64
8.4-RELEASE-p26	i386	iojs42_8_4_i386
9.3-RELEASE-p10	amd64	iojs_9_3_amd64
9.3-RELEASE-p10	amd64	nodejs010_9_3_amd64
9.3-RELEASE-p10	amd64	nodejs_9_3_amd64
9.3-RELEASE-p10	i386	iojs_9_3_i386
9.3-RELEASE-p10	i386	nodejs010_9_3_i386
9.3-RELEASE-p10	i386	nodejs_9_3_i386
9.3-RELEASE-p12	amd64	iojs42_9_3_amd64
9.3-RELEASE-p12	i386	iojs42_9_3_i386

#How to update and maintain the custom node-sass port

• Update libsass and sassc ports. Test them on different architectures. Usually trying to run sassc in the virtual environment is enough to check if all needed C++ libraries are in place.

• Check if www/npm port has been updated in the main ports tree. Update custom npm accordingly if this happened.

• To bootstrap the process we provide a stub package.json file to pre-download all the dependencies in advance. This file must contain all dependencies and the current version, but must not contain node-sass custom build scripts.

• Push the release version into the FreeBSD git tree. Note the last commit identifier.

• Delete old distfiles by running make distclean and delete the distinfo file.

• Update the Makefile to reflect the new version and the commit identifier.

• Run make makesum to fetch the new .tar.gz file. GitHub sometimes has trouble to create a .tar.gz distribution right after pushing the new commit into the branch. distinfo is recreated at this point.

• Check if there are updates to the FreeBSD www/iojs, www/node010 and www/node ports. If yes, prepare a new release of the misc/node-gyp-headers port. The easiest way is to start building node-sass by hand (not using poudriere) as root and see how node-gyp updates /usr/local//share/node-gyp directory. Package that directory into the new "release" of misc/node-gyp-headers.

(...)

• Update version number in the nodesass-repack.sh script.

#Building process

1. A shell script is used to start poudriere run for i386 and amd64 to build FreeBSD binaries for 10.1-RELEASE.

2. After the packages are built and placed into /usr/local/poudriere/data/packages/iojs42_10_1_i386-exp/All/node-sass-3.0.0.b7.txz (and friends) we are using a nodesass-repack.sh script to extract the binaries only and put them into a checked out directory of https://github.com/sass/node-sass-binaries repository.