Fail to build sharp

[djacu]

I am trying to package an astro theme to use for a personal website and the theme depends on sharp.

I found #162 and that gave me a tip for setting the npm_config_sharp_libvips_local_prebuilds environment variable because it was trying to fetch sources during the install. But now the error it returns has me stuck.

Seems like these two are offending lines and are related but it is in the package file so I'm not sure why it cannot be found.

npm ERR! sh: line 1: /build/node_modules/.bin/prebuild-install: cannot execute: required file not found

npm ERR! ../src/common.cc:24:10: fatal error: vips/vips8: No such file or directory
npm ERR!    24 | #include <vips/vips8>
npm ERR!       |          ^~~~~~~~~~~~
npm ERR! compilation terminated.

Any advice?

short log:

% nix develop .#                                                                             
trace: warning: [npmlock2nix] You are using the new v2 beta api. The interface isn't stable yet. Please report any issues at https://github.com/nix-community/npmlock2nix/issues
error: builder for '/nix/store/d14zfviif81614860qc6i8pczw74h19a-Astro-Modern-Personal-Website-2.2.0.drv' failed with exit code 1;
       last 10 log lines:
       > npm ERR! gyp ERR! System Linux 5.4.0-139-generic
       > npm ERR! gyp ERR! command "/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/bin/node" "/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
       > npm ERR! gyp ERR! cwd /build/node_modules/sharp
       > npm ERR! gyp ERR! node -v v18.16.0
       > npm ERR! gyp ERR! node-gyp -v v9.3.1
       > npm ERR! gyp ERR! not ok
       > 
       > npm ERR! A complete log of this run can be found in:
       > npm ERR!     /build/.npm/_logs/2023-05-15T23_33_07_430Z-debug-0.log
       > 
       For full logs, run 'nix log /nix/store/d14zfviif81614860qc6i8pczw74h19a-Astro-Modern-Personal-Website-2.2.0.drv'.
error: 1 dependencies of derivation '/nix/store/6ccmf5whzryz5asy8aqm0455bj0zz7cw-nix-shell-env.drv' failed to build

full log:

% nix log /nix/store/d14zfviif81614860qc6i8pczw74h19a-Astro-Modern-Personal-Website-2.2.0.drv
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
[#########.........] - reify:@esbuild/android-arm: timing reifyNode:node_module^M[#########.........] \ reify:@esbuild/android-arm: timing reifyNode:node_module^M[#########.........] / reif>
added 534 packages in 4s

210 packages are looking for funding
  run `npm fund` for details
patching script interpreter paths in node_modules/.bin
npm ERR! code 1
npm ERR! path /build/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Using local libvips from /nix/store/0bwij9y62n5l7nxv6kj7ql4k8075fglc-libvips/v8.13.3/libvips-8.13.3-linux-x64.tar.br
npm ERR! sharp: Integrity check passed for linux-x64
npm ERR! make: Entering directory '/build/node_modules/sharp/build'
npm ERR!   CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o
npm ERR! rm -f Release/obj.target/../node-addon-api/nothing.a Release/obj.target/../node-addon-api/nothing.a.ar-file-list; mkdir -p `dirname Release/obj.target/../node-addon-api/nothing.a`
npm ERR! ar crs Release/obj.target/../node-addon-api/nothing.a @Release/obj.target/../node-addon-api/nothing.a.ar-file-list
npm ERR!   COPY Release/nothing.a
npm ERR!   TOUCH Release/obj.target/libvips-cpp.stamp
npm ERR!   CXX(target) Release/obj.target/sharp-linux-x64/src/common.o
npm ERR! make: Leaving directory '/build/node_modules/sharp/build'
npm ERR! sh: line 1: /build/node_modules/.bin/prebuild-install: cannot execute: required file not found
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.10.11 found at "/nix/store/95cxzy2hpizr23343b8bskl4yacf4b3l-python3-3.10.11/bin/python3"
npm ERR! gyp WARN read config.gypi ENOENT: no such file or directory, open '/nix/store/z2vp3brb6bkfgfvlllvdpmvg624m5qfn-node-sources-18.16.0/include/node/config.gypi'
npm ERR! gyp info spawn /nix/store/95cxzy2hpizr23343b8bskl4yacf4b3l-python3-3.10.11/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/build/node_modules/sharp/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/nix/store/z2vp3brb6bkfgfvlllvdpmvg624m5qfn-node-sources-18.16.0/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/nix/store/z2vp3brb6bkfgfvlllvdpmvg624m5qfn-node-sources-18.16.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/nix/store/z2vp3brb6bkfgfvlllvdpmvg624m5qfn-node-sources-18.16.0/$(Configuration)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/build/node_modules/sharp',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! ../src/common.cc:24:10: fatal error: vips/vips8: No such file or directory
npm ERR!    24 | #include <vips/vips8>
npm ERR!       |          ^~~~~~~~~~~~
npm ERR! compilation terminated.
npm ERR! make: *** [sharp-linux-x64.target.mk:135: Release/obj.target/sharp-linux-x64/src/common.o] Error 1
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:203:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Linux 5.4.0-139-generic
npm ERR! gyp ERR! command "/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/bin/node" "/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/lib/node_modules/npm/node_modules>
npm ERR! gyp ERR! cwd /build/node_modules/sharp
npm ERR! gyp ERR! node -v v18.16.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /build/.npm/_logs/2023-05-15T23_33_07_430Z-debug-0.log

Relevant flake:

{
  description = "djacu's personal site";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.npmlock2nix.url = "github:nix-community/npmlock2nix";
  inputs.npmlock2nix.flake = false;
  inputs.astro-theme-src.url = "github:manuelernestog/astro-modern-personal-website";
  inputs.astro-theme-src.flake = false;

  outputs = {
    self,
    nixpkgs,
    flake-utils,
    npmlock2nix,
    astro-theme-src,
  }:
    flake-utils.lib.eachDefaultSystem (
      system: let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [
            (self: super: {
              npmlock2nix = pkgs.callPackage npmlock2nix {};
            })
          ];
        };

        astro-shell = pkgs.npmlock2nix.v2.shell {
          src = astro-theme-src;
          nodejs = pkgs.nodejs;
          node_modules_mode = "copy";

          node_modules_attrs.npm_config_sharp_libvips_local_prebuilds = "${lib-vips}";
          npm_config_sharp_libvips_local_prebuilds = "${lib-vips}";

          node_modules_attrs.nativeBuildInputs = [ pkgs.python3 ];
        };

        lib-vips = pkgs.stdenvNoCC.mkDerivation rec {
          name = "libvips";
          version = "8.13.3";
          arch = "linux-x64";
          src = builtins.fetchurl {
            url = "https://github.com/lovell/sharp-libvips/releases/download/v${version}/${name}-${version}-${arch}.tar.br";
            sha256 = "1jgw7dknfk9w1cwzcj6k98jrnncg1fwxf6jf1hlxpnixrn0a2hdk";
          };
          unpackPhase = ''
            mkdir source
            cp $src ./source/
          '';
          patchPhase = ''
            echo "nothing to patch"
          '';
          configurePhase = ''
            echo "nothing to configure"
          '';
          buildPhase = ''
            echo "nothing to build"
          '';
          installPhase = ''
            mkdir -p $out/v${version}
            cp -r ./source/* $out/v${version}/${name}-${version}-${arch}.tar.br
          '';
        };
      in {
        devShells.default = astro-shell;
        packages.default = lib-vips;
      }
    );
}

[icewind1991]

I was able to get sharp to work with the system libvips by explicitly adding sharp as a dependency and using

node_modules_attrs = {
  buildInputs = with pkgs; [ vips ];
  nativeBuildInputs = with pkgs; [ pkg-config python3 ];
  postBuild = ''
    npm rebuild sharp --nodedir=${nodeSource pkgs.nodejs_20}
  '';
};

to trigger the build scripts.

with nodeSource copied from npmlock2nix:

nodeSource = nodejs: pkgs.runCommand "node-sources-${nodejs.version}"
  { } ''
  tar --no-same-owner --no-same-permissions -xf ${nodejs.src}
  mv node-* $out
'';