Skip to content
This repository has been archived by the owner on Apr 14, 2021. It is now read-only.

Strange lockfile behaviour with bundle update/install and platform-specific gems #6491

Closed
nilsding opened this issue Apr 17, 2018 · 3 comments · Fixed by #6495
Closed

Strange lockfile behaviour with bundle update/install and platform-specific gems #6491

nilsding opened this issue Apr 17, 2018 · 3 comments · Fixed by #6495
Assignees

Comments

@nilsding
Copy link
Contributor

nilsding commented Apr 17, 2018

Hey everyone,
I recently stumbled upon some weird behaviour with bundle update and bundle install and some platform-specific gem dependencies in a few of our projects. I successfully managed to track it down to the following:

  • The pry gem, which has at least two gem versions: one for any platform (I'll call that one ruby from now on) and one for java
  • That java platform version has an additional dependency on spoon, which has a dependency on ffi
  • Any bundle command which modifies the Gemfile which is not bundle install (e.g. bundle update or bundle lock) adds the ffi dependency for the ruby platform.
  • bundle install will correctly remove the ffi gem for the ruby platform from the Gemfile.lock, as it determined it's apparently not needed for that platform.

I have created a git repository with each step in its own commit to reproduce this behaviour: https://github.com/nilsding/bundler-lockfile-issue/commits/master.


Output of bundle env, if it's of any help:

Environment

Bundler             1.16.1
  Platforms         ruby, universal-java-1.8
Ruby                2.3.3p0 (2018-02-21 revision 56859) [java]
  Full Path         /Users/nilsding/.rvm/rubies/jruby-9.1.16.0/bin/jruby
  Config Dir        /Users/nilsding/.rvm/rubies/jruby-9.1.16.0/etc
RubyGems            2.6.14.1
  Gem Home          /Users/nilsding/.rvm/gems/jruby-9.1.16.0
  Gem Path          /Users/nilsding/.rvm/gems/jruby-9.1.16.0:/Users/nilsding/.rvm/gems/jruby-9.1.16.0@global
  User Path         /Users/nilsding/.gem/jruby/2.3.0
  Bin Dir           /Users/nilsding/.rvm/gems/jruby-9.1.16.0/bin
Tools
  Git               2.16.2
  RVM               1.29.3 (master)
  rbenv             not installed
  chruby            not installed
  rubygems-bundler  (1.4.4)

Bundler Build Metadata

Built At          2017-12-21
Git SHA           0034ef341
Released Version  true

Bundler settings

jobs
  Set for the current user (/Users/nilsding/.bundle/config): "8"
gem.test
  Set for the current user (/Users/nilsding/.bundle/config): "rspec"
gem.mit
  Set for the current user (/Users/nilsding/.bundle/config): false
gem.coc
  Set for the current user (/Users/nilsding/.bundle/config): true

Gemfile

Gemfile

source "https://rubygems.org"

gem "pry"
gem "empyrean", "= 0.1.1"

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    coderay (1.1.2)
    empyrean (0.1.1)
    ffi (1.9.23-java)
    method_source (0.9.0)
    pry (0.11.3)
      coderay (~> 1.1.0)
      method_source (~> 0.9.0)
    pry (0.11.3-java)
      coderay (~> 1.1.0)
      method_source (~> 0.9.0)
      spoon (~> 0.0)
    spoon (0.0.6)
      ffi

PLATFORMS
  java
  ruby

DEPENDENCIES
  empyrean (= 0.1.1)
  pry

BUNDLED WITH
   1.16.1
@segiddins
Copy link
Member

Weird, on master even bundle install doesn't seem to be removing ffi.

Thanks so much for the amazing repro case.

@segiddins
Copy link
Member

I think #6495 should fix this!

@nilsding
Copy link
Contributor Author

nilsding commented Jun 6, 2018

@segiddins yes, #6495 indeed fixes this! No more unnecessary platform-specific gems on bundle lock or bundle updates 🎉

bundlerbot added a commit that referenced this issue Jul 22, 2018
…ckfile, r=segiddins

[Definition] Filter out unneeded gem platforms after resolving

### What was the end-user problem that led to this PR?

The problem was the lockfile would contain platform-specific gems that it didn't need

Closes #6491

### What was your diagnosis of the problem?

My diagnosis was the resolver sometimes activates platforms it doesn't need, since it can't know it won't need them

### What is your fix for the problem, implemented in this PR?

My fix is to use `SpecSet#for` to filter out gems that won't ever be used

### Why did you choose this fix out of the possible options?

I chose this fix because it isn't re-inventing the wheel!
colby-swandale pushed a commit that referenced this issue Oct 5, 2018
…ckfile, r=segiddins

[Definition] Filter out unneeded gem platforms after resolving

### What was the end-user problem that led to this PR?

The problem was the lockfile would contain platform-specific gems that it didn't need

Closes #6491

### What was your diagnosis of the problem?

My diagnosis was the resolver sometimes activates platforms it doesn't need, since it can't know it won't need them

### What is your fix for the problem, implemented in this PR?

My fix is to use `SpecSet#for` to filter out gems that won't ever be used

### Why did you choose this fix out of the possible options?

I chose this fix because it isn't re-inventing the wheel!

(cherry picked from commit 7b603f3)
colby-swandale pushed a commit that referenced this issue Oct 5, 2018
…ckfile, r=segiddins

[Definition] Filter out unneeded gem platforms after resolving

### What was the end-user problem that led to this PR?

The problem was the lockfile would contain platform-specific gems that it didn't need

Closes #6491

### What was your diagnosis of the problem?

My diagnosis was the resolver sometimes activates platforms it doesn't need, since it can't know it won't need them

### What is your fix for the problem, implemented in this PR?

My fix is to use `SpecSet#for` to filter out gems that won't ever be used

### Why did you choose this fix out of the possible options?

I chose this fix because it isn't re-inventing the wheel!

(cherry picked from commit 7b603f3)
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Dec 17, 2018
pkgsr change
* Remove @Prefix@ from ALTERNATIVES file.

## 1.17.2 (2018-12-11)

 - Add compatability for bundler merge with Ruby 2.6

## 1.17.1 (2018-10-25)

 - Convert `Pathname`s to `String`s before sorting them, fixing #6760 and #6758 ([#6761](rubygems/bundler#6761), @alexggordon)

## 1.17.0 (2018-10-25)

No new changes.

## 1.17.0.pre.2 (2018-10-13)

Features:

  - Configure Bundler home, cache, config and plugin directories with `BUNDLE_USER_HOME`, `BUNDLE_USER_CACHE`, `BUNDLE_USER_CONFIG` and `BUNDLE_USER_PLUGIN` env vars ([#4333](rubygems/bundler#4333), @gwerbin)
  - Add `--all` option to `bundle binstubs` that will generate an executable file for all gems with commands in the bundle
  - Add `bundle remove` command to remove gems from the Gemfile via the CLI
  - Improve checking file permissions and asking for `sudo` in Bundler when it doesn't need to
  - Add error message to `bundle add` to check adding duplicate gems to the Gemfile
  - When asking for `sudo`, Bundler will show a list of folders/files that require elevated permissions to write to.

The following new features are available but are not enabled by default. These are intended to be tested by users for the upcoming release of Bundler 2.

  - Improve deprecation warning message for `bundle show` command
  - Improve deprecation warning message for the `--force` option in `bundle install`

## 1.17.0.pre.1 (2018-09-24)

Features:

  - Check folder/file permissions of the Bundle home directory in the `bundle doctor` command ([#5786](rubygems/bundler#5786), @ajwann)
  - Remove compiled gem extensions when running `bundle clean` ([#5596](rubygems/bundler#5596), @akhramov)
  - Add `--paths` option to `bundle list` command ([#6172](rubygems/bundler#6172), @colby-swandale)
  - Add base error class to gems generated from `bundle gem` ([#6260](rubygems/bundler#6260), @christhekeele)
  - Correctly re-install gem extensions with a git source when running `bundle pristine` ([#6294](rubygems/bundler#6294), @wagenet)
  - Add config option to disable platform warnings ([#6124](rubygems/bundler#6124), @agrim123)
  - Add `--skip-install` option to `bundle add` command to add gems to the Gemfile without installation ([#6511](rubygems/bundler#6511), @agrim123)
  - Add `--only-explicit` option to `bundle outdated` to list only outdated gems in the Gemfile ([#5366](rubygems/bundler#5366), @peret)
  - Support adding multiple gems to the Gemfile with `bundle add` ([#6543](rubygems/bundler#6543), @agrim123)
  - Make registered plugin events easier to manage in the Plugin API (@jules2689)
  - Add new gem install hooks to the Plugin API (@jules2689)
  - Add `--optimistic` and `--strict` options to `bundle add` ([#6553](https://github.com/bundler/bundler/issues/6553), @agrim123)
  - Add `--without-group` and `--only-group` options to `bundle list` ([#6564](rubygems/bundler#6564), @agrim123)
  - Add `--gemfile` option to the `bundle exec` command ([#5924](rubygems/bundler#5924), @ankitkataria)

The following new features are available but are not enabled by default. These are intended to be tested by users for the upcoming release of Bundler 2.

  - Make `install --path` relative to the current working directory ([#2048](rubygems/bundler#2048), @igorbozato)
  - Auto-configure job count ([#5808](rubygems/bundler#5808), @segiddins)
  - Use the Gem Version Promoter for major gem updates ([#5993](rubygems/bundler#5993), @segiddins)
  - Add config option to add the Ruby scope to `bundle config path` when configured globally (@segiddins)

## 1.16.6 (2018-10-05)

Changes:

  - Add an error message when adding a gem with `bundle add` that's already in the bundle ([#6341](rubygems/bundler#6341), @agrim123)
  - Add Homepage, Source Code and Chanagelog URI metadata fields to the `bundle gem` gemspec template (@walf443)

Bugfixes:

  - Fix issue where updating a gem resulted in the gem's version being downgraded when `BUNDLE_ONLY_UPDATE_TO_NEWER_VERSIONS` was set ([#6529](rubygems/bundler#6529), @theflow)
  - Fix some rescue calls that don't specifiy error type (@utilum)
  - Fix an issue when the Lockfile would contain platform-specific gems that it didn't need ([#6491](rubygems/bundler#6491), @segiddins)
  - Improve handlding of adding new gems with only a single group to the Gemfile in `bundle add` (@agrim123)
  - Refactor check for OpenSSL in `bundle env` (@voxik)
  - Remove an unnecessary assignment in Metadata (@voxik)

Documentation:

  - Update docs to reflect revised guidance to check in Gemfile.lock into version control for gems ([#5879](https://github.com/bundler/bundler/issues/5879), @arbonap)
  - Add documentation for the `--all` flag in `bundle update` (@agrim123)
  - Update README to use `bundle add` in usage examples (@hdf1986)
hsbt pushed a commit to rubygems/bundler-graph that referenced this issue Oct 19, 2021
…ckfile, r=segiddins

[Definition] Filter out unneeded gem platforms after resolving

### What was the end-user problem that led to this PR?

The problem was the lockfile would contain platform-specific gems that it didn't need

Closes rubygems/bundler#6491

### What was your diagnosis of the problem?

My diagnosis was the resolver sometimes activates platforms it doesn't need, since it can't know it won't need them

### What is your fix for the problem, implemented in this PR?

My fix is to use `SpecSet#for` to filter out gems that won't ever be used

### Why did you choose this fix out of the possible options?

I chose this fix because it isn't re-inventing the wheel!
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants