-
-
Notifications
You must be signed in to change notification settings - Fork 2k
Nested bundle exec support is broken on latest rubygems/bundler combo #4602
Comments
/cc @segiddins |
Confirmed issue, exact bit by bit same image with rubygems |
I'm unable to reproduce this locally, |
@segiddins sounds like this might be linux-specific? All the examples I have seen are on Ubuntu. |
@SamSaffron I need better repro instructions, the commands you gave don't work for me:
|
I don't have a linux machine, so someone else will have to debug then ¯_(ツ)_/¯ |
I'm perfectly happy to debug in a VM if I can figure out how to repro :P |
@indirect oops was typing that from memory ... you want
|
I was able to reproduce the However, to make the steps completely reproducible, I uninstalled my (rbenv managed) versions of ruby first and then re-installed them from scratch. Afterwards, the problem went away. Even though I had the latest rubygems and bundler versions prior to the uninstall, I'm not sure what really changed during the reinstall of ruby. Nonetheless, I thought this might be worth mentioning. Reinstalling ruby, rubygems, and bundler in my particular environment made the issue go away. For posterity, here were the steps I took when I was able to reproduce this (even though I can no longer reproduce them after a fresh ruby install):
|
@indirect Have you tried to reproduce the issue from an official ruby docker image? e.g. Here is the related issue on the docker repo: docker-library/ruby#73 |
+1, I have same issue |
Some more information on reproducing the issue (on Ubuntu 14.04 LTS x86_64, single Ruby version 2.1.2 installed system-wide under It turns out that the issue manifests itself only if I've installed/updated Bundler after I've updated Rubygems, not before. Having Bundler 1.12.5 installed on Rubygems 2.6.1 and then updating Rubygems to 2.6.4 seems to work. However, installing Bundler on Rubygems 2.6.4 leads to the issue manifesting itself. I suspect it might be related to the gem binstubs/shims as - load Gem.bin_path('bundler', 'bundle', version) # When installed in Rubygems 2.6.1
+ load Gem.activate_bin_path('bundler', 'bundle', version) # When installed in Rubygems 2.6.4 So the Bundler shim with Steps to reproduce on Ubuntu 14.04
I can't reproduce it on Mac OS with Ruby via rbenv, though. Might be related to me using WorkaroundThis is the one-liner I used to revert back to the old Bundler shim, fixing the issue on our servers, without permanently downgrading Rubygems to 2.6.1: gem uninstall bundler bundle rubygems-update --all -x && \
gem update --system 2.6.1 && \
gem install bundler && \
gem uninstall rubygems-update --all -x && \
gem update --system 2.6.4 Note that Rubygems 2.6.1 is not compatible with gems installed in Rubygems 2.6.4 and |
rubygems 2.6.6 and bundler 1.12.5 here. After some minor digging i also found that's related to the change on the I have noticed that we can't run bundler inside bundler, eg: With a rails project for example, it works because you call the rails binstub in the app folder
So, just to be clear, the command: I have setup a docker image to easily reproduce this bug, you just have to pull and run it
Interesting fact, if you rename the Gemfile, it works :) :
Here's the Gemfile and the Dockerfile used: Gemfile:
Dockerfile:
|
Based on the answer above from @mitio I managed to implement a workaround thats a little simpler:
After running that |
OSX 10.11.6 Confirm that the fix by @srbry works for me. |
Can you check if running
|
I just builded a docker image from the ruby 2.3 dockerfile but with bundler version |
I'm having this problem with the "official" docker images too so I'm testing with that, but it makes life really difficult due to a weird configuration. It looks like bundler 1.12.7 is installed in /usr/local/lib/ruby/gems/2.3.0 by default but there are loads of env variables that are then set when you use the container yourself that change it's behaviour. Particularly GEM_HOME=/usr/local/bundle Anyway, with much faffing around I managed to confirm that if I install 1.13.0.rc.1 in the same weird way they install 1.12.7, then it fixes this problem. But I thought it worth noting that I think part of the problem here is the weird way these images are installing and configuring bundler in the first place. |
The combination of bundler 1.12.5 and rubygems 1.6.2 doesn't play well at all when trying to run gems such as foreman where bundler is used to run bundler. Just upgrading to the latest bundler rc doesn't fix it and changing rubygems causes a massive rebuild. Issues: - rubygems/bundler#4402 - rubygems/bundler#4576 - rubygems/bundler#4602 - docker-library/ruby#73 This PR patches bundler to work around the issue as highlighted here and unbreaks everything for me: rubygems/bundler#4602 (comment)
tl;dr `--deployment` will copy _all_ gems to the `--path` now and a proper fix is in place for nested bundle execs when `--path` is set and RubyGems >=2.6.2 is being used. Fixes rubygems#4974. There's two problems blended in here. Let's trace the events here from the beginning, shall we? First off, from probably the first versions of Bundler, when `disable_shared_gems` is true, the `GEM_PATH` gets internally overridden and initialized to an empty string. This is done to make sure that later in the `Bundler.setup` process it's expanded to ONLY the Bundler `--path` setting, otherwise it expands to include the system gems path. In 1.12, `bundle exec` was changed to use `Kernel.load` in some cases, and that new code path calls `require "bundler/setup"`. Later, issue rubygems#4592 was filed, showing that in cases like `--deployment` where `disable_shared_gems` is true, Bundler couldn't find itself, because Bundler never lives in the `--path` but only in system gems. And as it would later be discovered, was not a problem with RubyGems 2.6.1, but was a problem with >=2.6.2 (and older RubyGems it would seem, though those weren't as thoroughly investigated). We fixed rubygems#4592 (see PR rubygems#4701) in 1.13.0 by changing the oooold code initializing `GEM_PATH` to be initialized to `nil` instead of empty string in all `disable_shared_gems` cases. But it created another bug, filed as rubygems#4974, wherein system gems were no longer copied to the `--path` when `disable_shared_gems` was true. In this commit here (rubygems#4992) we've reverted the change so that `GEM_PATH` is now back to being initialized to an empty string instead of `nil`. That fixes rubygems#4974, but regresses rubygems#4592, and we needed a new way to fix it. After a few tortured attempts at this, I ran across issue rubygems#4602, a similar report of nested bundle execs being broken, rubygems#4602 itself an extension of rubygems#4381 reporting the same problem. It brought to light the role the Rubygems version played in the problem. When the `bundler` gem is installed and the wrapper is generated for any gem executables, the contents of this wrapper are determined by the Rubygems version. Up until RubyGems 2.6.1 the last line of this wrapper calls `Gem.bin_path`. Bundler replaces the Rubygems implementation of `Gem.bin_path` with its own, and has for a long time made a special exception for the Bundler gem itself, short-circuiting with the contents of a special ENV variable called `BUNDLE_BIN_PATH`. In Rubygems 2.6.2, `bin_path` was superseded by a new `Gem.activate_bin_path` method which did what `bin_path` did but also activated the gem. Bundler 1.13 added support for this, but it didn't include the same short-circuit for bundler itself. (Alert user @taoza even noticed this here rubygems@fcaab35#r56665282). This commit also includes that short circuit for Rubygems >=2.6.2 now and nested bundle exec should continue to work. Thx to @miros for filing rubygems#4974 and isolating the problem in 1.12, @segiddins for many contributions and better ideas, and everyone else for their bug reports and patience :)
…ismo Fix GEM_PATH regression in 1.13. _nth time is a charm ... here's the latest latest latest commit message_: Fix disable_shared_gems bug & keep nested exec tl;dr `--deployment` will copy _all_ gems to the `--path` now and a proper fix is in place for nested bundle execs when `--path` is set and RubyGems >=2.6.2 is being used. Fixes #4974. There's two problems blended in here. Let's trace the events here from the beginning, shall we? First off, from probably the first versions of Bundler, when `disable_shared_gems` is true, the `GEM_PATH` gets internally overridden and initialized to an empty string. This is done to make sure that later in the `Bundler.setup` process it's expanded to ONLY the Bundler `--path` setting, otherwise it expands to include the system path. In 1.12, `bundle exec` was changed to use `Kernel.load` in some cases, and that new code path calls `require "bundler/setup"`. Later, issue #4592 was filed, showing that in cases like `--deployment` where `disable_shared_gems` is true, Bundler couldn't find itself, because Bundler never lives in the `--path` but only in system gems. And as it would later be discovered, was not a problem with RubyGems 2.6.1, but was a problem with >=2.6.2 (and older RubyGems it would seem, though those weren't as thoroughly investigated). We fixed #4592 (see PR #4701) in 1.13.0 by changing the oooold code initializing `GEM_PATH` to be initialized to `nil` instead of empty string in all `disable_shared_gems` cases. But it created another bug, filed as #4974, wherein system gems were no longer copied to the `--path` when `disable_shared_gems` is true. In this commit here (#4992) we've reverted the change so that `GEM_PATH` is now back to being initialized to an empty string instead of `nil`. That fixes #4974, but regresses #4592, and we needed a new way to fix it. After a few tortured attempts at this, I ran across issue #4602, a similar report of nested bundle execs being broken, #4602 itself an extension of #4381 reporting the same problem. It brought to light the role the Rubygems version played in the problem. When the `bundler` gem is installed and the wrapper is generated for any gem executables, the contents of this wrapper are determined by the Rubygems version. Up until RubyGems 2.6.1 the last line of this wrapper calls `Gem.bin_path`. Bundler replaces the Rubygems implementation of `Gem.bin_path` with its own, and has for a long time made a special exception for the Bundler gem itself, short-circuiting with the contents of a special ENV variable called `BUNDLE_BIN_PATH`. In Rubygems 2.6.2, `bin_path` was superseded by a new `Gem.activate_bin_path` method which did what `bin_path` did but also activated the gem. Bundler 1.13 added support for this, but it didn't include the same short-circuit for bundler itself. (Alert user @taoza even noticed this here fcaab35#r56665282). This commit also includes that short circuit for Rubygems >=2.6.2 now and nested bundle exec should continue to work.
…ismo Fix GEM_PATH regression in 1.13. _nth time is a charm ... here's the latest latest latest commit message_: Fix disable_shared_gems bug & keep nested exec tl;dr `--deployment` will copy _all_ gems to the `--path` now and a proper fix is in place for nested bundle execs when `--path` is set and RubyGems >=2.6.2 is being used. Fixes #4974. There's two problems blended in here. Let's trace the events here from the beginning, shall we? First off, from probably the first versions of Bundler, when `disable_shared_gems` is true, the `GEM_PATH` gets internally overridden and initialized to an empty string. This is done to make sure that later in the `Bundler.setup` process it's expanded to ONLY the Bundler `--path` setting, otherwise it expands to include the system path. In 1.12, `bundle exec` was changed to use `Kernel.load` in some cases, and that new code path calls `require "bundler/setup"`. Later, issue #4592 was filed, showing that in cases like `--deployment` where `disable_shared_gems` is true, Bundler couldn't find itself, because Bundler never lives in the `--path` but only in system gems. And as it would later be discovered, was not a problem with RubyGems 2.6.1, but was a problem with >=2.6.2 (and older RubyGems it would seem, though those weren't as thoroughly investigated). We fixed #4592 (see PR #4701) in 1.13.0 by changing the oooold code initializing `GEM_PATH` to be initialized to `nil` instead of empty string in all `disable_shared_gems` cases. But it created another bug, filed as #4974, wherein system gems were no longer copied to the `--path` when `disable_shared_gems` is true. In this commit here (#4992) we've reverted the change so that `GEM_PATH` is now back to being initialized to an empty string instead of `nil`. That fixes #4974, but regresses #4592, and we needed a new way to fix it. After a few tortured attempts at this, I ran across issue #4602, a similar report of nested bundle execs being broken, #4602 itself an extension of #4381 reporting the same problem. It brought to light the role the Rubygems version played in the problem. When the `bundler` gem is installed and the wrapper is generated for any gem executables, the contents of this wrapper are determined by the Rubygems version. Up until RubyGems 2.6.1 the last line of this wrapper calls `Gem.bin_path`. Bundler replaces the Rubygems implementation of `Gem.bin_path` with its own, and has for a long time made a special exception for the Bundler gem itself, short-circuiting with the contents of a special ENV variable called `BUNDLE_BIN_PATH`. In Rubygems 2.6.2, `bin_path` was superseded by a new `Gem.activate_bin_path` method which did what `bin_path` did but also activated the gem. Bundler 1.13 added support for this, but it didn't include the same short-circuit for bundler itself. (Alert user @taoza even noticed this here fcaab35#r56665282). This commit also includes that short circuit for Rubygems >=2.6.2 now and nested bundle exec should continue to work.
1.13.2 is out now and this should still be fixed, but the 1.13.0 fix was reverted so as to solve #4992 and replaced with a better fix. Possibly we can close this now? But it's a long issue with many participants, so if there's a case here I'm overlooking (which could easily be true), please chime in! |
FWIW I think I experienced a version of this issue running Homebrew's
|
I recently updated to 1.13.2 and ran into this issue. I uninstalled bundler and then installed 1.12.5 to get things working again. This was with Ruby 2.1.6 and Ruby Gems 2.2.3 |
Is there any hint of a solution that allows the latest Bunder to be combined with the latest Rubygems, or is this still in limbo? |
This looks like a duplicate of #4402, or at least very similar. |
I think there's two issues afoot here. One is a bundler bug (see #4992) involving The other problem, as seen in @MatzFan's comment above, is We should probably make a new issue for the latter item here and close out this one (4602). I'm going to do some more research on the shims issue first. |
@chrismo I'm pretty sure my issue is the |
@bfad ok - I don't see any other mention of |
to confirm - the bug fix in #4992 handles this case with RubyGems 2.6.2+ and Bundler 1.12:
the other, similar symptom, but not #4992 is with RubyGems 2.6.1 and Bundler 1.12:
I need to find if there's a RG case for this latter problem. Neither of these cases occur for me with 1.13. Those of you with 1.13 problems similar to these, if you offer up some steps to reproduce I'll take a look. |
@chrismo First, #4992 says it fixes it for 1.13, not 1.12. Also, bundler 1.12.5 is working fine for me, it's 1.13.2 that I reported that I had a problem with nested Based on your suggestions above, I ran these commands to update my version of Ruby Gems and re-install bundler: $> gem uninstall rubygems-update -a -x --force
$> gem uninstall bundler -a -x --force
$> gem update --system
$> gem install bundler This gave me Ruby Gem version 2.6.8 and Bundler version 1.13.6 and fixed my issue. Thank you so much for the pointer to update my Ruby Gems version. I hope this helps other people too. |
I have the following setup: bundler + fastlane + danger. The fastlane lane declares all required variables to simulate Xcode Server CI environment with a PR number for danger and to declare credentials to a BitbucketServer. In a forked Bundler repo I enabled BitbucketServer support for XcodeServerCI. Gemfile # frozen_string_literal: true
source "https://rubygems.org"
gem 'cocoapods', '1.1.1'
gem 'fastlane', '~> 1.107.0'
# DangerCI
gem 'danger', '4.0.1', :git => 'https://github.com/krin-san/danger.git'
gem 'xcpretty', '~> 0.2.4' # Dependency for 'danger-xcode_summary'
gem 'xcpretty-json-formatter', '>= 0.1.0' # Dependency for 'danger-xcode_summary'
gem 'danger-xcode_summary' Fastfile lane :pull_request do
ENV['XCS_BOT_NAME'] = 'BuildaBot [Project/repo_name] PR #41'
ENV['XCS_ERROR_COUNT'] = '0'
ENV['XCS_TEST_FAILURE_COUNT'] = '0'
ENV['DANGER_BITBUCKETSERVER_HOST'] = 'bitbucket-server.company.com'
ENV['DANGER_BITBUCKETSERVER_USERNAME'] = 'username'
ENV['DANGER_BITBUCKETSERVER_PASSWORD'] = 'password'
sh("cd .. && bundle exec danger --verbose")
end @bfad, I tried your solution and have the same rubygems and Bundler versions you have. The direct call to danger with all env variables works ok: XCS_BOT_NAME='BuildaBot [Project/repo_name] PR #41' XCS_ERROR_COUNT='0' XCS_TEST_FAILURE_COUNT='0' DANGER_BITBUCKETSERVER_HOST='bitbucket-server.company.com' DANGER_BITBUCKETSERVER_USERNAME='username' DANGER_BITBUCKETSERVER_PASSWORD='password' bundle exec danger --verbose but when I try to call it via pull_request lane with # @return [String] The path to the local gem directory
def self.gem_path
if Gem::Specification.find_all_by_name(GEM_NAME).empty? # Empty array returned here
raise "Couldn't find gem directory for 'danger'"
end
return Gem::Specification.find_by_name(GEM_NAME).gem_dir
end Error log
|
@krin-san thank you for reporting this! unfortunately your post doesn't contain enough information for us to reproduce the problem. Please check ISSUES and open a new ticket with a list of steps that anyone can follow to recreate the problem. Thanks! |
@indirect, Ok, I'll try to make a demo repo with this issue reproducible. |
@krin-san if you do go back to this - let's open a new ticket, I think this one has done its duty, and it your symptom doesn't appear to be entirely the same. |
@chrismo, I tried to make a clean setup to reproduce the bug above but I can't. Not sure, maybe I had something locally that breaks normal behavior. |
…ecific order as mentioned in [rubygems/bundler#4602 (comment)]. This solved the problem to: [$ bundle exec bundle version —> /rubygems.rb `find_spec_for_exe': can't find gem bundler (>= 0.a) (Gem::GemNotFoundException)]
I tried a couple of the above solutions and none solved the problem. I have
Now I get a similar error message when i run
Please advise. Answer: https://github.com/guard/guard-minitest#install |
@indirect asked me to open a new ticket on this: #4381 (comment)
Bundler version: 1.12.4
Rubygems version: 2.6.4
The text was updated successfully, but these errors were encountered: