Faster, more correct bundler gem and Gemfile install #497

bf4 · 2015-01-22T20:49:36Z

About 3x faster when gems are available
Does not return false positive when other gems on system have 'bundler' in name

Updated: per discussion, to use gem install bundler --conservative as it works with rbenv and will only install the gem if not installed.
Tested on my 2015 macbook pro, Mavericks, with RVM and Ruby 2.1.5 running via bash

Regarding the gem uninstall -aIx gem_name command used below

 -a  Uninstall all matching versions
 -I  Ignore dependency requirements while uninstalling.
 -x  Uninstall applicable executables without confirmation.

Bundler not installed

gem list bundler --installed, *0.619**

gem uninstall -aIx bundler
time gem list bundler --installed; echo $?

true

real    0m0.619s
user    0m0.538s
sys     0m0.077s
0

but note it is falsely returning 'true' because I have other gems on the system with 'bundler' in the name

gem list ^bundler$ --installed, 0.604

time gem list ^bundler$ --installed; echo $?

false

real    0m0.604s
user    0m0.525s
sys     0m0.075s
1

command -v bundle, 0.000

time command -v bundle; echo $?

real    0m0.000s
user    0m0.000s
sys     0m0.000s
1

Bundler not installed, installing bundler

gem list bundler --installed || gem install bundler, *0.621**

gem uninstall -aIx bundler
time eval "gem list bundler --installed || gem install bundler"; echo $?

true

real    0m0.621s
user    0m0.539s
sys     0m0.076s
0

again, it is falsely only executing the LHS because I have other gems on the system with 'bundler' in the name

gem list ^bundler$ --installed || gem install bundler, 4.695

gem uninstall -aIx bundler
time eval "gem list ^bundler$ --installed || gem install bundler"; echo $?

false
Fetching: bundler-1.7.12.gem (100%)
Successfully installed bundler-1.7.12
Parsing documentation for bundler-1.7.12
Installing ri documentation for bundler-1.7.12
Done installing documentation for bundler after 1 seconds
1 gem installed

real    0m4.695s
user    0m2.983s
sys     0m0.348s
0

command -v bundle || gem install bundler, 4.051

gem uninstall -aIx bundler
time eval "command -v bundle || gem install bundler"; echo $?

Fetching: bundler-1.7.12.gem (100%)
Successfully installed bundler-1.7.12
Parsing documentation for bundler-1.7.12
Installing ri documentation for bundler-1.7.12
Done installing documentation for bundler after 1 seconds
1 gem installed

real    0m4.051s
user    0m2.459s
sys     0m0.269s
0

Bundler installed, Gemfile containing "gem 'dotenv-rails', '1.0.2'"

Gems Not Installed

Without Gemfile.lock

bundle, 2.706

rm Gemfile.lock; gem uninstall -aIx dotenv-rails dotenv
time bundle; echo $?

Fetching gem metadata from https://rubygems.org/....
Resolving dependencies...
Installing dotenv 1.0.2
Installing dotenv-rails 1.0.2
Using bundler 1.7.12
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

real    0m2.706s
user    0m0.739s
sys     0m0.103s
0

bundle check || bundle install, 2.423

rm Gemfile.lock; gem uninstall -aIx dotenv-rails dotenv
time eval "bundle check || bundle install"; echo $?

Resolving dependencies...
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Fetching gem metadata from https://rubygems.org/....
Resolving dependencies...
Installing dotenv 1.0.2
Installing dotenv-rails 1.0.2
Using bundler 1.7.12
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

real    0m2.423s
user    0m1.264s
sys     0m0.171s
0

With Gemfile.lock

bundle, 3.176

bundle; gem uninstall -aIx dotenv-rails dotenv
time bundle; echo $?

Fetching gem metadata from https://rubygems.org/...
Installing dotenv 1.0.2
Installing dotenv-rails 1.0.2
Using bundler 1.7.12
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

real    0m3.176s
user    0m0.755s
sys     0m0.105s
0

bundle check || bundle install, 2.880

bundle; gem uninstall -aIx dotenv-rails dotenv
time eval "bundle check || bundle install"; echo $?

The following gems are missing
 * dotenv (1.0.2)
  * dotenv-rails (1.0.2)
  Install missing gems with `bundle install`
  Fetching gem metadata from https://rubygems.org/...
  Installing dotenv 1.0.2
  Installing dotenv-rails 1.0.2
  Using bundler 1.7.12
  Your bundle is complete!
  Use `bundle show [gemname]` to see where a bundled gem is installed.

  real    0m2.880s
  user    0m1.277s
  sys     0m0.178s
  0

Gems Installed

As the time to resolve and install gems increases, bundle check || bundle's speed
advantage becomes more impressive. As a bonus, bundle check will update the Gemfile.lock
per changes in the Gemfile and all dependent gem versions are available locally.

Without Gemfile.lock

bundle, 1.519

bundle; rm Gemfile.lock
time bundle; echo $?

Fetching gem metadata from https://rubygems.org/....
Resolving dependencies...
Using dotenv 1.0.2
Using dotenv-rails 1.0.2
Using bundler 1.7.12
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

real    0m1.519s
user    0m0.651s
sys     0m0.092s
0

bundle check || bundle install, 0.612

bundle; rm Gemfile.lock
time eval "bundle check || bundle install"; echo $?

Resolving dependencies...
The Gemfile's dependencies are satisfied

real    0m0.612s
user    0m0.535s
sys     0m0.073s
0

With Gemfile.lock

bundle, 0.669

``bash
bundle
time bundle; echo $?


```plain
Using dotenv 1.0.2
Using dotenv-rails 1.0.2
Using bundler 1.7.12
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

real    0m0.669s
user    0m0.582s
sys     0m0.086s
0

bundle check || bundle install, 0.596

bundle
time eval "bundle check || bundle install"; echo $?

The Gemfile's dependencies are satisfied

real    0m0.596s
user    0m0.521s
sys     0m0.071s
0

derekprior · 2015-01-22T21:26:16Z

templates/bin_setup.erb

@@ -7,8 +7,8 @@
 set -e

 # Set up Ruby dependencies via Bundler
-gem list bundler --installed > /dev/null || gem install bundler
-bundle install
+command -v bundle > /dev/null || gem install bundler


command -v is insufficient if you are an rbenv user. It will pass if you have rbenv installed in any ruby, as the shim for bundler will be in the path.

Ah, good to know. I cargo-culted it from lower down in the script.

I can change the PR to then

add bundle check

apply the regex fix to gem list

@jferris suggested gem install bundler --conservative but I think that is unrelated to this PR, which was originally just about bundle check'ing "all the things" :)

I think it is related.

The gem list bundler line being changed here is to make sure Bundler gets installed. It looks like RubyGems has that functionality built-in (install a gem, only if it's missing), and the flag for it is --conservative.

We use gem install --conservative bundler in Paperback's bin/setup
script. Maybe @trevororeilly has thoughts on it.

On Thu, Jan 22, 2015 at 2:10 PM, Joe Ferris [email protected]
wrote:

In templates/bin_setup.erb
#497 (comment):

@@ -7,8 +7,8 @@
set -e

Set up Ruby dependencies via Bundler

-gem list bundler --installed > /dev/null || gem install bundler
-bundle install
+command -v bundle > /dev/null || gem install bundler

I think it is related.

The gem list bundler line being changed here is to make sure Bundler gets
installed. It looks like RubyGems has that functionality built-in (install
a gem, only if it's missing), and the flag for it is --conservative.

—
Reply to this email directly or view it on GitHub
https://github.com/thoughtbot/suspenders/pull/497/files#r23414815.

Ah, I accidentally the gem docs.

derekprior · 2015-01-22T21:30:39Z

bundle check seems inline with the idempotentcy changes we've made recently, but gem list is still more likely to be correct than command -v for everyone. I'm not really sweating half a second in bin/setup.

jferris · 2015-01-22T21:59:39Z

My local numbers are much faster for each case, and I agree with @derekprior that it's more important to be correct than half a second faster in this case.

One other option we haven't explored is the --conservative option:

gem install bundler --conservative

- About 3x faster when gems are available - Does not return false positive when other gems on system have 'bundler' in name Tested on my 2015 macbook pro, Mavericks, with RVM and Ruby 2.1.5 running via bash See details in thoughtbot#497

bf4 · 2015-01-22T23:18:00Z

Updated per discussion. Also removed the now incorrect and wordy comments from the commit message.

trevororeilly · 2015-01-23T17:03:18Z

I first noticed the --conservative flag in the default Rails bin/setup script:

https://github.com/rails/rails/blob/v4.2.0/railties/lib/rails/generators/rails/app/templates/bin/setup#L11

Looks like the same approach as what's proposed here.

jferris · 2015-01-26T16:21:41Z

The current version looks pretty good to me. @croaky any thoughts?

- About 3x faster when gems are available - Does not return false positive when other gems on system have 'bundler' in name Tested on my 2015 Macbook Pro, Mavericks, with RVM and Ruby 2.1.5 running via bash See details in #497

croaky · 2015-01-26T23:58:58Z

Thanks, @bf4. Merged as e5d5e43.

bf4 · 2015-05-29T14:19:23Z

Turn out, bundle install --local might be a better option ref: https://blog.engineyard.com/2011/bundler-pro-tip

tute · 2015-05-29T17:26:04Z

I'm surprised --local is not default behavior.

jferris · 2015-05-29T19:49:44Z

I think that, when generating a brand new Rails project, we probably want to fetch the gems index so that we install with the latest gems.

tute · 2015-05-29T20:31:24Z

I think that, when generating a brand new Rails project, we probably want to fetch the gems index so that we install with the latest gems.

Indeed, I thought we were talking about suspenders' CI.

See thoughtbot/suspenders#497

jferris · 2015-06-01T13:50:22Z

Hmm, yeah, it looks like generating a Suspenders app does its own bundle install.

Using --local will fail if the gems aren't available, though, so unless you're vendoring and committing gems, that flag will cause the bundle install to fail for other developers.

- About 3x faster when gems are available - Does not return false positive when other gems on system have 'bundler' in name Tested on my 2015 Macbook Pro, Mavericks, with RVM and Ruby 2.1.5 running via bash See details in thoughtbot/suspenders#497

derekprior reviewed Jan 22, 2015
View reviewed changes

bf4 force-pushed the patch-1 branch from 0c7340d to 6eef942 Compare January 22, 2015 23:17

croaky closed this Jan 26, 2015

bf4 deleted the patch-1 branch May 7, 2015 21:13

bf4 mentioned this pull request May 28, 2015

Big gems from git hang during every install doing... something rubygems/bundler#2846

Closed

c-lliope added a commit to c4lliope/dotfiles that referenced this pull request May 31, 2015

Install bundler after installing a new Ruby

828b4a9

See thoughtbot/suspenders#497

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Faster, more correct bundler gem and Gemfile install #497

Faster, more correct bundler gem and Gemfile install #497

bf4 commented Jan 22, 2015

derekprior Jan 22, 2015

bf4 Jan 22, 2015

jferris Jan 22, 2015

croaky Jan 22, 2015

bf4 Jan 22, 2015

derekprior commented Jan 22, 2015

jferris commented Jan 22, 2015

bf4 commented Jan 22, 2015

trevororeilly commented Jan 23, 2015

jferris commented Jan 26, 2015

croaky commented Jan 26, 2015

bf4 commented May 29, 2015

tute commented May 29, 2015

jferris commented May 29, 2015

tute commented May 29, 2015

jferris commented Jun 1, 2015

Faster, more correct bundler gem and Gemfile install #497

Faster, more correct bundler gem and Gemfile install #497

Conversation

bf4 commented Jan 22, 2015

Bundler not installed

gem list bundler --installed, 0.619*

gem list ^bundler$ --installed, 0.604

command -v bundle, 0.000

Bundler not installed, installing bundler

gem list bundler --installed || gem install bundler, 0.621*

gem list ^bundler$ --installed || gem install bundler, 4.695

command -v bundle || gem install bundler, 4.051

Bundler installed, Gemfile containing "gem 'dotenv-rails', '1.0.2'"

Gems Not Installed

Without Gemfile.lock

bundle, 2.706

bundle check || bundle install, 2.423

With Gemfile.lock

bundle, 3.176

bundle check || bundle install, 2.880

Gems Installed

Without Gemfile.lock

bundle, 1.519

bundle check || bundle install, 0.612

With Gemfile.lock

bundle, 0.669

bundle check || bundle install, 0.596

derekprior Jan 22, 2015

Choose a reason for hiding this comment

bf4 Jan 22, 2015

Choose a reason for hiding this comment

jferris Jan 22, 2015

Choose a reason for hiding this comment

croaky Jan 22, 2015

Choose a reason for hiding this comment

Set up Ruby dependencies via Bundler

bf4 Jan 22, 2015

Choose a reason for hiding this comment

derekprior commented Jan 22, 2015

jferris commented Jan 22, 2015

bf4 commented Jan 22, 2015

trevororeilly commented Jan 23, 2015

jferris commented Jan 26, 2015

croaky commented Jan 26, 2015

bf4 commented May 29, 2015

tute commented May 29, 2015

jferris commented May 29, 2015

tute commented May 29, 2015

jferris commented Jun 1, 2015

gem list bundler --installed, *0.619**

gem list bundler --installed || gem install bundler, *0.621**