Merge pull request #1294 from efficientcloud/bundler

---
Make fetch_specs faster

fetch_specs merges installed, cached and remote_specs together, using Index.use.
Index.use has acceptable performance for small indexes, but when adding a large Index as generated by remote_specs, it takes ages.

This change reorders fetch_specs to base its index off of the index generated by remote_specs and then replaces dupes with what cached_specs and installed_specs come up with.

Measured timing approximates, with this test program: https://gist.github.com/1067157

MRI 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]

    1-0-stable       17sec
    + this            9sec
    + this + #1288    9sec

MRI 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]

    1-0-stable       11sec
    + this            6sec
    + this + #1288    4sec

lib/bundler/index.rb

@@ -78,10 +78,13 @@ def each(&blk)
       end
     end
 
-    def use(other)
+    def use(other, override_dupes = false)
       return unless other
       other.each do |s|
-        next if search_by_spec(s).any?
+        if (dupes = search_by_spec(s)) && dupes.any?
+          next unless override_dupes
+          @specs[s.name] -= dupes
+        end
         @specs[s.name] << s
       end
       self

lib/bundler/source.rb

@@ -158,11 +158,17 @@ def normalize_uri(uri)
       end
 
       def fetch_specs
-        Index.build do |idx|
-          idx.use installed_specs
-          idx.use cached_specs if @allow_cached || @allow_remote
-          idx.use remote_specs if @allow_remote
+        # remote_specs usually generates a way larger Index than the other
+        # sources, and large_idx.use small_idx is way faster than
+        # small_idx.use large_idx.
+        if @allow_remote
+          idx = remote_specs.dup
+        else
+          idx = Index.new
         end
+        idx.use(cached_specs, :override_dupes) if @allow_cached || @allow_remote
+        idx.use(installed_specs, :override_dupes)
+        idx
       end
 
       def installed_specs