Merge #887

887: Reduce the amount of generic code for ParallelExtend r=cuviper a=cuviper

For unindexed parallel itererators, we've implemented `ParallelExtend`
for most collections using an intermediate `LinkedList<Vec<T>>` like:

```rust
    par_iter
        .into_par_iter()
        .fold(Vec::new, vec_push)
        .map(as_list)
        .reduce(LinkedList::new, list_append)
```

However, this introduces `Fold`, `Map`, and `Reduce` types that are all
dependent on the input iterator type. When it comes to very complicated
cases like nested tuple unzips, this can add up quickly. For example, in
rust-lang/rust#68926 an 8-way unzip leads to 3.7GB of LLVM IR, with
lines up to 67K characters in long generic types.

Now we add a new `ListVecConsumer` that is not generic at all itself,
and implements `Consumer<T>` etc. generic only on the item type. So each
collection now gets the same `LinkedList<Vec<T>>` as before with:

```rust
    par_iter.into_par_iter().drive_unindexed(ListVecConsumer);
```

Each implementation now also separates the code that doesn't need to be
iterator-specific to a separate function, for their `reserve` and final
`extend` from the list data.

That 8-way unzip is now _only_ 1.5GB with lines up to 17K characters.
Compile time drops from 12.8s to 7.7s debug, 32.1s to 26.9s release.


Co-authored-by: Josh Stone <[email protected]>

src/iter/collect/mod.rs

@@ -1,4 +1,4 @@
-use super::{IndexedParallelIterator, IntoParallelIterator, ParallelExtend, ParallelIterator};
+use super::{IndexedParallelIterator, ParallelIterator};
 use std::mem::MaybeUninit;
 use std::slice;
 
@@ -33,7 +33,7 @@ where
 /// *any* `ParallelIterator` here, and `CollectConsumer` has to also implement
 /// `UnindexedConsumer`.  That implementation panics `unreachable!` in case
 /// there's a bug where we actually do try to use this unindexed.
-fn special_extend<I, T>(pi: I, len: usize, v: &mut Vec<T>)
+pub(super) fn special_extend<I, T>(pi: I, len: usize, v: &mut Vec<T>)
 where
     I: ParallelIterator<Item = T>,
     T: Send,
@@ -141,33 +141,3 @@ impl<'c, T: Send + 'c> Collect<'c, T> {
         unsafe { slice::from_raw_parts_mut(tail_ptr, len) }
     }
 }
-
-/// Extends a vector with items from a parallel iterator.
-impl<T> ParallelExtend<T> for Vec<T>
-where
-    T: Send,
-{
-    fn par_extend<I>(&mut self, par_iter: I)
-    where
-        I: IntoParallelIterator<Item = T>,
-    {
-        // See the vec_collect benchmarks in rayon-demo for different strategies.
-        let par_iter = par_iter.into_par_iter();
-        match par_iter.opt_len() {
-            Some(len) => {
-                // When Rust gets specialization, we can get here for indexed iterators
-                // without relying on `opt_len`.  Until then, `special_extend()` fakes
-                // an unindexed mode on the promise that `opt_len()` is accurate.
-                special_extend(par_iter, len, self);
-            }
-            None => {
-                // This works like `extend`, but `Vec::append` is more efficient.
-                let list = super::extend::collect(par_iter);
-                self.reserve(super::extend::len(&list));
-                for mut vec in list {
-                    self.append(&mut vec);
-                }
-            }
-        }
-    }
-}