Tracking issue for Vec::extract_if and LinkedList::extract_if

[Gankra]

Feature gate: #![feature(extract_if)] (previously drain_filter)

This is a tracking issue for Vec::extract_if and LinkedList::extract_if, which can be used for random deletes using iterators.

Public API

pub mod alloc {
    pub mod vec {
        impl<T, A: Allocator> Vec<T, A> {
            pub fn extract_if<F>(&mut self, filter: F) -> ExtractIf<'_, T, F, A>
            where
                F: FnMut(&mut T) -> bool,
            {
            }
        }

        #[derive(Debug)]
        pub struct ExtractIf<'a, T, F, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global>
        where
            F: FnMut(&mut T) -> bool, {}

        impl<T, F, A: Allocator> Iterator for ExtractIf<'_, T, F, A>
        where
            F: FnMut(&mut T) -> bool,
        {
            type Item = T;
            fn next(&mut self) -> Option<T> {}
            fn size_hint(&self) -> (usize, Option<usize>) {}
        }

        impl<T, F, A: Allocator> Drop for ExtractIf<'_, T, F, A>
        where
            F: FnMut(&mut T) -> bool,
        {
            fn drop(&mut self) {}
        }
    }

    pub mod collections {
        pub mod linked_list {
            impl<T> LinkedList<T> {
                pub fn extract_if<F>(&mut self, filter: F) -> ExtractIf<'_, T, F>
                where
                    F: FnMut(&mut T) -> bool,
                {
                }
            }

            pub struct ExtractIf<'a, T: 'a, F: 'a>
            where
                F: FnMut(&mut T) -> bool, {}

            impl<T, F> Iterator for ExtractIf<'_, T, F>
            where
                F: FnMut(&mut T) -> bool,
            {
                type Item = T;
                fn next(&mut self) -> Option<T> {}
                fn size_hint(&self) -> (usize, Option<usize>) {}
            }

            impl<T, F> Drop for ExtractIf<'_, T, F>
            where
                F: FnMut(&mut T) -> bool,
            {
                fn drop(&mut self) {}
            }

            impl<T: fmt::Debug, F> fmt::Debug for ExtractIf<'_, T, F>
            where
                F: FnMut(&mut T) -> bool,
            {
                fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {}
            }
        }
    }
}

Steps / History

• Implementation: Add Vec::drain_filter #43245
• removed drain-on-drop behavior, renamed to extract_if
  • Don't drain-on-drop in DrainFilter impls of various collections. #104455
  • Remove drain-on-drop behavior from various DrainFilter iterators libs-team#136
• Stabilization PR

Unresolved Questions

• What should the method be named?
• Should extract_if accept a Range argument?
• Missing Send+Sync impls on linked list's ExtractIf, see comment

See #43244 (comment) for a more detailed summary of open issues.

[RalfJung]

Related issues:
#25477
#34265

[bluss]

Maybe this doesn't need to include the kitchen sink, but it could have a range parameter, so that it's like a superset of drain. Any drawbacks to that? I guess adding bounds checking for the range is a drawback, it's another thing that can panic. But drain_filter(.., f) can not.

[rustonaut]

Is there any chance this will stabilize in some form in the not to far future?

[rustonaut]

If the compiler is clever enough to eliminate the bounds checks
in the drain_filter(.., f) case I would opt for doing this.

( And I'm pretty sure you can implement it in a way
which makes the compiler clever eneugh, in the worst
case you could have a "in function specialization",
basically something like if Type::of::<R>() == Type::of::<RangeFull>() { dont;do;type;checks; return } )

[jonhoo]

I know this is bikeshedding to some extent, but what was the reasoning behind naming this drain_filter rather than drain_where? To me, the former implies that the whole Vec will be drained, but that we also run a filter over the results (when I first saw it, I thought: "how is this not just .drain(..).filter()?"). The former on the other hand indicates that we only drain elements where some condition holds.

[rustonaut]

No idea, but drain_where sounds much better and is much more intuitive.
Is there still a chance to change it?

[bluss]

.remove_if has been a prior suggestion too

[rustonaut]

I think drain_where does explains it the best. Like drain it returns values, but it does not drain/remove all values but just such where a given condition is true.

remove_if sounds a lot like a conditional version of remove which just removes a single item by index if a condition is true e.g. letters.remove_if(3, |n| n < 10); removes the letter at index 3 if it's < 10.

drain_filter on the other hand is slightly ambiguous, does it drain then filter in a more optimized way (like filter_map) or does if drain so that a iterator is returned comparble to the iterator filter would return,
and if so shouldn't it be called filtered_drain as the filter get logically used before...

[Gankra]

There is no precedent for using _where or _if anywhere in the standard library.

[jonhoo]

@gankro is there a precedent for using _filter anywhere? I also don't know that that is that a reason for not using the less ambiguous terminology? Other places in the standard library already use a variety of suffixes such as _until and _while.

[crlf0710]

The "said equivalent" code in the comment is not correct... you have to minus one from i at the "your code here" site, or bad things happens.

[thegranddesign]

IMO it's not filter that's the issue. Having just searched for this (and being a newbie), drain seems to be fairly non-standard compared to other languages.

Again, just from a newbie perspective, the things I would search for if trying to find something to do what this issue proposes would be delete (as in delete_if), remove, filter or reject.

I actually searched for filter, saw drain_filter and kept searching without reading because drain didn't seem to represent the simple thing that I wanted to do.

It seems like a simple function named filter or reject would be much more intuitive.

[thegranddesign]

On a separate note, I don't feel as though this should mutate the vector it's called on. It prevents chaining. In an ideal scenario one would want to be able to do something like:

        vec![
            "",
            "something",
            a_variable,
            function_call(),
            "etc",
        ]
            .reject(|i| { i.is_empty() })
            .join("/")

With the current implementation, what it would be joining on would be the rejected values.

I'd like to see both an accept and a reject. Neither of which mutate the original value.

[rpjohnst]

You can already do the chaining thing with filter alone. The entire point of drain_filter is to mutate the vector.

[thegranddesign]

@rpjohnst so I searched here, am I missing filter somewhere?

[rpjohnst]

Yes, it's a member of Iterator, not Vec.

[Gankra]

Drain is novel terminology because it represented a fourth kind of ownership in Rust that only applies to containers, while also generally being a meaningless distinction in almost any other language (in the absence of move semantics, there is no need to combine iteration and removal into a single ""atomic"" operation).

Although drain_filter moves the drain terminology into a space that other languages would care about (since avoiding backshifts is relevant in all languages).

[polarathene]

I came across drain_filter in docs as a google result for rust consume vec. I know that due to immutability by default in rust, filter doesn't consume the data, just couldn't recall how to approach it so I could manage memory better.

drain_where is nice, but as long as the user is aware of what drain and filter do, I think it's clear that the method drains the data based on a predicate filter.

[jonhoo]

I still feel as though drain_filter implies that it drains (i.e., empties) and then filters. drain_where on the other hand sounds like it drains the elements where the given condition holds (which is what the proposed function does).

[tmccombs]

Shouldn't linked_list::DrainFilter implement Drop as well, to remove any remaining elements that match the predicate?

[Gankra]

Yes

[NexusXe]

Is it possible to point to this feature when a user is trying to use the drain_filter method? This seems like it could cause some confusion.

[rsalmei]

I'm eagerly anticipating the addition of this feature to the standard library. Could it be stabilized this year?

[ebkalderon]

Quick question for anyone familiar with the internals of the unstable extract_if() methods: I noticed that all existing ExtractIf<'a, ...> iterators don't implement the FusedIterator trait. Since these iterators supposedly visit each element exactly once (unless the iterator is dropped partway through iteration, of course), it would seem that they are suitable candidates for implementing FusedIterator. Is this understanding correct? Or is there a reason these iterators aren't fused? Thanks!

[the8472]

They shouldn't suddenly restart, no. So it most likely is an oversight.

[joshtriplett]

@rfcbot merge

@rfcbot concern vec-needs-a-range-argument

We discussed this in today's @rust-lang/libs-api meeting. We were in favor of stabilizing these two, and we have consensus on the current names. However, we would like to have a range parameter to Vec::extract_if.

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

Concerns:

• vec-needs-a-range-argument resolved by Tracking issue for Vec::extract_if and LinkedList::extract_if #43244 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[cuviper]

Not necessarily a blocking concern, but can we remove the where F: FnMut constraint on struct ExtractIf and its Drop and Debug? The drain-on-drop behavior was removed a while ago, so now we only need to be callable for Iterator.

[the8472]

Sure I'll add that to my PR.

[Amanieu]

Also another note from the libs-api meeting is that we are still interested in solving more exotic use cases such as ExtractMapped proposed in #43244 (comment). However this may be better done via some form of cursor API rather than an iterator: you would need to manually increment the cursor, but it would give you precise control at each point whether to mutate or remove the value currently being pointed to by the cursor.

Such an API should be proposed as a separate ACP, with inspiration from linked list cursors (#58533) and B-Tree cursors (#107540).

[joshtriplett]

Resolving the concern since the corresponding work is in progress.

@rfcbot resolved vec-needs-a-range-argument

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.