-
Notifications
You must be signed in to change notification settings - Fork 542
Adds as_slice
and friends to basic slice and vec iterators
#1255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This commit adds the methods `as_slice`, `as_mut_slice`, `into_slice` and `AsRef<[T]>` on the appropriate iterators for slice iterators and vec iterators. This allows custom producers to use these iterators to decide on custom split points based on the remaining data in the slice.
Do you have an example of code where you would like to use this? Unlike an
That's good, we like to match precedent from the standard library -- although |
I only just noticed this, so this doesn't actually help me accomplish what I'm trying to do. I'm trying to create a parallel iterator of a join like iterator that works like this (assumes both iterators are sorted): impl<I, J> Iterator for Pairs<I, J>
where
I: Iterator<Item = u64>,
J: Iterator<Item = u64>,
{
type Item = (Option<u64>, Option<u64>);
fn next(&mut self) -> Option<Self::Item> {
match (self.left.peek(), self.right.peek()) {
(None, None) => None,
(Some(_), None) => Some((self.left.next(), None)),
(None, Some(_)) => Some((None, self.right.next())),
(Some(left), Some(right)) => match left.cmp(&right) {
Ordering::Equal => Some((self.left.next(), self.right.next())),
Ordering::Less => Some((self.left.next(), None)),
Ordering::Greater => Some((None, self.right.next())),
},
}
}
} After thinking about it for a little while, I noticed that if you split one of the two iterators you must split the other iterator at a specific point, otherwise you produce duplicates. But actually doing all of this with rayon seems impossible currently. Since the producer types of slices and vecs are all private so even if they have an Any help is greatly appreciated. |
I can't think of a way to do that with generic impl ParallelIterator for ParallelPairs {
type Item = (Option<u64>, Option<u64>);
fn drive_unindexed<C>(self, consumer: C) -> C::Result
where
C: UnindexedConsumer<Self::Item>
{
rayon::iter::split(self, |pairs| todo!("find your split points and do it"))
.flat_map_iter(|pairs| todo!("convert into the serial Pairs"))
.drive_unindexed(consumer)
}
} |
Thanks for the suggestion, I still wanted it to be generic over anything that contains a slice, such as Vec. Regarding this PR, as you said this doesn't really help my use case at all, regardless if anyone thinks this would be beneficial I'll gladly continue working on this PR until it is ready to be merged. |
This commit adds the methods
as_slice
,as_mut_slice
,into_slice
andAsRef<[T]>
on the appropriate iterators for slice iterators and vec iterators.This allows custom producers to use these iterators to decide on custom split points based on the remaining data in the slice.
These impls are based on the std impls on std::slice::Iter, std::slice::IterMut, and std::vec::IntoIter.