Currently, we can't implement `FromIterator`. There are a couple of
issues with this trait in the kernel, namely:
- Rust's specialization feature is unstable. This prevents us to
optimze for the special case where `I::IntoIter` equals `Vec`'s
`IntoIter` type.
- We also can't use `I::IntoIter`'s type ID either to work around this,
since `FromIterator` doesn't require this type to be `'static`.
- `FromIterator::from_iter` does return `Self` instead of
`Result<Self, AllocError>`, hence we can't properly handle allocation
failures.
- Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
additional allocation flags.
Instead, provide `IntoIter::collect`, such that we can at least convert
`IntoIter` into a `Vec` again.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index 3b79f977b65e..ad96f4c3af9e 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
fn as_raw_mut_slice(&mut self) -> *mut [T] {
ptr::slice_from_raw_parts_mut(self.ptr, self.len)
}
+
+ fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
+ let me = ManuallyDrop::new(self);
+ let ptr = me.ptr;
+ let buf = me.buf;
+ let len = me.len;
+ let cap = me.cap;
+ (ptr, buf, len, cap)
+ }
+
+ /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
+ ///
+ /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
+ /// in the kernel, namely:
+ ///
+ /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
+ /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
+ /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
+ /// doesn't require this type to be `'static`.
+ /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
+ /// we can't properly handle allocation failures.
+ /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
+ /// flags.
+ ///
+ /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
+ /// `Vec` again.
+ ///
+ /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
+ /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
+ ///
+ /// # Examples
+ ///
+ /// ```
+ /// let v = kernel::kvec![1, 2, 3]?;
+ /// let mut it = v.into_iter();
+ ///
+ /// assert_eq!(it.next(), Some(1));
+ ///
+ /// let v = it.collect(GFP_KERNEL);
+ /// assert_eq!(v, [2, 3]);
+ ///
+ /// # Ok::<(), Error>(())
+ /// ```
+ pub fn collect(self, flags: Flags) -> Vec<T, A> {
+ let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
+ let has_advanced = ptr != buf.as_ptr();
+
+ if has_advanced {
+ // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.
+ // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
+ // guaranteed to be smaller than `buf.add(cap)`.
+ unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
+ ptr = buf.as_ptr();
+ }
+
+ // This can never fail, `len` is guaranteed to be smaller than `cap`.
+ let layout = core::alloc::Layout::array::<T>(len).unwrap();
+
+ // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
+ // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
+ // it as it is.
+ ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
+ // If we fail to shrink, which likely can't even happen, continue with the existing
+ // buffer.
+ Err(_) => ptr,
+ Ok(ptr) => {
+ cap = len;
+ ptr.as_ptr().cast()
+ }
+ };
+
+ // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
+ // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
+ // to point to the start of the backing buffer. `cap` is either the original capacity or,
+ // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
+ // `into_iter` has been called on the original `Vec`.
+ unsafe { Vec::from_raw_parts(ptr, len, cap) }
+ }
}
impl<T, A> Iterator for IntoIter<T, A>
--
2.46.0On 16.08.24 02:10, Danilo Krummrich wrote:
> Currently, we can't implement `FromIterator`. There are a couple of
> issues with this trait in the kernel, namely:
>
> - Rust's specialization feature is unstable. This prevents us to
> optimze for the special case where `I::IntoIter` equals `Vec`'s
> `IntoIter` type.
> - We also can't use `I::IntoIter`'s type ID either to work around this,
> since `FromIterator` doesn't require this type to be `'static`.
> - `FromIterator::from_iter` does return `Self` instead of
> `Result<Self, AllocError>`, hence we can't properly handle allocation
> failures.
> - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
> additional allocation flags.
>
> Instead, provide `IntoIter::collect`, such that we can at least convert
> `IntoIter` into a `Vec` again.
>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
> rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 78 insertions(+)
>
> diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> index 3b79f977b65e..ad96f4c3af9e 100644
> --- a/rust/kernel/alloc/kvec.rs
> +++ b/rust/kernel/alloc/kvec.rs
> @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
> fn as_raw_mut_slice(&mut self) -> *mut [T] {
> ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> }
> +
> + fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
> + let me = ManuallyDrop::new(self);
> + let ptr = me.ptr;
> + let buf = me.buf;
> + let len = me.len;
> + let cap = me.cap;
> + (ptr, buf, len, cap)
> + }
> +
> + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> + ///
> + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> + /// in the kernel, namely:
> + ///
> + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> + /// doesn't require this type to be `'static`.
> + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> + /// we can't properly handle allocation failures.
> + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> + /// flags.
> + ///
> + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> + /// `Vec` again.
I think it's great that you include this in the code, but I don't think
that it should be visible in the documentation, can you move it under
the `Examples` section and turn it into normal comments?
> + ///
> + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> + /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> + ///
> + /// # Examples
> + ///
> + /// ```
> + /// let v = kernel::kvec![1, 2, 3]?;
> + /// let mut it = v.into_iter();
> + ///
> + /// assert_eq!(it.next(), Some(1));
> + ///
> + /// let v = it.collect(GFP_KERNEL);
> + /// assert_eq!(v, [2, 3]);
> + ///
> + /// # Ok::<(), Error>(())
> + /// ```
> + pub fn collect(self, flags: Flags) -> Vec<T, A> {
> + let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
> + let has_advanced = ptr != buf.as_ptr();
> +
> + if has_advanced {
> + // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.
This first sentence should not be part of the SAFETY comment.
> + // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
> + // guaranteed to be smaller than `buf.add(cap)`.
This doesn't justify all the requirements documented in [1].
[1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety
> + unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
> + ptr = buf.as_ptr();
> + }
> +
> + // This can never fail, `len` is guaranteed to be smaller than `cap`.
> + let layout = core::alloc::Layout::array::<T>(len).unwrap();
> +
> + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
> + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
> + // it as it is.
> + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
> + // If we fail to shrink, which likely can't even happen, continue with the existing
> + // buffer.
> + Err(_) => ptr,
> + Ok(ptr) => {
> + cap = len;
> + ptr.as_ptr().cast()
> + }
> + };
> +
> + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
> + // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
> + // to point to the start of the backing buffer. `cap` is either the original capacity or,
> + // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
> + // `into_iter` has been called on the original `Vec`.
Turn this into bullet points please.
---
Cheers,
Benno
> + unsafe { Vec::from_raw_parts(ptr, len, cap) }
> + }
> }
>
> impl<T, A> Iterator for IntoIter<T, A>
> --
> 2.46.0
>
On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > Currently, we can't implement `FromIterator`. There are a couple of
> > issues with this trait in the kernel, namely:
> >
> > - Rust's specialization feature is unstable. This prevents us to
> > optimze for the special case where `I::IntoIter` equals `Vec`'s
> > `IntoIter` type.
> > - We also can't use `I::IntoIter`'s type ID either to work around this,
> > since `FromIterator` doesn't require this type to be `'static`.
> > - `FromIterator::from_iter` does return `Self` instead of
> > `Result<Self, AllocError>`, hence we can't properly handle allocation
> > failures.
> > - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
> > additional allocation flags.
> >
> > Instead, provide `IntoIter::collect`, such that we can at least convert
> > `IntoIter` into a `Vec` again.
> >
> > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> > rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 78 insertions(+)
> >
> > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> > index 3b79f977b65e..ad96f4c3af9e 100644
> > --- a/rust/kernel/alloc/kvec.rs
> > +++ b/rust/kernel/alloc/kvec.rs
> > @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
> > fn as_raw_mut_slice(&mut self) -> *mut [T] {
> > ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> > }
> > +
> > + fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
> > + let me = ManuallyDrop::new(self);
> > + let ptr = me.ptr;
> > + let buf = me.buf;
> > + let len = me.len;
> > + let cap = me.cap;
> > + (ptr, buf, len, cap)
> > + }
> > +
> > + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> > + ///
> > + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> > + /// in the kernel, namely:
> > + ///
> > + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> > + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> > + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> > + /// doesn't require this type to be `'static`.
> > + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> > + /// we can't properly handle allocation failures.
> > + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> > + /// flags.
> > + ///
> > + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> > + /// `Vec` again.
>
> I think it's great that you include this in the code, but I don't think
> that it should be visible in the documentation,
Why not? I think this information is valuable for users of this API.
> can you move it under
> the `Examples` section and turn it into normal comments?
>
> > + ///
> > + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> > + /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> > + ///
> > + /// # Examples
> > + ///
> > + /// ```
> > + /// let v = kernel::kvec![1, 2, 3]?;
> > + /// let mut it = v.into_iter();
> > + ///
> > + /// assert_eq!(it.next(), Some(1));
> > + ///
> > + /// let v = it.collect(GFP_KERNEL);
> > + /// assert_eq!(v, [2, 3]);
> > + ///
> > + /// # Ok::<(), Error>(())
> > + /// ```
> > + pub fn collect(self, flags: Flags) -> Vec<T, A> {
> > + let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
> > + let has_advanced = ptr != buf.as_ptr();
> > +
> > + if has_advanced {
> > + // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.
>
> This first sentence should not be part of the SAFETY comment.
>
> > + // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
> > + // guaranteed to be smaller than `buf.add(cap)`.
>
> This doesn't justify all the requirements documented in [1].
>
> [1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety
>
> > + unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
> > + ptr = buf.as_ptr();
> > + }
> > +
> > + // This can never fail, `len` is guaranteed to be smaller than `cap`.
> > + let layout = core::alloc::Layout::array::<T>(len).unwrap();
> > +
> > + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
> > + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
> > + // it as it is.
> > + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
> > + // If we fail to shrink, which likely can't even happen, continue with the existing
> > + // buffer.
> > + Err(_) => ptr,
> > + Ok(ptr) => {
> > + cap = len;
> > + ptr.as_ptr().cast()
> > + }
> > + };
> > +
> > + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
> > + // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
> > + // to point to the start of the backing buffer. `cap` is either the original capacity or,
> > + // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
> > + // `into_iter` has been called on the original `Vec`.
>
> Turn this into bullet points please.
>
> ---
> Cheers,
> Benno
>
> > + unsafe { Vec::from_raw_parts(ptr, len, cap) }
> > + }
> > }
> >
> > impl<T, A> Iterator for IntoIter<T, A>
> > --
> > 2.46.0
> >
>
On 11.09.24 02:22, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
>>> + ///
>>> + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
>>> + /// in the kernel, namely:
>>> + ///
>>> + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
>>> + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
>>> + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
>>> + /// doesn't require this type to be `'static`.
>>> + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
>>> + /// we can't properly handle allocation failures.
>>> + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
>>> + /// flags.
>>> + ///
>>> + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
>>> + /// `Vec` again.
>>
>> I think it's great that you include this in the code, but I don't think
>> that it should be visible in the documentation,
>
> Why not? I think this information is valuable for users of this API.
If you want to keep it, then I don't mind, but I would still move it
underneath `Examples` and add a section header `# Implementation
Details` or similar.
---
Cheers,
Benno
>> can you move it under
>> the `Examples` section and turn it into normal comments?
>>
>>> + ///
>>> + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
>>> + /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
>>> + ///
>>> + /// # Examples
>>> + ///
>>> + /// ```
>>> + /// let v = kernel::kvec![1, 2, 3]?;
>>> + /// let mut it = v.into_iter();
>>> + ///
>>> + /// assert_eq!(it.next(), Some(1));
>>> + ///
>>> + /// let v = it.collect(GFP_KERNEL);
>>> + /// assert_eq!(v, [2, 3]);
>>> + ///
>>> + /// # Ok::<(), Error>(())
>>> + /// ```
>>> + pub fn collect(self, flags: Flags) -> Vec<T, A> {
On Wed, Sep 11, 2024 at 08:53:24AM +0000, Benno Lossin wrote:
> On 11.09.24 02:22, Danilo Krummrich wrote:
> > On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> >>> + ///
> >>> + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> >>> + /// in the kernel, namely:
> >>> + ///
> >>> + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> >>> + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> >>> + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> >>> + /// doesn't require this type to be `'static`.
> >>> + /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> >>> + /// we can't properly handle allocation failures.
> >>> + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> >>> + /// flags.
> >>> + ///
> >>> + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> >>> + /// `Vec` again.
> >>
> >> I think it's great that you include this in the code, but I don't think
> >> that it should be visible in the documentation,
> >
> > Why not? I think this information is valuable for users of this API.
>
> If you want to keep it, then I don't mind, but I would still move it
> underneath `Examples` and add a section header `# Implementation
> Details` or similar.
Sure, we can do that.
>
> ---
> Cheers,
> Benno
>
> >> can you move it under
> >> the `Examples` section and turn it into normal comments?
> >>
> >>> + ///
> >>> + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> >>> + /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> >>> + ///
> >>> + /// # Examples
> >>> + ///
> >>> + /// ```
> >>> + /// let v = kernel::kvec![1, 2, 3]?;
> >>> + /// let mut it = v.into_iter();
> >>> + ///
> >>> + /// assert_eq!(it.next(), Some(1));
> >>> + ///
> >>> + /// let v = it.collect(GFP_KERNEL);
> >>> + /// assert_eq!(v, [2, 3]);
> >>> + ///
> >>> + /// # Ok::<(), Error>(())
> >>> + /// ```
> >>> + pub fn collect(self, flags: Flags) -> Vec<T, A> {
>
© 2016 - 2026 Red Hat, Inc.