You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The address of the first element of an ndarray::Array is not necessarily the same as the start of the array's allocation. The IntoPyArray implementation for ArrayBase<OwnedRepr<A>, D> does not account for this, so the resulting NumPy array can contain the wrong portion of the data or even memory outside of the original allocation.
Here's an example with a negative stride which results in the NumPy array containing data out-of-bounds of the allocation:
(I ran these examples with the latest version of the main branch.)
Currently, the best way to get a boxed slice of an array's data and a pointer to the first element is like this:
let orig_ptr = arr.as_ptr();let offset_is_zero = arr.is_empty() || std::mem::size_of::<A>() == 0;let v = arr.into_raw_vec();let offset = if offset_is_zero {0}else{unsafe{ orig_ptr.offset_from(v.as_ptr())asusize}};letmut boxed = v.into_boxed_slice();let ptr = unsafe{ boxed.as_mut_ptr().add(offset)};// From here, `boxed` is the allocation, and `ptr` is a pointer to the first element of the n-D array.
Note that:
We need to check for empty arrays and zero-sized elements because the array's pointer is somewhat arbitrary in those cases.
The offset to the first element is computed before converting into a boxed slice, because .into_boxed_slice() may reallocate.
It seems like we could do better than just providing a tricky way to do this.
What's the interface that PyO3 requires here? What would it do if the array was non-contiguous? I can see that it's nice if reallocation and copying could be avoided, but perhaps it would be useful to have an ndarray method that could convert the array into a contiguous one with non-negative strides (and otherwise unchanged, if possible)?
The address of the first element of an
ndarray::Array
is not necessarily the same as the start of the array's allocation. TheIntoPyArray
implementation forArrayBase<OwnedRepr<A>, D>
does not account for this, so the resulting NumPy array can contain the wrong portion of the data or even memory outside of the original allocation.Here's an example with a negative stride which results in the NumPy array containing data out-of-bounds of the allocation:
The assertion fails with the following message:
Here's an example with all positive strides where the NumPy array contains an incorrect portion of the original data:
The assertion fails with the following message:
(I ran these examples with the latest version of the
main
branch.)Currently, the best way to get a boxed slice of an array's data and a pointer to the first element is like this:
Note that:
.into_boxed_slice()
may reallocate.I've created rust-ndarray/ndarray#994 to provide a better way to get the correct offset.
The text was updated successfully, but these errors were encountered: