Tracking Issue for RFC 2948: Portable SIMD

[calebzulawski]

Feature gate: #![feature(portable_simd)]

This is a tracking issue for the future feature chartered in RFC 2977, with the intent of creating something akin to the design in RFC 2948 (rust-lang/rfcs#2948): a portable SIMD library (std::simd).

Portable SIMD project group: https://github.com/rust-lang/project-portable-simd
Implementation: https://github.com/rust-lang/portable-simd

More discussion can be found in the #project-portable-simd zulip stream.

Steps

• Implement the experimental feature behind a feature gate:
  • pub use core::simd; #89167
• Define the semantics explicitly.
• Write up an RFC detailing the design and defined semantics.
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

• What will the overall design be?
• What are the ideal semantics for Masks?
• Are there any limits or vector sizes we should not support?
• How should these types interop with types like Saturating, NonZero, etc.?

Implementation History

• pub use core::simd; #89167
• Add [T]::as_simd(_mut) #91479

[HannesGitH]

Feature gate: #![feature(portable_simd)]

i'm sorry if this is the wrong place to ask but im rather new to rust and stumbled upon this issues as my compiler told me to

if i want to use this feature as soon as my compiler supports it can i gate it like:

#[cfg(feature = "portable_simd")]
use std::simd::Simd;

or is that only for feautures regarding my package (set in toml or passed to cargo?) if so what would be the appropriate way to use simd as soon as this issue is resolved?

[Lokathor]

The #![feature(portable_simd)] part goes at the top of a binary or library.

It's a language feature not a cargo feature so it works a little differently.

It's unfortunate that they're both just "feature". Rust is often too terse when it counts.

[HannesGitH]

ok thanks a lot!

just to make sure this means there is no (easy*) way to use this language feature if my compiler supports it and fall back to a custom implementation otherwise?

_{*easy as in compile time guards / attribute-like macros or creating a custom wrapper module that either provides rusts simd or my own fallback or something else in that level of skill}

---

for anyone else stumbling upon this:

language features are (unstable) features you can opt-in when using nightly rust (by putting the specified flag in your library root, the whole project will then be compiled with a compiler that uses this feature)

[CarlKCarlK]

@agausmann

• To enable the experimental feature flag on nightly,

#![rustversion::attr(nightly, feature(portable_simd))]

@safinaskar

Unfortunately, this particular code doesn't work

This worked for me:

#![cfg_attr(feature = "from_slice", feature(portable_simd))]

where "from_slice" is the name of my the-other-kind-of-feature, defined in Cargo.toml, that uses portable_simd.

[features]
from_slice = []

So, I run tests, for example, via cargo test --features=from_slice.

[GlenDC]

Is this on the 2024 edition roadmap, or will it be only for after that? I know it’s not related, but gives me a timeline range.

[calebzulawski]

I don't think anyone has a specific timeline, but we still need to draft a new RFC and go through the approval process, which can take some time.

[jhpratt]

Is there a particular reason that Simd does not implement Deref and DerefMut? I don't see any reason the impls would restrict the ability to do anything.

[Lokathor]

Like deref into a slice? Usually that's not done because it's a huge performance footgun.

[Firstyear]

It may be good to document what that footgun is and why the choice was made because people will ask this again in future.

[Lokathor]

So, to add more detail: the problem is that (depending on SIMD used) you can't in general index to a particular lane of a SIMD register. So if you view the SIMD data as a slice and operate on an element of the slice, what the hardware must do is have the CPU stop the current SIMD processing, write the register to the stack, work on the stack value (however the slice is adjusted), and then load that back into a SIMD register. This is, in general, a performance disaster. As usual, the optimizer might be able to cut out this stall in the pipeline, in some cases, depending on circumstances, etc etc. But you should expect that the SIMD handling is totally stalled when trying to treat the data as a slice.

[jhpratt]

I figured there was a reason, but I'm not familiar with how SIMD works under the hood. Given that indexing is the problem, why implement Index and IndexMut then?

[Lokathor]

Oh, uh, well I haven't looked in a while! I guess I'm out of the loop on the current API details.

I'm surprised that Index is in if Deref is out. Either both should be in or both should be out, would be my expectation.

[calebzulawski]

The basic idea is that we want a clear marker of the boundary between SIMD and non-SIMD operations. When using Index (vector[i]) there is an obvious sign that you are no longer using SIMD operations. Likewise with arrays and slices, we implement AsRef and the to_array function because these are explicit. The concern with Deref is that the automatic inclusion of all slice functions makes it harder to tell which operations are SIMD. For example, you may expect is_ascii to be vectorized, but instead it is simply a scalar implementation inherited from slices.

[Lokathor]

vector[i] isn't particularly more obvious, I would say.

Maybe we should just always make people convert to an array to index elements?

[calebzulawski]

A while ago we didn't implement Index and we got requests for it, but this is the first time Deref has come up, so I think it's a good compromise. Maybe it's not particularly obvious that Index is the boundary, but Deref is completely invisible without consulting the docs.

[ZagButNoZig]

There are certain types of instructions where the output data type is different from the input data type like: _mm256_maddubs_epi16. I don't think there is a way to do that in portable simd without casting first which is slower? Are there any plans to support these instructions. Similar instructions also exist on arch: vdotq_s32

[abysssol]

Hi, I was wondering if there had been any discussion or consideration of making a dynamically sized api for vector operations. The current api seems to be analogous to arrays, but perhaps a more elegant and convenient solution would be analogous to slices.

I learned about this idea when researching risc-v's vector extension. Both this article and this one (fully rendered here) are good references on the motivation, from the perspective of an ISA.

While the current api is already much better than traditional simd instructions, it seems to me that the logical conclusion is a runtime sized type; maybe a wrapper around &mut [T], or a type like Vec<T>, or perhaps a modification to Vec<T> that guarantees simd optimization if T is a numeric primitive.

Hopefully this can spark a useful discussion on the best design of simd/vector types and operations. Thank you for your consideration.

[Lokathor]

That could be some additional API that lives along aside the fixed sized SIMD types, but for the main CPU arches a fixed sized simd type is what generally works best with optimizations.

[colejohnson66]

Curious how ARM SVE and RISC-V V are meant to be used in Rust. The fixed-length abstraction is a nice one, and it's what .NET is going with in .NET 9 (Vector<T> for SVE is 128-bit, at least for now), but variable-length vectors are here to stay.

[dead-claudia]

Curious how ARM SVE and RISC-V V are meant to be used in Rust. The fixed-length abstraction is a nice one, and it's what .NET is going with in .NET 9 (Vector<T> for SVE is 128-bit, at least for now), but variable-length vectors are here to stay.

RISC-V offers extensions like Zvl128b that provide hard guarantees on minimum vector size. It should be possible to leverage this in the interim while RISC-V figures out their P extension (which isn't very far along).

Edit: fix extension name

[Salabar]

Would it it make sense to add a family of functions like "load_base_*" that take a slice and an isize index? It would account for buffer underflow as well as overflow. With this you can write things such as convolution with nice clean loops that don't have account for edge cases.

for i in 0..image.len(){
   let mut result = 0.;

  for j in 1..kernel.radius() / N {
    let left = Simd::<N>::load_base_or(image, i - j * N, splat(image[0]);
   //... 
 }
  for j in 0..kernel.radius() / N {
    let right= Simd::<N>::load_base_or(image, i + j * N, splat(image.last());
   //... 
}
  image[i] = result;
}

[DXist]

Is it possible to move Mask inherent methods into a trait like SimdMask and add this trait as a bound to associated type Mask of other traits, e.g. SimdPartialEq?

This will help to write generic code that works for different primitive types.

Got this idea while writing a fixed index map data structure that is expected to work with unsigned integer keys regardless of the width.

Without the trait bound for Mask associated type I have to wrap my implementation into macros and explicitly apply it to u8, u16, u32, u64 and usize.

[calebzulawski]

I think you should probably be able to do what you want:

fn generic<T>(v: Simd<T, 4>, m: Mask<T::Mask, 4>) -> bool
where
    T: SimdElement + Default,
    Simd<T, 4>: SimdPartialEq<Mask = Mask<T::Mask, 4>>,
{
    (v.simd_eq(Simd::splat(Default::default())) ^ m).all()
}

However, it would be nice if there were an easier way to do this without requiring that extra bound.

[DXist]

@calebzulawski , thank you!

It worked along with a couple of bounds from num-traits crate.

Maybe an example with generic code will be a useful demo of bounds usage.