Tracking Issue for NEON intrinsics

[Amanieu]

Feature gate: #![feature(neon_intrinsics)]

This is a tracking issue for NEON SIMD intrinsics on ARM and AArch64. These intrinsics can be found in the std::arch::arm and std::arch::aarch64 modules.

Public API

The NEON intrinsics that are covered by this tracking issue consist of all the ARM/AArch64 intrinsics in those modules that start with the letter v. These are checked against the equivalent Clang intrinsics using automated verifiers.

Unresolved Questions

• None yet.

[SparrowLii]

Could we clarify exactly what we're going to do? For example, the feature needs to be added to the rustc?

[Amanieu]

I created an updated list of intrinsics that are missing/broken compared to Clang and the ARM intrinsic database CSV in rust-lang/stdarch#1258. Thanks to @JamieCunliffe for writing the tool to automate the testing of the intrinsics.

AArch64

This looks pretty good. We are only missing:

• ARMv8.3 complex number intrinsics.
• ARMv8.4 dot product intrinsics.
• All intrinsics which use f16 and bf16 types, due to lack of rustc support. (These are not listed in missing_aarch64.txt)
• The vqshlu* intrinsics currently cause an ICE in LLVM. This needs to be investigated and fixed.

I think we can start the process of stabilizing the AArch64 intrinsics now, conditional on fixing the LLVM ICEs. The missing intrinsics can be added later.

ARM

The situation here is a bit uglier and I don't think we can stabilize this as it is.

• Clang and GCC disagree on the type of poly64_t on A32: it's a signed integer in Clang and an unsigned integer in GCC. On AArch64 both compilers use an unsigned integer. I believe GCC has the correct behavior here, but I am not sure. cc @adamgemmell @JamieCunliffe
• The same missing intrinsics in AArch64 are also missing here (ARMv8.3 complex number, ARMv8.4 dot product, f16, bf16). This is not a blocker for stabilization.
• We are missing quite a few intrinsics ("Implemented in stdarch for A64 only, Clang supports both A32/A64"). This must be fixed prior to stabilization.
• We have some intrinsics that seem to be incorrectly implemented ("Failing tests: stdarch has incorrect results compared to Clang"). This must be fixed prior to stabilization.
• Finally we also have a different set of intrinsics which cause an ICE in LLVM. This also needs to be investigated and fixed.

[Amanieu]

As per the report above, I would like to propose the stabilization of all the currently implemented NEON intrinsics on AArch64 only.

@rfcbot fcp merge

[rfcbot]

Team member @Amanieu has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se
• @yaahc

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[JamieCunliffe]

I would say GCC is correct here, in https://github.com/ARM-software/acle/blob/main/main/acle.rst#scalar-data-types poly64_t is an unsigned integer type.

For the LLVM ICE is there a bug for this? I'll take a look into it if no one else has.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[Amanieu]

The next step is to stabilize the NEON intrinsics on ARM. Looking at the list of intrinsics which are failing the tests, I consider the following entries to be blockers:

• # Failing tests: stdarch has incorrect results compared to Clang
• # Implemented in stdarch for A64 only, Clang support both A32/A64

Once these are fixed, we should be able to move forward with stabilizing the ARM intrinsics and also resolve rust-lang/stdarch#1268.

[Shnatsel]

rust-lang/stdarch#1266 has landed and shipped in 1.59. Should this issue be closed, or is there something else left to do?

[hkratz]

rust-lang/stdarch#1266 has landed and shipped in 1.59. Should this issue be closed, or is there something else left to do?

Only aarch64/armv8 neon support was stabilized, this issue also tracks armv7 neon.

[marmeladema]

I am trying to understand what exactly has been stabilized and what target feature is needed to use the simd intrinsics on aarch64 but when I go to https://doc.rust-lang.org/core/arch/aarch64/fn.vdupq_n_u8.html it says that the intrinsic is still unstable:

🔬 This is a nightly-only experimental API. (stdsimd [#48556](https://github.com/rust-lang/rust/issues/48556))

Additionally, it says that the target feature neon is required, however I tried and I am able to use them without declaring any target feature:

bash-5.1# cat neon.rs 
use std::arch::aarch64::*;

pub fn main() {
    let a = unsafe { vdupq_n_u8(7) };
    println!("a = {:?}", a);
}

bash-5.1# rustc neon.rs 

bash-5.1# ./neon 
a = uint8x16_t(7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7)

bash-5.1# rustc --version
rustc 1.59.0 (9d1b2106e 2022-02-23)

For posterity, here is a screenshot of what you can currently see for libcore aarch64 simd intrinsics:

[CryZe]

This is a rustdoc bug. The intrinsics that are shared with the 32-bit ARM targets are basically partially stable; stable on AArch64, unstable on 32-bit. Rustdoc doesn't seem to be able to handle that. As to why you don't need to specify neon: It's an active by default target feature of AArch64. However even if it wasn't, you would still be able to unsafely call it, it would just be "UB" (I think) to call the instruction without having ensured (at least at runtime) the CPU actually supports it.

[marmeladema]

Thank you for your answer, that explains everything.

Even though, from a user perspective, I think it's very confusing. At least, it was very confusing for me.

Is there an issue opened for the rustdoc bug already? If not, I can create one.

For the neon target feature, it's a bit weird to advertise it in the doc as a regular target feature that looks like needed to be enabled. Would it be possible to add a note saying it's enabled by default for aarch64? I can provide the PR, I just don't know where to look.

[marmeladema]

Additionally, I tried this small snippet:

use std::arch::aarch64::*;

#[target_feature(enable = "neon")]
unsafe fn aarch64_intrinsic() {
    let a = unsafe { vdupq_n_u8(7) };
    println!("a = {:?}", a);
}

pub fn main() {
        unsafe { aarch64_intrinsic() };
}

Which doesn't compile because the neon target feature is considered unstable:

error[E0658]: the target feature `neon` is currently unstable
 --> neon.rs:3:18
  |
3 | #[target_feature(enable = "neon")]
  |                  ^^^^^^^^^^^^^^^
  |
  = note: see issue #44839 <https://github.com/rust-lang/rust/issues/44839> for more information

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.

Now, I understand even less what has been stabilized.

[IceTDrinker]

Hello, any news on this?

And if contributions are possible to push towards stabilization what's currently missing?

[workingjubilee]

@IceTDrinker Most NEON intrinsics were stabilized for AArch64. To resolve this for 32-bit Armv7 requires implementation, debugging their apparent lack of support, and any bugs with them mentioned in rust-lang/stdarch. In addition, some things are still not implemented, and to finish this out, we would want to implement the last barrage of intrinsics.

Basically, just covering everything mentioned in #90972 (comment)

[IceTDrinker]

@IceTDrinker Most NEON intrinsics were stabilized for AArch64. To resolve this for 32-bit Armv7 requires implementation, debugging their apparent lack of support, and any bugs with them mentioned in rust-lang/stdarch. In addition, some things are still not implemented, and to finish this out, we would want to implement the last barrage of intrinsics.

Basically, just covering everything mentioned in #90972 (comment)

Thanks, I was mainly asking for aarch64 intrisics, will take a look at the link and failing tests, but I'm afraid I don't have an arm32 machine to take a shot at fixing those (plus I'm more than a beginner contributing to a language/compiler)

[betelgeuse]

rust-lang/stdarch#1266 has landed and shipped in 1.59. Should this issue be closed, or is there something else left to do?

Only aarch64/armv8 neon support was stabilized, this issue also tracks armv7 neon.

I would add this information to the issue title.

[tarcieri]

It seems like this is the tracking issue for NEON on 32-bit ARM: #111800

[Amanieu]

Yes, this tracking issue is now complete and AArch32 NEON support is tracked in #111800.