extern "C" fn on sparc64 targets does not respect repr(transparent)

[RalfJung]

I tried the following code:

#![feature(rustc_attrs)]

#[rustc_abi(debug)]
extern "C" fn test1(_x: [u8; 0]) {}

#[repr(transparent)]
struct Wrap<T>(T);

#[rustc_abi(debug)]
extern "C" fn test2(_x: Wrap<[u8; 0]>) {}

When running this with --target sparc64-unknown-linux-gnu (using Miri), the first function reports an argument pass mode of

                   mode: Indirect {
                       attrs: ArgAttributes {
                           regular: NoAlias | NoCapture | NonNull | NoUndef,
                           arg_ext: None,
                           pointee_size: Size(0 bytes),
                           pointee_align: Some(
                               Align(1 bytes),
                           ),
                       },
                       extra_attrs: None,
                       on_stack: false,
                   },

but the 2nd function reports

                   mode: Cast(
                       CastTarget {
                           prefix: [
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                           ],
                           rest: Uniform {
                               unit: Reg {
                                   kind: Integer,
                                   size: Size(8 bytes),
                               },
                               total: Size(0 bytes),
                           },
                           attrs: ArgAttributes {
                               regular: (empty),
                               arg_ext: None,
                               pointee_size: Size(0 bytes),
                               pointee_align: None,
                           },
                       },
                       false,
                   ),

I'm not an expert in interpreting PassMode, but Indirect and Cast are not compatible, are they?

Cc @bjorn3 @eddyb @psumbera

[RalfJung]

While this particular case might be fixed by having the ABI adjustment logic explicitly unwrap all repr(transparent) wrappers, that will not work in more complicated cases where it is unclear which argument is being wrapped:

struct Zst;

#[repr(transparent)]
struct Wrap(Zst, [u8; 0]);

A Wrap must be ABI-compatible with both Zst and [u8; 0] and that seems not possible?

Maybe we have to say that empty arrays do not count as 1-ZST...

[RalfJung]

Arguably this is a soundness issue, since unsafe code could rely on repr(transparent) behaving as advertised.

[RalfJung]

This affects all arrays, not just empty arrays: [u8; 8] has pass mode

                   mode: Indirect {
                       attrs: ArgAttributes {
                           regular: NoAlias | NoCapture | NonNull | NoUndef,
                           arg_ext: None,
                           pointee_size: Size(8 bytes),
                           pointee_align: Some(
                               Align(1 bytes),
                           ),
                       },
                       extra_attrs: None,
                       on_stack: false,
                   },

but Wrap<[u8; 8]> has

                   mode: Cast(
                       CastTarget {
                           prefix: [
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                           ],
                           rest: Uniform {
                               unit: Reg {
                                   kind: Integer,
                                   size: Size(8 bytes),
                               },
                               total: Size(8 bytes),
                           },
                           attrs: ArgAttributes {
                               regular: (empty),
                               arg_ext: None,
                               pointee_size: Size(0 bytes),
                               pointee_align: None,
                           },
                       },
                       false,
                   ),

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-high

[thejpster]

Does this also affect 32-bit SPARCV7 in sparc-unknown-none-elf?

[RalfJung]

No, this is specific to sparc64. (Well there might be other ABIs, I didn't check all of them. This PR should tell us once it reaches CI. But this here for sparc non-64 targets is completely harmless.)

[RalfJung]

The only way I see to fix this is to say that the actual presence of repr(transparent) affects ABI computation, and before checking the field shape a function like unfold_transparent defined in this PR needs to be applied. After all, repr(transparent) structs do not have to be compatible with their C counterparts, only repr(C) structs do.

... but that won't be enough for the ZST issue, since we still have the problem that

#[repr(transparent)]
struct Wrap((), [u8; 0]);

must be ABI-compatible with both of its fields.

[RalfJung]

MIPS64 is also affected: #115404.

And the non-zero-sized-array case also affects the sparc64-unknown-netbsd, sparc64-unknown-openbsd targets. I reached out to their maintainers via email.

[psumbera]

@RalfJung sorry for stupid question. What exactly does mean "using Miri"? I would like to reproduce it on Linux...

[bjorn3]

Running cargo miri run --target sparc64-unknown-linux-gnu after installing the miri component (rustup component add miri, needs nightly rustc)

[RalfJung]

You don't need to do this any more though, the test in tests/ui/abi/compatibility.rs now works on other targets even without Miri. You can see that some tests are disabled on sparc, e.g.

rust/tests/ui/abi/compatibility.rs

Lines 269 to 270 in cfb47ca

	#[cfg(not(any(target_arch = "sparc64")))]
	test_abi_compatible!(zst_array, Zst, [u8; 0]);

After enabling them, ./x.py test ui --test-args abi/compatibility will fail. (You can run this command on any host, no sparc CPU required.)