Use function reference to get cpu capability vector at the C-level

[torben-hansen]

Issues:

CryptoAlg-1577

Description of changes:

The goal of this PR is to close a performance gap between FIPS static and FIPS shared build. The latter is more performant in some cases.

Background

OPENSSL_ia32cap_P is a global, initialised, mutable variable. And hence, will typically go to .bss. However, the location of the .bss segment is not known before program load-time. Hence, dereferences of OPENSSL_ia32cap_P in code must be filled in by the loader. This modifies .text before it's mapped into memory, in turn, would make any hash over (part of) .text unpredictable. This is bad for FIPS, which requires a deterministic hash over part of .text to implement the integrity check.

All of this is already known though and there already exist an implement solution! We are only concerned about the static FIPS build. The shared build uses a different technique that doesn't require in-line modifications of textural assembly code, and therefore is not subject to the same performance degradation explained below.

The FIPS static build fix-up dereferences of OPENSSL_ia32cap_P in a pretty straightforward way:

• First add a "trampoline" label OPENSSL_ia32cap_addr_delta storing an offset to OPENSSL_ia32cap_P.
• Replace dereferences of OPENSSL_ia32cap_P by first loading the address of OPENSSL_ia32cap_addr_delta (which is known relative to RIP, and then add the offset stored under that label.

Note, that the latter step requires an addition. The addition might modify carry-flags. This is problematic, because the in-line modification of the assembly performed by the delocator script is not really context-aware. It just simply uses the destination register. But injecting an instruction that potentially modifies the carry-flag could break soundness/correctness because the subsequent assembly code was not written with this in mind.

Therefore, the delocator script push the cpu flags onto the stack and pops them after the injected code: https://github.com/aws/aws-lc/blob/main/util/fipstools/delocate/delocate.go#L1367-L1369. This restores the carry-flag state to what it was before the injected code ensuring soundness/correctness.

Now to the actual issue: It was found that this push/pop of carry-flags introduced a small latency, that when accumulated affects performance. For example, AES-256-XTS init and encrypt (256 bytes) can perform 4706548 invocations per second for the fips static build, but 6735574 invocations per second for the fips shared build (c6i.4xlarge); in other words, fips shared build does ~43% more invocations per second than the fips static build. Performance differences can be seen for various other algorithms e.g. P384 ops, SHA-256, AEAD-AES-128-GCM, etc.

Solution

This is the first PR out of two PRs that bridge the performance gap.

This PR use a function reference instead of directly dereferences the global variable at the C-level. Some instances of dereferencing the variable directly were added in https://github.com/aws/aws-lc/pull/330/files, removing the function reference. Others seem to just have been dereferencing the global variable always. Make this consistent and only use the functional reference at the C-level. This avoids the push/pop of the carry-flags. And performance benchmark shows that this recovers some of the performance gap.

Next PR will tackle the assembly level code, which can't directly use the functional reference: again because the delocator script is not context-aware, so adding a function call, will modify state that later assembly code has not been written to account for. For example, the assembly code might use registers that the function call by definition assumes are callee owned.

See CryptoAlg-1577 for performance gain information.

Testing:

CI passes over all relevant compilers for fips build. CryptoAlg-1577 has information on benchmarks.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and
the ISC license.

[Taffer]

Code looks good to me. 👍

[torben-hansen]

Missed some:

grep --exclude=\*.S --exclude=\*.pl -r "OPENSSL_ia32cap_P" ./crypto                                         8:35
./crypto/fipsmodule/cpucap/cpucap.c:HIDDEN uint32_t OPENSSL_ia32cap_P[4] = {0};
./crypto/fipsmodule/cpucap/internal.h:// OPENSSL_ia32cap_P contains the Intel CPUID bits when running on an x86 or
./crypto/fipsmodule/cpucap/internal.h:extern uint32_t OPENSSL_ia32cap_P[4];
./crypto/fipsmodule/cpucap/internal.h:  return OPENSSL_ia32cap_P;
./crypto/fipsmodule/cpucap/cpu_intel.c:extern uint32_t OPENSSL_ia32cap_P[4];
./crypto/fipsmodule/cpucap/cpu_intel.c:  OPENSSL_ia32cap_P[0] = edx;
./crypto/fipsmodule/cpucap/cpu_intel.c:  OPENSSL_ia32cap_P[1] = ecx;
./crypto/fipsmodule/cpucap/cpu_intel.c:  OPENSSL_ia32cap_P[2] = extended_features[0];
./crypto/fipsmodule/cpucap/cpu_intel.c:  OPENSSL_ia32cap_P[3] = extended_features[1];
./crypto/fipsmodule/cpucap/cpu_intel.c:  // The first value determines OPENSSL_ia32cap_P[0] and [1]. The second [2]
./crypto/fipsmodule/cpucap/cpu_intel.c:  handle_cpu_env(&OPENSSL_ia32cap_P[0], env1);
./crypto/fipsmodule/cpucap/cpu_intel.c:    handle_cpu_env(&OPENSSL_ia32cap_P[2], env2 + 1);

[torben-hansen]

Missed some:
grep --exclude=*.S --exclude=*.pl -r "OPENSSL_ia32cap_P" ./crypto 8:35
./crypto/fipsmodule/cpucap/cpucap.c:HIDDEN uint32_t OPENSSL_ia32cap_P[4] = {0};
./crypto/fipsmodule/cpucap/internal.h:// OPENSSL_ia32cap_P contains the Intel CPUID bits when running on an x86 or
./crypto/fipsmodule/cpucap/internal.h:extern uint32_t OPENSSL_ia32cap_P[4];
./crypto/fipsmodule/cpucap/internal.h: return OPENSSL_ia32cap_P;
./crypto/fipsmodule/cpucap/cpu_intel.c:extern uint32_t OPENSSL_ia32cap_P[4];
./crypto/fipsmodule/cpucap/cpu_intel.c: OPENSSL_ia32cap_P[0] = edx;
./crypto/fipsmodule/cpucap/cpu_intel.c: OPENSSL_ia32cap_P[1] = ecx;
./crypto/fipsmodule/cpucap/cpu_intel.c: OPENSSL_ia32cap_P[2] = extended_features[0];
./crypto/fipsmodule/cpucap/cpu_intel.c: OPENSSL_ia32cap_P[3] = extended_features[1];
./crypto/fipsmodule/cpucap/cpu_intel.c: // The first value determines OPENSSL_ia32cap_P[0] and [1]. The second [2]
./crypto/fipsmodule/cpucap/cpu_intel.c: handle_cpu_env(&OPENSSL_ia32cap_P[0], env1);
./crypto/fipsmodule/cpucap/cpu_intel.c: handle_cpu_env(&OPENSSL_ia32cap_P[2], env2 + 1);

Don't replace these:

• OPENSSL_ia32cap_get() returns a constant pointer...
• This code is only called once, so cost is suuuuper negligible.