Use SipHash-1-3 instead of SipHash-2-4 for StableHasher

[thomcc]

Noticed this, and it seems easy and likely a perf win. IIUC we don't need DDOS resistance (just collision) so we ideally would have an even faster hash, but it's hard to beat this SipHash impl here, since it's been so highly tuned for the interface.

It wouldn't surprise me if there's some subtle reason changing this sucks, as it's so obvious it seems likely to have been done. Still, SipHash-1-3 seems to still have the guarantees StableHasher should need (and seemingly more), and is clearly less work. So it's worth a shot.

Not fully tested locally.

[rustbot]

r? @eholk

(rustbot has picked a reviewer for you, use r? to override)

[thomcc]

@bors try @rust-timer queue

[bors]

⌛ Trying commit eacb450ac4c9466e963eb83b0369b9f8c03ad767 with merge 4582a54eeee75d64929d904e543e349ee8262268...

[thomcc]

(Oof, kicked off perf too soon...)

[Noratrieb]

cc @michaelwoerister @cjgillot

[thomcc]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 7d93880711f3e2300d32861a4b9fabb98c37fd66 with merge aa405e0569494aa0d8259bc2457fc38b186105c7...

[bors]

☀️ Try build successful - checks-actions
Build commit: aa405e0569494aa0d8259bc2457fc38b186105c7 (aa405e0569494aa0d8259bc2457fc38b186105c7)

[rust-timer]

Finished benchmarking commit (aa405e0569494aa0d8259bc2457fc38b186105c7): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.9%	[2.9%, 2.9%]	1
Improvements ✅ (primary)	-2.4%	[-4.1%, -0.2%]	177
Improvements ✅ (secondary)	-2.5%	[-25.3%, -0.3%]	160
All ❌✅ (primary)	-2.4%	[-4.1%, -0.2%]	177

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.1%	[1.1%, 1.1%]	1
Regressions ❌ (secondary)	4.1%	[1.7%, 6.6%]	2
Improvements ✅ (primary)	-3.5%	[-3.5%, -3.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.2%	[-3.5%, 1.1%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.2%	[-4.1%, -1.2%]	85
Improvements ✅ (secondary)	-4.4%	[-25.8%, -1.6%]	43
All ❌✅ (primary)	-2.2%	[-4.1%, -1.2%]	85

[WaffleLapkin]

There is only a single perf regression (which is likely due to noise) and a lot of perf wins, so I guess
@rustbot label +perf-regression-triaged

[cjgillot]

I don't know anything about hashing functions, but I agree that we don't need DDOS resistance.
The perf results are pretty convincing.
@bors r+

[bors]

📌 Commit 7d93880711f3e2300d32861a4b9fabb98c37fd66 has been approved by cjgillot

It is now in the queue for this repository.

[thomcc]

There are also 0 regressions under the "Cycles" view and similar speedups, which is nice to see (it's noisier but more closely corresponds to what users will experience IIUC).

---

As for whether or not this causes problems for us due to being weaker, my understanding is that it should not. I'm not a cryptographer, so someone is free to correct me, but I've done enough work on software involving cryptography to be fairly confident about this much. (Still, @tarcieri might want to weigh in to make sure I'm not blundering here)

My understanding is that the the only valuable property we want out of this hash is a low collision rate when used with a fixed key. This is required to avoid soundness issues, miscompilations, ICEs, and other problems. This means that key-recovery attacks and distinguishers (some of which do apply to other SipHash round choices, but not 1-3) do not matter at all here.

According to these papers, this change allows certain attacks¹ to find a collision in the internal state in less time.

Specifically, for SipHash-1-* there are attacks which cause a collision with probability 2^-167, but for SipHash-2-* the best known attack is probability 2^-236. (Note that the internal state is 256 bits, and in SipHash-c-d the c is the number of ARX rounds used when integrating the input with the internal state, and the d is the number of rounds used to produce the final output, so for a collision in the internal state only the first number is relevant).

However, we are using a 128 bits of hash output, so the chance of a collision (due to the birthday problem) is 2^-64, which is true no matter how good of a hash function we use. This means that the change doesn't materially increase collision likelihood — it's still staggeringly unlikely before and after.

---

This does make me wonder why it wasn't done before, but who knows — that code clearly has been optimized quite a bit already, and this seems like the first thing you'd try.

Also, I suspect we could also reduce the output rounds even more, but I suspect a better hash algorithm is a more useful investigation to do, it just has to be implemented really carefully to be an improvement given the way StableHasher is used (an off-the-shelf implementation is almost certainly not going to be an improvement from something this carefully tuned for it's use case).

Footnotes

1. These attacks require actively searching for collisions, rather than just achieving them randomly, so they should just be taken as a bound on the probability. search algorithm driven by the output of the hasher as well, so they're bounds on the actual probability — real-world use should have even fewer collisions. ↩

[thomcc]

Oh it got r+ while i was writing that, lol.

[scottmcm]

I think this is the easiest "If you can justify the regressions found in this try perf run" I've ever seen 😆

[michaelwoerister]

This was discussed a few years ago (see #41215). Back then the measured difference between the 1-3 and the 2-4 was very small -- but a lot has changed since then. We just want collision probability to stay roughly the same.

[lqd]

@bors r=cjgillot

[bors]

📌 Commit 46a3d28 has been approved by cjgillot

It is now in the queue for this repository.

[bors]

⌛ Testing commit 46a3d28 with merge 2e486be...

[bors]

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing 2e486be to master...

[rust-timer]

Finished benchmarking commit (2e486be): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-4.0%, -0.2%]	184
Improvements ✅ (secondary)	-2.6%	[-32.1%, -0.2%]	153
All ❌✅ (primary)	-2.3%	[-4.0%, -0.2%]	184

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.1%	[1.1%, 1.1%]	1
Regressions ❌ (secondary)	1.5%	[1.5%, 1.5%]	1
Improvements ✅ (primary)	-1.5%	[-2.6%, -0.5%]	2
Improvements ✅ (secondary)	-2.1%	[-4.0%, -1.1%]	5
All ❌✅ (primary)	-0.7%	[-2.6%, 1.1%]	3

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.8%	[-3.4%, -0.9%]	79
Improvements ✅ (secondary)	-3.7%	[-28.9%, -1.7%]	42
All ❌✅ (primary)	-1.8%	[-3.4%, -0.9%]	79