Add support for OS locking #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Sep 29, 2023
Merged
Jake-Shadle
added a commit
that referenced
this pull request
Sep 29, 2023
#33 introduced a compile error when targeting (at least) `-musl` but possibly other unixes.
Jake-Shadle
added a commit
to EmbarkStudios/cargo-deny
that referenced
this pull request
Sep 29, 2023
This updates tame-index, which includes EmbarkStudios/tame-index#33, which allows cargo-deny to use OS file locking for safer interactions with cargo, as well as improved safety and robustness for read/fetching advisory databases. This doesn't add new tests as I'm _pretty_ confident the OS file locking code works correctly, which means #537 is basically covered since it's more robust now. Resolves: #537
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As pointed out in #30, gix's locking mechanism is insufficient in the face of SIGKILL or other process interruption that can't be caught by gix's signal handler, in addition to the giant footgun that tame-index doesn't actually hook that signal handler for you, meaning applications using it that forget to hook it also run into issues.
In addition, I raised #17 while doing crate-ci/cargo-release#693 (https://github.com/crate-ci/cargo-release/pull/693/files/012d3e9a7be23db14096e6a0d41cea7528f9348c#r1301688472) as right now tame-index can perform mutation concurrently with cargo itself, or potentially read partial data while it is being written.
This PR resolves both of these issues by forcing all R/W operations on indexes to take a
&FileLockargument, as well as providing aLockOptions"builder" to create file locks. These locks are created using flock on unix and LockFileEx on windows, meaning they can properly be cleaned up by the OS in all situations, including SIGKILL and power loss etc, unlike gix's locks, and is the same mechanism that cargo uses for its global package lock, meaning downstream users can ensure they play nicely with cargo.The lock facilities are part of the public API of tame-index as I opted to roll my own implementation instead of using fslock, as it is very outdated, and doesn't support timeouts. This does mean a lot of unsafe has been added, but it is tested and not too bad. This can potentially be moved out to a separate crate in the future, but is fine for now. This means it could be used to resolve rustsec/rustsec#1011, and is something I will use in cargo-deny for the same thing, protecting access to the advisory database during mutation.
It should also be noted that one can also just construct a
FileLock::unlocked()to satisfy the API, without actually performing any locking, for cases where it's not needed/testing/etc.Resolves: #17
Resolves: #30