Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metadata::created() not working on Linux with statx #59743

Closed
oxalica opened this issue Apr 5, 2019 · 0 comments · Fixed by #65094
Closed

Metadata::created() not working on Linux with statx #59743

oxalica opened this issue Apr 5, 2019 · 0 comments · Fixed by #65094
Labels
C-enhancement Category: An issue proposing an enhancement or a PR with one. O-linux Operating system: Linux T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@oxalica
Copy link
Contributor

oxalica commented Apr 5, 2019

First of all, it now returns an Err with "not available on this platform currently".
But I found that there is a system call statx available in Linux kernel 4.11, which can get the birth time.

As the doc, the function uses stat on Unix platforms. But the stat result just never contains birthtime.

Shouldn't it try statx when available?
@jonas-schievink jonas-schievink added C-enhancement Category: An issue proposing an enhancement or a PR with one. O-linux Operating system: Linux T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Apr 7, 2019
@ariasuni ariasuni mentioned this issue May 31, 2019
Closed
6 tasks
@oxalica oxalica mentioned this issue Oct 4, 2019
Merged
Centril added a commit to Centril/rust that referenced this issue Oct 9, 2019
@Centril
Rollup merge of rust-lang#65094 - oxalica:linux-statx, r=alexcrichton
6846eb7 
Prefer statx on linux if available

This PR make `metadata`-related functions try to invoke `statx` first on Linux if available,
making `std::fs::Metadata::created` work on Linux with `statx` supported.

It follows the discussion in rust-lang#61386 , and will fix rust-lang#59743

The implementation of this PR is simply converting `struct statx` into `struct stat64` with
extra fields for `btime` if `statx` succeeds, since other fields are not currently used.

---

I also did a separated benchmark for `fs::metadata`, `stat64`, `statx`, and `statx` with conversion to `stat64`.
It shows that `statx` with conversion is even more faster than pure `statx`.
I think it's due to `sizeof stat64 == 114` but `sizeof statx == 256`.

Anyway, the bare implementation of `statx` with conversion is only about 0.2% slower than the original impl (`stat64`-family).
With heap-allocation counted (~8.5% of total cost), the difference between `stat` and `statx` (with or without conversion) is just nothing.

Therefore, I think it is not urgent to use bare `struct statx` as underlying representation now.
There is no need to break `std::os::linux::fs::MetadataExt::as_raw_stat` (rust-lang#61386 (comment))

[Separated bare benchmarks](https://gist.github.com/oxalica/c4073ecb202c599fe41b7f15f86dc79c):
```
metadata_ok             time:   [529.41 ns 529.77 ns 530.19 ns]
metadata_err            time:   [538.71 ns 539.39 ns 540.35 ns]
stat64_ok               time:   [484.32 ns 484.53 ns 484.75 ns]
stat64_err              time:   [481.77 ns 482.00 ns 482.24 ns]
statx_ok                time:   [488.07 ns 488.35 ns 488.62 ns]
statx_err               time:   [487.74 ns 488.00 ns 488.27 ns]
statx_cvt_ok            time:   [485.05 ns 485.28 ns 485.53 ns]
statx_cvt_err           time:   [485.23 ns 485.45 ns 485.67 ns]
```

r? @alexcrichton
Centril added a commit to Centril/rust that referenced this issue Oct 15, 2019
@Centril
Rollup merge of rust-lang#65094 - oxalica:linux-statx, r=alexcrichton
07ccdcf 
Prefer statx on linux if available

This PR make `metadata`-related functions try to invoke `statx` first on Linux if available,
making `std::fs::Metadata::created` work on Linux with `statx` supported.

It follows the discussion in rust-lang#61386 , and will fix rust-lang#59743

The implementation of this PR is simply converting `struct statx` into `struct stat64` with
extra fields for `btime` if `statx` succeeds, since other fields are not currently used.

---

I also did a separated benchmark for `fs::metadata`, `stat64`, `statx`, and `statx` with conversion to `stat64`.
It shows that `statx` with conversion is even more faster than pure `statx`.
I think it's due to `sizeof stat64 == 114` but `sizeof statx == 256`.

Anyway, the bare implementation of `statx` with conversion is only about 0.2% slower than the original impl (`stat64`-family).
With heap-allocation counted (~8.5% of total cost), the difference between `stat` and `statx` (with or without conversion) is just nothing.

Therefore, I think it is not urgent to use bare `struct statx` as underlying representation now.
There is no need to break `std::os::linux::fs::MetadataExt::as_raw_stat` (rust-lang#61386 (comment))

[Separated bare benchmarks](https://gist.github.com/oxalica/c4073ecb202c599fe41b7f15f86dc79c):
```
metadata_ok             time:   [529.41 ns 529.77 ns 530.19 ns]
metadata_err            time:   [538.71 ns 539.39 ns 540.35 ns]
stat64_ok               time:   [484.32 ns 484.53 ns 484.75 ns]
stat64_err              time:   [481.77 ns 482.00 ns 482.24 ns]
statx_ok                time:   [488.07 ns 488.35 ns 488.62 ns]
statx_err               time:   [487.74 ns 488.00 ns 488.27 ns]
statx_cvt_ok            time:   [485.05 ns 485.28 ns 485.53 ns]
statx_cvt_err           time:   [485.23 ns 485.45 ns 485.67 ns]
```

r? @alexcrichton
Centril added a commit to Centril/rust that referenced this issue Oct 16, 2019
@Centril
Rollup merge of rust-lang#65094 - oxalica:linux-statx, r=alexcrichton
1089caf 
Prefer statx on linux if available

This PR make `metadata`-related functions try to invoke `statx` first on Linux if available,
making `std::fs::Metadata::created` work on Linux with `statx` supported.

It follows the discussion in rust-lang#61386 , and will fix rust-lang#59743

The implementation of this PR is simply converting `struct statx` into `struct stat64` with
extra fields for `btime` if `statx` succeeds, since other fields are not currently used.

---

I also did a separated benchmark for `fs::metadata`, `stat64`, `statx`, and `statx` with conversion to `stat64`.
It shows that `statx` with conversion is even more faster than pure `statx`.
I think it's due to `sizeof stat64 == 114` but `sizeof statx == 256`.

Anyway, the bare implementation of `statx` with conversion is only about 0.2% slower than the original impl (`stat64`-family).
With heap-allocation counted (~8.5% of total cost), the difference between `stat` and `statx` (with or without conversion) is just nothing.

Therefore, I think it is not urgent to use bare `struct statx` as underlying representation now.
There is no need to break `std::os::linux::fs::MetadataExt::as_raw_stat` (rust-lang#61386 (comment))

[Separated bare benchmarks](https://gist.github.com/oxalica/c4073ecb202c599fe41b7f15f86dc79c):
```
metadata_ok             time:   [529.41 ns 529.77 ns 530.19 ns]
metadata_err            time:   [538.71 ns 539.39 ns 540.35 ns]
stat64_ok               time:   [484.32 ns 484.53 ns 484.75 ns]
stat64_err              time:   [481.77 ns 482.00 ns 482.24 ns]
statx_ok                time:   [488.07 ns 488.35 ns 488.62 ns]
statx_err               time:   [487.74 ns 488.00 ns 488.27 ns]
statx_cvt_ok            time:   [485.05 ns 485.28 ns 485.53 ns]
statx_cvt_err           time:   [485.23 ns 485.45 ns 485.67 ns]
```

r? @alexcrichton
Centril added a commit to Centril/rust that referenced this issue Oct 16, 2019
@Centril
Rollup merge of rust-lang#65094 - oxalica:linux-statx, r=alexcrichton
4800c24 
Prefer statx on linux if available

This PR make `metadata`-related functions try to invoke `statx` first on Linux if available,
making `std::fs::Metadata::created` work on Linux with `statx` supported.

It follows the discussion in rust-lang#61386 , and will fix rust-lang#59743

The implementation of this PR is simply converting `struct statx` into `struct stat64` with
extra fields for `btime` if `statx` succeeds, since other fields are not currently used.

---

I also did a separated benchmark for `fs::metadata`, `stat64`, `statx`, and `statx` with conversion to `stat64`.
It shows that `statx` with conversion is even more faster than pure `statx`.
I think it's due to `sizeof stat64 == 114` but `sizeof statx == 256`.

Anyway, the bare implementation of `statx` with conversion is only about 0.2% slower than the original impl (`stat64`-family).
With heap-allocation counted (~8.5% of total cost), the difference between `stat` and `statx` (with or without conversion) is just nothing.

Therefore, I think it is not urgent to use bare `struct statx` as underlying representation now.
There is no need to break `std::os::linux::fs::MetadataExt::as_raw_stat` (rust-lang#61386 (comment))

[Separated bare benchmarks](https://gist.github.com/oxalica/c4073ecb202c599fe41b7f15f86dc79c):
```
metadata_ok             time:   [529.41 ns 529.77 ns 530.19 ns]
metadata_err            time:   [538.71 ns 539.39 ns 540.35 ns]
stat64_ok               time:   [484.32 ns 484.53 ns 484.75 ns]
stat64_err              time:   [481.77 ns 482.00 ns 482.24 ns]
statx_ok                time:   [488.07 ns 488.35 ns 488.62 ns]
statx_err               time:   [487.74 ns 488.00 ns 488.27 ns]
statx_cvt_ok            time:   [485.05 ns 485.28 ns 485.53 ns]
statx_cvt_err           time:   [485.23 ns 485.45 ns 485.67 ns]
```

r? @alexcrichton
Centril added a commit to Centril/rust that referenced this issue Oct 17, 2019
@Centril
Rollup merge of rust-lang#65094 - oxalica:linux-statx, r=alexcrichton
c61a15d 
Prefer statx on linux if available

This PR make `metadata`-related functions try to invoke `statx` first on Linux if available,
making `std::fs::Metadata::created` work on Linux with `statx` supported.

It follows the discussion in rust-lang#61386 , and will fix rust-lang#59743

The implementation of this PR is simply converting `struct statx` into `struct stat64` with
extra fields for `btime` if `statx` succeeds, since other fields are not currently used.

---

I also did a separated benchmark for `fs::metadata`, `stat64`, `statx`, and `statx` with conversion to `stat64`.
It shows that `statx` with conversion is even more faster than pure `statx`.
I think it's due to `sizeof stat64 == 114` but `sizeof statx == 256`.

Anyway, the bare implementation of `statx` with conversion is only about 0.2% slower than the original impl (`stat64`-family).
With heap-allocation counted (~8.5% of total cost), the difference between `stat` and `statx` (with or without conversion) is just nothing.

Therefore, I think it is not urgent to use bare `struct statx` as underlying representation now.
There is no need to break `std::os::linux::fs::MetadataExt::as_raw_stat` (rust-lang#61386 (comment))

[Separated bare benchmarks](https://gist.github.com/oxalica/c4073ecb202c599fe41b7f15f86dc79c):
```
metadata_ok             time:   [529.41 ns 529.77 ns 530.19 ns]
metadata_err            time:   [538.71 ns 539.39 ns 540.35 ns]
stat64_ok               time:   [484.32 ns 484.53 ns 484.75 ns]
stat64_err              time:   [481.77 ns 482.00 ns 482.24 ns]
statx_ok                time:   [488.07 ns 488.35 ns 488.62 ns]
statx_err               time:   [487.74 ns 488.00 ns 488.27 ns]
statx_cvt_ok            time:   [485.05 ns 485.28 ns 485.53 ns]
statx_cvt_err           time:   [485.23 ns 485.45 ns 485.67 ns]
```

r? @alexcrichton
@bors bors closed this as completed in 22eec92 Oct 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Category: An issue proposing an enhancement or a PR with one. O-linux Operating system: Linux T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Prefer statx on linux if available oxalica/rust
Use statx on Linux ariasuni/rust
2 participants
@jonas-schievink @oxalica