read_dir has unexpected behavior for ""

[justinlovinger]

Path::parent returns Some("") for relative paths like foo.txt, but read_dir throws an error when given "", so code like if let Some(parent) = path.parent() { std::fs::read_dir(parent) } fails for files in the working directory.

Additionally, calling read_dir on a relative file without a ./ prefix returns file paths without ./ prefixes, but there is no way to achieve similar behavior for the working directory because read_dir("") throws an error. read_dir(".") returns file paths with ./ prefixes and is not a solution.

This was tested on Unix, and may be Unix-specific, but I lack a Windows machine to easily test against.

[ilyvion]

The behavior of Path::parent() you're describing is documented:

Returns the Path without its final component, if there is one.

This means it returns Some("") for relative paths with one component.

Returns None if the path terminates in a root or prefix, or if it's the empty string.

On (my) Windows, read_dir("") appears to read the current directory. This difference in behavior is also somewhat accounted for, under the "Platform-specific behavior" header:

This function currently corresponds to the opendir function on Unix and the FindFirstFile function on Windows.

Though if anything, I find the Windows behavior to be the one that's odd of the two. "" is not a meaningful path. I'm having a hard time imagining a scenario where having paths prefixed by ./ could be problematic when APIs like Path::file_name exist.

[ChrisDenton]

Tbh, I would agree this looks like a bug in the Windows implementation of read_dir and should be a simple fix.

[justinlovinger]

I'm having a hard time imagining a scenario where having paths prefixed by ./ could be problematic when APIs like Path::file_name exist.

Paths prefixed by ./ are a problem for a utility I am writing that does a lot of high-performance string parsing on paths. It would need extra logic to account for the . in ./ not being a directory, where every other string followed by / is a directory.

The behavior of Path::parent() you're describing is documented:

I do not consider the behavior of Path::parent() to be unexpected. It is the interaction with read_dir that I consider a problem. I expect to always be able to read the contents of the directory containing a file by calling read_dir on path.parent() for the path of that file.

Tbh, I would agree this looks like a bug in the Windows implementation of read_dir and should be a simple fix.

Do you mean a bug in the Unix implementation?

[ChrisDenton]

No I meant the Windows implementation. Empty paths are always considered invalid. The fact that it isn't in this one implementation of this one function was not intentional.

[ChrisDenton]

As this is a request to change the behaviour of read_dir to accept empty paths as meaning the current directory, I've tagged this as waiting on a libs-api decision.

[justinlovinger]

Empty paths are always considered invalid.

If empty paths are always considered invalid, why does Path::parent sometimes return an invalid path?

[ChrisDenton]

Sorry, I was imprecise. It's invalid at the point of use. OS APIs will reject this but due to a quirk in the Windows API we're using it happens to work in that case (the string we pass to the OS is never empty).

[ChrisDenton]

This was discussed in a recent libs-api meeting. There was broad agreement that the failure of if let Some(parent) = path.parent() { std::fs::read_dir(parent) } is a problem.

However, there was no consensus on the proposed solution to interpret an empty path as the current directory for read_dir. It was decided that discussion should continue here.

---

There were also a few related things discussed:

• Path::parent has some quite weird and somewhat inconsistent behaviour. Depending on the path, it can sometimes return . and other times return the empty path. Some felt it should return an enum rather than an Option but we can't change that now in any case.
• DirEntry::path creates a path by joining the file name to the path that was given to read_dir. It is therefore possible to do the same manually using different logic.
• There was discussion on if read_dir is special or if the proposed solution should be generalized to all filesystem APIs. I.e. the empty path becomes generally accepted as meaning the same as ..

[m-ou-se]

I personally think that read_dir("") should list the current directory (on all platforms). It's true that the underlying opendir("") (on Unix) gives 'not found' for "", but our read_dir functionality is more than just a wrapper around the syscall: it also Path::join()s the returned file names to obtain a full path as part of DirEntry::path(), which is why I think we need to support "" to support unprefixed paths (i.e. that won't start with ./).

I agree that if we see read_dir(x) as "give me all files in directory x", it feels wrong for "" to refer to the current directory. But I think we should instead see read_dir(x) as "give me all files with parent path x", which comes down to pretty much the same thing, except it makes the situation with "", ".", "./." etc. clear:

• read_dir("") gives "Cargo.toml", "src", etc.
• read_dir(".") gives "./Cargo.toml", "./src", etc.
• read_dir("./.") gives "././Cargo.toml", "././src", etc.

In all cases, the .parent() of the returned paths is exactly the input.

Perhaps it's just weird that Path::parent() returns Some("") for filenames without a parent, but that's not something we can really change. Assuming this will stay unchanged, I think it makes perfect sense for read_dir("") to give the files for which the parent is "": the files in the current directory.

---

In the team meeting, I was asked: "why not just use .file_name() instead of .path() if you don't want the ./ prefix?" My reasoning is that I might want to write code that works for both the current directory and a subdirectory, without adding unnecessary ./ prefixes:

for parent in ["", "src"] {
    for f in std::fs::read_dir(parent)? {
        println!("{}", f?.path().display());
    }
}

(Output: Cargo.toml, target, src, src/main.rs, etc.)

I could of course just use .file_name() and manually .join() that to the parent path, but that seems a bit odd given that we already have .path() on the DirEntry.

The follow-up question was why not just canonicalize or normalize the path before using it, to get rid of the ./ prefixes, to which I answered that I think there is value in preserving the exact parent path from the input, even if it contains redundant parts like ./ or a/../. For example, if I run a tool with --manifest-path a/../b/./Cargo.toml and it goes to list the files in the same directory as that Cargo.toml, I'd expect any messages/errors about other files to exactly start with a/../b/./, without any transformations or symlink resolutions. Similarly, the ./ should be preserved if the argument was ./Cargo.toml, and no prefix should be added the input was just Cargo.toml.

---

And last but not least, read_dir("") already works (lists the current directory) on Windows. Although that appears to have happened by accident, fixing that bug and breaking the behaviour on Windows might be more trouble than making the same work on other platforms.

That said, I'm curious to hear more about potential downsides. Will accepting "" for the current directory lead to any subtle bugs, for example?

[ChrisDenton]

Btw, @BurntSushi should probably be specifically cc'd here, both as a libs-api member and as the maintainer of the popular walkdir crate. I think whatever is decided here should also be reflected in the wider ecosystem and it'd be nice to coordinate any changes if possible.

[justinlovinger]

I would also like to note, I first became aware of this issue because read_dir("") worked in FakeFileSystem from the filesystem crate, so at least some parts of the ecosystem were already built under the assumption read_dir("") works.

[the8472]

I personally think that read_dir("") should list the current directory (on all platforms). It's true that the underlying opendir("") (on Unix) gives 'not found' for "", but our read_dir functionality is more than just a wrapper around the syscall: it also Path::join()s the returned file names to obtain a full path as part of DirEntry::path(), which is why I think we need to support "" to support unprefixed paths (i.e. that won't start with ./).

That said, I'm curious to hear more about potential downsides. Will accepting "" for the current directory lead to any subtle bugs, for example?

All other APIs that take a path do error on "" and being able to read_dir() a path but not being able to call canonicalize, symlink_metadata or similar at the same time would be confusing and require special handling anyway.
Being able to pass the same path to more than just read_dir is important for anything that has more complex preconditions than "can I open it".

But I think we should instead see read_dir(x) as "give me all files with parent path x"

I think that's more the semantics that I'd ascribe to something like walkdir, not a shallow enumeration. ReadDir does not provide all files under this prefix.

read_dir("") gives "Cargo.toml", "src", etc.

Well, except that this errors on unix. So it never worked portably, so there shouldn't be any software relying on this behavior.

In the team meeting, I was asked: "why not just use .file_name() instead of .path() if you don't want the ./ prefix?" My reasoning is that I might want to write code that works for both the current directory and a subdirectory, without adding unnecessary ./ prefixes:

for parent in ["", "src"] {
   for f in std::fs::read_dir(parent)? {
       println!("{}", f?.path().display());
   }
}

This seems like a somewhat narrow case. Anything more complex and I'd already reach for walkdir.
And it is also a merely aesthetic issue, src and ./src both are correct as far as filesystem APIs are concerned.
So if prettyfication is needed that can be solved with a method that strips leading ./ when it is ok to do so.
Ideally strip_prefix would be less strict and act more like the relative_to method that some other language libraries provide then it cold be used for this purpose, alas...

---

Overall I think read_dir behaves consistently with all other FS APIs here (on unix) and windows is the odd one out.

While path.parent() returning Some("") is kind of weird but I assume this is meant to distinguish relative paths from absolute ones. I think it would have been better to return an enum that distinguishes the reached-absolute-root vs. reached-relative-root cases. But I'll file this under the general "Path is a pile of small mistakes" category, not as a reason to adjust read_dir.

[ChrisDenton]

For whatever it's worth, I went looking at other standard libraries and tools to see what they do and they mostly seem to treat "" as an invalid path for their read_dir equivalents (e.g. bash ls "" or C++ filesystem). There are some special cases though.

Python's pathlib.Path type automatically converts "" to . but this is not specific to listing a directory. It applies to constructing the Path, which has the weird effect of os.listdir("") being an error while os.listdir(Path("")) works.

Powershell will error for ls -LiteralPath "". But ls -Path "" defaults to the current directory. However, it takes a pattern (e.g. wildcards like *) so it's not quite the same as Rust's API. I'd assume (but don't know) that the reason for this was the same as the reason Rust-on-Windows accepts the empty path (it defaults to appending * before passing to the underlying OS API).

I can also think of a few examples in other contexts where the empty path is treated as the current directory. I.e. Solaris treats an empty symlink the same as . and an empty in the unix PATH environment variable is also often taken as the current directory. Though again, these aren't quite the same thing.

I've not fully surveyed every language so it's possible I'm missing some precedent.

[BurntSushi]

I do personally find it odd to treat the empty string as equivalent to the current working directory. The empty string isn't a valid path, so it makes sense to me that it would return an error. The interaction with path.parent() seems unfortunate, but I don't think it's bad enough to warrant making read_dir("") work.

If we did end up deciding to make read_dir("") work despite me not signing off on it, I would still make the change to walkdir to ensure it's consistent. (That is, I won't hold inconsistency with walkdir as a hostage to avoid making the change in the first place.)

[Amanieu]

We discussed this in the libs-api meeting today and the team was still somewhat split on the issue. With that in mind, and taking @BurntSushi's comments into account, we decided to not add support for treating "" as the current directory.

As a consequence #116606 will make Windows match the current behavior on Unix.

@rfcbot fcp close

[rfcbot]

Team member @Amanieu has proposed to close this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to close, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

[dtolnay]

Fixed by #116606.