Skip to content

Commit

Permalink
pythonGH-101357: Suppress OSError from pathlib.Path.exists() and …
Browse files Browse the repository at this point in the history
…`is_*()`

Suppress all `OSError` exceptions from `pathlib.Path.exists()` and `is_*()`
rather than a selection of more common errors as we do presently. Also
adjust the implementations to call `os.path.exists()` etc, which are much
faster on Windows thanks to pythonGH-101196.
  • Loading branch information
barneygale committed Apr 24, 2024
1 parent 5865fa5 commit f1eb06c
Show file tree
Hide file tree
Showing 4 changed files with 66 additions and 103 deletions.
32 changes: 19 additions & 13 deletions Doc/library/pathlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@ Methods
^^^^^^^

Concrete paths provide the following methods in addition to pure paths
methods. Many of these methods can raise an :exc:`OSError` if a system
methods. Some of these methods can raise an :exc:`OSError` if a system
call fails (for example because the path doesn't exist).

.. versionchanged:: 3.8
Expand All @@ -886,6 +886,12 @@ call fails (for example because the path doesn't exist).
instead of raising an exception for paths that contain characters
unrepresentable at the OS level.

.. versionchanged:: 3.14

The methods given above now return ``False`` instead of raising an
:exc:`OSError` exception from the operating system. In previous versions,
some kinds of :exc:`OSError` exception are raised, and others suppressed.


.. classmethod:: Path.cwd()

Expand Down Expand Up @@ -1071,8 +1077,8 @@ call fails (for example because the path doesn't exist).
Return ``True`` if the path points to a directory, ``False`` if it points
to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
``False`` is also returned if the path doesn't exist, or is a broken
symlink, or is inaccessible for any other reason.

This method normally follows symlinks; to exclude symlinks to directories,
add the argument ``follow_symlinks=False``.
Expand All @@ -1086,8 +1092,8 @@ call fails (for example because the path doesn't exist).
Return ``True`` if the path points to a regular file, ``False`` if it
points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
``False`` is also returned if the path doesn't exist, or is a broken
symlink, or is inaccessible for any other reason.

This method normally follows symlinks; to exclude symlinks, add the
argument ``follow_symlinks=False``.
Expand Down Expand Up @@ -1125,8 +1131,8 @@ call fails (for example because the path doesn't exist).

Return ``True`` if the path points to a symbolic link, ``False`` otherwise.

``False`` is also returned if the path doesn't exist; other errors (such
as permission errors) are propagated.
``False`` is also returned if the path doesn't exist or is inaccessible for
any other reason.


.. method:: Path.is_socket()
Expand All @@ -1143,26 +1149,26 @@ call fails (for example because the path doesn't exist).
Return ``True`` if the path points to a FIFO (or a symbolic link
pointing to a FIFO), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
``False`` is also returned if the path doesn't exist, or is a broken
symlink, or is inaccessible for any other reason.


.. method:: Path.is_block_device()

Return ``True`` if the path points to a block device (or a symbolic link
pointing to a block device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
``False`` is also returned if the path doesn't exist, or is a broken
symlink, or is inaccessible for any other reason.


.. method:: Path.is_char_device()

Return ``True`` if the path points to a character device (or a symbolic link
pointing to a character device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
``False`` is also returned if the path doesn't exist, or is a broken
symlink, or is inaccessible for any other reason.


.. method:: Path.iterdir()
Expand Down
10 changes: 3 additions & 7 deletions Lib/glob.py
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,7 @@ def __init__(self, sep, case_sensitive, case_pedantic=False, recursive=False):

# Low-level methods

lstat = staticmethod(os.lstat)
lexists = staticmethod(os.path.lexists)
scandir = staticmethod(os.scandir)
parse_entry = operator.attrgetter('path')
concat_path = operator.add
Expand Down Expand Up @@ -512,12 +512,8 @@ def select_exists(self, path, exists=False):
# Optimization: this path is already known to exist, e.g. because
# it was returned from os.scandir(), so we skip calling lstat().
yield path
else:
try:
self.lstat(path)
yield path
except OSError:
pass
elif self.lexists(path):
yield path

@classmethod
def walk(cls, root, top_down, on_error, follow_symlinks):
Expand Down
34 changes: 34 additions & 0 deletions Lib/pathlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -525,12 +525,46 @@ def stat(self, *, follow_symlinks=True):
"""
return os.stat(self, follow_symlinks=follow_symlinks)

def exists(self, *, follow_symlinks=True):
"""
Whether this path exists.
This method normally follows symlinks; to check whether a symlink exists,
add the argument follow_symlinks=False.
"""
if follow_symlinks:
return os.path.exists(self)
return os.path.lexists(self)

def is_dir(self, *, follow_symlinks=True):
"""
Whether this path is a directory.
"""
if follow_symlinks:
return os.path.isdir(self)
return _abc.PathBase.is_dir(self, follow_symlinks=follow_symlinks)

def is_file(self, *, follow_symlinks=True):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
if follow_symlinks:
return os.path.isfile(self)
return _abc.PathBase.is_file(self, follow_symlinks=follow_symlinks)

def is_mount(self):
"""
Check if this path is a mount point
"""
return os.path.ismount(self)

def is_symlink(self):
"""
Whether this path is a symbolic link.
"""
return os.path.islink(self)

def is_junction(self):
"""
Whether this path is a junction.
Expand Down
93 changes: 10 additions & 83 deletions Lib/pathlib/_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,37 +14,17 @@
import functools
import glob
import operator
from errno import ENOENT, ENOTDIR, EBADF, ELOOP, EINVAL
from errno import ENOTDIR, ELOOP
from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO

#
# Internals
#

_WINERROR_NOT_READY = 21 # drive exists but is not accessible
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself

# EBADF - guard against macOS `stat` throwing EBADF
_IGNORED_ERRNOS = (ENOENT, ENOTDIR, EBADF, ELOOP)

_IGNORED_WINERRORS = (
_WINERROR_NOT_READY,
_WINERROR_INVALID_NAME,
_WINERROR_CANT_RESOLVE_FILENAME)

def _ignore_error(exception):
return (getattr(exception, 'errno', None) in _IGNORED_ERRNOS or
getattr(exception, 'winerror', None) in _IGNORED_WINERRORS)


@functools.cache
def _is_case_sensitive(parser):
return parser.normcase('Aa') == 'Aa'


class Globber(glob._Globber):
lstat = operator.methodcaller('lstat')
lexists = operator.methodcaller('exists', follow_symlinks=False)
add_slash = operator.methodcaller('joinpath', '')

@staticmethod
Expand Down Expand Up @@ -472,12 +452,7 @@ def exists(self, *, follow_symlinks=True):
"""
try:
self.stat(follow_symlinks=follow_symlinks)
except OSError as e:
if not _ignore_error(e):
raise
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False
return True

Expand All @@ -487,14 +462,7 @@ def is_dir(self, *, follow_symlinks=True):
"""
try:
return S_ISDIR(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_file(self, *, follow_symlinks=True):
Expand All @@ -504,14 +472,7 @@ def is_file(self, *, follow_symlinks=True):
"""
try:
return S_ISREG(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_mount(self):
Expand Down Expand Up @@ -540,13 +501,7 @@ def is_symlink(self):
"""
try:
return S_ISLNK(self.lstat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_junction(self):
Expand All @@ -564,14 +519,7 @@ def is_block_device(self):
"""
try:
return S_ISBLK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_char_device(self):
Expand All @@ -580,14 +528,7 @@ def is_char_device(self):
"""
try:
return S_ISCHR(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_fifo(self):
Expand All @@ -596,14 +537,7 @@ def is_fifo(self):
"""
try:
return S_ISFIFO(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_socket(self):
Expand All @@ -612,14 +546,7 @@ def is_socket(self):
"""
try:
return S_ISSOCK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def samefile(self, other_path):
Expand Down

0 comments on commit f1eb06c

Please sign in to comment.