Python's "shutil.rmtree()" fails

[buhtz]

Hello,
I am upstream maintainer of Back In Time investigating an issue a distro maintainer from Fedora reported to me (backintime#1911). I also postet about the problem on the python-list.

I am not familiar with Fedora, rpm or mock.
The test suite of "Back In Time" does read and write to the real file system.
One test fails because a temporary directory is cleaned up using shutil.rmtree(). Please see full output below. This is the line of interest causing the error I think.

assert func is os.lstat

Do you have any idea what could cause this or what is behind it?

Do you mind givin me a minmal tutorial how to "create" such a mock environment on my own. In this case I could start a Python REPL or a minimal Python script investigating the problem further.

Thanks in advance,
Christian Buhtz

__________________________ General.test_ctor_defaults __________________________
self = <test.test_uniquenessset.General testMethod=test_ctor_defaults>
    def test_ctor_defaults(self):
        """Default values in constructor."""
      with TemporaryDirectory(prefix='bit.') as temp_name:
test/test_uniquenessset.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.13/tempfile.py:946: in __exit__
    self.cleanup()
/usr/lib64/python3.13/tempfile.py:950: in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
/usr/lib64/python3.13/tempfile.py:930: in _rmtree
    _shutil.rmtree(name, onexc=onexc)
/usr/lib64/python3.13/shutil.py:763: in rmtree
    _rmtree_safe_fd(stack, onexc)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
stack = []
onexc = <function TemporaryDirectory._rmtree.<locals>.onexc at 0xffffb39bc860>
    def _rmtree_safe_fd(stack, onexc):
        # Each stack item has four elements:
        # * func: The first operation to perform: os.lstat, os.close or os.rmdir.
        #   Walking a directory starts with an os.lstat() to detect symlinks; in
        #   this case, func is updated before subsequent operations and passed to
        #   onexc() if an error occurs.
        # * dirfd: Open file descriptor, or None if we're processing the top-level
        #   directory given to rmtree() and the user didn't supply dir_fd.
        # * path: Path of file to operate upon. This is passed to onexc() if an
        #   error occurs.
        # * orig_entry: os.DirEntry, or None if we're processing the top-level
        #   directory given to rmtree(). We used the cached stat() of the entry to
        #   save a call to os.lstat() when walking subdirectories.
        func, dirfd, path, orig_entry = stack.pop()
        name = path if orig_entry is None else orig_entry.name
        try:
            if func is os.close:
                os.close(dirfd)
                return
            if func is os.rmdir:
                os.rmdir(name, dir_fd=dirfd)
                return

            # Note: To guard against symlink races, we use the standard
            # lstat()/open()/fstat() trick.
          assert func is os.lstat
E           AssertionError
/usr/lib64/python3.13/shutil.py:663: AssertionError

[xsuchy]

Hmm, I checkout the repo and checked the test and tried to somehow isolate the reproducer with:

# mock install python
# mock shell
...
<mock-chroot> sh-5.2# python3
Python 3.13.0 (main, Oct  8 2024, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tempfile import TemporaryDirectory
>>> TemporaryDirectory(prefix='bit.')
<TemporaryDirectory '/tmp/bit.pad24mg1'>

@hannes101 can you share your src.rpm so I can try with that?

[hannes101]

Hey @xsuchy I think it's easiest, if you are using the one from the build here:
https://kojipkgs.fedoraproject.org//work/tasks/5843/125095843/backintime-1.5.3-0.1.fc41.src.rpm

[xsuchy]

I tried to build this package:

mock -r fedora-rawhide-x86_64 backintime-1.5.3-0.1.fc41.src.rpm

and it works (minus the issue with groups).
I checked the Koji build again and noticed that it run on aarch64 builder. So I tried:

mock -r fedora-rawhide-aarch64 backintime-1.5.3-0.1.fc41.src.rpm

and I got an error. It is different one though:

=================================== FAILURES ===================================
_________________ General.test_autoExit_other_running_process __________________

self = <test.test_applicationinstance.General testMethod=test_autoExit_other_running_process>

    def test_autoExit_other_running_process(self):
        pid = self._createProcess()
        procname = tools.processName(pid)
    
        # create file with pid and process name
        with open(self.file_name, "wt") as file_with_pid:
            file_with_pid.write(str(pid) + "\n")
            file_with_pid.write(procname)
    
>       with self.assertRaises(SystemExit):
E       AssertionError: SystemExit not raised

test/test_applicationinstance.py:299: AssertionError
_____________ General.test_existing_process_with_correct_procname ______________

self = <test.test_applicationinstance.General testMethod=test_existing_process_with_correct_procname>

    def test_existing_process_with_correct_procname(self):
        """
        Test the check function with an existing process with correct process
        name
        """
        pid = self._createProcess()
        procname = tools.processName(pid)
    
        # create file with pid and process name
        with open(self.file_name, "wt") as file_with_pid:
            file_with_pid.write(str(pid) + "\n")
            file_with_pid.write(procname)
    
        # Execute test
>       self.assertFalse(self.app_instance.check())
E       AssertionError: True is not false

test/test_applicationinstance.py:124: AssertionError
____________________ General.test_killing_existing_process _____________________

self = <test.test_applicationinstance.General testMethod=test_killing_existing_process>

    def test_killing_existing_process(self):
        """
        Test the check function with an existing process with correct process
        name
        """
        pid = self._createProcess()
        procname = tools.processName(pid)
    
        # create file with pid and process name
        with open(self.file_name, "wt") as file_with_pid:
            file_with_pid.write(str(pid) + "\n")
            file_with_pid.write(procname)
    
>       self.assertFalse(self.app_instance.check())
E       AssertionError: True is not false

test/test_applicationinstance.py:193: AssertionError

So this may be arch related. Quick'n'dirty fix may be to resubmit to Koji again and hope that you get x86_64 builder.
Why is this failing on aarch64 is beyond my knowledge.
You can submit the build to Copr and log to the builders https://frostyx.cz/posts/ssh-access-to-copr-builders

[hannes101]

Ah sorry, it might be that I dropped the initially offending tests from that build by removing the test from the common/test folder before build.
Sorry.

[buhtz]

FYI: I realized that the related test does not use the real file system but a PyFakeFS file system. Investigation is ongoing...