Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZTS: include microsecond timestamps on all output #17045

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

robn
Copy link
Member

@robn robn commented Feb 12, 2025

[Sponsors: Klara, Inc., Wasabi Technology, Inc.]

Motivation and Context

When reviewing test output after a failure, it's often quite difficult to work out the order and timing of events, and to correlate test suite output with kernel logs.

Description

This adds timestamps to ZTS output to help correlate events:

  • all of the standard log_XXX functions ultimately end up in _printline, which now prefixes output with a timestamp.
  • the test runner logging function log() also now prefixes its output with a timestamp.
  • on failure, when capturing the kernel log in zfs_dmesg.ksh, the iso time format is requested. This is gated behind a platform check, as FreeBSD's dmesg has no switches to control timestamps.

How Has This Been Tested?

Manually, running a test I know fails in my test VMs and considering the output.

$ zfs-tests.sh -DvKx -t zfs_load-key_https
...
/usr/local/share/zfs/test-runner/bin/test-runner.py  -D  -K -c "/var/tmp/zfs-tests.481.run" -T "functional" -i "/usr/local/share/zfs/zfs-tests" -I "1"
[    6.960295] ZTS run /usr/local/share/zfs/zfs-tests/tests/functional/cli_root/zfs_load-key/setup
[2025-02-12T01:05:34.353115] NOTE: begin default_setup_noexit
[2025-02-12T01:05:34.398547] SUCCESS: zpool create -f testpool loop0
[2025-02-12T01:05:34.432899] SUCCESS: zfs create testpool/testfs
[2025-02-12T01:05:34.463772] SUCCESS: zfs set mountpoint=/var/tmp/testdir testpool/testfs
[2025-02-12T01:05:35.520185] SUCCESS: openssl req -x509 -newkey rsa:4096 -sha256 -days 1 -nodes -keyout /testpool/snakeoil.key -out /testpool/snakeoil.crt -subj /CN=localhost
Traceback (most recent call last):
  File "<string>", line 16, in <module>
  File "<string>", line 10, in <module>
  File "/usr/lib/python3.11/socketserver.py", line 456, in __init__
    self.server_bind()
  File "/usr/lib/python3.11/http/server.py", line 136, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/lib/python3.11/socketserver.py", line 472, in server_bind
    self.socket.bind(self.server_address)

[2025-02-12T01:06:25.290486] NOTE: Performing test-fail callback (/usr/local/share/zfs/zfs-tests/callbacks/zfs_dbgmsg.ksh)
...
[2025-02-12T01:06:25.315555] NOTE: Performing test-fail callback (/usr/local/share/zfs/zfs-tests/callbacks/zfs_dmesg.ksh)
=================================================================
 Tailing last 200 lines of dmesg log
=================================================================
2025-02-12T01:05:28,971544+00:00 .... node  #0, CPUs:      #1 #2 #3 #4 #5
2025-02-12T01:05:28,971544+00:00 smp: Brought up 1 node, 6 CPUs
2025-02-12T01:05:28,971544+00:00 smpboot: Max logical packages: 1
2025-02-12T01:05:28,971544+00:00 smpboot: Total of 6 processors activated (23904.00 BogoMIPS)
2025-02-12T01:05:29,090172+00:00 node 0 deferred pages initialised in 120ms
...
=================================================================
 End of dmesg log
=================================================================
[2025-02-12T01:06:25.334713] NOTE: Performing test-fail callback (/usr/local/share/zfs/zfs-tests/callbacks/zfs_mmp.ksh)
[2025-02-12T01:06:25.340432] Test: /usr/local/share/zfs/zfs-tests/tests/functional/cli_root/zfs_load-key/setup (run as root) [00:51] [FAIL]
[2025-02-12T01:06:25.345063] Test: /usr/local/share/zfs/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_https.ksh (run as root) [00:00] [SKIP]
[   58.006623] ZTS run /usr/local/share/zfs/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup
[2025-02-12T01:06:25.551120] SUCCESS: zpool destroy -f testpool
[2025-02-12T01:06:25.583079] SUCCESS: rm -rf /var/tmp/testdir
[2025-02-12T01:06:25.594580] Test: /usr/local/share/zfs/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup (run as root) [00:00] [PASS]
...

(fyi, this test fails because the VM has no network support at all, not even a loopback. It's no big deal)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

When reviewing test output after a failure, it's often quite difficult
to work out the order and timing of events, and to correlate test suite
output with kernel logs.

This adds timestamps to ZTS output to help with this, in three places:

- all of the standard log_XXX functions ultimately end up in _printline,
  which now prefixes output with a timestamp.

- the test runner logging function log() also now prefixes its output
  with a timestamp.

- on failure, when capturing the kernel log in zfs_dmesg.ksh, the "iso"
  time format is requested.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
@robn robn force-pushed the zts-timestamp-prefix branch from dad344b to c73849f Compare February 12, 2025 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant