Not working with php-fpm

[akademic]

Hi. I've built binary from git master 7e0b062.

Trying to inspect my php-fpm process, but it fails.

Output:

>>./phpspy -V73 -p 30516

objdump: '/proc/30516/root//usr/local/sbin/php-fpm': No such file
popen_read_line: No stdout; cmd=objdump -p /proc/30516/root//usr/local/sbin/php-fpm | awk '/LOAD/{print $5; exit}'
get_php_base_addr: Failed to get virt_addr

PHP version:

>> php -v

PHP 7.3.4 (cli) (built: Apr 20 2019 04:26:18) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.4, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.4, Copyright (c) 1999-2018, by Zend Technologies

OS version:

>> uname -a
Linux ai-w5 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Any suggestions?

[adsr]

Hello, thanks for the report. From the error it sounds like phpspy is having trouble locating the php-fpm binary. It is looking for /usr/local/sbin/php-fpm inside your container which appears to not exist. This path is probably coming from the get_php_bin_path function. Can you share the output of readlink /proc/$(pgrep -n php-fpm)/exe from inside your container?

[akademic]

Hello, here is the output:

>>readlink /proc/$(pgrep -n php-fpm)/exe
/usr/local/sbin/php-fpm

And path /usr/local/sbin/php-fpm' does exist.
But path /proc/$(pgrep -n php-fpm)//usr/local/sbin/php-fpm doesn't.

I think it is connected with php-fpm configuration.
All php-fpm pools are chrooted.

But for php-fpm master process I have this result:

>>ls -sh /proc/10065/root//usr/local/sbin/php-fpm
49M /proc/10065/root//usr/local/sbin/php-fpm

[adsr]

But path /proc/$(pgrep -n php-fpm)//usr/local/sbin/php-fpm doesn't.

Not sure if you made a typo, but phpspy will look for the binary at /proc/<pid>/root/<bin_path> where <bin_path> is what it found at /proc/<pid>/exe.

I see the following note in procfs man page which might be relevant? Wild guess.

In a multithreaded process, the contents of the
/proc/[pid]/root symbolic link are not available if the main
thread has already terminated (typically by calling
pthread_exit(3)).

Can you provide more details about your setup so I can try to reproduce? Thanks.

[akademic]

Sorry for long answer.

I've made docker-compose.yml for reproducing this issue: https://github.com/akademic/phpspy-php-fpm-test

There are two processes: master and worker.
For master you can run phpspy without any errors, but it does not process requests.
For worker (it has pool www description in ps output) you will get error I described in first message of this issue.

[sj-i]

The process is chrooted, so /proc/<pid>/root/ shows only its docroot.

@adsr
Maybe adding fallback to the /proc/<pid>/exe without readlink works in this case.
To support mod_php in chrooted process, it's not so easy though,

[adsr]

Thank you @akademic and @sj-i. Tracking in #109.

[xtrime-ru]

Hi! I ve got similar issue:

Output

> ./bin/phpspy --max-depth=60 --time-limit-ms=59000 --threads=8 -P '-x "php-fpm8.0"'

popen_read_line: No stdout; cmd=objdump -p /proc/2007773/root//usr/sbin/php-fpm8.0 | awk '/LOAD/{print $5; exit}'
get_php_base_addr: Failed to get virt_addr
sh: 1: objdump: not found
popen_read_line: No stdout; cmd=objdump -p /proc/742/root//usr/sbin/php-fpm8.0 | awk '/LOAD/{print $5; exit}'
get_php_base_addr: Failed to get virt_addr

PHP Version

PHP 8.0.13 (cli) (built: Nov 19 2021 06:34:49) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.13, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.13, Copyright (c), by Zend Technologies

OS version:

Ubuntu 20.04 in Docker

root@88f9a31df65d /zoon/zprof$ uname -a
Linux 88f9a31df65d 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Is there any solution? How to trace php-fpm?

[adsr]

Hi @xtrime-ru, I think if you install objdump in your container it may work. (Noticed sh: 1: objdump: not found.) That is package binutils in Debian/Ubuntu I think.

[xtrime-ru]

Thanks! apt install binutils solved the issue!