Visualize performance of the NPU device

[xiangyang-95]

How can I get the utilizations or log for the NPU device?

[jwludzik]

The utilization is not available, but it is a feature that we plan to add. No deadline for it for today

What do you mean by log for the NPU device? There are kernel logs from intel_vpu module

[xiangyang-95]

@jwludzik Thanks for the information. I am looking to view the utilization of NPU. Please let me know if the feature is available.

[mecattaf]

Would be great to have an intel_npu_top tool to visualize utilization as mentioned in this thread

[Qubitium]

Can we at least get a single int based utilization rate exposed in sys fs?

[jwludzik]

There is work in progress in nokyan/resources#302 to get a tool to show the utilization lately. You can get utilization using sys fs today, it is explained in nokyan/resources#302 (comment)

[jwludzik]

The nokyan/resources today offers the utilization monitor in Ubuntu24.10 for NPU

[junruizh2021]

There is work in progress in nokyan/resources#302 to get a tool to show the utilization lately. You can get utilization using sys fs today, it is explained in nokyan/resources#302 (comment)有工作正在进行中 nokyan/resources#302 获取一个显示最近利用率的工具。您今天可以使用 sys fs 来获得利用率，它在中进行了解释 nokyan/resources#302 (comment)

@jwludzik Can the NPU utilization be reflected by sys_fs counting npu busytime? Can the memory usage of the NPU be reflected? If we want to monitor how much NPU computing power/memory the model occupies, how should we do?

[m-falkowski]

There is work in progress in nokyan/resources#302 to get a tool to show the utilization lately. You can get utilization using sys fs today, it is explained in nokyan/resources#302 (comment)有工作正在进行中 nokyan/resources#302 获取一个显示最近利用率的工具。您今天可以使用 sys fs 来获得利用率，它在中进行了解释 nokyan/resources#302 (comment)

@jwludzik Can the NPU utilization be reflected by sys_fs counting npu busytime?

I don't think so. The method used here to calculate utilization is
to get a ratio busy / sampling time. Sysfs npu_busy_time_us already
returns a busy value so one just need to get a delta of it and divide
with a chosen sample value.

Can the memory usage of the NPU be reflected? If we want to monitor how much NPU computing power/memory the model occupies, how should we do?

Yes, it can. There needs to be either a driver feature for it or external tool used like some tracing tool.
We have already prepared memory usage feature and it is going to be upstreamed very soon.
For power the issue is much more complicated and there is no way to calculate it reliably.

[junruizh2021]

@m-falkowski Thanks for your explanation! Can I understand that the increment of sys_fs reflects the utilization of the NPU, which is similar to the cpu utilization seen by the top command. But the memory usage function reflects the memory water level.

Is NPU memory shared with CPU memory? For example, my host has 16G of memory. Is part of it reserved for the NPU?

[jlo62]

@junruizh2021 The NPU does not have it's own Memory like a dedicated GPU, but works similar to a iGPU, which has a dedicated/reserved part of the RAM and can "swap out" to regular RAM (GTT/GART).
However, it doesn't have a reserved/dedicated RAM part.
Fun fact, both systems work so similar that the code base or a interface or a API or whatever (DRM) is shared.

[m-falkowski]

@m-falkowski Thanks for your explanation! Can I understand that the increment of sys_fs reflects the utilization of the NPU, which is similar to the cpu utilization seen by the top command. But the memory usage function reflects the memory water level.

Is NPU memory shared with CPU memory?

Yes, they share host memory.

For example, my host has 16G of memory. Is part of it reserved for the NPU?

There is no memory reserved for the NPU, it is allocated when needed. There is a bit of memory allocated during NPU initialization for a firmware and data structures.