Add more support for intel Gaudi accelerators

[YangQun1]

Motivation

We already have initial support for Intel Gaudi accelerators in sglang.
This PR aims to add more supports (like hpu memory capacity getter and some minor changes to use generic torch device module apis) to make the offline_batch_inference.py example run e2e successfully on x1 or x2 Gaudi2 cards.

# x1
python examples/runtime/engine/offline_batch_inference.py --device hpu --model-path meta-llama/Meta-Llama-3.1-8B-Instruct

# x2
python examples/runtime/engine/offline_batch_inference.py --device hpu --tp-size=2 --model-path meta-llama/Meta-Llama-3.1-8B-Instruct

Modifications

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[YangQun1]

Hi @merrymercy , could you help to take a review?

[merrymercy]

This looks good!