Force Transformers engine to return float logits

[JC1DA]

Problem: Logits cannot be converted directly to CPU numpy if its type is in BFloat16

File "/home/jc1da/repos/ML/guidance_loc/guidance/models/transformers/_transformers.py", line 513, in get_logits
    model_out.logits[0, -1, : len(self.tokenizer.tokens)].cpu().numpy()
TypeError: Got unsupported ScalarType BFloat16

Example Code:

model = "Qwen/Qwen2-0.5B-Instruct"
lm = guidance.models.Transformers(
    model,
    device_map="auto",
    trust_remote_code=True,
    chat_template=QWen2_ChatTemplate,
    torch_dtype=torch.bfloat16,
)

lm += """1 + 1 = add(1, 1) = 2
3 + 5 = add(3, 5) = 8
11 + 9 = add"""

@hudson-ai

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.31%. Comparing base (50a5c39) to head (da37f6f).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1086      +/-   ##
==========================================
- Coverage   66.61%   65.31%   -1.30%     
==========================================
  Files          65       65              
  Lines        5173     5173              
==========================================
- Hits         3446     3379      -67     
- Misses       1727     1794      +67     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hudson-ai]

LGTM, thanks