Move streaming to StreamCallbacks.jl

[svilupp]

I would like to propose moving streaming functionality to a separate package (made by me): https://github.com/svilupp/StreamCallbacks.jl

Why?

• provides an implementation that can be shared across other providers (Anthropic, OpenAI, etc)
• Implements more of the SSE "standard" (eg, includes event names, does proper splitting, catches message overflow between "masterchunks", etc)
• Allows easy extensibility for users (to choose the sink, logging, etc.)

Downsides

• Opinionated
• Extra-dependency (lightweight though)
• By default, it builds the standard Response afterwards (to allow standard downstream functionality as well), which might be wasteful for some users/workflows (but the cost and time are sooo tiny compared to the generation time)

I've been using it in PromptingTools.jl for a few releases already and it seems to work well.

[svilupp]

Bump here.

Is there any interest in refactoring the functionality with a new dep? The current implementation is not correct and will lead to more errors.

[cpfiffer]

I would say it's probably better to separate them out -- I'd be happy to see StreamingCallbacks.jl implemented here. As to this point:

By default, it builds the standard Response afterwards (to allow standard downstream functionality as well), which might be wasteful for some users/workflows (but the cost and time are sooo tiny compared to the generation time)

Not a concern on my end, if anything that strikes me as pretty ergonomic.

[roryl23]

If the current implementation is not correct and will lead to errors, I say full steam ahead.