Return data type

[AsafMah]

Again, this is an issue to help me with the scenario of streaming without really implementing streaming.

I want a type, that when encountered, will stop parsing completely, and will give me the parts of the object already parsed, and the offset to the next point in the buffer.

For encoding, you can specify the type it will encode as.

This is to avoid saving a really big type, accepted by streaming for example, into memory completely.

Unlike #27, there doesn't need to be a resume login, let me as the client handle it.

What I ask for is something like this:

class Get(msgspec.Struct, tag=True):
    key: str

class Put(msgspec.Struct, tag=True):
    key: str
    val: str

class Huge(msgspec.Struct, tag=True):
    key: str
    some: int
    some2: str
    huge: Return[str]
    some_other: int

msgspec.json.encode([Get("1"), Put("1","2"), Huge("1", 2, "3", "4567", 5)])
# b'[{"type":"Get","key":"1"},{"type":"Put","key":"1","val":"2"},{"type":"Huge","key":"1","some":2,"some2":"3","huge":"4567", "some_other": 5}]'


msgspec.json.decode(d, type=List[Union[Get,Put,Huge]])
# [Get(key='1'), Put(key='1', val='2'), Return(key='1', some=2, some2='3', missing=["some_other"], offset: 114 )]

[jcrist]

In this case I think the type you're looking for is msgspec.Raw (https://jcristharif.com/msgspec/usage.html#raw)? In this case, it returns a Raw object that provides a memoryview (not copy) into the original buffer, but doesn't decode that buffer immediately. This would let you avoid decoding Huge.huge immediately and handle it however you want to.

[AsafMah]

The scenario was to get it without holding it all in memory, but since that's a non-goal I'm closing.