Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve "Input data was truncated" error message #168

Closed
AsafMah opened this issue Aug 2, 2022 · 2 comments
Closed

Improve "Input data was truncated" error message #168

AsafMah opened this issue Aug 2, 2022 · 2 comments

Comments

@AsafMah
Copy link

AsafMah commented Aug 2, 2022

Consider:

msgspec.json.decode(b"[[1,2,3,4,5], [3,4,5,6,7], [8,")

Will throw:

DecodeError: Input data was truncated

Which is pretty lacking in information.

By the point its throwing it (correct me if i'm wrong), we've already parsed some of the data.

It would be useful to know:

  1. What object was failed to parse? what was its type, and where it began? (in the example: the location of the [ before the 8, and then the first [)
  2. What are the objects that have been parsed so far? (in the example: [1,2,3,4,5], [3,4,5,6,7], and 8)

The scenario I'm looking for is implementing streaming-like parsing for a very specific case, and I know streaming isn't planned (#27), so I'm trying to find other small features that might be possible and will help me.

In this case, resuming to parse after more data arrives, without re-parsing everything.

@jcrist
Copy link
Owner

jcrist commented Aug 2, 2022

We can definitely add more context to the truncated error message to help with debugging. We won't ever provide a way to access half-parsed messages though.

If possible, can you open a new issue with your exact real problem? It's easier for me as a maintainer to suggest solutions to the actual issue you're dealing with, not workaround ideas you've come up with.

@jcrist jcrist changed the title Input Truncated Error - getting parsed so far, or some option to resume parsing with more data Improve "Input data was truncated" error message Aug 2, 2022
@AsafMah
Copy link
Author

AsafMah commented Aug 4, 2022

The scenario is to use it to parse data that might still be incomplete, and is still coming from the server. Since that's a non-goal, I'm closing.

@AsafMah AsafMah closed this as completed Aug 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants