Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fastpath for encoding unsorted dict to JSON #95385

Closed
aivarsk opened this issue Jul 28, 2022 · 2 comments
Closed

Fastpath for encoding unsorted dict to JSON #95385

aivarsk opened this issue Jul 28, 2022 · 2 comments
Labels
3.12 bugs and security fixes performance Performance or resource usage type-feature A feature request or enhancement

Comments

@aivarsk
Copy link
Contributor

aivarsk commented Jul 28, 2022

JSON encoder can produce output sorted by key (json.dumps(sort_keys=True)) The default value for sort_keys is False. The code always creates a list of dictionary items, sorts them if needed, and then iterates through them.
However, if sorting is not requested we can use PyDict_Next to avoid allocating a list of tuples.

pyperformance on my machine before changes:

### json_dumps ###
Mean +- std dev: 10.6 ms +- 0.1 ms

pyperformance after changes:

### json_dumps ###
Mean +- std dev: 9.00 ms +- 0.08 ms
@aivarsk aivarsk added the type-feature A feature request or enhancement label Jul 28, 2022
@mdboom mdboom added the performance Performance or resource usage label Jul 28, 2022
@aivarsk
Copy link
Contributor Author

aivarsk commented Jul 29, 2022

Patched pyperformance to use json.dumps(sort_keys=True)

before changes

### json_dumps ###
Mean +- std dev: 14.0 ms +- 0.2 ms

after changes

### json_dumps ###
Mean +- std dev: 13.6 ms +- 0.2 ms

Probably because the new code does not create an iterator from the list and PyList_GET_ITEM is a bit faster than PyIter_Next (extra incref/decref)?

@serhiy-storchaka
Copy link
Member

It caused a regression for OrderedDict. See #100272.

@serhiy-storchaka serhiy-storchaka moved this to Done in JSON issues Dec 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes performance Performance or resource usage type-feature A feature request or enhancement
Projects
Status: Done
Development

No branches or pull requests

4 participants