Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add np.datetime64 serialization and tests #1036

Merged
merged 22 commits into from
Jul 12, 2022

Conversation

jverswijver
Copy link
Contributor

@jverswijver jverswijver commented Jun 15, 2022

supersedes #1019

I have added the ability to serialize numpy.datetime64 types into datajoint. This pr includes a pack/unpack test as well as a test that checks to see if serializing/deserializing numpy.datetime64 is still faster than regular datetime.

fix #1022
fix #1035

@jverswijver jverswijver marked this pull request as ready for review June 15, 2022 20:22
datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
@jverswijver jverswijver marked this pull request as draft June 16, 2022 14:46
datajoint/blob.py Outdated Show resolved Hide resolved
@jverswijver
Copy link
Contributor Author

Alright I have moved the datetime64 array serialization to use the scalar mapping dictionary instead of the object mapping dictionary, this solution should be complete now @dimitri-yatsenko. I have some extra time so I will work on refactoring the naming like we discussed.

@jverswijver jverswijver marked this pull request as ready for review June 21, 2022 15:21
datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
datajoint/blob.py Outdated Show resolved Hide resolved
@jverswijver
Copy link
Contributor Author

Seems like datetime64 is no longer experimental, see this github thread. I think we are good to go ahead and merge @guzman-raphael

Copy link
Collaborator

@guzman-raphael guzman-raphael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jverswijver Fantastic work! 💪

That is an awesome performance boost! 1000x
🏁 🏎️

datajoint/blob.py Outdated Show resolved Hide resolved
datajoint/blob.py Show resolved Hide resolved
@guzman-raphael guzman-raphael merged commit b0f5dc4 into datajoint:master Jul 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants