Array compression with collections in Mixed #7412
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"Bugfix release"
Optimize storage of Decimal128 properties so that the individual values will take up 0 bits (if all nulls or all zero), 32 bits, 64 bits or 128 bits depending on what is needed.
* temporary disable failing c api decimal test
update next major to core v13.4.2
"Feature/bugfix release"
Introduce a new class - ColletionParent - which a collection will refer to as its owner. This class can be specialized as an Obj if the nesting level is 0 or a CollectionList if the collection is nested.
Basic support for nested collections
Make it clear that when an Obj is the owner, then the index must be a ColKey
Co-authored-by: Jørgen Edelbo <[email protected]>
Co-authored-by: Jørgen Edelbo <[email protected]>
"Bugfix release"
* Handle nullifying links in nested collections * Clear backlinks related to nested collections
"Bugfix release"
* dump to json support info about nested collections for schema * reuse logic for printing nested collections * main logic for expanding nested collections to json, requires to be polished * more testing for nested containers * complete algo for printing nested collections in json format * add testing json files to project * generate json files option set to false * run whole test suite * test nested collections with links * format checks * Move out_mixed_json... functionality to Mixed class * Remove not needed template parameter from CollectionBaseImpl * Delegate to_json to collections * remove commented code * fix audit conflicts --------- Co-authored-by: Jørgen Edelbo <[email protected]>
"Bugfix release"
* Cleanup naming and consolidate update strategy * Allow a Mixed containing a ref to be stores in ArrayMixed
nicola-cab
commented
Mar 5, 2024
| auto rot = leaf.get_as_ref_or_tagged(i); | ||
| if (rot.is_ref() && rot.get_as_ref()) { | ||
| // entries 0-2 are integral and can be compressed, entry 3 is strings and not compressed (yet) | ||
| bool do_compress = compress && i < 3; | ||
| // collections in mixed are stored at position 4. | ||
| bool do_compress = false; // (i < 3 || i == 4) ? true : false; |
There was a problem hiding this comment.
@finnschiermer disabling Mixed compression for now, first merge this then new PR for compressing mixed and collections in mixed.
finnschiermer
approved these changes
Mar 5, 2024
Once CI passes I will do it :-) .. |
nicola-cab
force-pushed
the
nc/array_compression_with_nested_collections
branch
from
613d3e1
to
4837a65
Compare
Pull Request Test Coverage Report for Build nicola.cabiddu_1458Details
💛 - Coveralls |
nicola-cab
force-pushed
the
nc/array_compression_with_nested_collections
branch
11 times, most recently
from
331a0dc
to
eaa7546
Compare
nicola-cab
force-pushed
the
nc/array_compression_with_nested_collections
branch
from
eaa7546
to
4040fd7
Compare
* tentative fix for 32 bit archs * removed wrong cast to size_t from bf_iterator::set_value() * fix inverted condition in 'unsigned_to_num_bits' * fix inverted condition in 'unsigned_to_num_bits' --------- Co-authored-by: Nicola Cabiddu <[email protected]>
This reverts commit 7ac0073.
….com:realm/realm-core into nc/array_compression_with_nested_collections
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What, How & Why?
Prepare Array classification + compression to core 14.0.0 which contains collections in mixed and requires some more handling in order to compress those arrays.
☑️ ToDos
bindgen/spec.yml, if public C++ API changed