Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Improved performance of iterator of Utf8Array and BinaryArray (3-4x) #427

Merged
merged 2 commits into from
Sep 20, 2021

Conversation

jorgecarleitao
Copy link
Owner

iter_values 2^20        time:   [1.6717 ms 1.6744 ms 1.6774 ms]                              
                        change: [-75.148% -75.071% -74.997%] (p = 0.00 < 0.05)
iter 2^20               time:   [2.2270 ms 2.2301 ms 2.2336 ms]                       
                        change: [-66.754% -66.678% -66.604%] (p = 0.00 < 0.05)

This has implications on almost every kernel over these arrays, as well as users of the iterator.

@jorgecarleitao jorgecarleitao added the enhancement An improvement to an existing feature label Sep 19, 2021
@jorgecarleitao jorgecarleitao changed the title Improved performance of iterator of values of Utf8Array and BinaryArray (3-4x) Improved performance of iterator of Utf8Array and BinaryArray (3-4x) Sep 19, 2021
@codecov
Copy link

codecov bot commented Sep 19, 2021

Codecov Report

Merging #427 (4ea7607) into main (24f6194) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   80.78%   80.78%           
=======================================
  Files         372      372           
  Lines       22645    22639    -6     
=======================================
- Hits        18294    18290    -4     
+ Misses       4351     4349    -2     
Impacted Files Coverage Δ
src/array/binary/mod.rs 79.68% <100.00%> (-1.48%) ⬇️
src/array/specification.rs 81.25% <100.00%> (+11.25%) ⬆️
src/array/utf8/mod.rs 80.59% <100.00%> (-0.84%) ⬇️
src/array/utf8/mutable.rs 89.91% <100.00%> (ø)
tests/it/array/binary/mod.rs 95.71% <0.00%> (-2.86%) ⬇️
src/types/mod.rs 29.20% <0.00%> (+0.88%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24f6194...4ea7607. Read the comment docs.

@jorgecarleitao
Copy link
Owner Author

Good catch, @Dandandan , I also rebased incorrectly and removed the simdutf8, so great to have reviewed it again :)

@ritchie46
Copy link
Collaborator

Very nice perf. improvement! Could you explain what you did? I've been browsing a bit, but I can't find it yet. Is it this line?

let slice = self.values.get_unchecked(start..end);

Copy link
Owner Author

@jorgecarleitao jorgecarleitao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the main changes, @ritchie46


let slice = &self.values.as_slice()[offset..offset + length];
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line @ritchie46


&self.values[offset..offset + length]
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line @ritchie46


&self.values[offset..offset + length]
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line

let length = (offset_1 - offset).to_usize();
let offset = offset.to_usize();

let slice = &self.values.as_slice()[offset..offset + length];
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this

@jorgecarleitao jorgecarleitao merged commit fa5d9ea into main Sep 20, 2021
@jorgecarleitao jorgecarleitao deleted the fast_iter branch September 20, 2021 19:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement An improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants