Ensure even distribution of versions across interval/range #54
Conversation
There was a problem hiding this comment.
As I mentioned in #53, sorry for the long wait. 🙏
The approach makes a lot of sense to me!
identical percentiles for versions generated by the same pipeline across different ranges. So if a pipeline has multiple resources defined that span a 4 hour range or are on a 4 hour interval, all of those resources will generate the exact same offset for each resource instance (🤔 is this is a problem?)
It is indeed a problem, but not one we need to solve today, and probably not a huge deal anyway. In the future, I plan to address this by modeling time as a var source, rather than a resource, which will allow for per-job intervals. For now, per-pipeline is as good as it gets.
All in all, I think this looks good, but I do have some somewhat minor feedback before merging.
owenfarrell
force-pushed
the
offset
branch
3 times, most recently
from
f2912e2
to
8dede7b
Compare
offset_test.go
Outdated
| reference = time.Date(2012, time.November, 4, 13, now.Minute(), now.Second(), now.Nanosecond(), loc) | ||
| }) | ||
|
|
||
| dayDuration = 25 * time.Hour |
There was a problem hiding this comment.
These "naked" (out of BeforeEach) assignments to a shared var here are a little spooky since they require somewhat intimate knowledge of how things are evaluated in the test framework, while looking like the other regular old vars which are set during test setup/execution.
Maybe this could be passed as an argument to RunIntervalAndOrRangeTests instead? 🤔
There was a problem hiding this comment.
Honestly, I don't think there's a good reason for this assignment to live outside of a BeforeEach setup. Similarly, I moved the rangeDuration assignment in to a BeforeEach.
There was a problem hiding this comment.
Much better - thanks! Gonna go ahead and merge.
Signed-off-by: Owen Farrell <[email protected]>
This change implements a request by @vito to generate unique versions per pipeline.
This implementation hashes a combination of the
BUILD_TEAM_NAMEandBUILD_PIPELINE_NAMEenvironmental variables using the FNV-1a algorithm. I chose FNV-1a based as the "best bang for the buck" (best distribution for algorithms readily available in Go).The
Offsetfunction then determines what percentile the hashed value falls in, relative to the maximum possible value, and returns a time that is in the same percentile of the specified range/interval (as calculated by theTimeLordinstance). All math rounds down to the nearest minute to align with the existing precision provided by this resource.I think this approach creates:
I know the common approach to these kinds of requests is to do some bitwise manipulation of inputs. But I took this approach in hopes of implementing a mechanism that is good enough to meet the need while promoting readability and maintainability.
One implication to this percentile approach is that non-standard days (e.g. the start/end of DST) will trigger pipelines in the same order as any other day. If/when days are 23 or 25 hours, the time that each pipeline triggers will adjust slightly (up to ~57 minutes), but each pipeline will have a similar offset. There won't be any wild swings of multiple hours due to a locale-related time zone change.
NOTE: This implementation is intentionally not integrated in to any of the executables and does not address any documentation updates (in order to avoid confusion).