feat: priority recompute restructure #279
Conversation
pratikmishra356
force-pushed
the
priority-value-restructure
branch
9 times, most recently
from
14c0833
to
40cd334
Compare
| ALTER TABLE public.dimensions | ||
| add column position integer DEFAULT 1 NOT NULL; | ||
|
|
||
| ALTER TABLE public.contexts |
There was a problem hiding this comment.
Can we write a query to copy priority to weightage
There was a problem hiding this comment.
@Datron we can write , but we will anyway have to re populate dimension position and then recompute
migration approach: #277 (comment)
| @@ -0,0 +1,8 @@ | |||
| -- Your SQL goes here | |||
| ALTER TABLE public.dimensions | |||
There was a problem hiding this comment.
Can you backfill this based on priority? Can be computed from priority
There was a problem hiding this comment.
we can but , serial order is being not followed in all tenants and their values does not need be of 2's power
| @@ -11,6 +11,7 @@ actix-http = "3.3.1" | |||
| actix-web = { workspace = true } | |||
| anyhow = { workspace = true } | |||
| base64 = { workspace = true } | |||
| bigdecimal = { version = "0.3.1" , features= ["serde"]} | |||
There was a problem hiding this comment.
Yes @Datron we need to calculate higher value
| @@ -38,6 +42,12 @@ use crate::db::{ | |||
| }, | |||
| }; | |||
|
|
|||
| pub struct DimensionData { | |||
| @@ -38,6 +42,12 @@ use crate::db::{ | |||
| }, | |||
| }; | |||
|
|
|||
| pub struct DimensionData { | |||
| pub schema: JSONSchema, | |||
| pub priority: i32, | |||
There was a problem hiding this comment.
| pub priority: i32, | |
| pub weight: i32, |
There was a problem hiding this comment.
@Datron it will be priority only,
I refactored a bit here , changed from tuples to struct
| .get(dimension.as_str()) | ||
| .map(|x| x.position) | ||
| .ok_or_else(|| { | ||
| let msg = String::from("Dimension not found in Dimension schema map"); |
There was a problem hiding this comment.
Can we also log the name of the dimension we couldn't parse?
| add column position integer DEFAULT 1 NOT NULL; | ||
|
|
||
| ALTER TABLE public.contexts | ||
| add column weightage numeric(1000,0) DEFAULT 1 NOT NULL; |
There was a problem hiding this comment.
Should we call this weight?
There was a problem hiding this comment.
we can , lets check with team also
| @@ -527,7 +529,8 @@ async fn reduce_config( | |||
| .and_then(|value| value.to_str().ok().and_then(|s| s.parse::<bool>().ok())) | |||
| .unwrap_or(false); | |||
|
|
|||
| let dimensions_schema_map = get_all_dimension_schema_map(&mut conn)?; | |||
| let dimensions_vec = get_dimension_data(&mut conn)?; | |||
There was a problem hiding this comment.
Should get_dimension_data return a map?
There was a problem hiding this comment.
No , it does not need to be , i have separated it into 2 function and get_dimension_data_map this function returns only values which are needed
| pub old_weightage: BigDecimal, | ||
| pub new_weightage: BigDecimal, |
There was a problem hiding this comment.
| pub old_weightage: BigDecimal, | |
| pub new_weightage: BigDecimal, | |
| pub old_weight: BigDecimal, | |
| pub new_weight: BigDecimal, |
| impl Position { | ||
| fn validate_data(position_val: Option<i32>) -> Result<Self, String> { | ||
| if let Some(val) = position_val { | ||
| if val < 0 { |
There was a problem hiding this comment.
We should allow 0 right? 2^0 is 1
There was a problem hiding this comment.
yes we are allowing 0
pratikmishra356
force-pushed
the
priority-value-restructure
branch
3 times, most recently
from
286c0d0
to
ec03437
Compare
pratikmishra356
force-pushed
the
priority-value-restructure
branch
from
6307e54
to
94a06eb
Compare
pratikmishra356
force-pushed
the
priority-value-restructure
branch
from
94a06eb
to
a7fba91
Compare
Problem
User has to declare this exponential value, as value is exponential it has limitation on the total number of dimensions that can be defined
Solution
Postgresql's Numeric Type Approach
For dimension we can have another column named order which will store order of dimension's specificity ,
the higher order is means more specific
For context , we will have another column named weightage , where we will calculate the value just like we do now
it will summation of (2^ dimension's order value)
since context's weightage value can be too large , so we will use postgresql Numeric type
CREATE TABLE large_numbers (
id SERIAL PRIMARY KEY,
big_value NUMERIC
);
Rust/ Diesel supports BigDecimal type which is compatible with Numeric type
Environment variable changes
What ENVs need to be added or changed
Pre-deployment activity
Things needed to be done before deploying this change (if any)
Post-deployment activity
link for migration/backward compatibility: #277 (comment)
API changes
Possible Issues in the future
Describe any possible issues that could occur because of this change