Skip to content

Commit

Permalink
Time Partition Granularity and Model Cards (#105)
Browse files Browse the repository at this point in the history
## Overview
- `artifacts.proto`
  - add model card
  - add granularity to search request
- `artifact_id.proto`
  - define granularity and add to time partition
- Cherry picked commits from #4959

## Test Plan
- Deploy to staging and validate artifacts workflows from the artifact demo repo
- Also validate workflows in our p0 functional tests to insure no interference there

## Rollout Plan (if applicable)
Deploy out to staging tenants and run through above tests, merge in and roll to canary then validate tests there, out to prod

## Upstream Changes
Should this change be upstreamed to OSS (flyteorg/flyte)? If so, please check this box for auditing. Note, this is the responsibility of each developer. See [this guide](https://unionai.atlassian.net/wiki/spaces/ENG/pages/447610883/Flyte+-+Union+Cloud+Development+Runbook/#When-are-versions-updated%3F).
- [ ] To be upstreamed

## Checklist
* [ ] Added tests
* [ ] Ran a deploy dry run and shared the terraform plan
* [ ] Added logging and metrics
* [ ] Updated [dashboards](https://unionai.grafana.net/dashboards) and [alerts](https://unionai.grafana.net/alerting/list)
* [ ] Updated documentation
  • Loading branch information
squiishyy authored Mar 7, 2024
1 parent a59bf9d commit 42caed0
Show file tree
Hide file tree
Showing 22 changed files with 2,948 additions and 883 deletions.
74 changes: 57 additions & 17 deletions flyteidl/clients/go/assets/admin.swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -8764,20 +8764,6 @@
],
"default": "SINGLE"
},
"ComparisonExpressionOperator": {
"type": "string",
"enum": [
"EQ",
"NEQ",
"GT",
"GTE",
"LT",
"LTE"
],
"default": "EQ",
"description": "- GT: Greater Than\n - LT: Less Than",
"title": "Binary Operator for each expression"
},
"ConjunctionExpressionLogicalOperator": {
"type": "string",
"enum": [
Expand Down Expand Up @@ -11048,8 +11034,8 @@
"bind_to_time_partition": {
"type": "boolean"
},
"transform": {
"type": "string",
"time_transform": {
"$ref": "#/definitions/coreTimeTransform",
"title": "This is only relevant in the time partition case"
}
},
Expand Down Expand Up @@ -11317,7 +11303,7 @@
"type": "object",
"properties": {
"operator": {
"$ref": "#/definitions/ComparisonExpressionOperator"
"$ref": "#/definitions/coreComparisonExpressionOperator"
},
"left_value": {
"$ref": "#/definitions/coreOperand"
Expand All @@ -11328,6 +11314,20 @@
},
"description": "Defines a 2-level tree where the root is a comparison operator and Operands are primitives or known variables.\nEach expression results in a boolean result."
},
"coreComparisonExpressionOperator": {
"type": "string",
"enum": [
"EQ",
"NEQ",
"GT",
"GTE",
"LT",
"LTE"
],
"default": "EQ",
"description": "- GT: Greater Than\n - LT: Less Than",
"title": "Binary Operator for each expression"
},
"coreCompiledLaunchPlan": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -11605,6 +11605,18 @@
},
"description": "GateNode refers to the condition that is required for the gate to successfully complete."
},
"coreGranularity": {
"type": "string",
"enum": [
"UNSET",
"MINUTE",
"HOUR",
"DAY",
"MONTH"
],
"default": "UNSET",
"title": "- DAY: default"
},
"coreIOStrategy": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -11772,6 +11784,9 @@
},
"input_binding": {
"$ref": "#/definitions/coreInputBindingData"
},
"runtime_binding": {
"$ref": "#/definitions/coreRuntimeBinding"
}
}
},
Expand Down Expand Up @@ -12255,6 +12270,9 @@
},
"description": "Retry strategy associated with an executable unit."
},
"coreRuntimeBinding": {
"type": "object"
},
"coreRuntimeMetadata": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -12708,6 +12726,20 @@
"properties": {
"value": {
"$ref": "#/definitions/coreLabelValue"
},
"granularity": {
"$ref": "#/definitions/coreGranularity"
}
}
},
"coreTimeTransform": {
"type": "object",
"properties": {
"transform": {
"type": "string"
},
"op": {
"$ref": "#/definitions/flyteidlcoreOperator"
}
}
},
Expand Down Expand Up @@ -13401,6 +13433,14 @@
},
"description": "A generic key value pair."
},
"flyteidlcoreOperator": {
"type": "string",
"enum": [
"MINUS",
"PLUS"
],
"default": "MINUS"
},
"flyteidlcoreSchema": {
"type": "object",
"properties": {
Expand Down
11 changes: 10 additions & 1 deletion flyteidl/gen/pb-es/flyteidl/artifacts/artifacts_connect.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

183 changes: 182 additions & 1 deletion flyteidl/gen/pb-es/flyteidl/artifacts/artifacts_pb.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 42caed0

Please sign in to comment.