Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add rolling window support to 'Big Number with Trendline' viz #9107

Merged
merged 2 commits into from
Mar 10, 2020

Conversation

mistercrunch
Copy link
Member

@mistercrunch mistercrunch commented Feb 10, 2020

CATEGORY

SUMMARY

I was working on a simple dashboard and found myself needing to add support for rolling windows to the 'Big Number with Trendline' viz.

  • refactoring out apply_rolling
  • add unit test
  • some minor label changes

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Screen Shot 2020-02-10 at 8 23 57 AM

TEST PLAN

Added a unit test

REVIEWERS

Copy link
Member

@willbarrett willbarrett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor things.

tests/viz_tests.py Outdated Show resolved Hide resolved
tests/viz_tests.py Show resolved Hide resolved
@mistercrunch mistercrunch force-pushed the rolling_big_number branch 3 times, most recently from 5110b5b to af51356 Compare February 26, 2020 03:02
@mistercrunch
Copy link
Member Author

Comments have been addressed

@codecov-io
Copy link

codecov-io commented Feb 26, 2020

Codecov Report

Merging #9107 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #9107      +/-   ##
=========================================
- Coverage   58.92%   58.9%   -0.03%     
=========================================
  Files         373     373              
  Lines       12016   12026      +10     
  Branches     2948    2953       +5     
=========================================
+ Hits         7081    7084       +3     
- Misses       4756    4763       +7     
  Partials      179     179
Impacted Files Coverage Δ
...et-frontend/src/explore/controlPanels/sections.jsx 100% <ø> (ø) ⬆️
superset-frontend/src/explore/controls.jsx 40.14% <ø> (ø) ⬆️
...t-frontend/src/dashboard/actions/dashboardState.js 30.66% <0%> (-1.06%) ⬇️
superset-frontend/src/chart/Chart.jsx 10.41% <0%> (ø) ⬆️
...erset-frontend/src/dashboard/components/Header.jsx 41.79% <0%> (ø) ⬆️
...erset-frontend/src/datasource/DatasourceEditor.jsx 61.25% <0%> (ø) ⬆️
...d/src/dashboard/components/SliceHeaderControls.jsx 11.62% <0%> (ø) ⬆️
.../src/dashboard/components/gridComponents/Chart.jsx 67.41% <0%> (ø) ⬆️
...-frontend/src/dashboard/components/SliceHeader.jsx 24% <0%> (ø) ⬆️
...nd/src/dashboard/containers/DashboardComponent.jsx 92% <0%> (ø) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89109a1...fe24126. Read the comment docs.

@mistercrunch
Copy link
Member Author

@willbarrett can you take another look?

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments addressing the current structure of the examples code, not specifically this PR. LGTM, but would be a good idea to do some refactoring to the examples code at some point (not necessarily now).

@@ -97,31 +97,32 @@ def load_world_bank_health_n_pop(
db.session.commit()
tbl.fetch_metadata()

metric = "sum__SP_POP_TOTL"
metrics = ["sum__SP_POP_TOTL"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more a general comment relating to the current structure of examples, not specifically this PR. But given the fact that metrics is already defined above (line 80), it might be a good idea to disambiguate here. For example default_metrics, total_population_metrics or similar.

Another option, which I personally would prefer, would be to remove the legacy metrics above, and replace both metric and metrics here with adhoc metrics. Something like

total_population_metric = {
    ...
}

and later on where a metrics list is expected, just passing a [total_population_metric] to highlight that it's a single value list.

Comment on lines +102 to +112
secondary_metric = {
"aggregate": "SUM",
"column": {
"column_name": "SP_RUR_TOTL",
"optionName": "_col_SP_RUR_TOTL",
"type": "DOUBLE",
},
"expressionType": "SIMPLE",
"hasCustomLabel": True,
"label": "Rural Population",
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For readability later on in the code, calling this rural_population_metric or similar would be preferable.

Comment on lines +181 to +199
def apply_rolling(self, df):
fd = self.form_data
rolling_type = fd.get("rolling_type")
rolling_periods = int(fd.get("rolling_periods") or 0)
min_periods = int(fd.get("min_periods") or 0)

if rolling_type in ("mean", "std", "sum") and rolling_periods:
kwargs = dict(window=rolling_periods, min_periods=min_periods)
if rolling_type == "mean":
df = df.rolling(**kwargs).mean()
elif rolling_type == "std":
df = df.rolling(**kwargs).std()
elif rolling_type == "sum":
df = df.rolling(**kwargs).sum()
elif rolling_type == "cumsum":
df = df.cumsum()
if min_periods:
df = df[min_periods:]
return df
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a small bug during testing.

),
),
Slice(
slice_name="Genders",
viz_type="pie",
datasource_type="table",
datasource_id=tbl.id,
params=get_slice_json(defaults, viz_type="pie", groupby=["gender"]),
params=get_slice_json(
defaults, viz_type="pie", groupby=["gender"], metrics=metrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During testing I noticed that the Gender chart wasn't rendering correctly:
image
As the pie chart wants the singular metric, you need to change metrics=metrics to metric=metric here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, thanks for deep testing this

@mistercrunch
Copy link
Member Author

@villebro thank you for the review, addressed your comment!

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mistercrunch mistercrunch merged commit c04d616 into apache:master Mar 10, 2020
@mistercrunch mistercrunch deleted the rolling_big_number branch March 10, 2020 17:19
@villebro villebro added the v0.36 label Mar 10, 2020
etr2460 pushed a commit to airbnb/superset-fork that referenced this pull request Mar 16, 2020
etr2460 pushed a commit that referenced this pull request Mar 16, 2020
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.36.0 labels Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M v0.36 🚢 0.36.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants