Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(model, graphql, ingest): Dashboard usage #5397

Merged
merged 1 commit into from
Jul 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import com.linkedin.datahub.graphql.generated.CorpUserInfo;
import com.linkedin.datahub.graphql.generated.Dashboard;
import com.linkedin.datahub.graphql.generated.DashboardInfo;
import com.linkedin.datahub.graphql.generated.DashboardUserUsageCounts;
import com.linkedin.datahub.graphql.generated.DataFlow;
import com.linkedin.datahub.graphql.generated.DataJob;
import com.linkedin.datahub.graphql.generated.DataJobInputOutput;
Expand Down Expand Up @@ -81,6 +82,7 @@
import com.linkedin.datahub.graphql.resolvers.config.AppConfigResolver;
import com.linkedin.datahub.graphql.resolvers.container.ContainerEntitiesResolver;
import com.linkedin.datahub.graphql.resolvers.container.ParentContainersResolver;
import com.linkedin.datahub.graphql.resolvers.dashboard.DashboardUsageStatsResolver;
import com.linkedin.datahub.graphql.resolvers.dataset.DatasetHealthResolver;
import com.linkedin.datahub.graphql.resolvers.deprecation.UpdateDeprecationResolver;
import com.linkedin.datahub.graphql.resolvers.domain.CreateDomainResolver;
Expand Down Expand Up @@ -1015,13 +1017,19 @@ private void configureDashboardResolvers(final RuntimeWiring.Builder builder) {
})
)
.dataFetcher("parentContainers", new ParentContainersResolver(entityClient))
.dataFetcher("usageStats", new DashboardUsageStatsResolver(timeseriesAspectService))
);
builder.type("DashboardInfo", typeWiring -> typeWiring
.dataFetcher("charts", new LoadableTypeBatchResolver<>(chartType,
(env) -> ((DashboardInfo) env.getSource()).getCharts().stream()
.map(Chart::getUrn)
.collect(Collectors.toList())))
);
builder.type("DashboardUserUsageCounts", typeWiring -> typeWiring
.dataFetcher("user", new LoadableTypeResolver<>(
corpUserType,
(env) -> ((DashboardUserUsageCounts) env.getSource()).getUser().getUrn()))
);
}

/**
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package com.linkedin.datahub.graphql.types.dashboard.mappers;

import com.linkedin.datahub.graphql.generated.DashboardUsageMetrics;
import com.linkedin.datahub.graphql.types.mappers.TimeSeriesAspectMapper;
import com.linkedin.metadata.aspect.EnvelopedAspect;
import com.linkedin.metadata.utils.GenericRecordUtils;
import javax.annotation.Nonnull;


public class DashboardUsageMetricMapper implements TimeSeriesAspectMapper<DashboardUsageMetrics> {

public static final DashboardUsageMetricMapper INSTANCE = new DashboardUsageMetricMapper();

public static DashboardUsageMetrics map(@Nonnull final EnvelopedAspect envelopedAspect) {
return INSTANCE.apply(envelopedAspect);
}

@Override
public DashboardUsageMetrics apply(EnvelopedAspect envelopedAspect) {
com.linkedin.dashboard.DashboardUsageStatistics gmsDashboardUsageStatistics =
GenericRecordUtils.deserializeAspect(envelopedAspect.getAspect().getValue(),
envelopedAspect.getAspect().getContentType(), com.linkedin.dashboard.DashboardUsageStatistics.class);

final com.linkedin.datahub.graphql.generated.DashboardUsageMetrics dashboardUsageMetrics =
new com.linkedin.datahub.graphql.generated.DashboardUsageMetrics();
dashboardUsageMetrics.setLastViewed(gmsDashboardUsageStatistics.getLastViewedAt());
dashboardUsageMetrics.setViewsCount(gmsDashboardUsageStatistics.getViewsCount());
dashboardUsageMetrics.setExecutionsCount(gmsDashboardUsageStatistics.getExecutionsCount());
dashboardUsageMetrics.setFavoritesCount(gmsDashboardUsageStatistics.getFavoritesCount());
dashboardUsageMetrics.setTimestampMillis(gmsDashboardUsageStatistics.getTimestampMillis());

return dashboardUsageMetrics;
}
}
160 changes: 159 additions & 1 deletion datahub-graphql-core/src/main/resources/entity.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -4121,6 +4121,12 @@ type Dashboard implements EntityWithRelationships & Entity {
"""
lineage(input: LineageInput!): EntityLineageResult


"""
Experimental (Subject to breaking change) -- Statistics about how this Dashboard is used
"""
usageStats(startTimeMillis: Long, endTimeMillis: Long, limit: Int): DashboardUsageQueryResult
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Questions: Don't you need an urn passed as input to know which dashboard to get usageStats for?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Usually we wrap all inputs into a single object, in this case UsageStatsInput and the output to be consistent should be called DashboardUsageStatsResult

Copy link
Collaborator Author

@mayurinehate mayurinehate Jul 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we don't need the resource urn as graphql input here. We can get that from environment get source.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you point to the place in the code please? It is usually not a good practice to have inconsistent APIs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's here


"""
Deprecated, use properties field instead
Additional read only information about the dashboard
Expand Down Expand Up @@ -5247,6 +5253,158 @@ type FieldUsageCounts {
count: Int
}

"""
Information about individual user usage of a Dashboard
"""
type DashboardUserUsageCounts {
"""
The user of the Dashboard
"""
user: CorpUser

"""
number of times dashboard has been viewed by the user
"""
viewsCount: Int

"""
number of dashboard executions by the user
"""
executionsCount: Int

"""
Normalized numeric metric representing user's dashboard usage
Higher value represents more usage
"""
usageCount: Int
}

"""
The result of a dashboard usage query
"""
type DashboardUsageQueryResult {
"""
A set of relevant time windows for use in displaying usage statistics
"""
buckets: [DashboardUsageAggregation]

"""
A set of rolled up aggregations about the dashboard usage
"""
aggregations: DashboardUsageQueryResultAggregations

"""
A set of absolute dashboard usage metrics
"""
metrics: [DashboardUsageMetrics!]

}

"""
A set of rolled up aggregations about the Dashboard usage
"""
type DashboardUsageQueryResultAggregations {
"""
The count of unique Dashboard users within the queried time range
"""
uniqueUserCount: Int

"""
The specific per user usage counts within the queried time range
"""
users: [DashboardUserUsageCounts]

"""
The total number of dashboard views within the queried time range
"""
viewsCount: Int

"""
The total number of dashboard executions within the queried time range
"""
executionsCount: Int

}


"""
A set of absolute dashboard usage metrics
"""
type DashboardUsageMetrics implements TimeSeriesAspect {
"""
The time at which the metrics were reported
"""
timestampMillis: Long!

"""
The total number of times dashboard has been favorited
FIXME: Qualifies as Popularity Metric rather than Usage Metric?
"""
favoritesCount: Int

"""
The total number of dashboard views
"""
viewsCount: Int

"""
The total number of dashboard execution
"""
executionsCount: Int

"""
The time when this dashboard was last viewed
"""
lastViewed: Long

}

"""
An aggregation of Dashboard usage statistics
"""
type DashboardUsageAggregation {
"""
The time window start time
"""
bucket: Long

"""
The time window span
"""
duration: WindowDuration

"""
The resource urn associated with the usage information, eg a Dashboard urn
"""
resource: String

"""
The rolled up usage metrics
"""
metrics: DashboardUsageAggregationMetrics
}

"""
Rolled up metrics about Dashboard usage over time
"""
type DashboardUsageAggregationMetrics {
"""
The unique number of dashboard users within the time range
"""
uniqueUserCount: Int

"""
The total number of dashboard views within the time range
"""
viewsCount: Int

"""
The total number of dashboard executions within the time range
"""
executionsCount: Int

}

"""
The duration of a fixed window of time
"""
Expand All @@ -5273,7 +5431,7 @@ enum WindowDuration {
}

"""
A time range used in fetching Dataset Usage statistics
A time range used in fetching Usage statistics
"""
enum TimeRange {
"""
Expand Down
Loading