Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Product Analytics): Introducing In-App Analytics Beta #2499

Merged
merged 35 commits into from
May 11, 2021

Conversation

jjoyce0510
Copy link
Collaborator

@jjoyce0510 jjoyce0510 commented May 5, 2021

Description

This PR introduces the first version of DataHub Product Analytics, it was the result of a design sprint led by the wonderful Maggie Hays @ Spothero!

With this PR, we are happy to introduce a set of analytics that can be found under the "Analytics [beta]" tab at the top of the home page. Currently these include some product analytics and some metadata analytics.

Product Analytics:

  • Weekly Active Users
  • Searches in the past week
  • Top search terms
  • Actions performed by entity type (writes)
  • Entity Details Section View views (past week)
  • Top viewed datasets (will become top viewed entities)

Metadata Analytics:

  • Number of Charts, dashboards, datasets, data tasks, data pipelines indexed by DataHub.

Changes

  1. Built on top of feat(React): React App Analytics (WIP) #2435 (superseded by this PR) to send collected frontend metrics to datahub-frontend server
  2. Adding collection endpoint to datahub-frontend which emits to a newly provisioned Kafka topic (DataHubUsageEvents_v1)
  3. Adding a DataHubUsageEvents consumer to listen to the product events, write into a new ElasticSearch index
  4. Extended the default GMS graphQL schema in datahub-frontend to include 2 new Queries: getAnalyticsCharts and getHighlights.
  5. Added an AnalyticsService to datahub-frontend to issue analytics queries to ElasticSearch & format responses
  6. Added an analytics page to the React app responsible for fetching / displaying highlights & charts

This is a first move on the analytics front, there is still much to do! Please let us know if you'd like to see any specific metrics in the dashboard :)

What it looks like
Screen Shot 2021-05-04 at 11 54 32 PM
Screen Shot 2021-05-04 at 11 54 39 PM

**

Acknowledgements

Big thanks to Dexter, Gabe, Shirshanka, Harshal & Maggie for the contributions!

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@@ -1,2 +1,3 @@
PUBLIC_URL=/assets
REACT_APP_THEME_CONFIG=theme_light.config.json
SKIP_PREFLIGHT_CHECK=true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended?

datahub-frontend/play.gradle Outdated Show resolved Hide resolved
datahub-frontend/build.gradle Show resolved Hide resolved
docker/datahub-frontend/Dockerfile Outdated Show resolved Hide resolved
docker/datahub-mce-consumer/env/docker.env Outdated Show resolved Hide resolved
docker/docker-compose.dev.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the reviews @dexter-mh-lee @hsheth2

@shirshanka shirshanka merged commit 29832e5 into datahub-project:master May 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants