feat: adding frontend and backend feature flags #817

fourtyplustwo · 2023-10-18T12:47:04Z

Feature or Bugfix

Feature

Detail

Adding frontend support for all feature flags defined in config.json with a new util method isFeatureEnabled
Adding a new flag preview_data in the datasets module to control whether previewing data is allowed
Adding a new flag glue_crawler in the datasets module to control whether running glue crawler is allowed
Updating environment features to be hidden or visible based on whether the module is active. Adding a new util isAnyFeatureModuleEnabled to check whether to render the entire feature box.

Relates

N/A

Security

Not relevant

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nd flags

fourtyplustwo · 2023-10-18T16:53:10Z

frontend/src/utils/helpers/moduleUtils.js

@@ -33,6 +33,15 @@ function isModuleEnabled(module) {
  return getModuleActiveStatus(module);
 }

+function isAnyFeatureModuleEnabled() {


I'm a little unsure about this method.. We basically have no nice way to tell if a module is a feature module or not. If we want to avoid this hardcoded if.. We could add a flag into the config.json that a module is considered a feature module...

Or perhaps make a new component like EnvironmentFeatures.js but with Toggles? And loop through the array? Let me know what you prefer.

To this issue we also have the challenge of managing dependencies. At the moment they are hardcoded in the isModuleEnabled function. Maybe we can think of a generic way of defining some module properties in the frontend as we do in the __init__ files for backend. I spoke about this with Mo as one of the future improvements for frontend. For the moment i think it is good

fourtyplustwo · 2023-10-19T09:49:08Z

@dlpzx @noah-paige can you take a look? thoughts on the comment?

frontend/src/modules/Environments/components/EnvironmentTeams.js

backend/dataall/modules/datasets/api/dataset/resolvers.py

backend/dataall/modules/datasets/api/table/resolvers.py

noah-paige · 2023-10-23T20:56:06Z

I tested in an AWS Deployment with the feature flags both on and off and is working as expected

One other thing I noted is that we use default useState('preview') in TableView.js to default to the Preview Tab. If that tab is disabled then when as a dataset owner navigating to the path /console/datasets/table/${table.tableUri} it will first have nothing but the tab options showing which I user will then have to select (see screenshot)

Alternatively, we can make overview the default Table tab opened (i.e. useState('overview')) - but trying to think through if this would cause any issues for a Requestor since when a requestor navigates to the catalog and clicks on a table to inspect from the catalog they also get navigated to /console/datasets/table/${table.tableUri}

cc: @dlpzx @zsaltys

fourtyplustwo · 2023-10-24T17:25:13Z

frontend/src/modules/Tables/views/TableView.js

@@ -157,7 +163,7 @@ const TableView = () => {
  const client = useClient();
  const navigate = useNavigate();
  const [table, setTable] = useState({});
-  const [currentTab, setCurrentTab] = useState('preview');
+  const [currentTab, setCurrentTab] = useState(tabs[0].value);


@noah-paige Actually this was a mistake on my side... When contributing back Mohit's changes I forgot to pick this up from our branch. This will now always default to the first tab on the UI.

noah-paige

Deployed these changes to AWS and tested with all the feature flags enabled and disabled - all working as expected

Also tested disabling a module and it is removed from the Env Feature Management card appropriately

noah-paige · 2023-10-24T20:41:06Z

Thank you @zsaltys for this PR - I will merge this through to v2m1m0 tomorrow barring any additional review comments found by @dlpzx

@dlpzx

### Feature or Bugfix - Feature - Bugfix - Refactoring ### Detail #### Features * Limit pivot role S3 permissions by @dlpzx in #780 * Limit pivot role KMS permissions by @dlpzx in #830 * Add configurable session timeout to IDP by @manjulaK in #786 * Allow to submit a share when you are both an approver and a requester by @zsaltys in #793 * Redirect upon creating a share request by @zsaltys in #799 * Handle Pre-filtering of tables by @anushka-singh in #811 * Email Notification on Share Workflow - Issue - 734 by @TejasRGitHub in #818 * Refactor notifications from core to modules by @dlpzx in #822 * Add frontend and backend feature flags by @zsaltys in #817 * Make hosted_zone_id optional by @lorchda in #812 #### Fixes * Add Additional Error Messages for KMS Key lookup on imported dataset by @noah-paige in #748 * Handle Environment Import of IAM service roles by @noah-paige in #749 * Build Compliant Names for Opensearch Resources by @noah-paige in #750 * Update Lambda runtime by @nikpodsh in #782 * Ensure valid environments for share request and other objects creation by @dlpzx in #781 * Fix shell true semgrep by @dlpzx in #760 * Add condition when there are no public subnets by @lorchda in #794 * Remove unused variable by @zsaltys in #815 * Check other share exists before clean up by @noah-paige in #769 ### Relates - v2.1.0 minor release ## New Contributors * @manjulaK made their first contribution in #786 * @zsaltys made their first contribution in #793 * @anushka-singh made their first contribution in #811 * @TejasRGitHub made their first contribution in #818 ### Security Please answer the questions below briefly where applicable, or write `N/A`. Based on [OWASP 10](https://owasp.org/Top10/en/). - Does this PR introduce or modify any input fields or queries - this includes fetching data from storage outside the application (e.g. a database, an S3 bucket)? - Is the input sanitized? - What precautions are you taking before deserializing the data you consume? - Is injection prevented by parametrizing queries? - Have you ensured no `eval` or similar functions are used? - Does this PR introduce any functionality or component that requires authorization? - How have you ensured it respects the existing AuthN/AuthZ mechanisms? - Are you logging failed auth attempts? - Are you using or adding any cryptographic features? - Do you use a standard proven implementations? - Are the used keys controlled by the customer? Where are they stored? - Are you introducing any new policies/roles/users? - Have you used the least-privilege principle? How? By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Noah Paige <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: jaidisido <[email protected]> Co-authored-by: mourya-33 <[email protected]> Co-authored-by: nikpodsh <[email protected]> Co-authored-by: MK <[email protected]> Co-authored-by: Zilvinas Saltys <[email protected]> Co-authored-by: Daniel Lorch <[email protected]> Co-authored-by: Anushka Singh <[email protected]> Co-authored-by: trajopadhye <[email protected]>

fourtyplustwo added 2 commits October 18, 2023 13:45

adding frontend feature flags and preview_data and glue_crawler backe…

9617eae

…nd flags

refactoring environment features

5c4e176

fourtyplustwo marked this pull request as draft October 18, 2023 16:20

simplifying isAnyFeatureModuleEnabled method

2c94c92

fourtyplustwo commented Oct 18, 2023

View reviewed changes

fourtyplustwo marked this pull request as ready for review October 19, 2023 09:49

marora90 mentioned this pull request Oct 19, 2023

Disabling UI features within modules #635

Closed

noah-paige reviewed Oct 23, 2023

View reviewed changes

frontend/src/modules/Environments/components/EnvironmentTeams.js Outdated Show resolved Hide resolved

noah-paige reviewed Oct 23, 2023

View reviewed changes

backend/dataall/modules/datasets/api/dataset/resolvers.py Show resolved Hide resolved

noah-paige reviewed Oct 23, 2023

View reviewed changes

backend/dataall/modules/datasets/api/table/resolvers.py Show resolved Hide resolved

fourtyplustwo added 3 commits October 24, 2023 17:35

fixing lint issues

1f1966d

fixing a bug where group remove button is being disabled

e026061

fixing an issue where preview tab is the default

3a4a086

fourtyplustwo closed this Oct 24, 2023

fourtyplustwo reopened this Oct 24, 2023

fourtyplustwo commented Oct 24, 2023

View reviewed changes

noah-paige linked an issue Oct 24, 2023 that may be closed by this pull request

Editable Environment Parameters with All Modules Disabled #807

Closed

noah-paige approved these changes Oct 24, 2023

View reviewed changes

noah-paige merged commit 48c32e5 into data-dot-all:v2m1m0 Oct 25, 2023

dlpzx modified the milestone: v2.1.0 Oct 30, 2023

dlpzx mentioned this pull request Oct 30, 2023

v2.1.0 features #840

Merged

noah-paige mentioned this pull request Jan 10, 2024

Generic way to toggle data.all features #473

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: adding frontend and backend feature flags #817

feat: adding frontend and backend feature flags #817

fourtyplustwo commented Oct 18, 2023 •

edited

Loading

fourtyplustwo Oct 18, 2023

dlpzx Oct 25, 2023

fourtyplustwo commented Oct 19, 2023

noah-paige commented Oct 23, 2023 •

edited

Loading

fourtyplustwo Oct 24, 2023

noah-paige left a comment

noah-paige commented Oct 24, 2023

feat: adding frontend and backend feature flags #817

feat: adding frontend and backend feature flags #817

Conversation

fourtyplustwo commented Oct 18, 2023 • edited Loading

Feature or Bugfix

Detail

Relates

Security

fourtyplustwo Oct 18, 2023

Choose a reason for hiding this comment

dlpzx Oct 25, 2023

Choose a reason for hiding this comment

fourtyplustwo commented Oct 19, 2023

noah-paige commented Oct 23, 2023 • edited Loading

fourtyplustwo Oct 24, 2023

Choose a reason for hiding this comment

noah-paige left a comment

Choose a reason for hiding this comment

noah-paige commented Oct 24, 2023

fourtyplustwo commented Oct 18, 2023 •

edited

Loading

noah-paige commented Oct 23, 2023 •

edited

Loading