Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Long DELETE project call #3360

Merged
merged 19 commits into from
Feb 2, 2024
Merged

Conversation

khvn26
Copy link
Member

@khvn26 khvn26 commented Jan 31, 2024

Thanks for submitting a PR! Please check the boxes below:

  • I have run pre-commit to check linting
  • I have added information to docs/ if required so people know about the feature!
  • I have filled in the "Changes" section below?
  • I have filled in the "How did you test this code" section below?
  • I have used a Conventional Commit title for this Pull Request

Changes

Closes #3302.

This introduces the following changes:

  1. Environment, Feature and Segment deletions are decoupled from project deletion.
  2. When project deletion is requested, environments, features and segments are deleted asynchronously.
  3. Fixed test_recurring_task_run_should_execute_first_run_at failing when one_hour_from_now parameter is next day.

How did you test this code?

Added unit tests; created and deleted large projects locally.

Copy link

vercel bot commented Jan 31, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 2, 2024 10:22am
flagsmith-frontend-preview ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 2, 2024 10:22am
flagsmith-frontend-staging ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 2, 2024 10:22am

Copy link
Contributor

github-actions bot commented Jan 31, 2024

Uffizzi Preview deployment-45519 was deleted.

@khvn26 khvn26 force-pushed the fix/long-large-project-deletions branch from 8afd597 to 11f24e0 Compare January 31, 2024 21:41
@khvn26 khvn26 force-pushed the fix/long-large-project-deletions branch from ce0b69a to c9d5564 Compare January 31, 2024 22:40
@khvn26 khvn26 force-pushed the fix/long-large-project-deletions branch from c9d5564 to 5188323 Compare January 31, 2024 23:10
@khvn26 khvn26 force-pushed the fix/long-large-project-deletions branch from 5188323 to 6eebc02 Compare January 31, 2024 23:23
@matthewelwell
Copy link
Contributor

The code looks good overall. I tried to find what the bug is with one of your failing tests and I couldn't figure it out with the review but test_audit_log_created_when_segment_created looks like a suspicious failure given the changes in this repo.

Yeah, it was a tricky one. The issue was here. The signals weren't getting added, so no AuditLog record was being created.

@github-actions github-actions bot removed the github label Feb 2, 2024
Comment on lines +33 to +39
environment, project = history_instance.instance.get_environment_and_project()
if project != history_instance.instance and (
(project and project.deleted_at)
or (environment and environment.project.deleted_at)
):
# don't trigger audit log records in deleted projects
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is arguably unrelated to this PR but I think it's a good improvement here to prevent unnecessary data in the AuditLog table.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's sorta unrelated but it's a good addition.

Comment on lines +36 to +39
from environments.tasks import delete_environment
from features.tasks import delete_feature
from projects.models import Project
from segments.tasks import delete_segment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, yeah, I tried to resolve this but I ended up just diving into a hell hole of circular dependencies. It annoyingly made testing a little harder because I can't mock the imports, but I do feel like it's not a major issue for tasks to have local imports given their place in the stack.

@@ -0,0 +1,20 @@
# Generated by Django 3.2.23 on 2024-02-01 12:12

from django.db import migrations, models
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the sqlmigrate output for these migrations

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❯ python manage.py sqlmigrate features 0063
BEGIN;
--
-- Alter field project on feature
--
COMMIT;
❯ python manage.py sqlmigrate features 0064
BEGIN;
--
-- Alter field project on feature
--
--
-- Alter field project on historicalfeature
--
COMMIT;

]

operations = [
migrations.AlterField(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sqlmigrate output:

❯ python manage.py sqlmigrate environments 0034
BEGIN;
--
-- Alter field project on environment
--
COMMIT;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Django migrations for this silly stuff is always a bit of an eye roll

]

operations = [
migrations.AlterField(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sqlmigrate output:

❯ python manage.py sqlmigrate segments 0020
BEGIN;
--
-- Alter field project on segment
--
COMMIT;

@zachaysan
Copy link
Contributor

The code looks good overall. I tried to find what the bug is with one of your failing tests and I couldn't figure it out with the review but test_audit_log_created_when_segment_created looks like a suspicious failure given the changes in this repo.

Yeah, it was a tricky one. The issue was here. The signals weren't getting added, so no AuditLog record was being created.

Well it's great that the test suite caught it.

Copy link
Contributor

@zachaysan zachaysan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Comment on lines +33 to +39
environment, project = history_instance.instance.get_environment_and_project()
if project != history_instance.instance and (
(project and project.deleted_at)
or (environment and environment.project.deleted_at)
):
# don't trigger audit log records in deleted projects
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's sorta unrelated but it's a good addition.

]

operations = [
migrations.AlterField(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Django migrations for this silly stuff is always a bit of an eye roll

@matthewelwell matthewelwell added this pull request to the merge queue Feb 2, 2024
Merged via the queue into main with commit aca0fc5 Feb 2, 2024
21 checks passed
@matthewelwell matthewelwell deleted the fix/long-large-project-deletions branch February 2, 2024 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Issue related to the REST API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can't delete a big project with hundreds of feature flags
5 participants