diff --git a/README.md b/README.md index 16e5ddc9..6cd414c2 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ src="https://raw.githubusercontent.com/afawcett/githubsfdeploy/master/deploy.png"> -Create fast, scalable custom rollups driven by Custom Metadata in your Salesforce org with `Rollup`. As seen on [Replacing DLRS With Custom Rollup](https://www.jamessimone.net/blog/joys-of-apex/replacing-dlrs-with-custom-rollup/)! +Create fast, scalable custom rollups driven by Custom Metadata in your Salesforce org with `Rollup`. As seen on [Replacing DLRS With Custom Rollup](https://www.jamessimone.net/blog/joys-of-apex/replacing-dlrs-with-custom-rollup/)! As of [v1.2.0](https://github.com/jamessimone/apex-rollup/tree/v1.2.0), `Rollup` also offers [the ability to roll directly up from children records to grandparent (or greater!) records](#grandparent-rollups), without needing complex hierarchical intermediate fields. ## Usage @@ -73,6 +73,7 @@ Within the `Rollup__mdt` custom metadata type, add a new record with fields: - `Is Full Record Set` (optional, defaults to `false`) - by default, if the records you are passing in comprise the full set of child items for a given lookup item but none of them "qualify" to be rolled up (either due to the use of the Calc Item Where Clause, Changed Fields On Calc Item, or a custom Evaluator), Rollup aborts early. If you know you have the exhaustive list of records to be used for a given lookup item **and** you stipulate the Full Recalculation Default Number (or String) Value, you can override the existing rollup item's amount by checking off this field - `Order By (First/Last)` (optional) - at present, only valid when FIRST/LAST is used as the Rollup Operation. This is the API name of a text/date/number-based field that you would like to order the calculation items by. **Note** unlike DLRS, this field is _not_ optional on a first/last operation; a validation rule will enforce that you supply a value here, even if the value used is the same as the field you are rolling up on. - `Is Rollup Started From Parent` (optional, defaults to `false`) - if the the records being passed in are the parent records, check this field off. `Rollup` will then go and retrieve the assorted children records before rolling the values up to the parents. +- `Grandparent Relationship Field Path` (optional) - if [you are rolling up to a grandparent (or greater) parent object](#grandparent-rollups), use this field to establish the full relationship name of the field, eg from Opportunity Line Items directly to an Account's Annual Revenue: `Opportunity.Account.AnnualRevenue` would be used here. The field name (after the last period) should match up with what is being used in `Rollup Field On Lookup Object`. For caveats and more information on how to setup rollups looking to use this functionality, please refer to the linked section. You can perform have as many rollups as you'd like per object/trigger — all operations are boxcarred together for optimal efficiency. @@ -96,8 +97,8 @@ When you install `Rollup`, you get two custom metadata types - `Rollup__mdt`, de These are the fields on the `Rollup Control` custom metadata type: - `Max Lookup Rows Before Batching` - if you are rolling up to an object that interacts in many different ways within the system, `Rollup` moves from using a Queueable based system (read: fast and light) to a Batched Apex approach (read: solid, sometimes slow). You can override the default for switching to Batch Apex by lowering the number of rows. Without an `Org_Default` record, this defaults to `3333` -- `Max Lookup Rows For Queueable` - if you haven't selected a Batch Apex override, defaults to `5000` -- `Rollup` lookup field to the `Rollup__mdt` metadata record. Optional. +- `Max Parent Rows Updated At Once` (defaults to 5000) - The maximum number of parent rows that can be updated in a single transaction. Otherwise, Rollup splits the parent items evenly and updates them in separate transactions. If you don't fill out this field (on the Org Defaults or specific Control records), defaults to half of the DML row limit. +- `Rollup` (optional) - lookup field to the `Rollup__mdt` metadata record. - `Should Abort Run` - if done at the `Org_Defaults` level, completely shuts down all rollup operations in the org. Otherwise, can be used on an individual rollup basis to turn on/off. - `Should Run As` - a picklist dictating the preferred method for running rollup operations. Possible values are `Queueable`, `Batchable`, or `Synchronous Rollup`. - `Trigger Or Invocable Name` - If you are using custom Apex, a schedulable, or rolling up by way of the Invocable action and can't use the `Rollup` lookup field. Use the pattern `trigger_fieldOnCalcItem_to_rollupFieldOnTarget_rollup` - for example: 'trigger_opportunity_stagename_to_account_name_rollup' (use lowercase on the field names). If there is a matching Rollup Limit record, those rules will be used. The first part of the string comes from how a rollup has been invoked - either by `trigger`, `invocable`, or `schedule`. A scheduled flow still uses `invocable`! @@ -109,8 +110,6 @@ These are the fields on the `Rollup Control` custom metadata type:
-**Important note for Record Triggered Flows as of 26 Februray 2021**: a patch to the Spring 21 release introduced a breaking change in the way that the Flow engine hands off Invocable variables to Apex. Currently, if you are using a record triggered flow, you have to use `Get Records` using the current record's Id in order to populate the `Records To Rollup` argument properly for both invocables. I will remove this notice when the functionality has been fixed, but there is a bug with simply adding the current record to a collection variable and passing that to the invocable action accordingly. Thank you for your attention to this notice! - I will touch only on Flows given that all indications from Salesforce would lead a person to believe they are the future of the "clicks" part in "clicks versus code": Invoking the `Rollup` process from a Flow, in particular, is a joy; with a Record Triggered Flow, you can do the up-front processing to take in only the records you need, and then dispatch the rollup operation to the `Rollup` invocable: @@ -138,8 +137,9 @@ Here are the arguments necessary to invoke `Rollup` from a Flow / Process Builde - `Is Full Record Set` (optional) - by default, if the records you are passing in comprise the full set of child items for a given lookup item but none of them "qualify" to be rolled up (either due to the use of the Calc Item Where Clause, Changed Fields On Calc Item, or a custom Evaluator), Rollup aborts early. If you know you have the exhaustive list of records to be used for a given lookup item **and** you stipulate the Full Recalculation Default Number (or String) Value, you can override the existing rollup item's amount by toggling this field - `Order By (First/Last)` (optional) - at present, only valid when FIRST/LAST is used as the Rollup Operation. This is the API name of a text/date/number-based field that you would like to order the calculation items by. **Note** unlike DLRS, this field is _not_ optional on a first/last operation; a validation rule will enforce that you supply a value here, even if the value used is the same as the field you are rolling up on. - `Defer Processing` (optional, default `false`) - when checked and set to `{!$GlobalConstant.True}`, you have to call the separate invocable method `Process Deferred Rollups` at the end of your flow. Otherwise, each invocable action kicks off a separate queueable/batch job. **Note** - for extremely large flows calling dozens of rollup operations, it behooves the end user / admin to occasionally call the `Process Deferred Rollups` to separate rollup operations into different jobs. You'll avoid running out of memory by doing so. -- `Is Rollup Started From Parent` (defaults to `{!$GlobalConstant.False}`) - set to `{!$GlobalConstant.True}` if collection being passed in is the parent SObject, and you want to recalculate the defined rollup operation for the passed in parent records. Used in conjunction with `Calc Item Type When Rollup Started From Parent` -- `Calc Item Type When Rollup Started From Parent` - only necessary to provide if `Is Rollup Started From Parent` field is enabled and set to `{!$GlobalConstant.True}`. Normally in this invocable, the calc item type is figured out by examining the passed-in collection - but when the collection is the parent records, we need the SObject API name of the calculation items explicitly defined. +- `Is Rollup Started From Parent` (optional, defaults to `{!$GlobalConstant.False}`) - set to `{!$GlobalConstant.True}` if collection being passed in is the parent SObject, and you want to recalculate the defined rollup operation for the passed in parent records. Used in conjunction with `Calc Item Type When Rollup Started From Parent` +- `Calc Item Type When Rollup Started From Parent` (optional) - only necessary to provide if `Is Rollup Started From Parent` field is enabled and set to `{!$GlobalConstant.True}`. Normally in this invocable, the calc item type is figured out by examining the passed-in collection - but when the collection is the parent records, we need the SObject API name of the calculation items explicitly defined. +- `Grandparent Relationship Field Path` (optional) - if [you are rolling up to a grandparent (or greater) parent object](#grandparent-rollups), use this field to establish the full relationship name of the field, eg from Opportunity Line Items directly to an Account's Annual Revenue: `Opportunity.Account.AnnualRevenue` would be used here. The field name should match up with what is being used in `Rollup Field On Lookup Object`. Please see the caveats in the linked section for more information on how to set up your rollups correctly when using this feature. Here is an example of the base action filled out (not shown, but also important - the assignment of the collection to the `Records to rollup` variable): @@ -172,7 +172,7 @@ In order to prevent blowing through the Flow Interview limit for each day, it's
-Use the included app and permission set (`See Rollup App`) permission set to uncover the `Rollup` app - a single-page-application where you can manually kick off rollup jobs. This is important because `Rollup` works on an ongoing basis; in order for your rollups to be correct, unless the child object you're starting to rollup has now rows when you implement `Rollup`, a one-off full recalculation is necessary. Here's how you would fill out the page to get things started: +Use the included app and permission set (`See Rollup App`) permission set to uncover the `Rollup` app - a single-page-application where you can manually kick off rollup jobs. This is important because `Rollup` works on an ongoing basis; in order for your rollups to be correct, unless the child object you're starting to rollup has no rows when you implement `Rollup`, a one-off full recalculation is necessary. Here's how you would fill out the page to get things started: ![Example of Rollup App](./media/joys-of-apex-rollup-app.png 'Manually kicking off rollup jobs') @@ -185,18 +185,51 @@ I would _highly_ recommend scheduling through Scheduled Flows. That being said, `Rollup` exposes the option to use Scheduled Jobs if that's more your style. You can use the following Anonymous Apex script to schedule rollups: ```java -// Method signature: (String jobName, String cronExp, String query, List rollupMetadataIds, Evaluator eval) +// Method signature: (String jobName, String cronExp, String query, String rollupObjectName, Evaluator eval) Rollup.schedule( 'My example job name', 'my cron expression, like 0 0 0 * * ?', 'my SOQL query, like SELECT Id, Amount FROM Opportunity WHERE CreatedDate > YESTERDAY', - new List{ 'The ids of Rollup__mdt records configuring the rollup operation' }, + 'The API name of the SObject associated with Rollup__mdt records configuring the rollup operation', null ); ``` That last argument - the `null` value - has to implement an interface called `Evaluator` (or it can just be left null). More on that below. +Note that the third argument - the `String rollupObjectName` should be one of two values: + +- the API name of the object(s) where rollups are started from the parent (where `Is Rollup Started From Parent` on `Rollup__mdt` is checked off) OR +- the API name of the object(s) where rollups are started from the child object + +In either case, the SOQL query needs to correspond to either the parent or the children records that you'd like to operate on. + +### Grandparent (Or Greater) Rollups + +
+ +It's not all that uncommon, especially with custom objects, to get into the practice of rolling up values from one object merely so that _another_ parent object can receive _those_ rolled up values; that is to say, we occasionally use intermediate objects in order to roll values up from a grandchild record to a grandparent (and there's no need to stop there; it's totally possible to want to roll up values from great-grandchildren to the great-grandparent record, and so on). `Rollup` offers the never-before-seen functionality of skipping the intermediate records so that you can go directly to the ultimate parent object. This is supported through the invocable rollup actions, as well as through the CMDT-based rollup approach by filling out the optional field `Grandparent Relationship Field Path`: + +![Example grandparent rollup](./media/example-grandparent-rollup.png) + +In this example, there are four objects in scope: + +- `ApplicationLog__c`, which has a lookup field `Application__c` +- `Application__c`, which has a lookup field `ParentApplication__c` +- `ParentApplication__c`, which has a lookup field `Account__c` +- `Account`, and the field we'd like to rollup to has the API name `AnnualRevenue` + +**Important things to note about grand(or greater)parent rollups:** + +- **super important** all intermediate objects in the chain (so, in this example, `Application__c`, and `ParentApplication__c`) must _also_ have the `Rollup.runFromTrigger()` snippet in those object's triggers (or the appropriate invocable built). This special caveat handles cases where the intermediate objects' lookup fields are updated; no big deal if the ultimate parent lookup hasn't changed, but _big_ deal if the ultimate parent lookup _has_ changed +- if your CMDT/invocable is set up with a relationship that is not the immediate parent and you don't fill out the `Grandparent Relationship Field Path`, it simply won't work. The field path is required because it's common for objects to have more than one lookup field to the same object +- if you are using `Grandparent Relationship Field Path` with a polymorphic standard field like `Task.WhatId` or `Task.WhoId`, you should also supply a `SOQL Where Clause` to ensure you are filtering the calculation items to only be related to one type of parent at a time. **Note** - at this time, the `SOQL Where Clause` only works on the fields present on the initial calculation items, and does not support cross-object filtering +- grandparent rollups respect [SOQL's map relationship-field hopping of 5 levels](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_relationships_query_limits.htm): + +> In each specified relationship, no more than five levels can be specified in a child-to-parent relationship. For example, Contact.Account.Owner.FirstName (three levels) + +While the base architecture for retrieving grand(or greater)parent items has no technical limit on the number of relationship field hops that can be made, correctly re-triggering the rollup calculations after an intermediate object has been updated made it necessary to respect this limit (for now). + ## Custom Apex Rollups If the CMDT-based or other solutions won't cut it and you need more customizability, there's an extensive API surface exposed by `Rollup` using public static helper methods: @@ -239,6 +272,8 @@ public static void runFromTrigger() public static void runFromCDCTrigger() // imperatively from Apex, relying on CMDT for additional rollup info +// if you are actually using this from WITHIN a trigger, the second argument should +// ALWAYS be the "Trigger.operationType" static variable global static void runFromApex(List calcItems, TriggerOperation rollupContext) // imperatively from Apex with arguments taking the place of values previously supplied by CMDT diff --git a/extra-tests/classes/RollupIntegrationTests.cls b/extra-tests/classes/RollupIntegrationTests.cls index 5ef09893..0540676c 100644 --- a/extra-tests/classes/RollupIntegrationTests.cls +++ b/extra-tests/classes/RollupIntegrationTests.cls @@ -1,7 +1,6 @@ @isTest private class RollupIntegrationTests { - // "Integration," in the sense that these include custom fields that shouldn't be installed - // we still don't need to actually update the records to prove the point + // "Integration," in the sense that these include custom fields / objects that shouldn't be installed @TestSetup static void setup() { Rollup.defaultControl = new RollupControl__mdt(ShouldAbortRun__c = true); @@ -142,6 +141,224 @@ private class RollupIntegrationTests { Test.stopTest(); ParentApplication__c updatedParent = [SELECT Engagement_Rollup__c FROM ParentApplication__c]; - System.assertEquals(45/3, updatedParent.Engagement_Rollup__c, 'Average should be calculated based off of all items for denominator, not just matching items'); + System.assertEquals(45/3, updatedParent.Engagement_Rollup__c, 'Average should be calculated based off of matching items'); + } + + @isTest + static void shouldSupportCustomObjectsWhenRollupTriggeredFromParent() { + ParentApplication__c parentApp = new ParentApplication__c(Name = 'Custom Object Parent App'); + insert parentApp; + + List apps = new List{ + new Application__c(Something_With_Underscores__c = 'We have and in the name', ParentApplication__c = parentApp.Id, Engagement_Score__c = 40), + new Application__c(Something_With_Underscores__c = 'We have and in the name', ParentApplication__c = parentApp.Id, Engagement_Score__c = 40), + new Application__c(Something_With_Underscores__c = 'Financial Services', ParentApplication__c = parentApp.Id, Engagement_Score__c = 30), + new Application__c(Something_With_Underscores__c = 'Backslashes/Too', ParentApplication__c = parentApp.Id, Engagement_Score__c = 5), + new Application__c(Something_With_Underscores__c = 'Something & Something Else', ParentApplication__c = parentApp.Id, Engagement_Score__c = 10) + }; + insert apps; + + Rollup.FlowInput input = new Rollup.FlowInput(); + input.lookupFieldOnCalcItem = 'ParentApplication__c'; + input.lookupFieldOnOpObject = 'Id'; + input.recordsToRollup = new List{ parentApp }; + input.rollupContext = 'INSERT'; + input.rollupFieldOnCalcItem = 'Engagement_Score__c'; + input.rollupFieldOnOpObject = 'Engagement_Rollup__c'; + input.rollupOperation = 'SUM'; + input.rollupSObjectName = 'ParentApplication__c'; + input.isRollupStartedFromParent = true; + input.calcItemTypeWhenRollupStartedFromParent = 'Application__c'; + + Test.startTest(); + Rollup.performRollup(new List{ + input + }); + Test.stopTest(); + + ParentApplication__c updatedParent = [SELECT Engagement_Rollup__c FROM ParentApplication__c]; + System.assertEquals(125, updatedParent.Engagement_Rollup__c, 'Custom fields should work when rollup started from parent!'); + } + + /** grandparent rollup tests */ + @isTest + static void shouldFindGreatGrandParentRelationshipBetweenCustomObjects() { + Account greatGrandparent = new Account(Name = 'Great-grandparent'); + Account secondGreatGrandparent = new Account(Name = 'Second great-grandparent'); + insert new List{ greatGrandparent, secondGreatGrandparent }; + + ParentApplication__c grandParent = new ParentApplication__c(Name = 'Grandparent', Account__c = greatGrandparent.Id); + ParentApplication__c nonMatchingGrandParent = new ParentApplication__c(Name = 'Non-matching grandparent'); + insert new List{ grandParent, nonMatchingGrandParent }; + + Application__c parent = new Application__c(Name = 'Parent', ParentApplication__c = grandParent.Id); + Application__c nonMatchingParent = new Application__c(Name = 'Non matching parent', ParentApplication__c = nonMatchingGrandParent.Id); + insert new List{ parent, nonMatchingParent }; + + ApplicationLog__c child = new ApplicationLog__c(Application__c = parent.Id, Name = 'Test Rollup Grandchildren'); + ApplicationLog__c nonMatchingChild = new ApplicationLog__c(Name = 'Non matching child', Application__c = nonMatchingParent.Id); + List appLogs = new List{ child, nonMatchingChild }; + insert appLogs; + + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder( + new RollupControl__mdt(MaxQueryRows__c = 1000), + 'Application__r.ParentApplication__r.Account__r.Name', + new Set{ 'Id', 'Name' }, + Account.SObjectType, + new Map() + ); + + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(appLogs); + System.assertEquals(true, traversal.getIsFinished(), 'Traversal should not have aborted early'); + System.assertEquals(greatGrandparent, traversal.retrieveParent(child.Id), 'Account should match!'); + + System.assertEquals( + null, + traversal.retrieveParent(nonMatchingChild.Id), + 'No matching records should be returned for relationship that does not go fully up the chain' + ); + + // ok, and can we access the great-grandparent if the lookup field is populated? + nonMatchingGrandParent.Account__c = secondGreatGrandparent.Id; + update nonMatchingGrandParent; + + // this also validates that the internal state of the finder is resilient; that it can be called more than once + traversal = finder.getParents(appLogs); + System.assertEquals(greatGrandparent, traversal.retrieveParent(child.Id), 'Should still match!'); + System.assertEquals(secondGreatGrandparent, traversal.retrieveParent(nonMatchingChild.Id), 'Should now match!'); + } + + @isTest + static void shouldNotBlowUpIfGrandparentsDontExist() { + Application__c app = new Application__c(Name = 'No grandparent app'); + insert app; + + List appLogs = new List{ + new ApplicationLog__c(Application__c = app.Id, Object__c = 'Lead'), + new ApplicationLog__c(Application__c = app.Id, Object__c = 'Account') + }; + + Rollup.records = appLogs; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'ApplicationLog__c', + RollupFieldOnCalcItem__c = 'Object__c', + LookupFieldOnCalcItem__c = 'Application__c', + LookupObject__c = 'Account', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'Name', + RollupOperation__c = 'CONCAT', + GrandparentRelationshipFieldPath__c = 'Application__r.ParentApplication__r.Account__r.Name' + ) + }; + Rollup.shouldRun = true; + Rollup.apexContext = TriggerOperation.AFTER_INSERT; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + // basically validates that traversal.isAbortedEarly correctly does its job in RollupRleationshipFieldFinder.cls + System.assert(true, 'Should make it here without exception being thrown'); + } + + @isTest + static void shouldRunCorrectlyForGrandparentReparenting() { + Account greatGrandparent = new Account(Name = 'Great-grandparent'); + Account secondGreatGrandparent = new Account(Name = 'Second great-grandparent'); + insert new List{ greatGrandparent, secondGreatGrandparent }; + + ParentApplication__c grandParent = new ParentApplication__c(Name = 'Grandparent', Account__c = greatGrandparent.Id); + ParentApplication__c secondGrandparent = new ParentApplication__c(Name = 'Second grandparent', Account__c = secondGreatGrandparent.Id); + insert new List{ grandParent, secondGrandparent }; + + Application__c parent = new Application__c(Name = 'Parent-level', ParentApplication__c = grandParent.Id); + Application__c secondParent = new Application__c(Name = 'Second parent-level', ParentApplication__c = secondGrandparent.Id); + insert new List{ parent, secondParent }; + + ApplicationLog__c child = new ApplicationLog__c(Application__c = secondParent.Id, Name = 'Test Rollup Grandchildren Reparenting'); + ApplicationLog__c secondChild = new ApplicationLog__c(Name = 'Reparenting deux', Application__c = parent.Id); + List appLogs = new List{ child, secondChild }; + insert appLogs; + + Rollup.records = appLogs; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'ApplicationLog__c', + RollupFieldOnCalcItem__c = 'Name', + LookupFieldOnCalcItem__c = 'Application__c', + LookupObject__c = 'Account', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'Name', + RollupOperation__c = 'CONCAT_DISTINCT', + GrandparentRelationshipFieldPath__c = 'Application__r.ParentApplication__r.Account__r.Name' + ) + }; + Rollup.shouldRun = true; + Rollup.apexContext = TriggerOperation.AFTER_UPDATE; + Rollup.oldRecordsMap = new Map{ + child.Id => new ApplicationLog__c(Id = child.Id, Application__c = parent.Id, Name = greatGrandparent.Name), + secondChild.Id => new ApplicationLog__c(Id = secondChild.Id, Application__c = secondParent.Id, Name = secondGreatGrandparent.Name) + }; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + Account updatedGreatGrandparent = [SELECT Name FROM Account WHERE Id = :greatGrandparent.Id]; + Account updatedGreatGrandparentTwo = [SELECT Name FROM Account WHERE Id = :secondGreatGrandparent.Id]; + + System.assertEquals(secondChild.Name, updatedGreatGrandparent.Name, 'CONCAT_DISTINCT and reparenting should have worked'); + System.assertEquals(child.Name, updatedGreatGrandparentTwo.Name, 'CONCAT_DISTINCT and reparenting should have worked again'); + } + + @isTest + static void shouldRunGrandparentRollupsWhenIntermediateObjectsAreUpdated() { + Account greatGrandparent = new Account(Name = 'Great-grandparent'); + Account secondGreatGrandparent = new Account(Name = 'Second great-grandparent'); + insert new List{ greatGrandparent, secondGreatGrandparent }; + + ParentApplication__c grandParent = new ParentApplication__c(Name = 'Grandparent', Account__c = greatGrandparent.Id); + ParentApplication__c secondGrandparent = new ParentApplication__c(Name = 'Second grandparent', Account__c = secondGreatGrandparent.Id); + List parentApps = new List{ grandParent, secondGrandparent }; + insert parentApps; + + Application__c parent = new Application__c(Name = 'Parent-level', ParentApplication__c = grandParent.Id); + Application__c secondParent = new Application__c(Name = 'Second parent-level', ParentApplication__c = secondGrandparent.Id); + insert new List{ parent, secondParent }; + + ApplicationLog__c child = new ApplicationLog__c(Application__c = secondParent.Id, Name = 'Test Rollup Grandchildren Reparenting'); + ApplicationLog__c secondChild = new ApplicationLog__c(Name = 'Reparenting deux', Application__c = parent.Id); + insert new List{ child, secondChild }; + + Rollup.records = parentApps; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'ApplicationLog__c', + RollupFieldOnCalcItem__c = 'Name', + LookupFieldOnCalcItem__c = 'Application__c', + LookupObject__c = 'Account', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'Name', + RollupOperation__c = 'CONCAT_DISTINCT', + GrandparentRelationshipFieldPath__c = 'Application__r.ParentApplication__r.Account__r.Name' + ) + }; + Rollup.shouldRun = true; + Rollup.apexContext = TriggerOperation.AFTER_UPDATE; + Rollup.oldRecordsMap = new Map{ + grandParent.Id => new ParentApplication__c(Id = grandParent.Id, Account__c = secondGreatGrandparent.Id), + secondGrandparent.Id => new ParentApplication__c(Id = secondGrandparent.Id, Account__c = greatGrandparent.Id) + }; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + Account updatedGreatGrandparent = [SELECT Name FROM Account WHERE Id = :greatGrandparent.Id]; + Account updatedGreatGrandparentTwo = [SELECT Name FROM Account WHERE Id = :secondGreatGrandparent.Id]; + + System.assertEquals(secondChild.Name, updatedGreatGrandparent.Name, 'Grandparent record should have retriggered greatgrandparent rollup!'); + System.assertEquals(child.Name, updatedGreatGrandparentTwo.Name, 'Grandparent record should have retriggered greatgrandparent rollup again!'); } } diff --git a/extra-tests/main/default/triggers/ApplicationTrigger.trigger b/extra-tests/triggers/ApplicationTrigger.trigger similarity index 100% rename from extra-tests/main/default/triggers/ApplicationTrigger.trigger rename to extra-tests/triggers/ApplicationTrigger.trigger diff --git a/extra-tests/main/default/triggers/ApplicationTrigger.trigger-meta.xml b/extra-tests/triggers/ApplicationTrigger.trigger-meta.xml similarity index 100% rename from extra-tests/main/default/triggers/ApplicationTrigger.trigger-meta.xml rename to extra-tests/triggers/ApplicationTrigger.trigger-meta.xml diff --git a/media/example-grandparent-rollup.png b/media/example-grandparent-rollup.png new file mode 100644 index 00000000..f24ff30d Binary files /dev/null and b/media/example-grandparent-rollup.png differ diff --git a/package.json b/package.json index b493815c..83dcbe3b 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "apex-rollup", - "version": "1.1.11", + "version": "1.2.0", "description": "Fast, configurable, elastically scaling custom rollup solution. Apex Invocable action, one-liner Apex trigger/CMDT-driven logic, and scheduled Apex-ready.", "repository": { "type": "git", diff --git a/rollup/main/default/classes/Rollup.cls b/rollup/main/default/classes/Rollup.cls index c7cb8daf..b487ce05 100644 --- a/rollup/main/default/classes/Rollup.cls +++ b/rollup/main/default/classes/Rollup.cls @@ -53,6 +53,7 @@ global without sharing virtual class Rollup implements Database.Batchable> lookupObjectToUniqueFieldNames; private List lookupItems; private RollupControl__mdt rollupControl; + private RollupRelationshipFieldFinder.Traversal traversal; /** * receiving an interface/subclass from a property get/set (from the book "The Art Of Unit Testing") is an old technique; @@ -183,6 +184,7 @@ global without sharing virtual class Rollup implements Database.Batchable> queryCountsToLookupIds = new Map>(); - for (Rollup rollup : this.rollups) { - rollup.isFullRecalc = this.isFullRecalc; - if (targetType == null) { - targetType = rollup.lookupObj; - } else if (rollup.lookupObj != targetType) { - hasMoreThanOneTarget = true; - } + Integer totalCountOfRecords = this.getLookupRecordsCount(hasMoreThanOneTarget); - Set uniqueIds = new Set(); - for (SObject calcItem : rollup.calcItems) { - String lookupKey = (String) calcItem.get(rollup.lookupFieldOnCalcItem); - if (String.isNotBlank(lookupKey)) { - uniqueIds.add(lookupKey); - } - } - - String countQuery = getQueryString(rollup.lookupObj, new List{ 'Count()' }, String.valueOf(rollup.lookupFieldOnLookupObject), '='); - if (queryCountsToLookupIds.containsKey(countQuery)) { - queryCountsToLookupIds.get(countQuery).addAll(uniqueIds); - } else { - queryCountsToLookupIds.put(countQuery, uniqueIds); - } - } - - Integer totalCountOfRecords = 0; - for (String countQuery : queryCountsToLookupIds.keySet()) { - Set objIds = queryCountsToLookupIds.get(countQuery); - totalCountOfRecords += getCountFromDb(countQuery, objIds); - } - - Boolean shouldRunAsBatch = - shouldRunAsBatch || (orgDefaults.ShouldRunAs__c == 'Batchable' && totalCountOfRecords >= orgDefaults.MaxLookupRowsBeforeBatching__c); + Boolean shouldBatch = shouldRunAsBatch || (orgDefaults.ShouldRunAs__c == 'Batchable' && totalCountOfRecords >= orgDefaults.MaxLookupRowsBeforeBatching__c); if (this.syncRollups.isEmpty() == false) { this.process(this.syncRollups); return 'Running rollups flagged to go synchronously'; - } else if (shouldRunAsBatch && hasMoreThanOneTarget == false) { + } else if (shouldBatch && hasMoreThanOneTarget == false) { // safe to batch because the QueryLocator will only return one type of SObject return Database.executeBatch(new Rollup(this), this.rollupControl.BatchChunkSize__c.intValue()); - } else if ( - shouldRunAsBatch == false && (orgDefaults.MaxLookupRowsForQueueable__c == null || totalCountOfRecords <= orgDefaults.MaxLookupRowsForQueueable__c) - ) { - return System.enqueueJob(this); } else { - throw new AsyncException( - 'Number of records that would be rolled up : ' + - totalCountOfRecords + - ' exceeds safety threshold, or you tried to run this in Batch mode with more than one target type: (' + - hasMoreThanOneTarget + - ')' - ); + return System.enqueueJob(this); } } @@ -305,16 +264,15 @@ global without sharing virtual class Rollup implements Database.Batchable objIds = new Set(); for (Rollup rollup : this.rollups) { - lookupFieldOnLookupObject = rollup.lookupFieldOnLookupObject.getDescribe().getName(); - objIds.addAll(this.getCalcItemsByLookupField(rollup).keySet()); - } - for (SObjectType sObjectType : this.lookupObjectToUniqueFieldNames.keySet()) { - query = getQueryString(sObjectType, new List(this.lookupObjectToUniqueFieldNames.get(sObjectType)), lookupFieldOnLookupObject, '='); + sObjectType = rollup.lookupObj; + lookupFieldForLookupObject = rollup.lookupFieldOnLookupObject.getDescribe().getName(); + objIds.addAll(this.getCalcItemsByLookupField(rollup, this.lookupObjectToUniqueFieldNames.get(sObjectType)).keySet()); } + String query = getQueryString(sObjectType, new List(this.lookupObjectToUniqueFieldNames.get(sObjectType)), lookupFieldForLookupObject, '='); return Database.getQueryLocator(query); } @@ -330,6 +288,17 @@ global without sharing virtual class Rollup implements Database.Batchable records; + public RollupAsyncSaver(List records) { + this.records = records; + } + + public void execute(QueueableContext context) { + new DMLHelper().doUpdate(this.records); + } + } + private class RollupAsyncProcessor extends Rollup implements System.Queueable { public RollupAsyncProcessor( List calcItems, @@ -368,12 +337,26 @@ global without sharing virtual class Rollup implements Database.Batchable getExistingLookupItems(Set objIds, Rollup rollup, Set uniqueQueryFieldNames) { if (objIds.isEmpty()) { return new List(); + } else { + List localLookupItems; + if (String.isNotBlank(rollup.metadata.GrandparentRelationshipFieldPath__c)) { + localLookupItems = rollup.traversal.getAllParents(); + // winnow the list, which would otherwise occur because of specifically only querying for the objIds passed in + for (Integer index = localLookupItems.size() - 1; index >= 0; index--) { + SObject lookupItem = localLookupItems[index]; + String key = (String) lookupItem.get(rollup.lookupFieldOnLookupObject); + if (objIds.contains(key) == false) { + localLookupItems.remove(index); + } + } + } else { + String queryString = getQueryString(rollup.lookupObj, new List(uniqueQueryFieldNames), String.valueOf(rollup.lookupFieldOnLookupObject), '='); + // non-obvious coupling between "objIds" and the computed "queryString", which uses dynamic variable binding + localLookupItems = Database.query(queryString); + } + this.initializeRollupFieldDefaults(localLookupItems, rollup); + return localLookupItems; } - // non-obvious coupling between "objIds" and the computed "queryString", which uses dynamic variable binding - String queryString = getQueryString(rollup.lookupObj, new List(uniqueQueryFieldNames), String.valueOf(rollup.lookupFieldOnLookupObject), '='); - List lookupItems = Database.query(queryString); - this.initializeRollupFieldDefaults(lookupItems, rollup); - return lookupItems; } public void execute(System.QueueableContext qc) { @@ -406,48 +389,19 @@ global without sharing virtual class Rollup implements Database.Batchable{ 'Count()' }, lookupFieldOnLookupObject, '!='); - - Set objIds = new Set(); // get everything that doesn't have a null Id - a pretty trick - Integer amountOfCalcItems = getCountFromDb(countQuery, objIds); - - // emptyRollup only used to call "getMaxQueryRows" below - Rollup emptyRollup = new Rollup(RollupInvocationPoint.FROM_LWC); - SObjectType lookupType = getSObjectTypeFromName(lookupSObjectName); - Rollup__mdt rollupInfo = new Rollup__mdt( - RollupFieldOnCalcItem__c = opFieldOnCalcItem, - LookupObject__c = lookupSObjectName, - LookupFieldOnCalcItem__c = lookupFieldOnCalcItem, - LookupFieldOnLookupObject__c = lookupFieldOnLookupObject, - RollupFieldOnLookupObject__c = rollupFieldOnLookupObject, - RollupOperation__c = operationName + QueryWrapper wrapper = new QueryWrapper(lookupSObjectName, lookupFieldOnLookupObject); + wrapper.setQuery(potentialWhereClause); + return performFullRecalculationInner( + opFieldOnCalcItem, + lookupFieldOnCalcItem, + lookupFieldOnLookupObject, + rollupFieldOnLookupObject, + lookupSObjectName, + calcItemSObjectName, + operationName, + wrapper, + null ); - Set queryFields = new Set{ 'Id', opFieldOnCalcItem, lookupFieldOnCalcItem }; - RollupEvaluator.WhereFieldEvaluator eval = new RollupEvaluator.WhereFieldEvaluator(potentialWhereClause, calcItemType); - queryFields.addAll(eval.getRelationshipFieldNames()); - String queryString = getQueryString(calcItemType, new List(queryFields), 'Id', '!=', potentialWhereClause); - if (amountOfCalcItems < emptyRollup.getMaxQueryRows()) { - List calcItems = Database.query(queryString); - Rollup thisRollup = getRollup( - new List{ rollupInfo }, - calcItemType, - calcItems, - new Map(calcItems), - eval, - RollupInvocationPoint.FROM_LWC - ); - thisRollup.isFullRecalc = true; - return thisRollup.runCalc(); - } else { - //batch to get calc items and then batch to rollup - return Database.executeBatch( - new RollupFullBatchRecalculator(queryString, RollupInvocationPoint.FROM_LWC, rollupInfo, calcItemType), - emptyRollup.rollupControl.BatchChunkSize__c.intValue() - ); - } } global class FlowInput { @@ -523,6 +477,9 @@ global without sharing virtual class Rollup implements Database.Batchable uniqueFieldNames = new Set{ 'Id' }; @@ -623,8 +588,8 @@ global without sharing virtual class Rollup implements Database.Batchable rollupMetadataIds, Evaluator eval) { - RollupSchedulable scheduledRollup = new RollupSchedulable(query, rollupMetadataIds, eval); + global static Id schedule(String jobName, String cronExp, String query, String rollupObjectName, Evaluator eval) { + RollupSchedulable scheduledRollup = new RollupSchedulable(query, rollupObjectName, eval); return System.schedule(jobName, cronExp, scheduledRollup); } @@ -1236,7 +1201,7 @@ global without sharing virtual class Rollup implements Database.Batchable{ rollupMetadata }, eval, getTriggerRecords(), getOldTriggerRecordsMap()); + return runFromApex(new List{ meta }, eval, getTriggerRecords(), getOldTriggerRecordsMap()); } global static void runFromCDCTrigger() { @@ -1262,15 +1227,15 @@ global without sharing virtual class Rollup implements Database.Batchable rollupMetadata = getTriggerRollupMetadata(sObjectType); - if (rollupMetadata.isEmpty()) { + List matchingMetadata = getRollupMetadataBySObject(sObjectType); + if (matchingMetadata.isEmpty()) { return; } Set uniqueFieldNames = new Set{ 'Id' }; - for (Rollup__mdt rollupInfo : rollupMetadata) { + for (Rollup__mdt rollupInfo : matchingMetadata) { uniqueFieldNames.add(getParedFieldName(rollupInfo.LookupFieldOnCalcItem__c)); uniqueFieldNames.add(getParedFieldName(rollupInfo.ROllupFieldOnCalcItem__c)); } @@ -1296,9 +1261,13 @@ global without sharing virtual class Rollup implements Database.Batchable(uniqueFieldNames), 'Id', '='); - Map cdcRecordsMap = new Map(Database.query(fullQuery)); + // getting the items back from the database before putting them into the map is an important step + // we COULD just initialize the map with the query, but then the map's .values() list doesn't return + // anything for .getSObjectType() - which we need, further downstream + List cdcCalcItems = Database.query(fullQuery); + Map cdcCalcItemsMap = new Map(cdcCalcItems); - Rollup rollupToReturn = runFromApex(rollupMetadata, null, cdcRecordsMap.values(), cdcRecordsMap); + Rollup rollupToReturn = runFromApex(rollupMetadata, null, cdcCalcItems, cdcCalcItemsMap); // because CDC is async, the DB will always be updated by the time we get there // for update, that means we always have to trigger a full recalc // the performance downsides should be negligible, given that we're already within an async context @@ -1308,8 +1277,8 @@ global without sharing virtual class Rollup implements Database.Batchable triggerRecords = getTriggerRecords(); - List rollupMetadata = getTriggerRollupMetadata(triggerRecords.getSObjectType()); - runFromApex(rollupMetadata, null, triggerRecords, getOldTriggerRecordsMap()).runCalc(); + List matchingMetadata = getRollupMetadataBySObject(triggerRecords.getSObjectType()); + runFromApex(matchingMetadata, null, triggerRecords, getOldTriggerRecordsMap()).runCalc(); } /** @@ -1319,6 +1288,10 @@ global without sharing virtual class Rollup implements Database.Batchable calcItems, TriggerOperation rollupContext) { shouldRun = true; @@ -1359,8 +1332,13 @@ global without sharing virtual class Rollup implements Database.Batchable{ 'Count()' }, lookupFieldOnLookupObject, '!=', queryWrapper.getQuery()); + + Set objIds = new Set(); // get everything that doesn't have a null Id - a pretty trick + Set recordIds = queryWrapper.recordIds; // also used below, bound to the "queryString" variable + Integer amountOfCalcItems = getCountFromDb(countQuery, objIds, recordIds); + + // emptyRollup used to call "getMaxQueryRows" below, as well as the default rollupControl.BatchChunkSize__c + Rollup emptyRollup = new Rollup(RollupInvocationPoint.FROM_LWC); + SObjectType lookupType = getSObjectFromName(lookupSObjectName).getSObjectType(); + Rollup__mdt rollupInfo = new Rollup__mdt( + RollupFieldOnCalcItem__c = opFieldOnCalcItem, + LookupObject__c = lookupSObjectName, + LookupFieldOnCalcItem__c = lookupFieldOnCalcItem, + LookupFieldOnLookupObject__c = lookupFieldOnLookupObject, + RollupFieldOnLookupObject__c = rollupFieldOnLookupObject, + RollupOperation__c = operationName, + GrandparentRelationshipFieldPath__c = potentialGrandparentRelationshipFieldPath + ); + Set queryFields = new Set{ 'Id', opFieldOnCalcItem, lookupFieldOnCalcItem }; + RollupEvaluator.WhereFieldEvaluator whereEval = new RollupEvaluator.WhereFieldEvaluator(queryWrapper.toString(), calcItemType); + queryFields.addAll(whereEval.getRelationshipFieldNames()); + String queryString = getQueryString(calcItemType, new List(queryFields), 'Id', '!=', queryWrapper.getQuery()); + if (amountOfCalcItems < emptyRollup.getMaxQueryRows()) { + List calculationItems = Database.query(queryString); + Rollup thisRollup = getRollup( + new List{ rollupInfo }, + calcItemType, + calculationItems, + new Map(calculationItems), + whereEval, + RollupInvocationPoint.FROM_LWC + ); + thisRollup.isFullRecalc = true; + return thisRollup.runCalc(); + } else { + // batch to get calc items and then batch to rollup + return Database.executeBatch( + new RollupFullBatchRecalculator(queryString, RollupInvocationPoint.FROM_LWC, rollupInfo, calcItemType, recordIds), + emptyRollup.rollupControl.BatchChunkSize__c.intValue() + ); + } + } + public static String getQueryString( SObjectType sObjectType, List uniqueQueryFieldNames, @@ -1446,13 +1504,13 @@ global without sharing virtual class Rollup implements Database.Batchable cachedMetadata; public static List getMetadataFromCache(SObjectType metadataType) { - List metadata; + List matchingMetadata; // CMDT is read-only when returned from the cache // use "deepClone" to get access to mutable versions // of the CMDT records. We also need to clean up the Entity Definition / Field Definition - // particles for custom objects - the final sweep through on custom fields will be done later + // particles for custom objects and custom fields if (metadataType == Rollup__mdt.SObjectType) { - if(cachedMetadata == null) { + if (cachedMetadata == null) { cachedMetadata = [ SELECT // we have to do transforms on these fields because custom objects/custom fields @@ -1474,6 +1532,7 @@ global without sharing virtual class Rollup implements Database.Batchable calcItems, Map oldCalcItems) { + if (isCDC) { + return new QueryWrapper(); + } + + SObjectType sObjectType = calcItems[0].getSObjectType(); + SObjectField fieldToken = getPartOfGrandparentChain(grandparentFieldPath, sObjectType); + String relationshipName = fieldToken.getDescribe().getRelationshipName(); + Integer relationshipIndex = grandparentFieldPath.indexOf(relationshipName) + relationshipName.length(); + String priorFieldPath = grandparentFieldPath.substring(0, relationshipIndex) + '.Id'; + QueryWrapper wrapper = new QueryWrapper('', priorFieldPath); + + for (SObject calcItem : calcItems) { + SObject oldCalcItem = oldCalcItems.containsKey(calcItem.Id) ? oldCalcItems.get(calcItem.Id) : calcItem; + String currentLookup = (String) calcItem.get(fieldToken); + String oldLookup = (String) oldCalcItem.get(fieldToken); + if (currentLookup != oldLookup) { + wrapper.addRecordId(currentLookup); + wrapper.addRecordId(oldLookup); + } + } + return wrapper; + } + + private class QueryWrapper { + private QueryWrapper() { + } + private QueryWrapper(String lookupObject, String lookupField) { + String base = String.isBlank(lookupObject) ? '' : lookupObject.replace('__c', '__r') + '.'; + this.query = base + lookupField + ' = :recordIds'; + } + + private Boolean hasQuery = false; + private String query; + public final Set recordIds = new Set(); + private final List stringifiedRecordIds = new List(); + + public void addRecordId(String recordId) { + if (String.isNotBlank(recordId)) { + this.hasQuery = true; + this.recordIds.add(recordId); + this.stringifiedRecordIds.add('\'' + recordId + '\''); + } + } + + public String getQuery() { + return this.hasQuery ? this.query : ''; + } + + public void setQuery(String query) { + if (String.isNotBlank(query)) { + this.hasQuery = true; + this.query = query; + } + } + + public override String toString() { + return this.hasQuery ? this.query.replace('= :recordIds', 'IN (' + String.join(this.stringifiedRecordIds, ',') + ')') : ''; } - return metadata; } - private static String getParentWhereClause(List calcItems, String lookupFieldOnLookupObject, String lookupObjectName) { - String parentWhereClause = ' IN ('; + private static QueryWrapper getParentWhereClause( + List calcItems, + String lookupFieldOnLookupObject, + String lookupObjectName, + String grandparentFieldPath + ) { + String fieldName = lookupFieldOnLookupObject; + if (String.isNotBlank(grandparentFieldPath)) { + lookupObjectName = ''; + fieldName = grandparentFieldPath.substringBeforeLast('.') + '.Id'; + } + + QueryWrapper wrapper = new QueryWrapper(lookupObjectName, fieldName); for (SObject calcItem : calcItems) { String lookupId = (String) calcItem.get(lookupFieldOnLookupObject); if (String.isNotBlank(lookupId)) { - parentWhereClause += '\'' + lookupId + '\','; + wrapper.addRecordId(lookupId); } } - // remove trailing comma, because this comment is less LOC than doing the for loop with an index and checking before appending the comma :) - parentWhereClause = parentWhereClause.substringBeforeLast(',') + ')'; - return lookupObjectName.replace('__c', '__r') + '.' + lookupFieldOnLookupObject + parentWhereClause; + return wrapper; } private static Integer getCountFromDb(String countQuery, Set objIds) { + return getCountFromDb(countQuery, objIds, null); + } + + private static Integer getCountFromDb(String countQuery, Set objIds, Set recordIds) { if (countQuery.contains('ALL ROWS')) { countQuery = countQuery.replace('ALL ROWS', ''); } @@ -1566,7 +1699,7 @@ global without sharing virtual class Rollup implements Database.Batchable getTriggerRollupMetadata(SObjectType sObjectType) { + private static List getRollupMetadataBySObject(SObjectType sObjectType) { String sObjectName = sObjectType.getDescribe().getName(); List rollupMetadatas = getMetadataFromCache(Rollup__mdt.SObjectType); for (Integer index = rollupMetadatas.size() - 1; index >= 0; index--) { - Rollup__mdt rollupMetadata = rollupMetadatas[index]; - if (rollupMetadata.CalcItem__c != sObjectName && rollupMetadata.IsRollupStartedFromParent__c == false) { + Rollup__mdt meta = rollupMetadatas[index]; + if (String.isNotBlank(meta.GrandparentRelationshipFieldPath__c)) { + if (getPartOfGrandparentChain(meta.GrandparentRelationshipFieldPath__c, sObjectType) != null) { + continue; + } + } else if (meta.CalcItem__c != sObjectName && meta.IsRollupStartedFromParent__c == false) { rollupMetadatas.remove(index); - } else if (rollupMetadata.IsRollupStartedFromParent__c && sObjectName != rollupMetadata.LookupObject__c) { + } else if (meta.IsRollupStartedFromParent__c && sObjectName != meta.LookupObject__c) { rollupMetadatas.remove(index); } } return rollupMetadatas; } + private static SObjectField getPartOfGrandparentChain(String grandParentFieldPath, SObjectType sObjectType) { + List validRelationshipNames = grandParentFieldPath.split('\\.'); + // remove the last field since it's not a relationship + validRelationshipNames.remove(validRelationShipNames.size() - 1); + DescribeSObjectResult describeObject = sObjectType.getDescribe(); + List fieldTokens = describeObject.fields.getMap().values(); + for (SObjectField fieldToken : fieldTokens) { + if (validRelationshipNames.contains(fieldToken.getDescribe().getRelationshipName())) { + return fieldToken; + } + } + return null; + } + private static Rollup getRollup( List rollupOperations, SObjectType sObjectType, @@ -1621,10 +1772,10 @@ global without sharing virtual class Rollup implements Database.Batchable lookupFieldNameToLookupFields = lookupObjectDescribe.fields.getMap(); SObjectField lookupFieldOnOpObject = getSObjectFieldByName(lookupObjectDescribe, rollupMetadata.LookupFieldOnLookupObject__c); @@ -1632,22 +1783,22 @@ global without sharing virtual class Rollup implements Database.Batchable(); } - private static SObjectType getSObjectTypeFromName(String sObjectName) { - return ((SObject) Type.forName(sObjectName).newInstance()).getSObjectType(); + private static SObject getSObjectFromName(String sObjectName) { + return ((SObject) Type.forName(sObjectName).newInstance()); } private static String getParedFieldName(String fullFieldName) { @@ -1809,6 +1958,9 @@ global without sharing virtual class Rollup implements Database.Batchable rollups) { - this.getFieldNamesForRollups(rollups); + this.getFieldNamesForRollups(rollups); // populates this.lookupObjectToUniqueFieldNames Map updatedLookupRecords = new Map(); - for (Integer index = 0; index < rollups.size(); index++) { + Map grandparentRollups = new Map(); + for (Rollup rollup : rollups) { // for each iteration, ensure we're not operating beyond the bounds of our query limits - Rollup rollup = rollups[index]; if (this.hasExceededCurrentRollupQueryLimit(rollup.rollupControl)) { this.deferredRollups.add(rollup); continue; } - Map> calcItemsByLookupField = this.getCalcItemsByLookupField(rollup); - List lookupItems = new List(); + + if (grandparentRollups.containsKey(rollup.lookupObj) && rollup.traversal == null) { + rollup.traversal = grandparentRollups.get(rollup.lookupObj); + } + + Map> calcItemsByLookupField = this.getCalcItemsByLookupField(rollup, this.lookupObjectToUniqueFieldNames.get(rollup.lookupObj)); + // some rollups may not finish retrieving all parent rows the first time around - and that's ok! we can keep + // trying until all necessary records have been retrieved + if (rollup.traversal != null && rollup.traversal.getIsFinished() == false) { + this.deferredRollups.add(rollup); + continue; + } else if (rollup.traversal != null && grandparentRollups.containsKey(rollup.lookupObj) == false) { + // cache the traversal for any future callers - because we queried for ALL unique grand(or greater)parent fields + // we don't need to re-traverse the whole object chain again if there are other grandparent rollups in the list + grandparentRollups.put(rollup.lookupObj, rollup.traversal); + } + + List localLookupItems = new List(); Set lookupItemKeys = new Set(calcItemsByLookupField.keySet()); for (String lookupId : calcItemsByLookupField.keySet()) { if (updatedLookupRecords.containsKey(lookupId)) { @@ -1865,16 +2034,32 @@ global without sharing virtual class Rollup implements Database.Batchable updatedParentRecords = this.getUpdatedLookupItemsByRollup(rollup, calcItemsByLookupField, lookupItems); + localLookupItems.addAll(this.getExistingLookupItems(lookupItemKeys, rollup, this.lookupObjectToUniqueFieldNames.get(rollup.lookupObj))); + List updatedParentRecords = this.getUpdatedLookupItemsByRollup(rollup, calcItemsByLookupField, localLookupItems); for (SObject updatedRecord : updatedParentRecords) { updatedLookupRecords.put(updatedRecord.Id, updatedRecord); } } + if (this.rollupControl.MaxParentRowsUpdatedAtOnce__c < updatedLookupRecords.size()) { + Integer maxIndexToRemove = updatedLookupRecords.size() / 2; + Integer removalIndex = 0; + List asyncUpdateList = new List(); + for (String lookupKey : updatedLookupRecords.keySet()) { + SObject lookupRecordToUpdate = updatedLookupRecords.get(lookupKey); + asyncUpdateList.add(lookupRecordToUpdate); + updatedLookupRecords.remove(lookupKey); + removalIndex++; + if (removalIndex >= maxIndexToRemove) { + break; + } + } + System.enqueueJob(new RollupAsyncSaver(asyncUpdateList)); + } + DML.doUpdate(updatedLookupRecords.values()); this.processDeferredRollups(); @@ -1888,10 +2073,12 @@ global without sharing virtual class Rollup implements Database.Batchable stackDepth; + this.rollups.clear(); this.rollups.addAll(this.deferredRollups); this.deferredRollups.clear(); + if (System.isBatch()) { Database.executeBatch(this, this.rollupControl.BatchChunkSize__c.intValue()); } else { @@ -1912,7 +2099,7 @@ global without sharing virtual class Rollup implements Database.Batchable> getCalcItemsByLookupField(Rollup rollup) { + private Map> getCalcItemsByLookupField(Rollup rollup, Set uniqueQueryFieldNames) { + if (String.isNotBlank(rollup.metadata.GrandparentRelationshipFieldPath__c)) { + if (rollup.traversal == null) { + rollup.traversal = new RollupRelationshipFieldFinder( + rollup.rollupControl, + rollup.metadata.GrandparentRelationshipFieldPath__c, + uniqueQueryFieldNames, + rollup.lookupObj, + rollup.oldCalcItems + ) + .getParents(rollup.calcItems); + } else if (rollup.traversal?.getIsFinished() == false) { + rollup.traversal.recommence(); + } + return rollup.traversal.getIsFinished() ? rollup.traversal.getParentLookupToRecords() : new Map>(); + } Map> lookupFieldToCalcItems = new Map>(); for (SObject calcItem : rollup.calcItems) { String key = (String) calcItem.get(rollup.lookupFieldOnCalcItem); @@ -1942,6 +2144,8 @@ global without sharing virtual class Rollup implements Database.Batchable orgDefaults.MaxLookupRowsBeforeBatching__c) { orgDefaults.MaxLookupRowsBeforeBatching__c = rollupSpecificControl.MaxLookupRowsBeforeBatching__c; } - if (rollupSpecificControl.MaxLookupRowsForQueueable__c > orgDefaults.MaxLookupRowsForQueueable__c) { - orgDefaults.MaxLookupRowsForQueueable__c = rollupSpecificControl.MaxLookupRowsForQueueable__c; + if (rollupSpecificControl.MaxParentRowsUpdatedAtOnce__c == null) { + rollupSpecificControl.MaxParentRowsUpdatedAtOnce__c = orgDefaults.MaxParentRowsUpdatedAtOnce__c; + } + } + } + + private Integer getLookupRecordsCount(Boolean hasMoreThanOneTarget) { + // we need to burn a few SOQL calls to consider how many records are going to be queried/updated + // then, using RollupControl__mdt and/or sensible defaults, we'll decide whether to queue up or batch (or fail - that's always an option) + // if there's more than one SObjectType involved we bail on retrieving the actual count + // because you can only return one list of SObjects from a batch job's QueryLocator + SObjectType targetType; + Map> queryCountsToLookupIds = new Map>(); + for (Rollup rollup : this.rollups) { + rollup.isFullRecalc = this.isFullRecalc; + if (targetType == null) { + targetType = rollup.lookupObj; + } else if (rollup.lookupObj != targetType) { + hasMoreThanOneTarget = true; + } else if (String.isNotBlank(rollup.metadata?.GrandparentRelationshipFieldPath__c)) { + // getting the count for grandparent (or greater) relationships will be handled further + // downstream; for our purposes, it isn't useful to try to get all of the records while + // we're still in a sync context + continue; + } + + if (hasMoreThanOneTarget) { + break; + } + + Set uniqueIds = new Set(); + for (SObject calcItem : rollup.calcItems) { + String lookupKey = (String) calcItem.get(rollup.lookupFieldOnCalcItem); + if (String.isNotBlank(lookupKey)) { + uniqueIds.add(lookupKey); + } + } + + String countQuery = getQueryString(rollup.lookupObj, new List{ 'Count()' }, String.valueOf(rollup.lookupFieldOnLookupObject), '='); + if (queryCountsToLookupIds.containsKey(countQuery)) { + queryCountsToLookupIds.get(countQuery).addAll(uniqueIds); + } else { + queryCountsToLookupIds.put(countQuery, uniqueIds); + } + } + + Integer totalCountOfRecords = 0; + if (hasMoreThanOneTarget == false) { + for (String countQuery : queryCountsToLookupIds.keySet()) { + Set objIds = queryCountsToLookupIds.get(countQuery); + totalCountOfRecords += getCountFromDb(countQuery, objIds); } } + return totalCountOfRecords; } private List getUpdatedLookupItemsByRollup(Rollup rollup, Map> calcItemsByLookupField, List lookupItems) { @@ -2007,40 +2256,52 @@ global without sharing virtual class Rollup implements Database.Batchable calcItems = calcItemsByLookupField.get(key); + List localCalcItems = calcItemsByLookupField.get(key); - // Check for reparented records - Map oldCalcItems = rollup.oldCalcItems; - for (Integer index = calcItems.size() - 1; index >= 0; index--) { - SObject calcItem = calcItems[index]; + for (Integer index = localCalcItems.size() - 1; index >= 0; index--) { + SObject calcItem = localCalcItems[index]; if (rollup.eval?.matches(calcItem) == false && rollup.metadata?.IsFullRecordSet__c == true) { // technically it should only be possible for a calc item that doesn't match // to still exist if it is a Full Record Set operation; this gives people the chance // to reset rollup values if none of the records passed in match the eval criteria - calcItems.remove(index); + localCalcItems.remove(index); continue; } - SObject oldCalcItem = oldCalcItems.get(calcItem.Id); + // Check for reparented records + SObject oldCalcItem = rollup.oldCalcItems.get(calcItem.Id); if (oldCalcItem == null) { continue; } String priorLookup = (String) oldCalcItem.get(rollup.lookupFieldOnCalcItem); + // if the lookup wasn't previously populated, there's nothing to update + if (String.isBlank(priorLookup)) { + continue; + } Object newLookup = calcItem.get(rollup.lookupFieldOnCalcItem); - if (newLookup != priorLookup) { - if (!oldLookupItems.containsKey(priorLookup)) { - oldLookupItems.put(priorLookup, new List()); + if (newLookup != priorLookup && rollup.traversal == null) { + this.populateOldLookupItems(priorLookup, oldCalcItem, oldLookupItems); + } else if (rollup.traversal != null && rollup.traversal.isUltimatelyReparented(calcItem, rollup.lookupFieldOnCalcItem.getDescribe().getName())) { + // slightly different, but with the same end result + // note that when the reparented record is not null + // it should be the same as the current "lookupRecord" + SObject reparentedRecord = rollup.traversal.retrieveParent(oldCalcItem.Id); + if (reparentedRecord != null) { + priorLookup = (String) reparentedRecord.get(rollup.lookupFieldOnLookupObject); + if (String.isNotBlank(priorLookup)) { + Id oldLookupId = rollup.traversal.getOldLookupId(calcItem, rollup.lookupFieldOnCalcItem.getDescribe().getName()); + oldCalcItem = this.reassignOldCalcItemIfValueChanged(oldLookupId, oldCalcItem, rollup); + this.populateOldLookupItems(priorLookup, oldCalcItem, oldLookupItems); + } } - List reparentedCalcItemsForKey = oldLookupItems.get(priorLookup); - reparentedCalcItemsForKey.add(oldCalcItem); } } // Check for changed values Object priorVal = lookupRecord.get(rollup.opFieldOnLookupObject); - Object newVal = this.getRollupVal(rollup, calcItems, priorVal, key, rollup.lookupFieldOnCalcItem); + Object newVal = this.getRollupVal(rollup, localCalcItems, priorVal, key, rollup.lookupFieldOnCalcItem); if (priorVal != newVal) { lookupRecord.put(rollup.opFieldOnLookupObject, newVal); recordsToUpdate.put(key, lookupRecord); @@ -2067,9 +2328,10 @@ global without sharing virtual class Rollup implements Database.Batchable> oldLookupItems) { + if (oldLookupItems.containsKey(priorLookup) == false) { + oldLookupItems.put(priorLookup, new List{ oldCalcItem }); + } else { + oldLookupItems.get(priorLookup).add(oldCalcItem); + } + } + + private SObject reassignOldCalcItemIfValueChanged(String lookupId, SObject oldCalcItem, Rollup rollup) { + if (String.isBlank(lookupId)) { + return oldCalcItem; + } + // truly terrible, but before we pass the old item through the reparenting code path, we need to validate that it's only + // the lookup field that has changed; otherwise, if the opFieldOnCalcItem has changed too, substitute the item whose value + // previously corresponded to the parent record + for (SObject otherOldCalcItem : rollup.oldCalcItems.values()) { + if (otherOldCalcItem.get(rollup.lookupFieldOnCalcItem) == lookupId) { + if (otherOldCalcItem.get(rollup.opFieldOnCalcItem) != oldCalcItem.get(rollup.opFieldOnCalcItem)) { + return otherOldCalcItem; + } + break; // break on the match, no matter what + } + } + return oldCalcItem; + } + @testVisible private virtual class DMLHelper { public virtual void doUpdate(List recordsToUpdate) { @@ -2103,46 +2391,24 @@ global without sharing virtual class Rollup implements Database.Batchable rollupMetadataIds; + private final SObjectType rollupObject; private final Evaluator eval; - public RollupSchedulable(String query, List rollupMetadataIds, Evaluator eval) { + public RollupSchedulable(String query, String rollupObjectName, Evaluator eval) { this.query = query; - this.rollupMetadataIds = rollupMetadataIds; + this.rollupObject = getSObjectFromName(rollupObjectName).getSObjectType(); this.eval = eval; try { Database.query(this.query); - } catch (Exception ex) { + } catch (QueryException ex) { throw new QueryException('There\'s a problem with your query: ' + ex.getMessage() + '\n' + ex.getStackTraceString()); } } public void execute(SchedulableContext sc) { - List allMetadata = getMetadataFromCache(Rollup__mdt.SObjectType); - Map> lookupObjectToMetadata = new Map>(); - for (Rollup__mdt meta : allMetadata) { - if (rollupMetadataIds.contains(meta.Id) == false) { - continue; - } else if (lookupObjectToMetadata.containsKey(meta.LookupObject__c)) { - lookupObjectToMetadata.get(meta.LookupObject__c).add(meta); - } else { - lookupObjectToMetadata.put(meta.LookupObject__c, new List{ meta }); - } - } - List rollupsOrderedByLookupItem = new List(); - for (String lookupObject : lookupObjectToMetadata.keySet()) { - rollupsOrderedByLookupItem.addAll(lookupObjectToMetadata.get(lookupObject)); - } + List metadata = getRollupMetadataBySObject(this.rollupObject); List calcItems = Database.query(this.query); - getRollup( - rollupsOrderedByLookupItem, - calcItems.getSObjectType(), - calcItems, - new Map(calcItems), - this.eval, - RollupInvocationPoint.FROM_SCHEDULED - ) - .runCalc(); + getRollup(metadata, calcItems.getSObjectType(), calcItems, new Map(calcItems), this.eval, RollupInvocationPoint.FROM_SCHEDULED).runCalc(); } } } diff --git a/rollup/main/default/classes/RollupCalculator.cls b/rollup/main/default/classes/RollupCalculator.cls index e8db4599..a5a8f7a5 100644 --- a/rollup/main/default/classes/RollupCalculator.cls +++ b/rollup/main/default/classes/RollupCalculator.cls @@ -681,10 +681,11 @@ public without sharing abstract class RollupCalculator { protected override void setReturnValue() { String trimmedDelimiter = this.concatDelimiter.trim(); String possibleReturnValue = this.stringVal.normalizeSpace(); - String withoutEndingComma = possibleReturnValue.endsWith(trimmedDelimiter) - ? possibleReturnValue.substringBeforeLast(trimmedDelimiter) - : possibleReturnValue; - this.returnVal = (withoutEndingComma.startsWith(trimmedDelimiter) ? withoutEndingComma.substring(1, withoutEndingComma.length()) : withoutEndingComma) + while(possibleReturnValue.endsWith(trimmedDelimiter)) { + possibleReturnValue = possibleReturnValue.substringBeforeLast(trimmedDelimiter).trim(); + } + + this.returnVal = (possibleReturnValue.startsWith(trimmedDelimiter) ? possibleReturnValue.substring(1, possibleReturnValue.length()) : possibleReturnValue) .trim(); } @@ -712,6 +713,9 @@ public without sharing abstract class RollupCalculator { public override void handleUpdateConcat(SObject calcItem, Map oldCalcItems) { Boolean isConcatDistinct = this.op.name().contains(Rollup.Op.CONCAT_DISTINCT.name()); String newVal = String.valueOf(calcItem.get(this.opFieldOnCalcItem)); + if(newVal == null) { + newVal = ''; // this.stringVal.contains throws for null, below + } String priorString = String.valueOf((oldCalcItems.containsKey(calcItem.Id) ? oldCalcItems.get(calcItem.Id).get(this.opFieldOnCalcItem) : newVal)); if (isConcatDistinct && this.stringVal.contains(newVal) == false || !isConcatDistinct) { this.stringVal = this.replaceWithDelimiter(this.stringVal, priorString, newVal); @@ -720,6 +724,9 @@ public without sharing abstract class RollupCalculator { public override void handleDeleteConcat(SObject calcItem) { String existingVal = String.valueOf(calcItem.get(this.opFieldOnCalcItem)); + if(existingVal == null) { + return; + } this.stringVal = this.replaceWithDelimiter(this.stringVal, existingVal, ''); } @@ -979,4 +986,4 @@ public without sharing abstract class RollupCalculator { return sorter.compare(value, ((ComparableItem) o).value); } } -} +} \ No newline at end of file diff --git a/rollup/main/default/classes/RollupEvaluator.cls b/rollup/main/default/classes/RollupEvaluator.cls index b440b2bc..37bd099a 100644 --- a/rollup/main/default/classes/RollupEvaluator.cls +++ b/rollup/main/default/classes/RollupEvaluator.cls @@ -99,6 +99,7 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato private final String originalWhereClause; private final String whereClause; private final List conditionalGroupings = new List(); + private final Set validRelationshipNames; public WhereFieldEvaluator(String whereClause, SObjectType calcItemSObjectType) { this.originalWhereClause = whereClause; @@ -110,6 +111,7 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato } this.whereClause = whereClause; this.createConditions(calcItemSObjectType); + this.validRelationshipNames = this.getValidRelationshipNames(calcItemSObjectType); } public List getRelationshipFieldNames() { @@ -117,19 +119,22 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato for (ConditionalGrouping conditionalGrouping : this.conditionalGroupings) { for (WhereFieldCondition condition : conditionalGrouping.innerConditions) { if (fieldNames.contains(condition.fieldName) == false) { - if(condition.fieldName.contains('__c.') == false) { + if (condition.fieldName.contains('__c.') == false && this.validRelationshipNames.contains(condition.fieldName.substringBefore('.'))) { fieldNames.add(condition.fieldName); } if (condition.fieldName.contains('.')) { List relationshipNames = condition.fieldName.split(RELATIONSHIP_FIELD_DELIMITER); String priorVal = ''; for (String relationshipName : relationshipNames) { - if (String.isNotBlank(priorVal)) { + if(String.isBlank(priorVal) && this.validRelationshipNames.contains(relationshipName) == false) { + continue; + // it's the first run, don't keep going if the relationship name doesn't exist + } else if (String.isNotBlank(priorVal)) { priorVal += '.'; } priorVal += relationshipName.replace('__c', '__r').trim(); } - if (fieldNames.contains(priorVal) == false) { + if (String.isNotBlank(priorVal) && fieldNames.contains(priorVal) == false) { fieldNames.add(priorVal); } } @@ -154,6 +159,18 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato return matches; } + private Set getValidRelationshipNames(SObjectType sObjectType) { + Set validRelationshipNames = new Set(); + List fields = sObjectType.getDescribe().fields.getMap().values(); + for(SObjectField field : fields) { + String relationshipName = field.getDescribe().getRelationshipName(); + if(String.isNotBlank(relationshipName)) { + validRelationshipNames.add(relationshipName); + } + } + return validRelationshipNames; + } + private void createConditions(SObjectType calcItemSObjectType) { List splitWheres = this.getSoqlWhereClauses(this.whereClause, calcItemSObjectType); try { @@ -271,7 +288,7 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato String fieldName = whereClause.substring(0, whereClause.indexOf(' ')); whereClause = whereClause.replace(fieldName, '').trim(); String criteria = whereClause.substring(0, whereClause.indexOf(' ')).trim(); - String value = this.getValue( whereClause.substringAfter(criteria)); + String value = this.getValue(whereClause.substringAfter(criteria)); if (value.startsWith('(') && value.endsWith(')')) { List values = value.substring(1, value.length() - 1).split(','); @@ -292,10 +309,10 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato if (this.originalWhereClause.contains(potentialReplacement)) { whereClausePiece = potentialReplacement; break; - } else if(this.originalWhereClause.containsIgnoreCase(whereClausePiece.replace(' = ', ' in '))) { + } else if (this.originalWhereClause.containsIgnoreCase(whereClausePiece.replace(' = ', ' in '))) { whereClausePiece = whereClausePiece.replace(' = ', ' in '); break; - } else if(this.originalWhereClause.containsIgnoreCase(whereClausePiece.replace(' != ', ' not in '))) { + } else if (this.originalWhereClause.containsIgnoreCase(whereClausePiece.replace(' != ', ' not in '))) { whereClausePiece = whereClausePiece.replace(' != ', ' not in '); break; } @@ -522,4 +539,4 @@ public without sharing abstract class RollupEvaluator implements Rollup.Evaluato } } } -} +} \ No newline at end of file diff --git a/rollup/main/default/classes/RollupEvaluator.cls-meta.xml b/rollup/main/default/classes/RollupEvaluator.cls-meta.xml index d75b0582..df1fe2f7 100644 --- a/rollup/main/default/classes/RollupEvaluator.cls-meta.xml +++ b/rollup/main/default/classes/RollupEvaluator.cls-meta.xml @@ -1,5 +1,5 @@ - 51.0 - Active + 51.0 + Active diff --git a/rollup/main/default/classes/RollupEvaluatorTests.cls b/rollup/main/default/classes/RollupEvaluatorTests.cls index 96b4c146..115e45c9 100644 --- a/rollup/main/default/classes/RollupEvaluatorTests.cls +++ b/rollup/main/default/classes/RollupEvaluatorTests.cls @@ -97,10 +97,7 @@ private class RollupEvaluatorTests { @isTest static void shouldFilterCalcItemsBasedOnWhereClauseForLists() { - List accIds = new List{ - '0016g0000000000001', - '0016g0000000000002' - }; + List accIds = new List{ '0016g0000000000001', '0016g0000000000002' }; String whereClause = 'AccountId IN ' + JSON.serialize(accIds).replace('[', '(').replace(']', ')').replaceAll('"', '\''); Opportunity oppOne = new Opportunity(AccountId = accIds[0]); @@ -289,8 +286,8 @@ private class RollupEvaluatorTests { @isTest static void shouldCorrectlyIdentifyIncludesForMultiSelectPicklists() { // QuickText.Channel is the only multi-select picklist in a vanilla Salesforce org - QuickText qt = new QuickText(Channel='AAA;BBB;CCC'); - QuickText nonMatch = new QuickText(Channel='AAA'); + QuickText qt = new QuickText(Channel = 'AAA;BBB;CCC'); + QuickText nonMatch = new QuickText(Channel = 'AAA'); String whereClause = 'Channel INCLUDES (\'AAA;CCC\')'; Rollup.Evaluator eval = new RollupEvaluator.WhereFieldEvaluator(whereClause, QuickText.SObjectType); @@ -353,7 +350,7 @@ private class RollupEvaluatorTests { Opportunity oldOpp = opp.clone(true, true); oldOpp.StageName = 'Old Name'; - Opportunity secondOpp = new Opportunity(Id = '0066g000000000000Z', Amount = 15); + Opportunity secondOpp = new Opportunity(Id = '0066g000000000000Z', Amount = 15); Rollup__mdt rollupMetadata = new Rollup__mdt(CalcItemWhereClause__c = 'Amount > 20'); @@ -389,6 +386,14 @@ private class RollupEvaluatorTests { System.assertNotEquals(true, eval.matches(secondOpp), 'Should not match since StageName has not changed'); } + @isTest + static void shouldNotReturnRelationshipFieldsIfTheyAreNotDirectlyRelated() { + String queryString = 'User.Id IN (\'0056g000002GeR0AAA\')'; + RollupEvaluator.WhereFieldEvaluator eval = new RollupEvaluator.WhereFieldEvaluator(queryString, Opportunity.SObjectType); + + System.assertEquals(new List(), eval.getRelationshipFieldNames()); + } + private class AlwaysTrueEval implements Rollup.Evaluator { public Boolean matches(Object calcItem) { return true; diff --git a/rollup/main/default/classes/RollupEvaluatorTests.cls-meta.xml b/rollup/main/default/classes/RollupEvaluatorTests.cls-meta.xml index d75b0582..811305d0 100644 --- a/rollup/main/default/classes/RollupEvaluatorTests.cls-meta.xml +++ b/rollup/main/default/classes/RollupEvaluatorTests.cls-meta.xml @@ -1,5 +1,5 @@ - 51.0 - Active - + 51.0 + Active + \ No newline at end of file diff --git a/rollup/main/default/classes/RollupFlowBulkProcessor.cls b/rollup/main/default/classes/RollupFlowBulkProcessor.cls index f57f8409..52a89744 100644 --- a/rollup/main/default/classes/RollupFlowBulkProcessor.cls +++ b/rollup/main/default/classes/RollupFlowBulkProcessor.cls @@ -54,6 +54,7 @@ global without sharing class RollupFlowBulkProcessor { input.rollupOperation = meta.RollupOperation__c; input.rollupSObjectName = meta.LookupObject__c; input.isRollupStartedFromParent = meta.IsRollupStartedFromParent__c; + input.grandparentRelationshipFieldPath = meta.GrandparentRelationshipFieldPath__c; // everything else is supplied from the invocable input.rollupContext = flowInput.rollupContext; input.recordsToRollup = flowInput.recordsToRollup; diff --git a/rollup/main/default/classes/RollupFullBatchRecalculator.cls b/rollup/main/default/classes/RollupFullBatchRecalculator.cls index 97daf8de..5fd4e862 100644 --- a/rollup/main/default/classes/RollupFullBatchRecalculator.cls +++ b/rollup/main/default/classes/RollupFullBatchRecalculator.cls @@ -2,15 +2,19 @@ public class RollupFullBatchRecalculator extends Rollup { private final String queryString; private final Rollup__mdt rollupInfo; private final SObjectType calcItemType; + private final Set recordIds; - public RollupFullBatchRecalculator(String queryString, RollupInvocationPoint invokePoint, Rollup__mdt rollupInfo, SObjectType calcItemType) { + public RollupFullBatchRecalculator(String queryString, RollupInvocationPoint invokePoint, Rollup__mdt rollupInfo, SObjectType calcItemType, Set recordIds) { super(invokePoint); this.queryString = queryString; this.rollupInfo = rollupInfo; + this.recordIds = recordIds; } public override Database.QueryLocator start(Database.BatchableContext bc) { Set objIds = new Set(); // necessary; there's a bind variable in the query string + // note - if the optional where clause was appended to the passed in query string, this.recordIds is also + // used as a bind variable return Database.getQueryLocator(this.queryString); } diff --git a/rollup/main/default/classes/RollupRelationshipFieldFinder.cls b/rollup/main/default/classes/RollupRelationshipFieldFinder.cls new file mode 100644 index 00000000..299b4a0c --- /dev/null +++ b/rollup/main/default/classes/RollupRelationshipFieldFinder.cls @@ -0,0 +1,279 @@ +/** + * Big caveat here - this class can only be used with lookup relationships. The larger Rollup framework + * accepts and even welcomes text-based keys being used, but here we accept that text-based keys + * are likely niche, anyway, and that people looking to make use of a grandchild -> grandparent (or greater!) + * rollup are likely operating using lookups anyway + */ +public without sharing class RollupRelationshipFieldFinder { + private final List originalParts; + private final Traversal traversal; + private final SObjectType ultimateParent; + private final RollupControl__mdt rollupControl; + private final Map oldRecords; + private final Set uniqueFinalFieldNames; + + private List recommencementRecords; + private List records; + private List relationshipParts; + private Boolean isFirstRun = true; + private String currentRelationshipName; + + public RollupRelationshipFieldFinder( + RollupControl__mdt rollupControl, + String relationshipPathName, + Set uniqueFinalFieldNames, + SObjectType ultimateParent, + Map oldRecords + ) { + this.traversal = new Traversal(this); + this.relationshipParts = relationshipPathName.split('\\.'); + this.rollupControl = rollupControl; + this.ultimateParent = ultimateParent; + this.oldRecords = oldRecords; + this.uniqueFinalFieldNames = uniqueFinalFieldNames; + + if (this.relationshipParts.size() == 1) { + this.relationshipParts.add(0, ultimateParent.getDescribe().getName()); + } + this.originalParts = new List(this.relationshipParts); + } + + private class CombinedHierarchy { + private List oldHierarchy; + private List currentHierarchy; + } + + public class Traversal { + private Boolean isFinished = false; + private Boolean isAbortedEarly = false; + + private final Map lookupIdToFinalRecords = new Map(); + private Map> lookupIdMap = new Map>(); + private final Map> hierarchy = new Map>(); + private final RollupRelationshipFieldFinder finder; + + private Traversal(RollupRelationshipFieldFinder finder) { + this.finder = finder; + } + + public Boolean getIsFinished() { + return this.isFinished; + } + + public SObject retrieveParent(Id descendantId) { + return this.lookupIdToFinalRecords.get(descendantId); + } + + public List getAllParents() { + if (this.isAbortedEarly) { + return new List(); + } + // not ideal, but because multiple parents can be tied to different descendants ... + return new List(new Set(this.lookupIdToFinalRecords.values())); + } + + public void recommence() { + this.finder.getParents(this.finder.recommencementRecords); + } + + public Map> getParentLookupToRecords() { + Map> parentToLookupRecords = new Map>(); + if (this.isAbortedEarly) { + return parentToLookupRecords; + } + for (SObject record : this.finder.records) { + SObject parentRecord = this.retrieveParent(record.Id); + if (parentToLookupRecords.containsKey(parentRecord.Id)) { + parentToLookupRecords.get(parentRecord.Id).add(record); + } else { + parentToLookupRecords.put(parentRecord.Id, new List{ record }); + } + } + return parentToLookupRecords; + } + + public Boolean isUltimatelyReparented(SObject record, String relationshipFieldName) { + Id currentLookupId = (Id) record.get(relationshipFieldName); + Id oldLookupId = (Id) (this.finder.oldRecords.containsKey(record.Id) + ? this.finder.oldRecords.get(record.Id).get(relationshipFieldName) + : currentLookupId); + if (currentLookupId == oldLookupId) { + return false; + } else if (currentLookupId == null || oldLookupId == null) { + // this is pretty cut and dry. if we are moving from having a lookup to not having one, or vice versa, it's a reparenting + return true; + } + CombinedHierarchy combinedHierarchy = this.getHierarchy(record, relationshipFieldName); + if (combinedHierarchy.currentHierarchy?.size() > 0 && combinedHierarchy.oldHierarchy?.size() > 0) { + // the last Ids present in the chain have to match, otherwise it's a reparenting + return combinedHierarchy.currentHierarchy[combinedHierarchy.currentHierarchy.size() - 1] != + combinedHierarchy.oldHierarchy[combinedHierarchy.oldHierarchy.size() - 1]; + } else { + // if there was only one hop, we can just compare the Ids. This comparison has to be last + // because it's possible (as explained below, where the hierarchy is created) + // that only the intermediate lookup fields have changed, and not the ultimate + // parent (which is what gets checked above). + // only if that isn't the case can we do the simple comparison below + return currentLookupId != oldLookupId; + } + } + + public Id getOldLookupId(SObject record, String relationshipFieldName) { + CombinedHierarchy combinedHierarchy = this.getHierarchy(record, relationshipFieldName); + return combinedHierarchy.oldHierarchy?.isEmpty() == false ? combinedHierarchy.currentHierarchy[0] : null; + } + + private CombinedHierarchy getHierarchy(SObject record, String relationshipFieldName) { + Id currentLookupId = (Id) record.get(relationshipFieldName); + Id oldLookupId = (Id) (this.finder.oldRecords.containsKey(record.Id) + ? this.finder.oldRecords.get(record.Id).get(relationshipFieldName) + : currentLookupId); + CombinedHierarchy combinedHierarchy = new CombinedHierarchy(); + combinedHierarchy.currentHierarchy = this.hierarchy.get(currentLookupId); + combinedHierarchy.oldHierarchy = this.hierarchy.get(oldLookupId); + return combinedHierarchy; + } + } + + public Traversal getParents(List records) { + if (records.isEmpty() || this.relationshipParts.isEmpty()) { + this.traversal.isFinished = true; + return this.traversal; + } else if ( + this.rollupControl.MaxQueryRows__c < Limits.getQueries() || + Limits.getLimitQueryRows() / 4 < Limits.getQueryRows() || + Limits.getLimitHeapSize() / 2 < Limits.getHeapSize() + ) { + // we pop fields off of the list while recursively iterating + // which means we need to re-add the last field used if we are stopping + // due to limits + this.relationshipParts.add(0, this.currentRelationshipName); + return this.traversal; + } + + // even before the recursion begins, the List won't be strongly typed + SObjectType baseSObjectType = records[0].getSObjectType(); + if (baseSObjectType == this.ultimateParent) { + this.prepFinishedObject(records); + return this.traversal; + } else { + return this.recurseThroughObjectChain(records, baseSObjectType); + } + } + + private SObjectField getField(Map fieldMap, String relationshipPart) { + for (String key : fieldMap.keySet()) { + SObjectField field = fieldMap.get(key); + if (field.getDescribe().getRelationshipName() == relationshipPart) { + return field; + } else if (field.getDescribe().getName() == relationshipPart) { + return field; + } + } + // effectively a throw; if there's no match, nothing else will work + return null; + } + + private Set getDescendantIds(Id lookupId, Set descendantIds) { + Boolean hasMatch = this.traversal.lookupIdMap.containsKey(lookupId); + if (hasMatch) { + List extraIds = this.traversal.lookupIdMap.get(lookupId); + for (Id descendantId : extraIds) { + descendantIds.addAll(this.getDescendantIds(descendantId, descendantIds)); + } + return descendantIds; + } + descendantIds.add(lookupId); + return descendantIds; + } + + private void prepFinishedObject(List records) { + this.traversal.isFinished = true; + for (SObject record : records) { + Set descendantIds = this.getDescendantIds(record.Id, new Set()); + for (Id descendantId : descendantIds) { + if (descendantId != record.Id) { + this.traversal.lookupIdToFinalRecords.put(descendantId, record); + } + } + } + this.traversal.isFinished = true; + this.relationshipParts = this.originalParts; // reset to initial state in case outer method is re-called + this.traversal.lookupIdMap = new Map>(); // try to spare the heap + } + + private Traversal recurseThroughObjectChain(List records, SObjectType baseSObjectType) { + // cache the latest records through in case we need to continue later + this.recommencementRecords = records; + this.currentRelationshipName = this.relationshipParts.remove(0); + Map fieldMap = baseSObjectType.getDescribe().fields.getMap(); + SObjectField field = this.getField(fieldMap, currentRelationshipName); + + Set lookupIds = new Set(); + Id firstId; + for (SObject record : records) { + Id lookupId = (Id) record.get(field); + if (firstId == null) { + firstId = lookupId; + } + if (String.isNotBlank(lookupId)) { + lookupIds.add(lookupId); + + if (this.traversal.lookupIdMap.containsKey(lookupId)) { + this.traversal.lookupIdMap.get(lookupId).add(record.Id); + } else { + this.traversal.lookupIdMap.put(lookupId, new List{ record.Id }); + } + + if (this.isFirstRun) { + // we need to keep track of potentially reparented lookups to aid with the note below + if (this.oldRecords.containsKey(record.Id)) { + Id oldLookupId = (Id) this.oldRecords.get(record.Id).get(field); + if (String.isNotBlank(oldLookupId) && oldLookupId != lookupId) { + lookupIds.add(oldLookupId); + this.traversal.hierarchy.put(oldLookupId, new List{ oldLookupId }); + } + } + this.traversal.hierarchy.put(lookupId, new List{ lookupId }); + } else if (this.traversal.hierarchy.containsKey(record.Id)) { + // track the hierarchy of objects to help in determining whether or not something + // has ultimately been reparented + // for example: + // * Object 1 -> Parent 1 -> Grandparent 1 could be updated to + // * Object 1 -> Parent 2 -> Grandparent 1 + // this would "traditionally" be a reparenting situation, but if we are skipping + // the intermediate objects for a rollup and the end result is the same, we need + // to avoid reporting false positives like this one + this.traversal.hierarchy.get(record.Id).add(lookupId); + } + } + } + // no matter how far up the chain we are, if we arrive at a point where there are no records, we're done + if (firstId == null) { + this.prepFinishedObject(records); + this.traversal.isAbortedEarly = true; + return this.traversal; + } + + String nextFieldToLookup = this.relationshipParts[0].replace('__r', '__c'); + SObjectType nextSObjectType = firstId.getSObjectType(); + SObjectField nextFieldToken = this.getField(nextSObjectType.getDescribe().fields.getMap(), nextFieldToLookup); + List fieldNames = new List(); + if (nextSObjectType == this.ultimateParent) { + fieldNames.addAll(this.uniqueFinalFieldNames); + } else { + fieldNames.add(nextFieldToken.getDescribe().getName()); + } + if (fieldNames.contains('Id') == false) { + fieldNames.add('Id'); + } + // NB - we only support one route through polymorphic fields such as Task.WhoId and Task.WhatId for this sort of thing + String query = 'SELECT ' + String.join(fieldNames, ',') + ' FROM ' + nextSObjectType.getDescribe().getName() + ' WHERE Id = :lookupIds'; + // recurse through till we get to the top/bottom of the chain + if (this.isFirstRun) { + this.records = records; + this.isFirstRun = false; + } + return this.getParents(Database.query(query)); + } +} diff --git a/rollup/main/default/classes/RollupRelationshipFieldFinder.cls-meta.xml b/rollup/main/default/classes/RollupRelationshipFieldFinder.cls-meta.xml new file mode 100644 index 00000000..2b461287 --- /dev/null +++ b/rollup/main/default/classes/RollupRelationshipFieldFinder.cls-meta.xml @@ -0,0 +1,5 @@ + + + 51.0 + Active + diff --git a/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls b/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls new file mode 100644 index 00000000..f1bdda6a --- /dev/null +++ b/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls @@ -0,0 +1,182 @@ +@isTest +private class RollupRelationshipFieldFinderTests { + static RollupControl__mdt control = new RollupControl__mdt(MaxQueryRows__c = 10000); + + @isTest + static void shouldFindParentRelationshipBetweenStandardObjects() { + Account parent = new Account(Name = 'Parent relationship between standard objects'); + insert parent; + + Opportunity opp = new Opportunity(AccountId = parent.Id, Name = 'Child opp', StageName = 'Prospecting', CloseDate = System.today()); + insert opp; + + Set uniqueFieldNames = new Set{ 'Name', 'Id' }; + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder( + control, + 'Account.Name', + uniqueFieldNames, + Account.SObjectType, + new Map() + ); + + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(new List{ opp }); + + System.assertEquals(parent, traversal.retrieveParent(opp.Id)); + + finder = new RollupRelationshipFieldFinder(control, 'Name', uniqueFieldNames, Account.SObjectType, new Map()); + traversal = finder.getParents(new List{ opp }); + + System.assertEquals(parent, traversal.retrieveParent(opp.Id)); + } + + @isTest + static void shouldFindGrandparentRelationshipBetweenStandardObjects() { + Account parent = new Account(Name = 'Parent account looking up to User'); + insert parent; + + Opportunity opp = new Opportunity(AccountId = parent.Id, Name = 'Child opp looking up to account', StageName = 'Prospecting', CloseDate = System.today()); + insert opp; + + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder( + control, + 'Account.Owner.Name', + new Set{ 'Name', 'Id' }, + User.SObjectType, + new Map() + ); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(new List{ opp }); + + parent = [SELECT OwnerId FROM Account WHERE Id = :parent.Id]; + System.assertEquals( + [SELECT Id, Name FROM User WHERE Id = :parent.OwnerId][0], + traversal.retrieveParent(opp.Id), + 'User should have been retrieved correctly!' + ); + } + + @isTest + static void shouldBailEarlyIfQueryCountExceedsControlCount() { + Account acc = new Account(Name = 'Parent to opp'); + insert acc; + + Opportunity opp = new Opportunity(AccountId = acc.Id, Name = 'Child opp'); + control.MaxQueryRows__c = 1; + + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder( + control, + 'Account.Owner.Name', + new Set{ 'Name', 'Id' }, + User.SObjectType, + new Map() + ); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(new List{ opp }); + + System.assertEquals(false, traversal.getIsFinished(), 'Should have bailed early!'); + } + + @isTest + static void shouldNotReportFalsePositiveIfUltimateParentStaysTheSame() { + Account intermediateOne = new Account(Name = 'Intermediate 1'); + Account intermediateTwo = new Account(Name = 'Intermediate 2'); + insert new List{ intermediateOne, intermediateTwo }; + + List updatedAccounts = [SELECT Id, OwnerId, Name FROM Account]; + if (updatedAccounts.size() == 2) { + // don't run the rest of the test if the org has some kind of ownership assignment going on that would invalidate + // the results + Account one = updatedAccounts[0]; + Account two = updatedAccounts[1]; + if (one.OwnerId != two.OwnerId) { + return; + } else { + intermediateOne = one.Id == intermediateOne.Id ? one : two; + intermediateTwo = two.Id == intermediateTwo.Id ? two : one; + } + } + + Opportunity opp = new Opportunity(AccountId = intermediateTwo.Id, Name = 'Child reparented', StageName = 'Prospecting', CloseDate = System.today()); + List opps = new List{ opp }; + insert opps; + + Map oldOpps = new Map{ opp.Id => new Opportunity(Id = opp.Id, AccountId = intermediateOne.Id) }; + + Set uniqueFieldNames = new Set{ 'Name', 'Id' }; + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder(control, 'Account.Owner.Name', uniqueFieldNames, User.SObjectType, oldOpps); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(opps); + + System.assertEquals(false, traversal.isUltimatelyReparented(opp, 'AccountId'), 'Should not report false positive!'); + System.assertEquals(intermediateTwo.Id, traversal.getOldLookupId(opp, 'AccountId')); + + finder = new RollupRelationshipFieldFinder(control, 'Account.Name', uniqueFieldNames, Account.SObjectType, oldOpps); + traversal = finder.getParents(opps); + + System.assertEquals(true, traversal.isUltimatelyReparented(opp, 'AccountId'), 'Should correctly report reparenting if ultimate lookup is different'); + } + + @isTest + static void shouldReportReparentingCorrectlyForNulls() { + Account intermediateOne = new Account(Name = 'Intermediate 1'); + insert new List{ intermediateOne }; + + Opportunity opp = new Opportunity(AccountId = intermediateOne.Id, Name = 'Child reparented', StageName = 'Prospecting', CloseDate = System.today()); + List opps = new List{ opp }; + insert opps; + + Map oldOpps = new Map{ opp.Id => new Opportunity(Id = opp.Id, AccountId = null) }; + + Set uniqueFieldNames = new Set{ 'Id', 'Name' }; + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder(control, 'Account.Owner.Name', uniqueFieldNames, User.SObjectType, oldOpps); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(opps); + + System.assertEquals(true, traversal.isUltimatelyReparented(opp, 'AccountId'), 'Should correctly report reparenting if old lookup null'); + + oldOpps.put(opp.Id, new Opportunity(Id = opp.Id, AccountId = intermediateOne.Id)); + opp.AccountId = null; + update opp; + + finder = new RollupRelationshipFieldFinder(control, 'Account.Owner.Name', uniqueFieldNames, User.SObjectType, oldOpps); + System.assertEquals(true, traversal.isUltimatelyReparented(opp, 'AccountId'), 'Should correctly report reparenting if new lookup is null'); + } + + @isTest + static void shouldReportReparentingCorrectlyForImmediateParent() { + Account parentOne = new Account(Name = 'Parent1'); + Account parentTwo = new Account(Name = 'Parent2'); + insert new List{ parentOne, parentTwo }; + + Opportunity oppOne = new Opportunity(AccountId = parentOne.Id, Name = 'Child1', StageName = 'Prospecting', CloseDate = System.today()); + Opportunity oppTwo = new Opportunity(AccountId = parentOne.Id, Name = 'Child1', StageName = 'Prospecting', CloseDate = System.today()); + List opps = new List{ oppOne, oppTwo }; + insert opps; + + Map oldOpps = new Map{ oppOne.Id => oppOne, oppTwo.Id => new Opportunity(AccountId = parentTwo.Id) }; + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder(control, 'Name', new Set{ 'Name', 'Id' }, Account.SObjectType, oldOpps); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(opps); + + System.assertEquals(true, traversal.isUltimatelyReparented(oppTwo, 'AccountId')); + System.assertEquals(false, traversal.isUltimatelyReparented(oppOne, 'AccountId')); + } + + @isTest + static void shouldTrackMultipleParents() { + Account parentOne = new Account(Name = 'SoloParent'); + insert parentOne; + + Opportunity oppOne = new Opportunity(AccountId = parentOne.Id, Name = 'FirstParentedChild', StageName = 'Prospecting', CloseDate = System.today()); + Opportunity oppTwo = new Opportunity(AccountId = parentOne.Id, Name = 'SecondParentedChild', StageName = 'Prospecting', CloseDate = System.today()); + List opps = new List{ oppOne, oppTwo }; + insert opps; + + RollupRelationshipFieldFinder finder = new RollupRelationshipFieldFinder( + control, + 'Name', + new Set{ 'Name', 'Id' }, + Account.SObjectType, + new Map() + ); + RollupRelationshipFieldFinder.Traversal traversal = finder.getParents(opps); + + System.assertEquals(parentOne, traversal.retrieveParent(oppOne.Id), 'First opp parent should not be exluded!'); + System.assertEquals(parentOne, traversal.retrieveParent(oppTwo.Id), 'Second opp should not have been excluded!'); + } +} diff --git a/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls-meta.xml b/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls-meta.xml new file mode 100644 index 00000000..2b461287 --- /dev/null +++ b/rollup/main/default/classes/RollupRelationshipFieldFinderTests.cls-meta.xml @@ -0,0 +1,5 @@ + + + 51.0 + Active + diff --git a/rollup/main/default/classes/RollupTests.cls b/rollup/main/default/classes/RollupTests.cls index d3f9fff9..f112692a 100644 --- a/rollup/main/default/classes/RollupTests.cls +++ b/rollup/main/default/classes/RollupTests.cls @@ -510,10 +510,7 @@ private class RollupTests { static void shouldRunDirectlyFromApex() { Account acc = [SELECT Id FROM Account]; - List opps = new List{ - new Opportunity(AccountId = acc.Id, Amount = 5), - new Opportunity(AccountId = acc.Id, Amount = 10) - }; + List opps = new List{ new Opportunity(AccountId = acc.Id, Amount = 5), new Opportunity(AccountId = acc.Id, Amount = 10) }; Rollup.records = opps; Rollup.rollupMetadata = new List{ @@ -651,22 +648,19 @@ private class RollupTests { } @isTest - static void shouldThrowExceptionWhenBatchedOperationWithMultipleSObjectTypes() { - DMLMock mock = loadAccountIdMock(new List{ new Opportunity(Amount = 50, Name = 'Test') }); + static void shouldDeferUpdateWhenMaxParentRowsLessThanCurrentUpdateRows() { + Opportunity opp = new Opportunity(Amount = 50, Name = 'MaxParentRows', StageName = 'Prospecting', CloseDate = System.today()); + DMLMock mock = loadAccountIdMock(new List{ opp }); Rollup.apexContext = TriggerOperation.AFTER_INSERT; Rollup.shouldRunAsBatch = true; + Rollup.defaultControl = new RollupControl__mdt(MaxParentRowsUpdatedAtOnce__c = 0, BatchChunkSize__c = 1); - Exception ex; - try { - Rollup.batch( - Rollup.sumFromApex(Opportunity.Amount, Opportunity.AccountId, Account.Id, Account.AnnualRevenue, Account.SObjectType), - Rollup.concatFromApex(Opportunity.Name, Opportunity.CreatedById, User.Id, User.Username, User.SObjectType) - ); - } catch (Exception e) { - ex = e; - } + Test.startTest(); + Rollup.sumFromApex(Opportunity.Amount, Opportunity.AccountId, Account.Id, Account.AnnualRevenue, Account.SObjectType).runCalc(); + Test.stopTest(); - System.assertNotEquals(null, ex); + Account acc = [SELECT AnnualRevenue FROM Account]; + System.assertEquals(50, acc.AnnualRevenue, 'Account should have been updated since the mock is not used async'); } @isTest @@ -2527,7 +2521,7 @@ private class RollupTests { static void shouldRunAsBatchableWhenSpecificRollupIsBatchable() { DMLMock mock = loadAccountIdMock(new List{ new Opportunity(Amount = 1) }); Rollup.apexContext = TriggerOperation.AFTER_INSERT; - Rollup.specificControl = new RollupControl__mdt(ShouldRunAs__c = 'Batchable', BatchChunkSize__c = 1); + Rollup.defaultControl = new RollupControl__mdt(ShouldRunAs__c = 'Batchable', BatchChunkSize__c = 1, MaxLookupRowsBeforeBatching__c = 0); Test.startTest(); Rollup.countFromApex(Opportunity.Amount, Opportunity.AccountId, Account.Id, Account.AnnualRevenue, Account.SObjectType).runCalc(); @@ -2541,7 +2535,7 @@ private class RollupTests { static void shouldNotRunAsBatchableWhenDefaultIsBatchableAndRecordsAreLessThanBatchableLimit() { DMLMock mock = loadAccountIdMock(new List{ new Opportunity(Amount = 1) }); Rollup.apexContext = TriggerOperation.AFTER_INSERT; - Rollup.defaultControl = new RollupControl__mdt(ShouldRunAs__c = 'Batchable', MaxLookupRowsBeforeBatching__c = 1000); + Rollup.defaultControl = new RollupControl__mdt(ShouldRunAs__c = 'Batchable', MaxLookupRowsBeforeBatching__c = 1000, BatchChunkSize__c = 50); Test.startTest(); Rollup.countFromApex(Opportunity.Amount, Opportunity.AccountId, Account.Id, Account.AnnualRevenue, Account.SObjectType).runCalc(); @@ -2573,7 +2567,7 @@ private class RollupTests { Exception ex; try { - Rollup.schedule('Test bad query', '0 0 0 0 0', veryBadQuery, new List(), null); + Rollup.schedule('Test bad query', '0 0 0 0 0', veryBadQuery, 'Account', null); } catch (Exception e) { ex = e; } @@ -2585,7 +2579,7 @@ private class RollupTests { static void shouldScheduleSuccessfullyForGoodQuery() { String goodQuery = 'SELECT Id, StageName FROM Opportunity WHERE CreatedDate > YESTERDAY'; - String jobId = Rollup.schedule('Test good query' + System.now(), '0 0 0 * * ?', goodQuery, new List(), null); + String jobId = Rollup.schedule('Test good query' + System.now(), '0 0 0 * * ?', goodQuery, 'Opportunity', null); System.assertNotEquals(null, jobId); } @@ -2595,7 +2589,13 @@ private class RollupTests { @isTest static void shouldNotFailForTruncatedTextFields() { Account acc = [SELECT Id FROM Account]; - Opportunity opp = new Opportunity(AccountId = acc.Id, Description = '0'.repeat(256), Name = 'Truncate', StageName = 'Prospecting', CloseDate = System.today()); + Opportunity opp = new Opportunity( + AccountId = acc.Id, + Description = '0'.repeat(256), + Name = 'Truncate', + StageName = 'Prospecting', + CloseDate = System.today() + ); insert opp; Test.startTest(); @@ -2911,6 +2911,119 @@ private class RollupTests { System.assertEquals('Completed', [SELECT Status FROM AsyncApexJob WHERE JobType = 'Queueable' LIMIT 1]?.Status); } + /** Grandparent rollups */ + @isTest + static void shouldAllowGrandparentRollups() { + Account acc = [SELECT Id FROM Account]; + List opps = new List{ + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'One', CloseDate = System.today()), + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'Two', CloseDate = System.today()) + }; + insert opps; + + DMLMock mock = new DMLMock(); + Rollup.DML = mock; + Rollup.shouldRun = true; + Rollup.records = opps; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'Opportunity', + RollupFieldOnCalcItem__c = 'Name', + LookupFieldOnCalcItem__c = 'AccountId', + LookupObject__c = 'User', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'AboutMe', + RollupOperation__c = 'CONCAT', + GrandparentRelationshipFieldPath__c = 'Account.Owner.AboutMe' + ) + }; + Rollup.apexContext = TriggerOperation.AFTER_INSERT; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + System.assertEquals(1, mock.Records.size(), 'Grandparent record should have been found!'); + User updatedUser = (User) mock.Records[0]; + System.assertEquals(opps[0].Name + ', ' + opps[1].Name, updatedUser.AboutMe, 'Grandparent rollup should have worked!'); + } + + @isTest + static void shouldAllowGrandparentRollupsFromParent() { + Account acc = [SELECT Id, OwnerId FROM Account]; + List opps = new List{ + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'One', CloseDate = System.today()), + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'Two', CloseDate = System.today()) + }; + insert opps; + + DMLMock mock = new DMLMock(); + Rollup.DML = mock; + Rollup.shouldRun = true; + Rollup.records = [SELECT Id, AboutMe FROM User WHERE Id = :acc.OwnerId]; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'Opportunity', + RollupFieldOnCalcItem__c = 'Name', + LookupFieldOnCalcItem__c = 'AccountId', + LookupObject__c = 'User', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'AboutMe', + RollupOperation__c = 'CONCAT', + GrandparentRelationshipFieldPath__c = 'Account.Owner.AboutMe', + IsRollupStartedFromParent__c = true + ) + }; + Rollup.apexContext = TriggerOperation.AFTER_INSERT; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + System.assertEquals(1, mock.Records.size(), 'Grandparent record should have been found!'); + User updatedUser = (User) mock.Records[0]; + System.assertEquals(opps[0].Name + ', ' + opps[1].Name, updatedUser.AboutMe, 'Grandparent rollup should have worked!'); + } + + @isTest + static void shouldDeferGrandparentRollupSafelyTillAllParentRecordsAreRetrieved() { + Account acc = [SELECT Id, OwnerId FROM Account]; + List opps = new List{ + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'One', CloseDate = System.today()), + new Opportunity(AccountId = acc.Id, StageName = 'Prospecting', Name = 'Two', CloseDate = System.today()) + }; + insert opps; + + DMLMock mock = new DMLMock(); + Rollup.defaultControl = new RollupControl__mdt(MaxQueryRows__c = 2, BatchChunkSize__c = 1, MaxRollupRetries__c = 100); + // start as synchronous rollup to allow for one deferral + Rollup.specificControl = new RollupControl__mdt(ShouldRunAs__c = 'Synchronous Rollup', MaxQueryRows__c = 2); + Rollup.DML = mock; + Rollup.shouldRun = true; + Rollup.records = opps; + Rollup.rollupMetadata = new List{ + new Rollup__mdt( + CalcItem__c = 'Opportunity', + RollupFieldOnCalcItem__c = 'Name', + LookupFieldOnCalcItem__c = 'AccountId', + LookupObject__c = 'User', + LookupFieldOnLookupObject__c = 'Id', + RollupFieldOnLookupObject__c = 'AboutMe', + RollupOperation__c = 'CONCAT', + GrandparentRelationshipFieldPath__c = 'Account.Owner.AboutMe' + ) + }; + Rollup.apexContext = TriggerOperation.AFTER_INSERT; + + Test.startTest(); + Rollup.runFromTrigger(); + Test.stopTest(); + + System.assertEquals(1, mock.Records.size(), 'Grandparent record should have been found!'); + User updatedUser = (User) mock.Records[0]; + System.assertEquals(opps[0].Name + ', ' + opps[1].Name, updatedUser.AboutMe, 'Grandparent rollup should have worked!'); + } + //** Helpers */ private static DMLMock loadAccountIdMock(List records) { diff --git a/rollup/main/default/customMetadata/RollupControl.Org_Defaults.md-meta.xml b/rollup/main/default/customMetadata/RollupControl.Org_Defaults.md-meta.xml index 0857dcdb..52619e06 100644 --- a/rollup/main/default/customMetadata/RollupControl.Org_Defaults.md-meta.xml +++ b/rollup/main/default/customMetadata/RollupControl.Org_Defaults.md-meta.xml @@ -11,7 +11,7 @@ 3000.0 - MaxLookupRowsForQueueable__c + MaxParentRowsUpdatedAtOnce__c 5000.0 @@ -22,10 +22,6 @@ MaxRollupRetries__c 100.0 - - MaximumRollupRetries__c - - ShouldAbortRun__c false diff --git a/rollup/main/default/layouts/RollupControl__mdt-Rollup Control Layout.layout-meta.xml b/rollup/main/default/layouts/RollupControl__mdt-Rollup Control Layout.layout-meta.xml index 138001aa..cc49c5f1 100644 --- a/rollup/main/default/layouts/RollupControl__mdt-Rollup Control Layout.layout-meta.xml +++ b/rollup/main/default/layouts/RollupControl__mdt-Rollup Control Layout.layout-meta.xml @@ -34,7 +34,7 @@ Edit - MaxLookupRowsForQueueable__c + MaxParentRowsUpdatedAtOnce__c Edit diff --git a/rollup/main/default/layouts/Rollup__mdt-Rollup Layout.layout-meta.xml b/rollup/main/default/layouts/Rollup__mdt-Rollup Layout.layout-meta.xml index e1e03c2b..00da3afc 100644 --- a/rollup/main/default/layouts/Rollup__mdt-Rollup Layout.layout-meta.xml +++ b/rollup/main/default/layouts/Rollup__mdt-Rollup Layout.layout-meta.xml @@ -56,6 +56,10 @@ Required CalcItem__c + + Edit + GrandparentRelationshipFieldPath__c + @@ -155,7 +159,7 @@ false false - 00h6g000007qgMp + 00h54000003XXMf 4 2 diff --git a/rollup/main/default/objects/RollupControl__mdt/fields/MaxLookupRowsForQueueable__c.field-meta.xml b/rollup/main/default/objects/RollupControl__mdt/fields/MaxLookupRowsForQueueable__c.field-meta.xml deleted file mode 100644 index 5dfb758f..00000000 --- a/rollup/main/default/objects/RollupControl__mdt/fields/MaxLookupRowsForQueueable__c.field-meta.xml +++ /dev/null @@ -1,14 +0,0 @@ - - - MaxLookupRowsForQueueable__c - Can be used in conjunction with Ma xLookup Rows Before Batching. Defaults to 5000. - false - DeveloperControlled - Can be used in conjunction with MaxLookupRowsBeforeBatching. Defaults to 5000. - - 18 - false - 0 - Number - false - diff --git a/rollup/main/default/objects/RollupControl__mdt/fields/MaxParentRowsUpdatedAtOnce__c.field-meta.xml b/rollup/main/default/objects/RollupControl__mdt/fields/MaxParentRowsUpdatedAtOnce__c.field-meta.xml new file mode 100644 index 00000000..4fad5e99 --- /dev/null +++ b/rollup/main/default/objects/RollupControl__mdt/fields/MaxParentRowsUpdatedAtOnce__c.field-meta.xml @@ -0,0 +1,14 @@ + + + MaxParentRowsUpdatedAtOnce__c + The maximum number of parent rows that can be updated in a single transaction. Otherwise, Rollup splits the parent items evenly and updates them in separate transactions. + false + DeveloperControlled + The maximum number of parent rows that can be updated in a single transaction. Otherwise, Rollup splits the parent items evenly and updates them in separate transactions. + + 18 + false + 0 + Number + false + diff --git a/rollup/main/default/objects/RollupControl__mdt/fields/MaximumRollupRetries__c.field-meta.xml b/rollup/main/default/objects/RollupControl__mdt/fields/MaximumRollupRetries__c.field-meta.xml deleted file mode 100644 index bc48dff1..00000000 --- a/rollup/main/default/objects/RollupControl__mdt/fields/MaximumRollupRetries__c.field-meta.xml +++ /dev/null @@ -1,15 +0,0 @@ - - - MaximumRollupRetries__c - 100 - Use in conjunction with Max Query Rows. This determines the maximum possible rollup jobs (either batched or queued) that can be spawned from a single overall rollup operation. Default is 100. - false - DeveloperControlled - Use in conjunction with Max Query Rows. This determines the maximum possible rollup jobs (either batched or queued) that can be spawned from a single overall rollup operation. Default is 100. - - 18 - false - 0 - Number - false - diff --git a/rollup/main/default/objects/Rollup__mdt/fields/GrandparentRelationshipFieldPath__c.field-meta.xml b/rollup/main/default/objects/Rollup__mdt/fields/GrandparentRelationshipFieldPath__c.field-meta.xml new file mode 100644 index 00000000..ec2369f6 --- /dev/null +++ b/rollup/main/default/objects/Rollup__mdt/fields/GrandparentRelationshipFieldPath__c.field-meta.xml @@ -0,0 +1,13 @@ + + + GrandparentRelationshipFieldPath__c + If you are rolling up from grandchildren -> grandparent (or greater), supply the field path to the ultimate parent field. As an example, if you were rolling up from Opp Line Items to Account AnnualRevenue, you would do "Opportunity.Account.AnnualRevenue" + false + DeveloperControlled + If you are rolling up from grandchildren -> grandparent (or greater), supply the field path to the ultimate parent field. As an example, if you were rolling up from Opp Line Items to Account AnnualRevenue, you would do "Opportunity.Account.AnnualRevenue" + + 255 + false + Text + false + diff --git a/rollup/main/default/profiles/Admin.profile-meta.xml b/rollup/main/default/profiles/Admin.profile-meta.xml index 69d466ac..2c7a0fd1 100644 --- a/rollup/main/default/profiles/Admin.profile-meta.xml +++ b/rollup/main/default/profiles/Admin.profile-meta.xml @@ -9,6 +9,14 @@ Rollup true + + RollupCalculator + true + + + RollupCalculatorTests + true + RollupEvaluator true @@ -25,10 +33,30 @@ RollupFieldInitializerTests true + + RollupFlowBulkProcessor + true + + + RollupFlowBulkProcessorTests + true + + + RollupFlowBulkSaver + true + RollupFullBatchRecalculator true + + RollupRelationshipFieldFinder + true + + + RollupRelationshipFieldFinderTests + true + RollupTests true @@ -54,7 +82,7 @@ true - RollupControl__mdt.MaxLookupRowsForQueueable__c + RollupControl__mdt.MaxParentRowsUpdatedAtOnce__c true @@ -97,6 +125,11 @@ Rollup__mdt.FullRecalculationDefaultStringValue__c true + + true + Rollup__mdt.GrandparentRelationshipFieldPath__c + true + true Rollup__mdt.IsFullRecordSet__c diff --git a/scripts/test.ps1 b/scripts/test.ps1 index 3a917f95..e5d1c8e8 100644 --- a/scripts/test.ps1 +++ b/scripts/test.ps1 @@ -1,7 +1,7 @@ # This is also the same script that runs on Github via the Github Action configured in .github/workflows - there, the # DEVHUB_SFDX_URL.txt file is populated in a build step -$testInvocation = 'sfdx force:apex:test:run -n "RollupTests, RollupEvaluatorTests, RollupFieldInitializerTests, RollupCalculatorTests, RollupIntegrationTests, RollupFlowBulkProcessorTests" -c -d ./tests/apex -r human -w 20' +$testInvocation = 'sfdx force:apex:test:run -n "RollupTests, RollupEvaluatorTests, RollupFieldInitializerTests, RollupCalculatorTests, RollupIntegrationTests, RollupFlowBulkProcessorTests, RollupRelationshipFieldFinderTests" -c -d ./tests/apex -r human -w 20' function Start-Tests() { # Run tests diff --git a/scripts/test.sh b/scripts/test.sh index a1b40934..41034ff3 100755 --- a/scripts/test.sh +++ b/scripts/test.sh @@ -16,7 +16,7 @@ fi echo "Copying deploy SFDX project json file to root directory, storing backup in /scripts" cp ./sfdx-project.json ./scripts/sfdx-project.json -cp ./scripts/deploy-sfdx-project.json ./sfdx-project.json +cp ./scripts/deploy-sfdx-project.json ./sfdx-project.json # Authorize Dev Hub using prior creds. There's some issue with the flags --setdefaultdevhubusername and --setdefaultusername both being passed when run remotely @@ -27,7 +27,7 @@ sfdx config:set defaultusername=james@sheandjim.com defaultdevhubusername=james@ # Also store test command shared between script branches, below scratchOrgAllotment=$(sfdx force:limits:api:display 2>/dev/null --json | jq -r '.result[] | select (.name=="DailyScratchOrgs").remaining') echo "Total remaining scratch orgs for the day: $scratchOrgAllotment" -testInvocation='sfdx force:apex:test:run -n "RollupTests,RollupEvaluatorTests,RollupFieldInitializerTests,RollupCalculatorTests,RollupIntegrationTests,RollupFlowBulkProcessorTests" -c -d ./tests/apex -r human -w 20' +testInvocation='sfdx force:apex:test:run -n "RollupTests,RollupEvaluatorTests,RollupFieldInitializerTests,RollupCalculatorTests,RollupIntegrationTests,RollupFlowBulkProcessorTests,RollupRelationshipFieldFinderTests" -c -d ./tests/apex -r human -w 20' echo "Test command to use: $testInvocation" if [ $scratchOrgAllotment -gt 0 ]; then @@ -53,7 +53,7 @@ else # Run tests $testInvocation echo "Tests finished running with success: $?" - + fi # If the priorUserName is not blank and we used a scratch org, reset to it @@ -64,7 +64,7 @@ if [ "$(echo $orgInfo | jq -r '.result.username' 2>/dev/null)" != "" ] && [ $use fi echo "Resetting SFDX project JSON at project root" -cp ./scripts/sfdx-project.json ./sfdx-project.json +cp ./scripts/sfdx-project.json ./sfdx-project.json rm ./scripts/sfdx-project.json echo "Build + testing finished successfully, preparing to upload code coverage"