Can I use test.loadData() with more than 10K records?












2















I'm writing a component to handle CampaignMembers, and it will be common to have more than 10k CampaignMembers. So I'm writing my SOQL to get the campaignMembers in batches of 10K, appending them to the list.



Now, I'm working on a unit test, so I want to generate more than 20K Leads, and then more than 20k corresponding CampaignMembers. (I want at least 3 SOQL batches to make sure the logic works.)



I searched here and elsewhere and couldn't find a good way to circumvent the 10K DML rows limitation in a test. So now, I'm trying to do it with test.loadData(). So far, I'm getting System.UnexpectedException -- so I think I'm probably running into the same 10k limit.



I'm thinking I'll add a dynamic batch size to the class, so if Test.IsRunningTest(), batch size will scale down to 100 in my dynamic SOQL and in my forLoop.



Any ideas of how to test this otherwise?










share|improve this question





























    2















    I'm writing a component to handle CampaignMembers, and it will be common to have more than 10k CampaignMembers. So I'm writing my SOQL to get the campaignMembers in batches of 10K, appending them to the list.



    Now, I'm working on a unit test, so I want to generate more than 20K Leads, and then more than 20k corresponding CampaignMembers. (I want at least 3 SOQL batches to make sure the logic works.)



    I searched here and elsewhere and couldn't find a good way to circumvent the 10K DML rows limitation in a test. So now, I'm trying to do it with test.loadData(). So far, I'm getting System.UnexpectedException -- so I think I'm probably running into the same 10k limit.



    I'm thinking I'll add a dynamic batch size to the class, so if Test.IsRunningTest(), batch size will scale down to 100 in my dynamic SOQL and in my forLoop.



    Any ideas of how to test this otherwise?










    share|improve this question



























      2












      2








      2








      I'm writing a component to handle CampaignMembers, and it will be common to have more than 10k CampaignMembers. So I'm writing my SOQL to get the campaignMembers in batches of 10K, appending them to the list.



      Now, I'm working on a unit test, so I want to generate more than 20K Leads, and then more than 20k corresponding CampaignMembers. (I want at least 3 SOQL batches to make sure the logic works.)



      I searched here and elsewhere and couldn't find a good way to circumvent the 10K DML rows limitation in a test. So now, I'm trying to do it with test.loadData(). So far, I'm getting System.UnexpectedException -- so I think I'm probably running into the same 10k limit.



      I'm thinking I'll add a dynamic batch size to the class, so if Test.IsRunningTest(), batch size will scale down to 100 in my dynamic SOQL and in my forLoop.



      Any ideas of how to test this otherwise?










      share|improve this question
















      I'm writing a component to handle CampaignMembers, and it will be common to have more than 10k CampaignMembers. So I'm writing my SOQL to get the campaignMembers in batches of 10K, appending them to the list.



      Now, I'm working on a unit test, so I want to generate more than 20K Leads, and then more than 20k corresponding CampaignMembers. (I want at least 3 SOQL batches to make sure the logic works.)



      I searched here and elsewhere and couldn't find a good way to circumvent the 10K DML rows limitation in a test. So now, I'm trying to do it with test.loadData(). So far, I'm getting System.UnexpectedException -- so I think I'm probably running into the same 10k limit.



      I'm thinking I'll add a dynamic batch size to the class, so if Test.IsRunningTest(), batch size will scale down to 100 in my dynamic SOQL and in my forLoop.



      Any ideas of how to test this otherwise?







      unit-test bulk






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Feb 16 at 20:39







      PatMcClellan__c

















      asked Feb 16 at 20:21









      PatMcClellan__cPatMcClellan__c

      791221




      791221






















          1 Answer
          1






          active

          oldest

          votes


















          4














          No. You cannot do it. What you might be able to do instead is use mocking. For example a simple query mock might look like:



          public virtual inherited sharing class Query
          {
          static Query instance = new Query();
          public static void setMock(Query mock) { instance = mock; }

          public static List<SObject> records(List<SObject> records)
          {
          return instance.getRecords(records);
          }

          protected virtual List<SObject> getRecords(List<SObject> records) { return records; }
          }


          Then, in your test, you can set a mock so that you can return records which are not in the database. That way, you can set up more than 10k records in memory, then have your mock return them.



          @IsTest
          class MyTest
          {
          class CampaignMemberMock extends Query
          {
          final List<CampaignMember> mockRecords = new List<CampaignMember>();
          CampaignMemberMock buildRecords(Integer recordCount)
          {
          //populate mockRecords
          return this;
          }
          protected override List<SObject> getRecords(List<SObject> records)
          {
          return mockRecords;
          }
          }
          @IsTest static void testMyMethod()
          {
          final Integer recordCount = Limit.getLimitDmlRows() + 1;
          Query mock = new CampaignMemberMock().buildRecords(recordCount);

          Test.startTest();
          Query.setMock(mock);
          List<CampaignMember> records = Query.records([
          SELECT ... FROM CampaignMember
          ]);
          Test.startTest();

          system.assertEquals(recordCount, records.size(), 'Many records returned');
          }
          }





          share|improve this answer
























          • that's a good approach -- well, best we can do, I guess.

            – PatMcClellan__c
            Feb 16 at 23:06











          • You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

            – Adrian Larson
            Feb 17 at 0:58











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "459"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f250588%2fcan-i-use-test-loaddata-with-more-than-10k-records%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          4














          No. You cannot do it. What you might be able to do instead is use mocking. For example a simple query mock might look like:



          public virtual inherited sharing class Query
          {
          static Query instance = new Query();
          public static void setMock(Query mock) { instance = mock; }

          public static List<SObject> records(List<SObject> records)
          {
          return instance.getRecords(records);
          }

          protected virtual List<SObject> getRecords(List<SObject> records) { return records; }
          }


          Then, in your test, you can set a mock so that you can return records which are not in the database. That way, you can set up more than 10k records in memory, then have your mock return them.



          @IsTest
          class MyTest
          {
          class CampaignMemberMock extends Query
          {
          final List<CampaignMember> mockRecords = new List<CampaignMember>();
          CampaignMemberMock buildRecords(Integer recordCount)
          {
          //populate mockRecords
          return this;
          }
          protected override List<SObject> getRecords(List<SObject> records)
          {
          return mockRecords;
          }
          }
          @IsTest static void testMyMethod()
          {
          final Integer recordCount = Limit.getLimitDmlRows() + 1;
          Query mock = new CampaignMemberMock().buildRecords(recordCount);

          Test.startTest();
          Query.setMock(mock);
          List<CampaignMember> records = Query.records([
          SELECT ... FROM CampaignMember
          ]);
          Test.startTest();

          system.assertEquals(recordCount, records.size(), 'Many records returned');
          }
          }





          share|improve this answer
























          • that's a good approach -- well, best we can do, I guess.

            – PatMcClellan__c
            Feb 16 at 23:06











          • You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

            – Adrian Larson
            Feb 17 at 0:58
















          4














          No. You cannot do it. What you might be able to do instead is use mocking. For example a simple query mock might look like:



          public virtual inherited sharing class Query
          {
          static Query instance = new Query();
          public static void setMock(Query mock) { instance = mock; }

          public static List<SObject> records(List<SObject> records)
          {
          return instance.getRecords(records);
          }

          protected virtual List<SObject> getRecords(List<SObject> records) { return records; }
          }


          Then, in your test, you can set a mock so that you can return records which are not in the database. That way, you can set up more than 10k records in memory, then have your mock return them.



          @IsTest
          class MyTest
          {
          class CampaignMemberMock extends Query
          {
          final List<CampaignMember> mockRecords = new List<CampaignMember>();
          CampaignMemberMock buildRecords(Integer recordCount)
          {
          //populate mockRecords
          return this;
          }
          protected override List<SObject> getRecords(List<SObject> records)
          {
          return mockRecords;
          }
          }
          @IsTest static void testMyMethod()
          {
          final Integer recordCount = Limit.getLimitDmlRows() + 1;
          Query mock = new CampaignMemberMock().buildRecords(recordCount);

          Test.startTest();
          Query.setMock(mock);
          List<CampaignMember> records = Query.records([
          SELECT ... FROM CampaignMember
          ]);
          Test.startTest();

          system.assertEquals(recordCount, records.size(), 'Many records returned');
          }
          }





          share|improve this answer
























          • that's a good approach -- well, best we can do, I guess.

            – PatMcClellan__c
            Feb 16 at 23:06











          • You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

            – Adrian Larson
            Feb 17 at 0:58














          4












          4








          4







          No. You cannot do it. What you might be able to do instead is use mocking. For example a simple query mock might look like:



          public virtual inherited sharing class Query
          {
          static Query instance = new Query();
          public static void setMock(Query mock) { instance = mock; }

          public static List<SObject> records(List<SObject> records)
          {
          return instance.getRecords(records);
          }

          protected virtual List<SObject> getRecords(List<SObject> records) { return records; }
          }


          Then, in your test, you can set a mock so that you can return records which are not in the database. That way, you can set up more than 10k records in memory, then have your mock return them.



          @IsTest
          class MyTest
          {
          class CampaignMemberMock extends Query
          {
          final List<CampaignMember> mockRecords = new List<CampaignMember>();
          CampaignMemberMock buildRecords(Integer recordCount)
          {
          //populate mockRecords
          return this;
          }
          protected override List<SObject> getRecords(List<SObject> records)
          {
          return mockRecords;
          }
          }
          @IsTest static void testMyMethod()
          {
          final Integer recordCount = Limit.getLimitDmlRows() + 1;
          Query mock = new CampaignMemberMock().buildRecords(recordCount);

          Test.startTest();
          Query.setMock(mock);
          List<CampaignMember> records = Query.records([
          SELECT ... FROM CampaignMember
          ]);
          Test.startTest();

          system.assertEquals(recordCount, records.size(), 'Many records returned');
          }
          }





          share|improve this answer













          No. You cannot do it. What you might be able to do instead is use mocking. For example a simple query mock might look like:



          public virtual inherited sharing class Query
          {
          static Query instance = new Query();
          public static void setMock(Query mock) { instance = mock; }

          public static List<SObject> records(List<SObject> records)
          {
          return instance.getRecords(records);
          }

          protected virtual List<SObject> getRecords(List<SObject> records) { return records; }
          }


          Then, in your test, you can set a mock so that you can return records which are not in the database. That way, you can set up more than 10k records in memory, then have your mock return them.



          @IsTest
          class MyTest
          {
          class CampaignMemberMock extends Query
          {
          final List<CampaignMember> mockRecords = new List<CampaignMember>();
          CampaignMemberMock buildRecords(Integer recordCount)
          {
          //populate mockRecords
          return this;
          }
          protected override List<SObject> getRecords(List<SObject> records)
          {
          return mockRecords;
          }
          }
          @IsTest static void testMyMethod()
          {
          final Integer recordCount = Limit.getLimitDmlRows() + 1;
          Query mock = new CampaignMemberMock().buildRecords(recordCount);

          Test.startTest();
          Query.setMock(mock);
          List<CampaignMember> records = Query.records([
          SELECT ... FROM CampaignMember
          ]);
          Test.startTest();

          system.assertEquals(recordCount, records.size(), 'Many records returned');
          }
          }






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Feb 16 at 21:59









          Adrian LarsonAdrian Larson

          109k19115247




          109k19115247













          • that's a good approach -- well, best we can do, I guess.

            – PatMcClellan__c
            Feb 16 at 23:06











          • You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

            – Adrian Larson
            Feb 17 at 0:58



















          • that's a good approach -- well, best we can do, I guess.

            – PatMcClellan__c
            Feb 16 at 23:06











          • You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

            – Adrian Larson
            Feb 17 at 0:58

















          that's a good approach -- well, best we can do, I guess.

          – PatMcClellan__c
          Feb 16 at 23:06





          that's a good approach -- well, best we can do, I guess.

          – PatMcClellan__c
          Feb 16 at 23:06













          You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

          – Adrian Larson
          Feb 17 at 0:58





          You could make the mock a bit more sophisticated by adding a per object cache (Map<SObjectType, List<SObject>>) so you can mock queries for more than one type of object.

          – Adrian Larson
          Feb 17 at 0:58


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Salesforce Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f250588%2fcan-i-use-test-loaddata-with-more-than-10k-records%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Index of /

          Tribalistas

          Listed building