Skip to content

fix: improve Subscription Billing performance for large datasets#7776

Open
jeffreybulanadi wants to merge 1 commit intomicrosoft:mainfrom
jeffreybulanadi:fix/7690-subscription-billing-performance
Open

fix: improve Subscription Billing performance for large datasets#7776
jeffreybulanadi wants to merge 1 commit intomicrosoft:mainfrom
jeffreybulanadi:fix/7690-subscription-billing-performance

Conversation

@jeffreybulanadi
Copy link
Copy Markdown

@jeffreybulanadi jeffreybulanadi commented Apr 21, 2026 •
edited by github-actions Bot
Loading

Fixes #7690

Summary

Subscription Billing document creation is unscalable at 4M+ subscription lines due to missing indexes, crashes in Job Queue, an O(n2) query pattern, redundant database reads, and no per-document transaction boundary. This PR addresses all root causes identified in the issue.

Root Causes Fixed

1. Missing database indexes

  • BillingLine: Added key SK7 on (Partner, Subscription Contract No., Subscription Contract Line No., Document Type, Document No.) covering GetBillingLineNo() on every line insertion
  • SubscriptionLine: Added key Key4 on (Partner, Subscription Contract No., Next Billing Date) covering ProcessContractServiceCommitments()
  • UsageDataBilling: Added key key2 on (Partner, Subscription Contract No., Subscription Contract Line No., Document Type, Document No.) mirroring the billing line lookup pattern
  • UsageDataGenericImport: Added key key2 on (Usage Data Import Entry No., Processing Status) covering status-filtered scans during import

2. Job Queue crash (GuiAllowed = false)

  • BillingProposal: ProposalWindow.Open/Update/Close now guarded with if GuiAllowed. The Count() call that feeds the progress dialog is also skipped when GuiAllowed = false.
  • CreateBillingDocuments: Window.Open/Update/Close already had an AutomatedBilling guard but was missing GuiAllowed checks. Both guards are now applied. ProcessingFinishedMessage exits early when not GuiAllowed.

3. Redundant GetBillingLineNo() calls inside inner loop

  • In InsertSalesLineFromTempBillingLine and InsertPurchaseLineFromTempBillingLine, GetBillingLineNo() was called inside a UsageDataBilling.FindSet() loop with constant parameters. Hoisted outside the loop.

4. Redundant contract Get() on every billing line

  • CreateTempBillingLines called CustomerContract.Get() and VendorContract.Get() for every billing line. Added cache variables CachedCustomerContractNo and CachedVendorContractNo to skip the read when the contract has not changed.

5. Single transaction for entire batch

  • CreateSalesDocumentsPerContract, CreatePurchaseDocumentsPerContract, CreateSalesDocumentsPerCustomer, and CreatePurchaseDocumentsPerVendor each call Commit() after completing each document. This limits the transaction scope to one document at a time, preventing lock escalation and enabling partial-batch recovery.

6. Progress visibility in BillingProposal

  • Added a Dialog progress window in CreateBillingProposal so users see per-contract progress during long proposal creation runs. Includes contract number, partner number, and a processed/total counter.

Files Changed

  • src/Apps/W1/Subscription Billing/App/Billing/Tables/BillingLine.Table.al -- added SK7
  • src/Apps/W1/Subscription Billing/App/Service Commitments/Tables/SubscriptionLine.Table.al -- added Key4
  • src/Apps/W1/Subscription Billing/App/Usage Based Billing/Tables/UsageDataBilling.Table.al -- added key2
  • src/Apps/W1/Subscription Billing/App/Usage Based Billing/Tables/UsageDataGenericImport.Table.al -- added key2
  • src/Apps/W1/Subscription Billing/App/Billing/Codeunits/BillingProposal.Codeunit.al -- progress dialog + GuiAllowed guards
  • src/Apps/W1/Subscription Billing/App/Billing/Codeunits/CreateBillingDocuments.Codeunit.al -- GuiAllowed guards, Commit() per document, hoisted GetBillingLineNo(), cached contract Get()

How to Test

  1. Set up a BC environment with a large number of Customer and Vendor Subscription Contracts (10,000+ lines recommended).
  2. Create a Billing Template and run recurring billing to generate a billing proposal -- verify the progress dialog shows contract progress.
  3. Run "Create Billing Documents" -- verify documents are created without error and the progress dialog updates correctly.
  4. Schedule the billing template via Job Queue (AutomatedBilling = true) -- verify the run completes without a "Dialog not allowed" error.
  5. With 4M+ subscription lines, verify that document creation completes without lock timeout or transaction size errors.
  6. After an interrupted run, verify that already-committed documents are present and no partial data loss occurred.

@jeffreybulanadi
fix: improve Subscription Billing performance for large datasets (mic…
7515abd 
…rosoft#7690)

Root causes addressed:
1. Missing database indexes causing full table scans on 4M+ billing lines
2. No progress dialog in billing proposal creation (unusable at scale)
3. Window.Open/Close/Update calls not guarded with GuiAllowed (Job Queue failures)
4. GetBillingLineNo() called inside UsageDataBilling inner loop (N^2 queries)
5. CustomerContract/VendorContract.Get() called for every billing line (millions of round-trips)
6. No per-document Commit() leaving entire run as one giant transaction

Changes:
- BillingLine.Table.al: add SK7 (Partner, ContractNo, ContractLineNo, DocType, DocNo)
  covering the GetBillingLineNo() and UsageDataBilling query patterns
- SubscriptionLine.Table.al: add Key4 (Partner, ContractNo, NextBillingDate)
  covering the ProcessContractServiceCommitments filter pattern
- UsageDataBilling.Table.al: add key2 (Partner, ContractNo, ContractLineNo, DocType, DocNo)
  enabling efficient Usage Based Billing document-save lookups
- UsageDataGenericImport.Table.al: add key2 (UsageDataImportEntryNo, ProcessingStatus)
  enabling efficient status-filtered queries on import lines
- BillingProposal.Codeunit.al: add progress dialog (BillingProposalProgressTxt) showing
  contract no., partner no., and processed/total count; guard all Message/Page.Run with
  if GuiAllowed to allow Job Queue execution
- CreateBillingDocuments.Codeunit.al:
  - Wrap all Window.Open/Close/Update with if GuiAllowed
  - Wrap ProcessingFinishedMessage with if GuiAllowed
  - Add Commit() after each completed billing document in all four document-creation
    procedures (PerContract/PerCustomer for Sales; PerContract/PerVendor for Purchase)
    to create recovery points and reduce transaction size
  - Hoist GetBillingLineNo() call outside the UsageDataBilling FindSet() loop in both
    InsertSalesLineFromTempBillingLine and InsertPurchaseLineFromTempBillingLine; the
    result is constant within a given document line so calling it once is correct
  - Cache CustomerContract.Get() and VendorContract.Get() in CreateTempBillingLines
    using a last-seen contract no. variable; reduces O(n) Get() calls to O(distinct
    contracts) with zero behavioral change
  - Extend ProgressTxt label with a third field showing contracts-processed count

Fixes microsoft#7690
@jeffreybulanadi jeffreybulanadi requested a review from a team as a code owner April 21, 2026 08:00
@github-actions github-actions Bot added AL: Apps (W1) Add-on apps for W1 From Fork Pull request is coming from a fork labels Apr 21, 2026
@JesperSchulz JesperSchulz added the Finance GitHub request for Finance area label Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

AL: Apps (W1) Add-on apps for W1 Finance GitHub request for Finance area From Fork Pull request is coming from a fork

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug][SubscriptionBilling]: Performance: Billing document creation does not scale to large datasets (4M+ subscription lines)

2 participants

@jeffreybulanadi @JesperSchulz