Bump supplementary end year if no annual bill run #875
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://eaflood.atlassian.net/browse/WATER-4403 We want our Billing & Data team to be able to generate supplementary bill runs all year round. However, at the moment they have to enforce a manual 'stop' on them until annual bill runs are completed. This is because of the 2 ways the engine operates. Annual just bills for the current charge versions. It doesn't look at replaced charge versions or what was previously billed. supplementary does do these things. So, if the order is 'Annual -> Supplementary' then the bill will be correct. For example, supplementary calculates a debit of £100 for a charge version (the licence was flagged for 'reasons'!) but sees the £100 debit from the annual bill. They cancel each other out so the customer sees nothing. If the order was `Supplementary -> Annual` though, annual doesn't look back at previous transactions. It would generate another debit to the customer for £100 leading to them being billed twice. We could guarantee annual bills were raised on April 1 we wouldn't have a problem. However, the additional work that goes into preparing them and validating what the engine generates, means it can take some time. That leaves a window where a supplementary could be raised and the charges for a customer get messed up. This has always been the case and billing & data have just had to deal with it by not generating supplementary. But that causes problems because bills start backing up that could be sent out. So, this year we intend to solve the problem. We will amend our supplementary billing engine to check if an annual bill run for the same region has been raised in the current billing period. If it has we'll carry on as usual. If it hasn't, we'll move the financial year end back to the first year where one was raised. This will allow B&D to crack on with supplementary bills, at least for previous years and prevent charging data from being messed up. The first step was to refactor [Refactor DetermineBillingPeriods to use type](#864). It can then use this information to generate the billing periods accordingly, rather than always from the current billing period. Now that service will work appropriately if given a type and year we can implement determining _what_ that year is.
Cruikshanks
requested review from
Demwunz,
robertparkinson,
Jozzey,
Beckyrose200 and
rvsiyad
robertparkinson
approved these changes
Apr 2, 2024
Cruikshanks
added a commit
that referenced
this pull request
Apr 2, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](#875) we added logic that would mean when creating a new supplementary bill run the engine could tell if an annual bill run existed for the same year and region. If one doesn't it will make a mess of the billing for customers if the supplementary happened before the annual. But we don't want to block users from working. So, instead we bump the financial end year to use back to the same as the last 'sent' annual bill run for that region. We even [Refactor DetermineBillingPeriods to use type](#864) so it would know to use the calculated year when determining the billing periods for the supplementary bill run. What we forgot to do was remove a [temporary fix](#859) we'd added until this work can be completed. This means the year we've calculated is being dropped and not passed to `DetermineBillingPeriods`. This change fixes that!
Cruikshanks
added a commit
that referenced
this pull request
Apr 4, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. So, what's the problem? In this explanation we keep referring to the last sent 'annual' bill run. But we've just spotted the logic which looks for the last sent bill run does not include batch type in the `where()` clause. Theoretically, we could create a two-part tariff bill run, send it and that would then match our query! This change corrects the query we use to find the last sent annual bill run for a region.
Cruikshanks
added a commit
that referenced
this pull request
Apr 4, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. So, what's the problem? In this explanation, we keep referring to the last sent 'annual' bill run. But we've just spotted the logic which looks for the last sent bill run does not include batch type in the `where()` clause. Theoretically, we could create a two-part tariff bill run, send it and that would then match our query! This change corrects the query we use to find the last sent annual bill run for a region.
Cruikshanks
added a commit
to DEFRA/water-abstraction-service
that referenced
this pull request
Apr 4, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](DEFRA/water-abstraction-system#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. The problem is this means our billing engine is working on the assumption there is _always_ at least one 'sent' annual bill run for a region. This is definitely the case in our live service but not in our test setup. This updates the `FixtureLoader` to generate the current and previous billing period start and end years. We then update our existing SROC fixtures to include creating a current annual bill run. We also create a copy of the fixture that will include creating an annual bill run for the previous year. These changes will mean our current supplementary test will start working again plus we have the scope to add a new test of this feature.
Cruikshanks
added a commit
to DEFRA/water-abstraction-acceptance-tests
that referenced
this pull request
Apr 5, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](DEFRA/water-abstraction-system#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. The problem is this means our billing engine is working on the assumption there is _always_ at least one 'sent' annual bill run for a region. This is definitely the case in our live service but not in our test setup. We've made changes to our [acceptance test fixture setup](DEFRA/water-abstraction-service#2484) to ensure we always create an annual as part of our SROC test fixture. These changes cover any other tweaks needed to get the existing tests working.
Cruikshanks
added a commit
to DEFRA/water-abstraction-acceptance-tests
that referenced
this pull request
Apr 5, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](DEFRA/water-abstraction-system#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. The problem is this means our billing engine is working on the assumption there is _always_ at least one 'sent' annual bill run for a region. This is the case in our live service but not in our test setup. We've made changes to our [acceptance test fixture setup](DEFRA/water-abstraction-service#2484) to ensure we always create an annual as part of our SROC test fixture. These changes cover any other tweaks needed to get the existing tests working. --- Notes on the changes - `cypress/e2e/internal/billing/supplementary/cancel-existing.cy.js` It was 'cheating' before by using a two-part tariff test fixture that would result in empty bill runs. We have only added the annual bill run entry to our SROC data fixture so we needed to switch it to use that and update the assertions accordingly. - `internal/billing/supplementary/journey.cy.js Having moved into 2024-25 we've hit a point where part of the PRESROC charge version is no longer included in the bill run. It starts on Jan 1 2019. When the billing period was 2023-24 five years before that year it would include the billing period 2018-19. This means there would be 4 supplementary bills with the first just being for 2019-01-01 to 2019-03-31. Now the billing period is 2024-25 the period 2018-2019 has dropped off. This means 1 less bill which affects the bill run total. - `internal/billing/supplementary/replace-cv-current-year-with-changes.cy.js` Moving into a new billing period means the number of billable days being calculated is different which affects the values being asserted. - `internal/billing/supplementary/change-billing-account-current-year.cy.js` This corrects a faulty assumption in the test, put there no doubt because of a copy and paste. Once we add the new charge version for the current year and then create the second SRO supplementary, with no other changes we should only see 2 bills; the credit for the previous one and the debit for the new account. But we were using the number returned from our logic which works out how many bills we expect to see based on the number of billing periods since 2022. It worked when running the tests in 2023-24 because the result was also 2. Now we are in 2024-25 it's exposed the faulty logic. - Fix broken annual billing We had overlooked that the annual billing tests were using the `sroc-billing-data` fixture as well. Adding an annual bill run to that fixture caused the annual tests to fail because of the existing bill run. We realised that we could use the 'previous' fixture we created as part of these changes both for our new supplementary test and the annual tests. This saves a bunch of duplication in the integration sets. It still creates a bill run but it is for the previous year. If anything it's a more robust way of checking that bill runs in previous billing periods shouldn't prevent a new one from being created. - Add new test for supplementary financial year bump This new test gives us a means to check that the supplementary end-year bump feature is working. It uses a new SROC fixture that is the same as our normal one only the annual bill run created is for the previous billing period. This means we can confirm when we create an SROC supplementary that its financial year end is also for the previous billing period.
Cruikshanks
added a commit
to DEFRA/water-abstraction-service
that referenced
this pull request
Apr 5, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](DEFRA/water-abstraction-system#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. The problem is this means our billing engine is working on the assumption there is _always_ at least one 'sent' annual bill run for a region. This is the case in our live service but not in our test setup. This updates the `FixtureLoader` to generate the current and previous billing period start and end years. We then update our existing SROC fixtures to include creating a current annual bill run. We also create a copy of the fixture that will include creating an annual bill run for the previous year. These changes will mean our current supplementary test will start working again plus we have the scope to add a new test of this feature.
Demwunz
pushed a commit
that referenced
this pull request
Apr 5, 2024
https://eaflood.atlassian.net/browse/WATER-4403 In [Bump supplementary end year if no annual bill run](#875) we amended our billing engine for supplementary. We made it possible for users to generate supplementary bill runs in years where no annual has yet been created and sent. Previously, the Billing & Data team would have to put a block on creating bill runs until the annuals were generated. This is because supplementary takes into account previous transactions whereas annual doesn't. Without the change creating a supplementary followed by an annual bill run would result in the customer getting charged twice. We now determine when the last annual bill run was sent and use its financial end year as the end year for the supplementary bill run. Annual created in this billing period? Then the supplementary will start there and work back 5 years. If the last sent annual was for the previous billing period the supplementary will start _there_ and work back 5 years. So, what's the problem? In this explanation, we keep referring to the last sent 'annual' bill run. But we've just spotted the logic which looks for the last sent bill run does not include batch type in the `where()` clause. Theoretically, we could create a two-part tariff bill run, send it and that would then match our query! This change corrects the query we use to find the last sent annual bill run for a region.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://eaflood.atlassian.net/browse/WATER-4403
We want our Billing & Data team to be able to generate supplementary bill runs all year round. However, at the moment they have to enforce a manual 'stop' on them until annual bill runs are completed. This is because of the 2 ways the engine operates.
Annual just bills for the current charge versions. It doesn't look at replaced charge versions or what was previously billed. supplementary does do these things. So, if the order is 'Annual -> Supplementary' then the bill will be correct. For example, supplementary calculates a debit of £100 for a charge version (the licence was flagged for 'reasons'!) but sees the £100 debit from the annual bill. They cancel each other out so the customer sees nothing.
If the order was
Supplementary -> Annualthough, annual doesn't look back at previous transactions. It would generate another debit to the customer for £100 leading to them being billed twice.We could guarantee annual bills were raised on April 1 we wouldn't have a problem. However, the additional work that goes into preparing them and validating what the engine generates, means it can take some time. That leaves a window where a supplementary could be raised and the charges for a customer get messed up.
This has always been the case and billing & data have just had to deal with it by not generating supplementary. But that causes problems because bills start backing up that could be sent out.
So, this year we intend to solve the problem. We will amend our supplementary billing engine to check if an annual bill run for the same region has been raised in the current billing period. If it has we'll carry on as usual. If it hasn't, we'll move the financial year end back to the first year where one was raised. This will allow B&D to crack on with supplementary bills, at least for previous years and prevent charging data from being messed up.
The first step was to refactor Refactor DetermineBillingPeriods to use type. It can then use this information to generate the billing periods accordingly, rather than always from the current billing period.
Now that service will work appropriately if given a type and year we can implement determining what that year is.