Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output legacy score_id when a processing failure happens #186

Merged
merged 2 commits into from
Dec 19, 2023

Conversation

peppy
Copy link
Member

@peppy peppy commented Dec 19, 2023

Until now it was very hard to figure out which score actually caused the failure, especially running against data.ppy.sh test data where the range of each batch can be 30 mil scores (due to holes between each score).

Tested using:

diff --git a/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs b/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs
index 42d6c45..c47f5e4 100644
--- a/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs
+++ b/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs
@@ -559,6 +559,9 @@ namespace osu.Server.Queues.ScoreStatisticsProcessor.Commands.Queue
                                 DefaultValueHandling = DefaultValueHandling.Ignore
                             });
 
+                            if (RNG.NextDouble() > 0.999)
+                                throw new InvalidCastException("I am a poop");
+
                             if (existingMapping != null)
                             {
                                 // Note that this only updates the `data` field. We could add others in the future as required.

Example output:

37  4553930413 - 4553951402: success
38  4553957715 - 4553930073: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553936766 failed. (I am a poop)))
39  4553930142 - 4553947895: success
40  4553964781 - 4553938420: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553967810 failed. (I am a poop)))
41  4553953221 - 4553963354: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553963004 failed. (I am a poop)))
42  4553969301 - 4553962128: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553968479 failed. (I am a poop)))
43  4553944688 - 4553930210: success
44  4553931276 - 4553951864: success
45  4553929039 - 4553927755: success
46  4553930150 - 4553927721: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553938019 failed. (I am a poop)))
47  4553928055 - 4553930994: success
48  4553929799 - 4553934363: success
49  4553960309 - 4553929107: success
50  4553964258 - 4553939406: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553969597 failed. (I am a poop)))
51  4553943792 - 4553945592: success
52  4553963500 - 4553928141: success
53  4553932730 - 4553958290: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553970142 failed. (I am a poop)))
54  4553929764 - 4553938582: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553934310 failed. (I am a poop)))
55  4553970750 - 4553959168: success
56  4553935306 - 4553962660: success
57  4553946923 - 4553945269: success
58  4553941394 - 4553927032: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553943057 failed. (I am a poop)))
59  4553934685 - 4553968714: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553935424 failed. (I am a poop)))
60  4553941879 - 4553954602: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553938436 failed. (I am a poop)))
61  4553946026 - 4553960447: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553960452 failed. (I am a poop)))
62  4553946717 - 4553930608: success
63  4553931525 - 4553945240: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553932795 failed. (I am a poop)))
64  4553962195 - 4553960053: success
65  4553960055 - 4553967704: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553939734 failed. (I am a poop)))
66  4553935968 - 4553933106: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553958907 failed. (I am a poop)))
67  4553964108 - 4553933566: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553946303 failed. (I am a poop)))
68  4553948810 - 4553967084: success
69  4553943855 - 4553929960: success
70  4553935863 - 4553945817: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553928311 failed. (I am a poop)))
71  4553948606 - 4553966015: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553966916 failed. (I am a poop)))
72  4553966022 - 4553936472: success
73  4553945928 - 4553963983: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553936405 failed. (I am a poop)))
74  4553933456 - 4553935622: success
75  4553957494 - 4553962249: success
76  4553963385 - 4553964174: success
77  4553938663 - 4553936161: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553932195 failed. (I am a poop)))
78  4553936415 - 4553935900: success
79  4553936074 - 4553949284: FAILED (One or more errors occurred. (Creating reference score for legacy score 4553950066 failed. (I am a poop)))

System.AggregateException: One or more errors occurred. (Creating reference score for legacy score 4553941679 failed. (I am a poop))
 ---> System.AggregateException: Creating reference score for legacy score 4553941679 failed. (I am a poop)
 ---> System.InvalidCastException: I am a poop
   at osu.Server.Queues.ScoreStatisticsProcessor.Commands.Queue.ImportHighScoresCommand.BatchInserter.run(HighScore[] scores) in /root/repos/osu-queue-score-statistics/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs:line 563
   --- End of inner exception stack trace ---
   at osu.Server.Queues.ScoreStatisticsProcessor.Commands.Queue.ImportHighScoresCommand.BatchInserter.run(HighScore[] scores) in /root/repos/osu-queue-score-statistics/osu.Server.Queues.ScoreStatisticsProcessor/Commands/Queue/ImportHighScoresCommand.cs:line 609
   --- End of inner exception stack trace ---

@smoogipoo smoogipoo merged commit 3730470 into ppy:master Dec 19, 2023
3 checks passed
@peppy peppy deleted the error-stuff branch January 24, 2024 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants