Move unnecessary properties from `BeatmapInfo` / realm to `IBeatmap` #28473

bdach · 2024-06-13T13:35:18Z

RFC. PRing for early feedback.

Closes Editor does not read properties ignored on BeatmapInfo correctly #20883
Supersedes / closes Move Ignored properties into IBeatmap #23418
Needs more testing probably, I've only checked that tests pass

This is basically #23418 but hopefully better (because it moves out more stuff, and hopefully contains less naming discussions). This removes everything that was feasible to remove from BeatmapInfo, therefore bypassing the reference handling nightmare that causes #20883.

I'm necroing that because it came up in recent discussions about persisting the new grid settings somewhere (see discord, cc @OliBomby).

Some things still remain:

BeatmapInfo.BeatDivisor - can't really be moved out of realm because it's used for filtering in song select
BeatmapInfo.EditorTimestamp could go live in the .osu file but I didn't want to make that change here as I would rather keep this diff strictly menial.

I'm not very happy about how much this makes IBeatmap grow but I don't see much wiggle room. Any effort to constrain interfaces any more to specific usages (diffcalc, editor) is basically killed by WorkingBeatmap / conversion and requires at least introduction of generic messes and multiple scoped overloads so I decided I'd rather not.

smoogipoo

I suppose that in principle I'm not super against this, but it does blurry the definition for what should and should not be put in BeatmapInfo a bit.

You could say that BeatmapInfo is everything to be databased, in which case I'd probably expect the epilepsy warning to be put there, since it's being used in a context where it's expected to be available relatively "free-of-cost"?

smoogipoo · 2024-06-17T04:20:42Z

osu.Game/Storyboards/Storyboard.cs

        public BeatmapInfo BeatmapInfo = new BeatmapInfo();
+        public IBeatmap Beatmap { get; set; } = new Beatmap();


This is kinda weird...

So is everything else being populated with empty objects in this class. I was just following precedent, if you will.

My problem here is two-fold. The first is that I'm struggling to follow the code to even ensure this is being written to right now, because it doesn't look like it is? The second is I think this goes above the "everything else" for me.

One thing I thought about but hesitated from posting is to split the properties out into a BeatmapProperties or maybe even into BeatmapMetadata. It feels wrong to be passing Beatmap around which historically has been used very close to gameplay...

To go further, both BeatmapInfo and Beatmap look wrong to me.

If we're not going to have a general "class of info" that we're passing around, then I think EpilepsyWarning/WidescreenStoryboard should be moved into Storyboard.

The remaining references to BeatmapInfo need to be removed - it looks like there's two:

ReplacesBackground (BeatmapInfo.Metadata.BackgroundFile) - probably BackgroundFile should be duplicated as a property here.

GetStoragePathFromStoryboardPath the directionality looks inverted. It should be DrawableStoryboard doing the lookups on resource stores, not what is effectively a model class otherwise.

Perhaps this would be best as a pre-pass.

One thing I thought about but hesitated from posting is to split the properties out into a BeatmapProperties or maybe even into BeatmapMetadata

That's what #23418 tried to do but got stunlocked on naming. If we can agree on a name or a place I don't see why not.

One thing I thought about but hesitated from posting is to split the properties out into a BeatmapProperties or maybe even into BeatmapMetadata. It feels wrong to be passing Beatmap around which historically has been used very close to gameplay...

I get this, but is there an actually requirement for the split? In stable, for instance, we have an .osu file. And we have a Beatmap. And they are 1:1. Should we not just stick with that for simplicity? Am I missing a case where passing "too much" in one class is an issue?

Also I'm still confused. I can't find any write usages of Storyboard.Beatmap or its properties (except one test), even though it's read from. @bdach can you clarify here?

Judging from master code, it looks like the Beatmap would need to be set around here to work correctly:

osu/osu.Game/Beatmaps/WorkingBeatmap.cs

Line 65 in 3978d4b

protected virtual Storyboard GetStoryboard() => new Storyboard { BeatmapInfo = BeatmapInfo };

Also I'm still confused. I can't find any write usages of Storyboard.Beatmap or its properties (except one test), even though it's read from.

Yeah that's just a bug. Good spot, although I did try to preempt this PR by saying it's basically a draft of the general shape with no testing other than checking that unit tests pass.

Judging from master code, it looks like the Beatmap would need to be set around here to work correctly:

Looks close, but the reality is uglier than that; see 1d4d806. tl;dr is that storyboards go through a complete separate decoding path:

osu/osu.Game/Beatmaps/WorkingBeatmapCache.cs

Lines 257 to 312 in 40b171f

protected override Storyboard GetStoryboard()

{

Storyboard storyboard;

if (BeatmapInfo.Path == null)

return new Storyboard();

try

{

string fileStorePath = BeatmapSetInfo.GetPathForFile(BeatmapInfo.Path);

var beatmapFileStream = GetStream(fileStorePath);

if (beatmapFileStream == null)

{

Logger.Log($"Beatmap failed to load (file {BeatmapInfo.Path} not found on disk at expected location {fileStorePath})", level: LogLevel.Error);

return new Storyboard();

}

using (var reader = new LineBufferedReader(beatmapFileStream))

{

var decoder = Decoder.GetDecoder<Storyboard>(reader);

Stream storyboardFileStream = null;

string mainStoryboardFilename = getMainStoryboardFilename(BeatmapSetInfo.Metadata);

if (BeatmapSetInfo?.Files.FirstOrDefault(f => f.Filename.Equals(mainStoryboardFilename, StringComparison.OrdinalIgnoreCase))?.Filename is string

storyboardFilename)

{

string storyboardFileStorePath = BeatmapSetInfo?.GetPathForFile(storyboardFilename);

storyboardFileStream = GetStream(storyboardFileStorePath);

if (storyboardFileStream == null)

Logger.Log($"Storyboard failed to load (file {storyboardFilename} not found on disk at expected location {storyboardFileStorePath})", level: LogLevel.Error);

}

if (storyboardFileStream != null)

{

// Stand-alone storyboard was found, so parse in addition to the beatmap's local storyboard.

using (var secondaryReader = new LineBufferedReader(storyboardFileStream))

storyboard = decoder.Decode(reader, secondaryReader);

}

else

storyboard = decoder.Decode(reader);

}

}

catch (Exception e)

{

Logger.Error(e, "Storyboard failed to load");

storyboard = new Storyboard();

}

storyboard.BeatmapInfo = BeatmapInfo;

return storyboard;

}

and thus all IBeatmap properties that Storyboard needs to read correctly must be populated by LegacyStoryboardDecoder.

I guess this is a relevant point of discussion to the general direction here, and I'm not sure what to make of that revelation. In a way this may be more correct because this separate decoding path reads both the .osb (if present) and the .osu as a fallback, so the resultant behaviour may be more correct here (needs stable cross-check) - but that's also a weak edge case that probably has never been an issue until now

The fact that decoding would need to be split and/or duplicated based on what beatmaps care about and storyboards care about could be seen as bad. Although that is arguably already a problem because that exists:

osu/osu.Game/Beatmaps/Formats/LegacyStoryboardDecoder.cs

Lines 72 to 74 in 40b171f

case "UseSkinSprites":

storyboard.UseSkinSprites = pair.Value == "1";

break;

smoogipoo · 2024-06-17T04:21:59Z

osu.Game/Screens/Play/PlayerLoader.cs

-            if (Beatmap.Value.BeatmapInfo.EpilepsyWarning)
+            if (Beatmap.Value.Beatmap.EpilepsyWarning)


Have you tested whether this is blocking on the beatmap load? This looks kinda scary to me.

I haven't tested this until now (see mention in OP that this probably needs more testing). I was interested in getting the shape of this right first and foremost, and was only going to go over specifics later. The extent of what is to be moved out is negotiable.

As far as the answer goes, I believe it to be "generally it will not be", because the game-wide async load process will have already completed by now:

osu/osu.Game/OsuGame.cs

Lines 799 to 803 in 57688c2

private void beatmapChanged(ValueChangedEvent<WorkingBeatmap> beatmap)

{

beatmap.OldValue?.CancelAsyncLoad();

beatmap.NewValue?.BeginAsyncLoad();

}

But sure, I can agree with not moving this out. I was in two minds about doing this myself, but am prepared to revert this part. It just seemed to me like it wasn't impossible to move it so I did it, to cover absolutely everything that can be moved without significant changes.

An argument here (just for hypothetical purposes, since i kinda want to nuke this warning completely) is that we may want this to be databased as metadata to show at song select or be searchable. In such a case, we wouldn't want to move it out of BeatmapInfo, correct?

To make it very clear, I'm still fine with it being moved out if it makes more sense (TBD).

we may want this to be databased as metadata to show at song select or be searchable. In such a case, we wouldn't want to move it out of BeatmapInfo, correct?

That is my understanding yes.

bdach · 2024-06-18T06:41:48Z

Generally I'll just bring up that while this is a draft I'd like to see movement on this or an alternative solution since the underlying issue is currently blocking editor improvements that users want to see too (like bookmarks which don't work because they get dropped by the dumb BeatmapInfo reference juggling logic).

OliBomby · 2024-06-18T09:16:28Z

I'll try to put in my 2 cents.

If the BeatmapInfo is for realm databased properties, I suggest renaming it to BeatmapDatabasedInfo or BeatmapRealmInfo. That will help convey its purpose and explain which properties you want in that class.

For the properties which should not be databased, I suggest putting them in classes like BeatmapGeneral for general properties and BeatmapEditor for editor properties. This mirrors the sections that exist in the .osu format and follows the precedent set by the BeatmapMetadata and BeatmapDifficulty classes.

peppy · 2024-06-22T18:48:50Z

Generally I'll just bring up that while this is a draft I'd like to see movement on this or an alternative solution since the underlying issue is currently blocking editor improvements that users want to see too (like bookmarks which don't work because they get dropped by the dumb BeatmapInfo reference juggling logic).

I have this on my review queue, if you need some assurance that it hasn't been forgotten.

peppy · 2024-07-23T09:25:16Z

osu.Game/Beatmaps/IBeatmap.cs

@@ -69,6 +70,43 @@ public interface IBeatmap
        /// </summary>
        double GetMostCommonBeatLength();

+        double AudioLeadIn { get; internal set; }


I guess my biggest concern is requiring these in the IBeatmap interface (and the delegation requirement of implementations). Which is probably not a huge concern at all.

I'm neutral on moving stuff into a separate class, will require further discussion to ascertain what we're looking for here, as touched on in other comment thread.

In my view this is a nice change, but do note that on its own it does nothing to fix ppy#29492, because of `BeatmapInfo` reference management foibles when opening the editor. See also: ppy#20883 (comment), ppy#28473.

peppy

Finally got around to doing a second pass on this.

Apart from the remaining comments in this review pass, I'm on board with getting this in (with whatever further testing @bdach wants to do) in the interest of moving forward.

I'm not 100% sure on the final home of a lot of these properties, but the first step is getting them out of BeatmapInfo, which is what this PR achieves.

peppy · 2024-11-22T09:57:54Z

osu.Game/Screens/Play/PlayerLoader.cs

-            if (Beatmap.Value.BeatmapInfo.EpilepsyWarning)
+            if (Beatmap.Value.Beatmap.EpilepsyWarning)


An argument here (just for hypothetical purposes, since i kinda want to nuke this warning completely) is that we may want this to be databased as metadata to show at song select or be searchable. In such a case, we wouldn't want to move it out of BeatmapInfo, correct?

To make it very clear, I'm still fine with it being moved out if it makes more sense (TBD).

osu.Game/Database/RealmAccess.cs

bdach · 2024-11-22T10:11:04Z

@smoogipoo before I move on this further, did you want to check this over again or is it fine to proceed with these changes as is?

Notably this will 100% conflict with #30826.

bdach · 2024-11-26T10:41:54Z

I have done my checks and only found 46d1f00, so undrafting.

At this point about I'm not sure how to proceed with the following:

Whether EpilepsyWarning should be moved out as per discussion in Move unnecessary properties from BeatmapInfo / realm to IBeatmap #28473 (comment)
Whether SpecialStyle should be moved out - we might need it in future for song select filtering when (once?) special style is implemented; mentioned on discord
Nobody really responded to Move unnecessary properties from BeatmapInfo / realm to IBeatmap #28473 (comment)

smoogipoo

@smoogipoo before I move on this further, did you want to check this over again or is it fine to proceed with these changes as is?

I'm not sure what sort of review you want on this (i.e. for merge or for if it's a good direction?), but I've done a shallow pass and am roughly fine with it.

smoogipoo · 2024-11-26T11:04:17Z

osu.Game/Beatmaps/IBeatmap.cs

@@ -69,6 +70,43 @@ public interface IBeatmap
        /// </summary>
        double GetMostCommonBeatLength();

+        double AudioLeadIn { get; internal set; }


Is the internal set; part temporary? Maybe just set; is fine.

I'm not sure what the future is for this interface so I'm not suggesting a change - just asking.

The spirit of that modifier was to have editor flows working (which depend on being able to mutate IBeatmaps) while keeping up a semblance of being immutable to external consumers. To what degree this matters given other things such as Breaks or ControlPointInfo are already doing away with that illusion and just doing { get; set; } is disputable I guess. set; would be fine by me if it's fine by everyone else.

bdach · 2024-11-26T11:16:42Z

I'm not sure what sort of review you want on this (i.e. for merge or for if it's a good direction?),

Mostly the former probably, just to ensure you're not completely in disagreement with a change to something this core or something of that sort.

smoogipoo · 2024-11-26T11:22:52Z

Something that's just come to mind... If we generally think of Beatmap as something that's parsed, and a BeatmapInfo as something that's databased, then I'm not sure the correct structure is for the former to contain the latter. Especially if we're going to run into these questions of whether a property should be databased or not.

It may be simpler to consider duplicating the properties in both - that is:

IBeatmapInfo
{
    bool EpilepsyWarning { get; }
}

Beatmap : IBeatmapInfo
{
    bool EpilepsyWarning { get; }
}

BeatmapInfo : IBeatmapInfo
{
    bool EpilepsyWarning { get; }
}

I'm not sure how to approach the above, just laying the seeds for something to think about.

peppy · 2024-11-27T06:50:11Z

It may be simpler to consider duplicating the properties in both - that is

makes logical sense on surface level, but dunno how annoying this would be to actually work with. not sure i'd push for that to be attempted as part of this PR or maybe something that comes later on if we figure there are reasons to do that.

bdach · 2024-11-27T07:00:23Z

not sure i'd push for that to be attempted as part of this PR

I didn't read it as required for this PR fwiw, just a follow up to think about.

bdach · 2024-12-02T07:09:04Z

Soooo is there anything I should be doing at this point to get this merged?

Not sure why I didn't do that in ppy#28473...

bdach added 15 commits June 12, 2024 13:04

Move AudioLeadIn out of BeatmapInfo

4339e2d

Move StackLeniency out of BeatmapInfo

0a4560a

Move SpecialStyle out of BeatmapInfo

011c2e3

Move LetterboxInBreaks out of BeatmapInfo

a6b7600

Move WidescreenStoryboard out of BeatmapInfo

1ab86eb

Move EpilepsyWarning out of BeatmapInfo

f64a062

Move SamplesMatchPlaybackRate out of BeatmapInfo

c216283

Move DistanceSpacing out of BeatmapInfo

3634307

Move GridSize out of BeatmapInfo

6685c5a

Move TimelineZoom out of BeatmapInfo

7f2a6f6

Move Countdown out of BeatmapInfo

d373f75

Move CountdownOffset out of BeatmapInfo

dd50d6f

Remove no longer applicable region marking

9fbf287

Bump schema version

c67e2dc

Fix TestBeatmap not transferring newly migrated properties

04527f3

bdach added type:code-quality realm deals with local realm database labels Jun 13, 2024

pull-request-size bot added the size/XL label Jun 13, 2024

bdach requested a review from a team June 13, 2024 13:36

smoogipoo reviewed Jun 17, 2024

View reviewed changes

bdach self-assigned this Jul 5, 2024

bdach mentioned this pull request Jul 10, 2024

Allow viewing bookmarks in Editor #28810

Closed

This was referenced Jul 17, 2024

Grid rotation does not save. (editor) #28895

Closed

Save grid rotation to beatmap #28973

Closed

kstefanowicz mentioned this pull request Jul 22, 2024

Save grid settings to beatmap via IBeatmap #29001

Closed

peppy self-requested a review July 23, 2024 03:02

Merge branch 'master' into beatmap-info-purge

d707e29

peppy reviewed Jul 23, 2024

View reviewed changes

Fix WidescreenStoryboard breakage after moving out of BeatmapInfo

1d4d806

bdach mentioned this pull request Aug 20, 2024

Fix crash on attempting to edit particular beatmaps #29534

Merged

peppy self-requested a review September 14, 2024 19:50

Merge branch 'master' into beatmap-info-purge

58fe502

bdach mentioned this pull request Sep 25, 2024

Implemented IDifficultyCalculatorBeatmap to move diffcalc-specific functions into specific classes #29989

Open

Merge branch 'master' into beatmap-info-purge

086a34f

peppy reviewed Nov 22, 2024

View reviewed changes

Fix incorrect comment

ead7e99

peppy previously approved these changes Nov 22, 2024

View reviewed changes

Merge branch 'master' into beatmap-info-purge

cf905d0

bdach dismissed peppy’s stale review via cf905d0 November 26, 2024 09:24

Fix Beatmap.Countdown not being copied on conversion

46d1f00

bdach marked this pull request as ready for review November 26, 2024 10:42

smoogipoo reviewed Nov 26, 2024

View reviewed changes

peppy approved these changes Dec 2, 2024

View reviewed changes

peppy merged commit 5b2558c into ppy:master Dec 2, 2024
9 of 10 checks passed

bdach deleted the beatmap-info-purge branch December 2, 2024 07:24

bdach added a commit to bdach/osu that referenced this pull request Dec 3, 2024

Move bookmarks out of BeatmapInfo

8089345

Not sure why I didn't do that in ppy#28473...

bdach mentioned this pull request Dec 3, 2024

Implement basic bookmark support in editor #30960

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move unnecessary properties from `BeatmapInfo` / realm to `IBeatmap` #28473

Move unnecessary properties from `BeatmapInfo` / realm to `IBeatmap` #28473

bdach commented Jun 13, 2024 •

edited

Loading

smoogipoo left a comment

smoogipoo Jun 17, 2024

bdach Jun 17, 2024

smoogipoo Jun 17, 2024 •

edited

Loading

bdach Jun 17, 2024 •

edited

Loading

peppy Jul 23, 2024

peppy Jul 23, 2024

bdach Jul 23, 2024

smoogipoo Jun 17, 2024

bdach Jun 17, 2024

peppy Nov 22, 2024

bdach Nov 22, 2024

bdach commented Jun 18, 2024

OliBomby commented Jun 18, 2024

peppy commented Jun 22, 2024

peppy Jul 23, 2024

peppy left a comment

peppy Nov 22, 2024

bdach commented Nov 22, 2024

bdach commented Nov 26, 2024 •

edited

Loading

smoogipoo left a comment •

edited

Loading

smoogipoo Nov 26, 2024

bdach Nov 26, 2024

bdach commented Nov 26, 2024 •

edited

Loading

smoogipoo commented Nov 26, 2024

peppy commented Nov 27, 2024

bdach commented Nov 27, 2024

bdach commented Dec 2, 2024

		public BeatmapInfo BeatmapInfo = new BeatmapInfo();
		public IBeatmap Beatmap { get; set; } = new Beatmap();

	protected override Storyboard GetStoryboard()
	{
	Storyboard storyboard;

	if (BeatmapInfo.Path == null)
	return new Storyboard();

	try
	{
	string fileStorePath = BeatmapSetInfo.GetPathForFile(BeatmapInfo.Path);
	var beatmapFileStream = GetStream(fileStorePath);

	if (beatmapFileStream == null)
	{
	Logger.Log($"Beatmap failed to load (file {BeatmapInfo.Path} not found on disk at expected location {fileStorePath})", level: LogLevel.Error);
	return new Storyboard();
	}

	using (var reader = new LineBufferedReader(beatmapFileStream))
	{
	var decoder = Decoder.GetDecoder<Storyboard>(reader);

	Stream storyboardFileStream = null;

	string mainStoryboardFilename = getMainStoryboardFilename(BeatmapSetInfo.Metadata);

	if (BeatmapSetInfo?.Files.FirstOrDefault(f => f.Filename.Equals(mainStoryboardFilename, StringComparison.OrdinalIgnoreCase))?.Filename is string
	storyboardFilename)
	{
	string storyboardFileStorePath = BeatmapSetInfo?.GetPathForFile(storyboardFilename);
	storyboardFileStream = GetStream(storyboardFileStorePath);

	if (storyboardFileStream == null)
	Logger.Log($"Storyboard failed to load (file {storyboardFilename} not found on disk at expected location {storyboardFileStorePath})", level: LogLevel.Error);
	}

	if (storyboardFileStream != null)
	{
	// Stand-alone storyboard was found, so parse in addition to the beatmap's local storyboard.
	using (var secondaryReader = new LineBufferedReader(storyboardFileStream))
	storyboard = decoder.Decode(reader, secondaryReader);
	}
	else
	storyboard = decoder.Decode(reader);
	}
	}
	catch (Exception e)
	{
	Logger.Error(e, "Storyboard failed to load");
	storyboard = new Storyboard();
	}

	storyboard.BeatmapInfo = BeatmapInfo;

	return storyboard;
	}

	case "UseSkinSprites":
	storyboard.UseSkinSprites = pair.Value == "1";
	break;

		if (Beatmap.Value.BeatmapInfo.EpilepsyWarning)
		if (Beatmap.Value.Beatmap.EpilepsyWarning)

	private void beatmapChanged(ValueChangedEvent<WorkingBeatmap> beatmap)
	{
	beatmap.OldValue?.CancelAsyncLoad();
	beatmap.NewValue?.BeginAsyncLoad();
	}

Move unnecessary properties from BeatmapInfo / realm to IBeatmap #28473

Move unnecessary properties from BeatmapInfo / realm to IBeatmap #28473

Conversation

bdach commented Jun 13, 2024 • edited Loading

smoogipoo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smoogipoo Jun 17, 2024 • edited Loading

Choose a reason for hiding this comment

bdach Jun 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdach commented Jun 18, 2024

OliBomby commented Jun 18, 2024

peppy commented Jun 22, 2024

Choose a reason for hiding this comment

peppy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdach commented Nov 22, 2024

bdach commented Nov 26, 2024 • edited Loading

smoogipoo left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdach commented Nov 26, 2024 • edited Loading

smoogipoo commented Nov 26, 2024

peppy commented Nov 27, 2024

bdach commented Nov 27, 2024

bdach commented Dec 2, 2024

Move unnecessary properties from `BeatmapInfo` / realm to `IBeatmap` #28473

Move unnecessary properties from `BeatmapInfo` / realm to `IBeatmap` #28473

bdach commented Jun 13, 2024 •

edited

Loading

smoogipoo Jun 17, 2024 •

edited

Loading

bdach Jun 17, 2024 •

edited

Loading

bdach commented Nov 26, 2024 •

edited

Loading

smoogipoo left a comment •

edited

Loading

bdach commented Nov 26, 2024 •

edited

Loading