[Spec] Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL

[achimmihca]

Suggestion

Melody Mania uses the #WEBSITE header field to specify a website that should be used in its embedded browser.

This way, it is possible to specify a youtube link for example that can be used as alternative to local audio and video files.

Note that the same approach might also work with other platforms such as Spotify and Apple Music.
Thus, I suggest to use WEBSITE instead of platform specific names such as YOUTUBE or similar.

Note further that WEBSITE fits with other header names of the format, e.g. VIDEO, BACKGROUND, and COVER. These header fields describe what the header value points to.

Use case

• Link to or embed the website for a preview of a fitting audio / video for the txt file (i.e. with matching GAP, VIDEOGAP, etc.)
• Use website as alternative to local audio and video files (e.g. in Melody Mania)

Extra info/examples/attachments

No response

[Baklap4]

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ?
If yes this can potentially lead to security/malware risks if no whitelist is made
Personally i'm more fan of using an enum/typed field since this is self-explainatory, easy to implement and not prone to beforementioned risks

[achimmihca]

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ?

Yes.

this can potentially lead to security/malware risks if no whitelist is made

I would consider this an implementation detail of karaoke players, not part of the spec.

For instance, Melody Mania loads only websites from supported hosts. And a new host can be supported by adding a corresponding JavaScript file.
So you have a whitelist mechanism there. But other players may use a different approach.

As I said, I would not consider whitelisting part of the spec.

i'm more fan of using an enum/typed field

New platforms may arrive in the future. Further, communities might want to host their own song server.
Thus, I don't think that an enum with a list of supported platforms is sufficient.

Which websites can be used should be a config issue of the player in one way or another. Not part of the spec.

[Baklap4]

How would this be combined with say for example our current #MP3 header or the newly introduced #AUDIO or in the later future maybe both #AUDIO and #INSTRUMENTAL? Would this field contain multiple values seperated by character or would we have multiple #WEBSITE tags

If we're going to introduce this tag it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file.

Now thinking of it... what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields?
If no http is found search local. If the string is found load from web? This way we don't need an extra field

Within performous we can for example specify a NAS through the network share \\NAS\Somefolder and it'll load everything just fine

[achimmihca]

Good question.
My goal is to provide a fallback mechanism when local files are not present.

it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file

Yes.

what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields

Nothing. Actually, Melody Mania and UltraStar Play accept an http URL in these fields that points to an audio / video / image file.

However, my goal is to provide a fallback mechanism when local files are not present. Thereby, it would be nice to stay compatible with offline play in existing UltraStar games.

Thus, I argue that a new field is required for this.

---

But you are right, other header fields could make use of an online fallback as well.
So maybe the field should accept a key value list, where the keys are other header fields.

Further, I would expect the fallback to be something online (e.g., a URI, YouTube video ID, or some other identifier).
So maybe a better name for the field would be #ONLINE or #CLOUD?

Example:

#ONLINE: audio="https://something.com/audio.ogg",cover="https://something.com/image.png"

I think this would be similar to what people are doing with new txt files on usdb.animux.de.
Their goal is also to make the txt file self-sufficient such that other stuff can be fetched when needed. At the moment they use the #VIDEO field for this, which breaks compatibility with local video files. A standardized approach for this would be nice.

[marwin89]

SUMMARY

So here is a summary for this issue up till now.

We add this tag to version 1.1.0:

#ONLINE: audio=1k8craCGpgs, video=yCC_b5WHLX0, cover=heart-skips-a-beat-57c3df3b913b0.jpg, background=4f9954bd84f67.jpg

The description in the spec (website) could be:

#ONLINE
This tag links to different online ressources and webservices. You can link for a preview/reference/embed of a fitting audio or video that matches with GAP, VIDEOGAP, in the txt-file. Use IDs or URLs from the webservices of your choice (for example: Youtube, Spotify, Apple Music, Vimeo etc.). Right now the tag can only be 255 characters long.
Allowed keys are: audio=, video=, cover=, background=
Allowed values are: ID (prefered), complete url

@bohning : could you help us out here with usdb syncer meta-tags ? That's a lot to document here. You also got advanced features: cropping, rotating and resizing of covers.

[Nianna]

Can't we use several flat tags for this?
e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have
#KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

[Baklap4]

I can see why it's been done for USDB_Syncer and the #VIDEO tag as the syncer processes it and makes a #KEY:VALUE header which is easy to parse (which i think is good). Introducing multi-key value within the text file without processing after can make things a little bit tricker and more complex

[bohning]

Can't we use several flat tags for this? e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have #KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

Then I would vote for #COVERURL, #VIDEOURLand #AUDIOURL (and sort them right after the respective offline tags), so that

#AUDIO:xyz.mp3
#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEO:xyz.mp4
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVER:xyz [CO].jpg
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUND:xyz [BG].jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

The main reason for using YT and fanart.tv IDs for usdb/usdb syncer is to save characters (max. line length = 255 chars) and to push people towards using those two platforms over others. If the source URLs get their own tag, then at least the first reason vanishes.

[achimmihca]

keeping a flat, easy-to-understand structure

I agree

• the first suggestion (maybe name it nested-key-value) would add a second key=value mechanism to the file format, which is redundant
  • if we decide to use this then syntax should be symmetric with [Spec] Add tags for online music databases #19
    • BTW: in this scenario, should we separate mulitple keys-value pairs with comma or ampersand? I suggest to use ampersand such that existing URL-parsing libs can be reused.
• @bohning's suggestion (maybe name it flat-key-value) also works for me
  • on first glance, it is simpler and more intuitive

---

In any case, I would prefer that we keep the file format consistent.
In the flat-key-value scenario, should we also prefer a flat-key-value style for #19 ? For example

#MUSICDBX:VALUEX
#MUSICDBY:VALUEY
...

[Nianna]

should we also prefer a flat-key-value style for #19

Yes, let's use flat structure for all tags.

[bohning]

We might want/need to add

#COVERROTATE:[angle in degrees ccw -> float]
#COVERCROP:[left-upper-width-height]
#COVERRESIZE:[width --> square: width=height]

if online resources are used. Similarly, for backgrounds (but not rotation needed):

#BACKGROUNDCROP:
#BACKGROUNDRESIZE:

These are features that the syncer uses to provide square covers and e.g. 16:9 backgrounds (furthermore, it has the option to increase the image contrast for covers --> #COVERCONTRAST:).

[marwin89]

2nd SUMMARY / VOTE NOW 📈 ✋

Thanks for the discussion. So here is the second summary for this issue. We are close to a final result.

Please vote for this issue till 15th nov 2023 with emoticon 🎉 / 🚀.

Options

🚀 A: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and Standard crop/resizing method in USDB SYNCER with central clipping

🎉 B: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and #COVERROTATE, #COVERCROP, #COVERRESIZE, #BACKGROUNDCROP, #BACKGROUNDSIZE

👀 C: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and tags are not flat anymore

❤️ D: Change value structure in #AUDIO/#MP3, #COVER, #VIDEO, #BACKGROUND from flat to complex (for txt-file hostings).

What?

• Adding # attributes to identify online files vom webservices that fits to txt-file.
• making txtfiles self sufficent and/or "streamable".
• Song Editors just have to enable editing the # attributes
• All downloading functionality is done by USDB SYNCER (and optionally by games)
• Games can use it for fallback or streaming, but don't have to

Why?

• USDB-SYNCER and USDB: the URLs/IDs help usdb-syncer to know the sources to load.
• FOR SONG REWORKS: It helps Song Creators to understand on which file basis a song was originally made at all, there are a lot of messy txt files in usdb that are build-on obscure mp3 with wrong bpm that nobody understands anymore. with audio/videourl this problems are history.
• PREVIEW/FALLBACK of missing local files in game and streamable alternative to local files (initial suggestion of achimmihca in this issue,)

---

details A 🚀:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

These tags are optional.

➕ Built-in standard method in USDB SYNCER that crops/resizes cover and background images with centered clipping to max Full-HD. Many CD Covers will not be cropped good.

---

details B 🎉:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#COVERROTATE:-0.3
#COVERCROP:0-200-1000-1000
#COVERRESIZE:1920
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDCROP:10-10-1920-1080
#BACKGROUNDRESIZE:1920-1080

These tags are optional.

details C 👀 :

Introduce complex tags

#AUDIOURL:https=asdfghjkloi
#VIDEOURL:https=asdfghjkloi
#COVERURL:https=asdfghjkloi,resize=500x500,crop=0-200-1000-1000,rotate=-0.1
#BACKGROUNDURL:https=asdfghjkloi,resize=500x500,crop=10-10-1920-1080

details D ❤️ :

We change the value structure in existing # Attributes in the next version v.1.1.0 from flat to complex.
(Only if usdb.animux gets an improvement from Alex! Or new USDB.) In the TXT-FILE Hostings the value structure would be

#AUDIO/#MP3:url
#COVER:url,rotate=,crop=,resize=
#BACKGROUND:url,resize=,crop=
#VIDEO:url,trim=,crop=

#Audio/#MP3 stays mandatory, the others stay optional. Games detect files in the specific song folder by file type for instance .mp3, .wav .ogg = Audio, mp4 m4a wmv = VIDEO, Cover = jpg / png file name contains [CO], Background = jpg png file name contains [BG].

---

[codello]

What is the state of this proposal? If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue. This could be similar to how HTTP handles headers:

• Single-Valued headers/tags can only be specified once. If a header is specified multiple times it is implementation dependent which value takes precedence (or we define it to be the first/last one).
• Multi-Valued headers can have multiple values separated by a comma. This is the consensus that was reached in [Spec] Multiple entries in #GENRE / #LANGUAGE / #CREATOR #30 for GENRE, LANGUAGE, etc.
• Multi-Valued headers can also be specified multiple times. Multiple occurrences are then concatenated with a comma. This solves the issue about a max line length.

(The same behavior should of course be applied to other multi-valued tags)

I think this would be a quite elegant solution for this proposal: The fields AUDIO, VIDEO, BACKGROUND, and COVER become multi-valued. The semantics of multiple values in a single field are that different values are to be tried in order. So a header like this:

#AUDIO:foobar.mp3
#AUDIO:file://nas.local/foobar.mp3
#AUDIO:https://youtube.com/…

Would semantically be equivalent to:

#AUDIO:foobar.mp3,file://nas.local/foobar.mp3,https://youtube.com/…

When loading such a song a program would first try to load the local foobar.mp3, then try the file located on a network share and as a final fallback try the YouTube URL (if supported by the client). Alternatively the spec could allow clients to choose a file by their own preference (which would be my preferred solution).

From a specification perspective each value would be a URL. If a value does not include a URL scheme it‘s understood to be a local file located relative to the txt file.

One drawback of this solution is that file names that contain a comma could not be used. I think this is an acceptable limitation but could be solved by implementing an escape sequence of some sort.

[DoubleDee73]

@codello For now, my implementation is single-valued only, but that doesn't mean, it couldn't be multi-valued. To be honest, personally, I'm not sure, what good it would do, to have multi-values anyway. The games would have to have some sort of fallback strategy, if a file was not found locally, to look into a NAS, or on youtube. The path to the NAS would be very user-specific anyway. I doubt, that would have a shared value.

[achimmihca]

If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue

We had the discussion of multi-value vs. single-value (aka. nested vs. flat) tags before.
We decided to stay with single-value (flat) tags. For details, see above comments.

[codello]

I think I may have worded that badly.

I read the whole discussion in this issue and I understand the nested vs. flat argument. On that I don't have a strong opinion either way but from what I gathered there seemed to be a tendency towards simpler structures (so flat tags).

To me the multi-value vs. single-value is a different point. The spec already includes multi-valued tags (see #30). I was thinking that maybe the same mechanism could be used to address the need for URL-based files. I'm not suggesting to introduce a complex sub-language for the media files. I just thought that maybe the existing multi-value system might me a good fit as it would alleviate the need for additional tags.

I'll try and phrase my suggestion a bit differently:

---

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

...
AUDIO:foobar.mp3
AUDIO:https://youtube.com/...
...

The exact semantics of this would have to be decided. I can think of two plausible approaches:

• The order of precedence is the order in which the tags appear in the file. So in the example above the local file foobar.mp3 would be the preferred audio source and if it does not exist or cannot be read, the YouTube link serves as a fallback.
• The order of precedence is an implementation detail and thereby defined by the application. I think I'd actually prefer this solution as it would give the end users as opposed to the song creators the choice of preferred media source.

In addition to this I'd like to unify this syntax of repeating a tag with the existing multi-valued tags in order to reduce the complexity of the spec. But this aspect can be discussed separately.

[Baklap4]

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

AUDIO:foobar.mp3
AUDIO:https://youtube.com/...

The web usually handles multiple same keys in this way: ?audio=foobar.mp3&audio=youtube.com/...

Basicly the same as @codello is suggesting. I'd choose @codello answere over any other mentioned solution

[achimmihca]

I like your suggestion in general.
However, I am unsure whether this is compatible with old implementations.

The solution with new tags has the advantage that it is backwards compatible because unknown tags are ignored.

[codello]

That's a good point. I don't think it's fundamentally incompatible, but in practice it would be implementation-dependent. There is currently no spec that mandates that headers/tags cannot occur multiple times but I think most implementations assume that's the case.

Specifically USDX should actually be compatible with this suggestion. USDX issues an Error log entry for every MP3 (or VIDEO etc.) entry that does not resolve to a file but if at least one MP3 value actually resolves to an existing file, the song will still load without problems. If multiple MP3 values resolve to files, the last one will be used by USDX.

I don't know how other implementations would deal with repeated headers, but the most straightforward handling would be to just use the last value. So if an implementation doesn't specifically forbid the use of repeated headers this should be backwards compatible.

If we combine this with the multi-valued header syntax used for #TAGS etc. then that part would not be backwards-compatible. So maybe it would be best to defer that part to v2 of the spec which contains breaking changes anyways (or defer even the repeated headers to v2, just to be safe).

[randompersona1]

Im late to the party, but, disregarding the tags to edit the images, what is the expected implementation?

• is a file/song considered valid if it does not have an #AUDIO tag, only a link? That would mean a check whether a song is valid could take as long as the timeout for the request is.
• should a client save these to the file system or even the library folder, or stream them every time? Streaming works kind of well as a fallback for melody mania (at least for video, haven't checked the other tags), but still requires manual intervention afterwards to actually get the local file as nobody wants to watch ads before every song. After that, why even have the tags?
• if a client (or the syncer, or the manager, etc.) detects a link is dead, is it safe to delete and save every future program from trying it again? Rhetorical question, obviously no. However, that means that, without user intervention, every program will retry the link every time. The standard should define what a program does here. I honestly don't see an elegant solution to this.

All of these are obviously implementation-related, but i do think it makes sense to clarify how these tags are supposed to be used.

I propose that the real benefit of these tags is actually updating. Someone finds a better cover image or a cleaner version of a song? No problem, edit txt on usdb, now anyone can grab the new links and update their local files. Of course, that would require a song link, not just the name of the website (not sure if that was the intention of #SOURCE in #35 ), and also some sort of versioning or supplied hash, but that seems doable, even in the tags themselves.

[basisbit]

is a file/song considered valid if it does not have an #AUDIO tag, only a link?

That is up to the developer of the specific implementation. This imho should not be part of the standard.

check whether a song is valid could take as long as the timeout for the request is.

Oh gosh please do not ever implement it like this. Please always keep in mind how many people there are who might be using the program and how few voluntary operators of the file hosting servers there are. Bandwidth is not free. Servers are not free. And song libraries with thousands of songs are not uncommon.

should a client save these to the file system or even the library folder, or stream them every time?

Cache them on your local system at least for a few weeks or until your cache size limit is reached (fifo). But this also is highly dependent on implementation specific requirements.

if a client (or the syncer, or the manager, etc.) detects a link is dead, is it safe to delete and save every future program from trying it again?

You are talking about the internet. Stuff goes 404 or unreachable for some unknown amounts of time occasionally, usually comes back again, until it at some point doesn't come back any more.
So: implementation specific, however please keep in mind to not overstress shared resources.
I'd suggest not deleting txt files with http 404 / unreachable stuff, but instead only hide or unavailable-mark those songs until the next time the game is opened for example.

[marwin89]

Final Result: Introduce URL-Header

Dear all, I've got lost in this long discussion and didn't see the actual result: We already had a consens for Option A (6 Votes). So forget my former post.

Here is the final result:

details A :
We add to the next version v.1.2.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg
These tags are optional.

Built-in standard method in USDB SYNCER that crops/resizes cover and background images with centered clipping to max Full-HD. Many CD Covers will not be cropped good.

With this I'll do the release of version v1.2.0 with the url-header in january (via website usdx.eu/format and spec.md)