Inconsistant result from playlist URI using Spotify.playlist_tracks

[fcusson]

Describe the bug
We use Spotipy for an Home Assistant custom component Spotscat. An issue was raised by a user that seems to come from Spotipy handling of URI using the ?si= element that was added by Spotify to comply with EU regulations. This means this bug is only reproduisible naturally from an EU based Spotify install. Otherwise, the ?si= will not be present in the URI.

If a user pass a URI that was provided from an EU Spotify client, the playlist_tracks method will return a different dictionary than if it was provided without see issue 267:

test	result
with ?si=	dict_keys(['collaborative', 'description', 'external_urls', 'followers', 'href', 'id', 'images', 'name', 'owner', 'primary_color', 'public', 'sharing_info', 'snapshot_id', 'tracks', 'type', 'uri'])
without ?si=	dict_keys(['href', 'items', 'limit', 'next', 'offset', 'previous', 'total'])

Your code

import spotipy
from spotipy.oauth2 import SpotifyClientCredentials

auth_manager = SpotifyClientCredentials()
sp = spotipy.Spotify(auth_manager=auth_manager)

# uri with ?si=
result_1 = sp.playlist_tracks('spotify:playlist:5rAhD02CGReFtSbinTHxkn?si=968337997b194202')
print(result_1.keys())

# uri without ?si=
result_2 = sp.playlist_tracks('spotify:playlist:37i9dQZF1DZ06evO3hsjg5')
print(result_2.keys())

# potential fix
assert result_1['tracks'].keys() == result_2.keys()

# current uses in Spotcast
print(result_2['total'] - 1)
print(result_1['total'] - 1) # provides the max index of the tracks, this will fail

Expected behavior
Both types of URI should return the same behavior (a dictionary containing the tracks of a playlist)

Output

>  python 267_test.py 
dict_keys(['collaborative', 'description', 'external_urls', 'followers', 'href', 'id', 'images', 'name', 'owner', 'primary_color', 'public', 'snapshot_id', 'tracks', 'type', 'uri'])
dict_keys(['href', 'items', 'limit', 'next', 'offset', 'previous', 'total'])
43
Traceback (most recent call last):
  File "/home/fcusson/Sources/spotcast/267_test.py", line 20, in <module>
    print(result_1['total'] - 1) # provides the max index of the tracks, this will fail
KeyError: 'total'

Environment:

• OS: Ubuntu 21.10
• Python version 3.9.7
• spotipy version 2.19.0
• your IDE (if using any) VS code (same behavior when running directly from bash

Additional context
The problem seems to be related to how spotipy manages the return from Spotify API. In my minimal code I check and realized that we seem to be one level up in the data provided. In the hierarchy of data, we normally have ... -> playlist -> tracks >> return as dictionary, but when receiving a EU Complient URI, we seem to stop at the playlist element, and we never extract the tracks portion.

[fcusson]

I think I found part of the problem. Using this code:

import spotipy
from spotipy.oauth2 import SpotifyClientCredentials

auth_manager = SpotifyClientCredentials()
sp = spotipy.Spotify(auth_manager=auth_manager)

# uri with ?si=
print(sp._get_id("playlist", "spotify:playlist:37i9dQZF1DZ06evO3hsjg5"))

print(sp._get_id("playlist", "spotify:playlist:5rAhD02CGReFtSbinTHxkn?si=968337997b194202"))

I get the following result

37i9dQZF1DZ06evO3hsjg5
5rAhD02CGReFtSbinTHxkn?si=968337997b194202

we can see the _get_id method simply slice the string ad returns whats is after the spotify::. That would explain the problem. A solution would be to check to remove anything following (and including) an interogation mark