-
Notifications
You must be signed in to change notification settings - Fork 11
[Import] Jellyfin : Strange syncing issues... #141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hi, i am writing this reply from my phone, From quick log view it seems you encountered NFO jellyfin bug for movies at least. if you could please try pulling newest version. After that import all your backends. Once that is done try running import for each server. if you notice huge records changes between each single backend import that could indicate problem with that backend. Im guessing here probably one of your backends lost access to your media library and the records got purged and after gaining access it pulled new metadata and now has mismatch. You can also run backend:library:mismatch for the problematic backend to see if we are able to determine which library item has mismatched data. |
Thanks :) I ran the individual imports a few times again, but they keep reporting huge records changes for sure. Even running just the Plex import a few times in a row keeps resulting in a huge number of record changes (similar to what I posted above).... time and time again. The mismatch query shows some interesting things. I ran it on JF first and it showed a few mismatched movies (two, to be exact). I rematched them in JF but still the same result (both movies still mismatched). Emby showed 4 mismatched movies. Again, matching them in Emby didn't correct it. Plex, whoa. At least 20 or 30 mismatched movies (I didn't attempt to correct those). |
No worries im now back at my pc i could take better look on logs, if you can please do the following, first move your database in docker exec -ti watchstate ash -c "console state:import -vvv --force-full --trace > /config/logs/import.full.log"
docker exec -ti watchstate ash -c "console state:import -vvv --trace -s plex > /config/logs/import.plex.log"
docker exec -ti watchstate ash -c "console state:import -vvv --trace -s jellyfin > /config/logs/import.jellyfin.log"
docker exec -ti watchstate ash -c "console state:import -vvv --trace -s emby > /config/logs/import.emby.log" once that done, compress the /config/logs/ folder and either attach link here or you can email them to me if you dont feel safe sharing them here. |
Haha yeah that sure took a while... but here you are: -snip- Again, thanks so much, I appreciate your support! |
Np, i downloaded the logs you can delete the logs now if you want. i'll take look at hopefully will manage to point to the problem. |
From first look as i expected it seems jellyfin screwing up your data for some reason, take for example these following lines
The metadata from your jellyfin backend seems to be corrupted, And is reporting themoviedb ids as if they are imdb ids, if you check your initial import log file you will notice most of jellyfin updates relates to something like Both emby/plex seems to agree on ids for the majority of your import while jellyfin disagree, this worth investigating more. do you happen to use NFO files for jellyfin? it seems there are bugs related to it in 10.7x |
Ah, yes, NFO is enabled. To tell you the truth, I'm not overly attached to Jellyfin. I still have a lifetime Plex and Emby subscription, and JF doesn't really enhance anything over those two. If the answer is to abandon it, I wouldn't cry ^_^ But if it helps you troubleshoot, I'd be more than willing to see if we can sort this. Is there anything you'd like me to do? |
I honestly wouldn't want to abandon jellyfin, it's safe bet for the future in case plex/emby do something stupid. Jellyfin NFO related bugs It seems it doesn't fix the external ids even after doing full metadata replace. You could continue using jellyfin, and remove your movies library from it as it seems thats the only problem related to NFO, otherwise your episodes listing seems to be fine. and reporting correct ids in majority of the cases as far as i can see. on my side i think I'll implement fix to reject the external ids from jellyfin if they dont match expected format for imdb. |
Ok, that's a good idea... it won't be so bad just replacing the movie library since that scans a lot faster than all those series. I'll do that and report back. Would you recommend starting WatchState from scratch again after that's done? |
It's always good idea to start fresh, data purity is important in this, as you might end up with duplicate records in the database thanks to those ids :-( |
Sure thing :) I'll report back in a day or so if it's behaving correctly then! |
No worries, i'll be implementing a fix for this hopefully today or tomorrow. Thanks for bringing this up |
Great! Meanwhile, I have deleted the Movies library from Jellyfin and re-added it without NFO. Then I reinstalled WatchState from scratch and ran the initial import. I guess this is what you intended, but here's a heads up of some of the errors it spits out:
|
Indeed thats the expected behavior until jellyfin fixes their NFO parser :-( to actually add your movies again you have to remove the library refresh jellyfin cache and then re-add the library. otherwise it will still pick up the incorrect ids as far as i know |
OH! Yeah I deleted the library but I don't think I refreshed the JF cache - I don't even know how to. I can certainly do it again, adding the library was a lot fast than I expected really. |
i think it's in the Scheduled Tasks tab |
Thank you :) I'll try that tomorrow and report back again! |
No worries. good night =) |
Okay as promised, a status report :) So, deleting the library, clearing the cache and re-adding it didn't solve it, because it adds the movies with exactly the same flaw as before. But at least now with the new WS version it doesn't bugger up anything else any longer. I do however still have one issue: watching things on for example Plex will mark it watched on the other services, but marking something watched manually doesn't do anything at all. I know this was a know issue with Plex, but it doesn't work with Emby or JF either. Perhaps unrelated but running |
--queue-show has become its own command For marking things on other services. Tr the following, mark on emby or jellyfin after that pull only changes from that backend docker exec -ti watchstate console state:pull -v -s emby and check the item in |
Ahh ok. So yeah, db:queue shows stage:pull -v -s emby shows
And finally, db:list shows only stuff that is synced between Plex and Jellyfin, whatever got marked in Emby didn't sync to either. The plot thickens 😆 |
sorry i meant the data shown in db:list reflect the last changes received to see full data entry try |
Hah, okay enjoy your leisure time! state:import with emby gives an error as well
As for marking things watched on both services - it seems to be a little erratic (works sometimes, doesn't work other times) so let me shut my big mouth until I have tested stuff in a little more detail with all the commands you just shared 🤣 |
i think i understand why it's happening, it seems emby also effected by the NFO bug or at least this one entry you got is using tmdb ids as imdb, anyway I'll be making new change to make it into an error instead of fetal exception. and continue parsing other ids. |
@TechPerplexed fix is out, pull the new container and try now. |
Aaallllllllllmost, lol. Sorry to be such a nuisance - I really appreciate your responsiveness and patience! So here's the latest. I waited for the queue to finish before reporting to you. I marked one episode watched on Emby and one on Jellyfin (since with Plex only actual watching seems to matter). Marked watched on JF: synced without issues to Emby and Plex So, Jellyfin works fine, so does Plex. |
Hmm, it might be possible you actually have import disabled for emby? could you try
|
This is what I have: ┌────────┬───────────────────────────────────────────┐ There are two Emby users, that is why I enabled user:true. As for |
Your config seems to be fine. Sorry it's |
Ah, no it doesn't get logged, however I do see a long list of errors like these:
Edit: oh wait, I do see the event of it RECEIVING the entry from Jellyfin:
|
Sorry im not following, do you mean emby events are being shown as jellyfin events i log? |
Oh I'm not sure - this is the output of |
haha, no problem, i think thats unrelated as it's show everything there, anyway, i need to see some request dump from emby, edit your emby webhook url to include this |
Wait, wait wait wait. You are magic. You are solving stuff without even updating anything ^_^ I just marked something watched on Emby and another on Jellyfin, and the docker logs that is still running just mentioned
For reference, that is one episode I marked watched on Emby, and one episode I marked watched on JF. But here's the real beauty: they BOTH synced with each other!!! Don't tell me it got solved by just talking about this 😆 |
Haha, i wish it was this easy, anyway because i dont have Emby premium it's hard for me to diagnose any webhook related problems there. anyway your logs helped and i discovered that i fat fingered something in Emby webhook parsing. a fix should be out shorty. |
That reminds me.... do you accept donations? It might buy you at least a few months of Emby (or a few beers, whichever you prefer). Anyway thanks for this really useful script, I really don't miss Trakt! Cheers, Sophia |
Oh.. dont worry about, the only reason i dont have Emby is i actually like Plex interface more and it works well with my apple tv setup, i plan on getting Emby premium just before release to check everything related to it. If you feel like donating please donate to any charity that take care of kids everywhere. ❤️ PS: new build is out and ready for testing. do the same again |
Awww, you're awesome :) Also, this log is more like it!!
|
Glad it's working for you now :-) Please don't hesitate to report any issues/problems that you may notice. 👍 |
Haha ok. Right now the most pressing issue is a hungry hubby and three cats that want feeding ^_^ Thanks again for everything! |
For some reason WatchState stopped syncing a few days ago, so I removed the container and started again from scratch. This is on Debian 11 with Docker. However, it suddenly marked thousands of random episodes as watched. I checked the import log and it constantly updates the same entries.
I'm also noticing some weird renaming such as
[2022-06-03T10:08:50.241361-07:00] logger.NOTICE: plex: Updating 'Dark Angel (2000) - 01x004'. State changed. {(guids: [ (guid_imdb: [ (old: tt0554559), (new: tt0554566) ]) ])}
Also it doesn't seem to sync in between services. Any idea what could be wrong? As always, thanks :)
The text was updated successfully, but these errors were encountered: