Level 4 decos, black bars, ignore updates (#36)

* likely black bar removal, lvl4 deco

* black bars checkbox in UI

* ignore main app updates

* fixes #34 by actually translating language

* You can now add "invalid" skills

* version bump & readme

* fixed UTF-8 loading issue with symspellpy (thanks W-1252)

.vscode/launch.json

@@ -4,6 +4,7 @@
     // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
     "version": "0.2.0",
     "configurations": [
+
         {
             "name": "Python: Current File",
             "type": "python",

PATCHNOTES.md

@@ -1,5 +1,25 @@
-# Patch Notes (Updated June 22nd 2022)
-- 1.6
+# Patch Notes
+- 1.7 (July 8th 2022)
+  - Added level 4 slot support (#35)
+  - Fixed restarting the app after setting a different game language (#34)
+  - You can now ignore an update
+  - You can now add invalid skills (or skills that are not in the list, don't come to me if the builder doesn't work)
+  - You can record in windowed mode (See next point)
+  - You can now let the app remove black bars from videos
+    - It is experimental. You should fix your recordings first if it doesn't work
+    - Black bars can be caused by not setting your TV size to 100% in the switch settings.
+
+# Older patch notes
+- 1.6.2 (July 5th 2022)
+  - Fixed an issue with the creation of the config file
+  - `--reset` commandline option now takes into account `-a` and `-l`
+- 1.6.1 (July 5th 2022)
+  - Tesseract location now goes through the entire list of OS-specific locations
+  - You can now set a custom tesseract location in the config (it has a higher priority)
+  - Added a few extra error messages
+  - New English Sunbreak skills and corrections are now bundled.
+  - The other languages use the english names as placeholders, sorry about that, feel free to submit the names.
+- 1.6 (June 22nd 2022)
   - Added an app language dropdown
   - Version Checker automatically updates language files
     - Other updates (main app, skills and corrections) are still "on demand" when there is an update 
@@ -8,10 +28,6 @@
   - Languages in language dropdowns should show up in the actual language
     - NOTE: They are probably wrong
   - Added a "Go to Set Searcher" button
-
-
-# Older patch notes
-
 - 1.5.3 (FEB 3rd 2022)
   - Updated where language packs are downloaded from, 
     - Github changed where "raw" data was fetched from and broke "fresh" installs 

README.md

@@ -12,27 +12,16 @@ This repo contains code that will allow you to extract all of your charms in Mon
 
 It's called Utsushi's charm because I thought it would be funny to make a complementary "Utsushi's Armor Search System", but [this armor set searcher](https://mhrise.wiki-db.com/sim/?hl=en) exists. I might still try to port Athena's ASS for MHW to MHR, but for now this works for me.
 
-# Patch Notes (Updated July 5th 2022)
-The next non-bugfix version will have the option to ignore an update for those who don't care or don't have issues. Sorry for the double update in one day
-
-- 1.6.2 (July 5th 2022)
-  - Fixed an issue with the creation of the config file
-  - `--reset` commandline option now takes into account `-a` and `-l`
-- 1.6.1 (July 5th 2022)
-  - Tesseract location now goes through the entire list of OS-specific locations
-  - You can now set a custom tesseract location in the config (it has a higher priority)
-  - Added a few extra error messages
-  - New English Sunbreak skills and corrections are now bundled.
-  - The other languages use the english names as placeholders, sorry about that, feel free to submit the names.
-- 1.6 (June 22nd 2022)
-  - Added an app language dropdown
-  - Version Checker automatically updates language files
-    - Other updates (main app, skills and corrections) are still "on demand" when there is an update 
-  - App and game language are now stored inside a config file
-  - Added a --reset cmd option to clear the config file
-  - Languages in language dropdowns should show up in the actual language
-    - NOTE: They are probably wrong
-  - Added a "Go to Set Searcher" button
+# Patch Notes (Updated July 8th 2022)
+- 1.7 (July 8th 2022)
+  - Added level 4 slot support (#35)
+  - Fixed restarting the app after setting a different game language (#34)
+  - You can now ignore an update
+  - You can now add invalid skills (or skills that are not in the list, don't come to me if the builder doesn't work)
+  - You can record in windowed mode (See next point)
+  - You can now let the app remove black bars from videos
+    - It is experimental. You should fix your recordings first if it doesn't work
+    - Black bars can be caused by not setting your TV size to 100% in the switch settings.
 
  The rest of the patch notes can be found [here](PATCHNOTES.md)
 
@@ -124,13 +113,20 @@ Having translations for the instructions for other languages might be useful, bu
   - I see no reason other capture cards should not work
   - 1080p adds extra processing due to a downscaling step to 720p (20-30% slower)
   - I also tested using an Mclassic without any issues.
+- Q: I record with a capture card but there are black bars
+  - 99% of the time this will be caused by not having the screen size set to 100% in the switch settings
+  - In the remaining cases, you should check your recorder settings.
+  - You can try using the "Remove black bars" feature.
 - Q: Does this Work for the PC version
   - Yes. As long as your resolution is 720p or above, and has a 16:9 ratio, it should
     - Basically, 1080p, 1440p, 4k and above should work without issues, albeit, the higher the resolution, the slower.
 - Q: I'm trying to enter a skill in the Skill Correction thing and it doesn't work
   - This might be because the PC version has a different name for the skill, add it to the corrections.lang.csv file, or tell me to do it.
     - An example of this would be `Quick Sheath`, which is named `Quick Sheathe` on PC
     - Make sure you give me both versions of the skill name if you think the PC version messed up again.
+- Q: Can I record in windowed mode
+  - Yes. 
+  - Yes, but if you have the "top bar" in your recording you will need the "Remove Black Bars" feature. You should try not recording the full window and only the game.
 
 # Notes
 - Version 1.5 is the last version that will officially support console mode, I may fix it from time to time if I need it for dev reasons, but I don't want to maintain a secondary workflow that will likely be used by nobody.

data/translations/eng.json

@@ -7,7 +7,7 @@
     "input-dir": "Input video location: ",
     "frame-dir": "Frame location: ",
     "skipping-frames": "Skipping frame extraction: ",
-    "skip-frames": "Skips frames extraction\n (Recover from previous run)",
+    "skip-frames": "Skip frames extraction\n (Recover from previous run)",
     "skip-charms": "Skip charm extraction (Why??)",
     "delete-frames": "Delete old frames",
     "deleting-frames": "Deleting existing frames: ",
@@ -68,5 +68,9 @@
     "tess-wait-5-retry": "Waiting 5 seconds before trying to download the language pack again...",
     "tess-url-error": "A URL error occured, this might be a DNS issue... retrying",
     "tess-cannot-download": "Unable to download Tesseract language pack.",
-    "tess-not-found": "Tesseract was not found. Please install it, and if it is installed, pass it as a command line option at least once. If you have done both and it still isn't found, please open an issue on Github."
+    "tess-not-found": "Tesseract was not found. Please install it, and if it is installed, pass it as a command line option at least once. If you have done both and it still isn't found, please open an issue on Github.",
+    "remove-black-bars": "Remove black bars\nMay work with windowed capture",
+    "remove-black-bars-info": "The app will attempt to remove black bars.\nIt should work if you are recording in windowed mode.\nThis should be useful if your switch isn't properly scaled (TV Size setting).\nThis is 100% experimental (and somewhat random). Fix your videos before you complain :)",
+    "upd-ignore": "Ignore",
+    "add-as-is": "Add skill as-is"
 }

data/translations/fra.json

@@ -7,7 +7,7 @@
     "input-dir": "Dossier des vidéos: ",
     "frame-dir": "Dossier des frames: ",
     "skipping-frames": "Sauter l'extraction des frames: ",
-    "skip-frames": "Ignore l'extraction des frames\n (Récupération d'une exécution échouée)",
+    "skip-frames": "Sauter l'extraction des frames\n(Récupération d'une exécution échouée)",
     "skip-charms": "Sauter l'extraction des charmes (Pourquoi??)",
     "delete-frames": "Supprimer les anciens frames",
     "deleting-frames": "Suppression des frames: ",
@@ -65,8 +65,12 @@
     "wiki-button": "Visiter le MHR Set Searcher",
     "wiki-clipboard": "Copier l'URL",
     "wiki-attempt": "Tentative d'ouverture du site web...",
-    "tess-wait-5-retry":"Pause de 5 secondes avant de ré-essayer le téléchargement du language pack...",
-    "tess-url-error":"Une erreur d'URL s'est produite, potentiellement causé par une erreur DNS...",
-    "tess-cannot-download":"Impossible de télécharger le language pack Tesseract.",
-    "tess-not-found": "Tesseract n'a pas été trouv. Veuillez l'installer, s'il est installé, passez le à travers le command line au moins une fois. Si cette erreur continue d'apparaître, veuillez créer une issue sur Github.."
+    "tess-wait-5-retry": "Pause de 5 secondes avant de ré-essayer le téléchargement du language pack...",
+    "tess-url-error": "Une erreur d'URL s'est produite, potentiellement causé par une erreur DNS...",
+    "tess-cannot-download": "Impossible de télécharger le language pack Tesseract.",
+    "tess-not-found": "Tesseract n'a pas été trouv. Veuillez l'installer, s'il est installé, passez le à travers le command line au moins une fois. Si cette erreur continue d'apparaître, veuillez créer une issue sur Github..",
+    "remove-black-bars": "Retirer les barres noires\nDevrait marcher avec le mode fenêtre",
+    "remove-black-bars-info": "L'application va essayer d'enlever les barres noires autour du vidéo.\n¨Ça devrait aussi fonctionner si vous enregistrer le jeu en mode fenêtre.\nC'est utile lorsque vous n'avez pas configuré la taille de la TV pour la switch.\nC'est 100% expérimental et un peu aléatoire.\nEnregistrez les vidéos comme il faut avant de vous plaindre :)",
+    "upd-ignore": "Ignorer",
+    "add-as-is": "Ajouter tel quel"
 }

data/versions.json

@@ -1,10 +1,10 @@
 {
-    "app": "1.6.2",
+    "app": "1.7",
     "skills": "3.1.1",
     "languages": {
-        "eng": "1.2",
+        "eng": "1.2.1",
         "jpn": 0,
-        "fra": "1.2",
+        "fra": "1.2.1",
         "ita": 0,
         "deu": 0,
         "spa": 0,

src/__main__.py

@@ -29,7 +29,6 @@
 )
 from .updater.updater_utils import (
     ask_main_update,
-    ask_language_update,
     ask_skill_update,
     ask_corrections_update,
 )
@@ -143,7 +142,7 @@ def run_in_console(args):
     charm_json = args.charm_json
     charm_encoded = args.charm_encoded
 
-    lang = get_language_code(args.language)
+    lang = get_game_language()
 
     os.makedirs(input_dir, exist_ok=True)
     os.makedirs(frame_dir, exist_ok=True)

src/frame_extraction.py

@@ -1,8 +1,11 @@
 import os
+import random
 import cv2
+
 from .utils import (
     apply_pre_crop_mask,
     get_frame_change_observation_section,
+    compare_pixel,
 )
 from tqdm import tqdm
 from math import floor
@@ -28,23 +31,111 @@ def resize_frame(frame):
     return frame
 
 
-def crop_frames(capture_device):
-    results = []
-    for i, f in read_frames(capture_device):
+def crop_frames(capture_device, remove_black_bars):
+    for i, f in read_frames(capture_device, remove_black_bars):
         yield i, crop_frame(f)
 
 
-def read_frames(capture_device):
+def _jitter_pos(pos: int, jitter_range: int = 10):
+    return pos + (random.randint(-jitter_range, jitter_range))
+
+
+# top, left, right, bottom
+def detect_black_bars(frame):
+    height, width = frame.shape[:2]
+
+    x_pos_base = floor(width / 2)
+    y_pos_base = floor(height / 2)
+    prev_pixel = None
+
+    i = 0
+    top = 5
+    left = 5
+    right = 5
+    bottom = 5
+    while True:
+        pixel = frame[top, _jitter_pos(x_pos_base)]
+        if prev_pixel is not None:
+            if compare_pixel(pixel, prev_pixel):
+                top += 1
+            else:
+                prev_pixel = None
+                break
+        else:
+            prev_pixel = pixel
+
+    while True:
+        pixel = frame[height - bottom - 1, _jitter_pos(x_pos_base)]
+        if prev_pixel is not None:
+            if compare_pixel(pixel, prev_pixel):
+                bottom += 1
+            else:
+                prev_pixel = None
+                break
+        else:
+            prev_pixel = pixel
+
+    while True:
+        pixel = frame[_jitter_pos(y_pos_base), left]
+        if prev_pixel is not None:
+            if compare_pixel(pixel, prev_pixel):
+                left += 1
+            else:
+                prev_pixel = None
+                break
+        else:
+            prev_pixel = pixel
+
+    while True:
+        pixel = frame[_jitter_pos(y_pos_base), width - right - 1]
+        if prev_pixel is not None:
+            if compare_pixel(pixel, prev_pixel):
+                right += 1
+            else:
+                prev_pixel = None
+                break
+        else:
+            prev_pixel = pixel
+
+    if top == 5:
+        top = 0
+    if left == 5:
+        left = 0
+    if right == 5:
+        right = 0
+    if bottom == 5:
+        bottom = 0
+
+    return (top, left, right, bottom)
+
+
+def crop_black_bars(frame, black_bars):
+    (top, left, right, bottom) = black_bars
+    height, width = frame.shape[:2]
+
+    y = top
+    y2 = height - bottom
+    x = left
+    x2 = width - right
+    return frame[y:y2, x:x2]
+
+
+def read_frames(capture_device, remove_black_bars=False):
     i = 0
     fps = capture_device.get(cv2.CAP_PROP_FPS)
+    black_bars = None
 
     while True:
         ret, frame = capture_device.read()
+        if i == 0:
+            black_bars = detect_black_bars(frame)
         if not ret:
             break
         if fps == 60 and (i % 2):
             pass
         else:
+            if remove_black_bars:
+                frame = crop_black_bars(frame, black_bars)
             yield i, resize_frame(frame)
         i += 1
 
@@ -68,6 +159,7 @@ def is_validated_video_format(video_name):
 def extract_unique_frames(
     input_dir,
     frame_dir,
+    remove_black_bars=False,
     _=lambda x: x,
     iter_wrapper=None,
     frame_callback=lambda x: None,
@@ -102,7 +194,7 @@ def extract_unique_frames(
 
         previous_charm_marker = None
         with iter_wrapper(
-            crop_frames(cap),
+            crop_frames(cap, remove_black_bars),
             total=frame_count,
             desc=_("fn-total-charm").format(f_name, frame_count),
         ) as frame_pbar: