Merge pull request #2046 from pupil-labs/develop

Pupil v2.6 Release Candidate 1

Author: papr

.travis.yml

@@ -22,13 +22,11 @@ jobs:
 
     - name: black formatting check
       language: python
+      before_install: pip install -U pip
       # Travis automatically runs `pip install -r requirements.txt` if such a file is present.
       # Source: https://docs.travis-ci.com/user/languages/python/#dependency-management
-      # Since we do not need the requirements to be installed, we overwrite 
-      install: ~
-      before_script:
-        - pip install -U pip
-        - pip install black
+      # Since we only need black to be installed, we overwrite with:
+      install: pip install black
       script:
         - >
           black . --check --exclude pupil_src/tests || (

docs/dependencies-macos.md

@@ -62,32 +62,12 @@ make && make install
 
 ### Install Python Libraries
 
-We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil.
+We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil. To install all Python dependencies, you can use the [`requirements.txt`](https://github.com/pupil-labs/pupil/blob/master/requirements.txt) file from the root of the `pupil` repository.
 
 ```sh
 # Upgrade pip to latest version. This is necessary for some dependencies.
-python -m pip install --upgrade pip
-
-pip install cysignals
-pip install cython
-pip install msgpack==0.5.6
-pip install numexpr
-pip install packaging
-pip install psutil
-pip install pyaudio
-pip install pyopengl
-pip install pyzmq
-pip install scikit-learn
-pip install scipy
-pip install glfw
-pip install git+https://github.com/zeromq/pyre
-
-pip install pupil-apriltags
-pip install pupil-detectors
-pip install git+https://github.com/pupil-labs/PyAV
-pip install git+https://github.com/pupil-labs/pyuvc
-pip install git+https://github.com/pupil-labs/pyndsi
-pip install git+https://github.com/pupil-labs/pyglui
+python -m pip install --upgrade pip wheel
+pip install -r requirements.txt
 ```
 
 **NOTE:** Installing **pyglui** might fail on newer versions of **macOS** due to missing OpenGL headers. In this case, you need to install Xcode which comes with the required header files.

docs/dependencies-ubuntu17.md

@@ -152,32 +152,12 @@ sudo ldconfig
 
 ### Install Python Libraries
 
-We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil.
+We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil. To install all Python dependencies, you can use the [`requirements.txt`](https://github.com/pupil-labs/pupil/blob/master/requirements.txt) file from the root of the `pupil` repository.
 
 ```sh
 # Upgrade pip to latest version. This is necessary for some dependencies.
-python -m pip install --upgrade pip
-
-pip install cysignals
-pip install cython
-pip install msgpack==0.5.6
-pip install numexpr
-pip install packaging
-pip install psutil
-pip install pyaudio
-pip install pyopengl
-pip install pyzmq
-pip install scikit-learn
-pip install scipy
-pip install glfw
-pip install git+https://github.com/zeromq/pyre
-
-pip install pupil-apriltags
-pip install pupil-detectors
-pip install git+https://github.com/pupil-labs/PyAV
-pip install git+https://github.com/pupil-labs/pyuvc
-pip install git+https://github.com/pupil-labs/pyndsi
-pip install git+https://github.com/pupil-labs/pyglui
+python -m pip install --upgrade pip wheel
+pip install -r requirements.txt
 ```
 
 **NOTE**: If you get the error `ImportError: No module named 'cv2'` when trying to run Pupil, please refer to the section [OpenCV Troubleshooting](#opencv-troubleshooting) above.

docs/dependencies-ubuntu18.md

@@ -63,32 +63,12 @@ sudo udevadm trigger
 
 ### Install Python Libraries
 
-We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil.
+We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil. To install all Python dependencies, you can use the [`requirements.txt`](https://github.com/pupil-labs/pupil/blob/master/requirements.txt) file from the root of the `pupil` repository.
 
 ```sh
 # Upgrade pip to latest version. This is necessary for some dependencies.
-python -m pip install --upgrade pip
-
-pip install cysignals
-pip install cython
-pip install msgpack==0.5.6
-pip install numexpr
-pip install packaging
-pip install psutil
-pip install pyaudio
-pip install pyopengl
-pip install pyzmq
-pip install scikit-learn
-pip install scipy
-pip install glfw
-pip install git+https://github.com/zeromq/pyre
-
-pip install pupil-apriltags
-pip install pupil-detectors
-pip install git+https://github.com/pupil-labs/PyAV
-pip install git+https://github.com/pupil-labs/pyuvc
-pip install git+https://github.com/pupil-labs/pyndsi
-pip install git+https://github.com/pupil-labs/pyglui
+python -m pip install --upgrade pip wheel
+pip install -r requirements.txt
 ```
 
 ### OpenCV Troubleshooting

docs/dependencies-windows.md

@@ -48,51 +48,17 @@ If you downloaded the linked installer:
 - Check the box `Add Python to PATH`. This will add Python to your System PATH Environment Variable.
 - Check the box `Install for all users`. **Note:** By default this will install Python to `C:\Program Files\Python36`. Some build scripts may fail to start Python due to spaces in the path name. So, you may want to consider installing Python to `C:\Python36` instead.
 
-
 ## Install Python Libraries
 
-We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil.
+We recommend using a [virtual environment](https://docs.python.org/3/tutorial/venv.html) for running Pupil. To install all Python dependencies, you can use the [`requirements.txt`](https://github.com/pupil-labs/pupil/blob/master/requirements.txt) file from the root of the `pupil` repository.
 
 ```sh
 # Upgrade pip to latest version. This is necessary for some dependencies.
-python -m pip install --upgrade pip
-
-pip install cython
-pip install msgpack==0.5.6
-pip install numexpr
-pip install opencv-python==3.*
-pip install packaging
-pip install psutil
-pip install pyaudio
-pip install pyopengl
-pip install pyzmq
-pip install scikit-learn
-pip install scipy
-pip install glfw
-pip install win_inet_pton
-pip install git+https://github.com/zeromq/pyre
-
-pip install pupil-apriltags
-pip install pupil-detectors
+python -m pip install --upgrade pip wheel
+pip install -r requirements.txt
 ```
 
-## Pupil Labs Python Wheels
-
-In addition to these libraries, you will need to install some Pupil-Labs support libraries. Since building them for Windows is also not automated yet, we provide some prebuilt wheels that you can use. If you want to build the support libraries yourself as well, you will have to look for install instructions on the respective GitHub repositories.
-
-Download the following Python wheels from Pupil Labs github repos:
-
-- [pyglui](https://github.com/pupil-labs/pyglui/releases/latest)
-- [pyav](https://github.com/pupil-labs/pyav/releases/latest)
-- [pyndsi](https://github.com/pupil-labs/pyndsi/releases/latest)
-- [pyuvc](https://github.com/pupil-labs/pyuvc/releases/latest)
-
-`pyuvc` requires that you download Microsoft Visual C++ 2010 Redistributable from [microsoft](https://www.microsoft.com/en-us/download/details.aspx?id=14632). The `pthreadVC2` lib, which is used by libuvc, depends on `msvcr100.dll`.
-
-Open your command prompt and `Run as administrator` in the directory where the wheels are downloaded.
-
-- Install all wheels with `pip install X` (where X is the name of the `.whl` file)
-- You can check that libs are installed with `python import X` statements in the command prompt where `X` is the name of the lib.
+**NOTE:** `pyuvc` requires that you download Microsoft Visual C++ 2010 Redistributable from [microsoft](https://www.microsoft.com/en-us/download/details.aspx?id=14632). The `pthreadVC2` lib, which is used by libuvc, depends on `msvcr100.dll`.
 
 ## Modifying Pupil to Work with Windows
 

pupil_src/launchables/eye.py

@@ -204,8 +204,30 @@ def get_timestamp():
         g_pool.get_timestamp = get_timestamp
         g_pool.get_now = get_time_monotonic
 
+        def load_runtime_pupil_detection_plugins():
+            from plugin import import_runtime_plugins
+            from pupil_detector_plugins.detector_base_plugin import PupilDetectorPlugin
+
+            plugins_path = os.path.join(g_pool.user_dir, "plugins")
+
+            for plugin in import_runtime_plugins(plugins_path):
+                if not isinstance(plugin, type):
+                    continue
+                if not issubclass(plugin, PupilDetectorPlugin):
+                    continue
+                if plugin is PupilDetectorPlugin:
+                    continue
+                yield plugin
+
         default_2d, default_3d, available_detectors = available_detector_plugins()
-        plugins = manager_classes + source_classes + available_detectors + [Roi]
+        runtime_detectors = list(load_runtime_pupil_detection_plugins())
+        plugins = (
+            manager_classes
+            + source_classes
+            + available_detectors
+            + runtime_detectors
+            + [Roi]
+        )
         g_pool.plugin_by_name = {p.__name__: p for p in plugins}
 
         preferred_names = [
@@ -513,6 +535,14 @@ def set_window_size():
             # with incorrect settings that were loaded from session settings.
             plugins_to_load.append(overwrite_cap_settings)
 
+        # Add runtime plugins to the list of plugins to load with default arguments,
+        # if not already restored from session settings
+        plugins_to_load_names = set(name for name, _ in plugins_to_load)
+        for runtime_detector in runtime_detectors:
+            runtime_name = runtime_detector.__name__
+            if runtime_name not in plugins_to_load_names:
+                plugins_to_load.append((runtime_name, {}))
+
         g_pool.plugins = Plugin_List(g_pool, plugins_to_load)
 
         if not g_pool.capture:
@@ -590,7 +620,8 @@ def window_should_update():
             glfw.swap_interval(0)
 
         # Event loop
-        while not glfw.window_should_close(main_window):
+        window_should_close = False
+        while not window_should_close:
 
             if notify_sub.new_data:
                 t, notification = notify_sub.recv()
@@ -657,6 +688,17 @@ def window_should_update():
                         )
                     except KeyError as err:
                         logger.error(f"Attempt to load unknown plugin: {err}")
+                elif (
+                    subject.startswith("stop_eye_plugin")
+                    and notification["target"] == g_pool.process
+                ):
+                    try:
+                        plugin_to_stop = g_pool.plugin_by_name[notification["name"]]
+                    except KeyError as err:
+                        logger.error(f"Attempt to load unknown plugin: {err}")
+                    else:
+                        plugin_to_stop.alive = False
+                        g_pool.plugins.clean()
 
                 for plugin in g_pool.plugins:
                     plugin.on_notify(notification)
@@ -724,13 +766,13 @@ def window_should_update():
                 for result in event.get(EVENT_KEY, ()):
                     pupil_socket.send(result)
 
-            cpu_graph.update()
-
             # GL drawing
             if window_should_update():
+                cpu_graph.update()
                 if is_window_visible(main_window):
                     consume_events_and_render_buffer()
                 glfw.poll_events()
+                window_should_close = glfw.window_should_close(main_window)
 
         # END while running
 

pupil_src/launchables/player.py

@@ -117,6 +117,7 @@ def player(
         from gaze_producer.gaze_from_offline_calibration import (
             GazeFromOfflineCalibration,
         )
+        from pupil_detector_plugins.detector_base_plugin import PupilDetectorPlugin
         from system_graphs import System_Graphs
         from system_timelines import System_Timelines
         from blink_detection import Offline_Blink_Detection
@@ -152,6 +153,9 @@ def interrupt_handler(sig, frame):
         signal.signal(signal.SIGINT, interrupt_handler)
 
         runtime_plugins = import_runtime_plugins(os.path.join(user_dir, "plugins"))
+        runtime_plugins = [
+            p for p in runtime_plugins if not issubclass(p, PupilDetectorPlugin)
+        ]
         system_plugins = [
             Log_Display,
             Seek_Control,

pupil_src/launchables/world.py

@@ -161,6 +161,7 @@ def detection_enabled_setter(is_on: bool):
 
         from gaze_mapping import registered_gazer_classes
         from gaze_mapping.gazer_base import GazerBase
+        from pupil_detector_plugins.detector_base_plugin import PupilDetectorPlugin
         from fixation_detector import Fixation_Detector
         from recorder import Recorder
         from display_recent_gaze import Display_Recent_Gaze
@@ -241,6 +242,9 @@ def get_timestamp():
         runtime_plugins = import_runtime_plugins(
             os.path.join(g_pool.user_dir, "plugins")
         )
+        runtime_plugins = [
+            p for p in runtime_plugins if not issubclass(p, PupilDetectorPlugin)
+        ]
         user_plugins = [
             Pupil_Groups,
             NetworkApiPlugin,

pupil_src/shared_modules/circle_detector.py

@@ -702,7 +702,7 @@ def bench():
             sts, img = cap.read()
             # img = cv2.imread('/Users/mkassner/Desktop/manual_calibration_marker-01.png')
             gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
-            print(len(find_concetric_circles(gray, visual_debug=img)))
+            print(len(find_concentric_circles(gray, visual_debug=img)))
             cv2.imshow("img", img)
             cv2.waitKey(1)
             # return

pupil_src/shared_modules/gaze_mapping/gazer_3d/gazer_headset.py

@@ -157,6 +157,10 @@ def _predict_single(self, x):
         gaze_3d = self._toWorld(gaze_point)
         normal_3d = np.dot(self.rotation_matrix, pupil_normal)
 
+        # Check if gaze is in front of camera. If it is not, flip direction.
+        if gaze_3d[-1] < 0:
+            gaze_3d *= -1.0
+
         g = {
             "eye_center_3d": eye_center.tolist(),
             "gaze_normal_3d": normal_3d.tolist(),
@@ -284,6 +288,10 @@ def _predict_single(self, x):
         if nearest_intersection_point is None:
             return None
 
+        # Check if gaze is in front of camera. If it is not, flip direction.
+        if nearest_intersection_point[-1] < 0:
+            nearest_intersection_point *= -1.0
+
         g = {
             "eye_centers_3d": {0: s0_center.tolist(), 1: s1_center.tolist()},
             "gaze_normals_3d": {0: s0_normal.tolist(), 1: s1_normal.tolist()},

pupil_src/shared_modules/pupil_detector_plugins/pye3d_plugin.py

@@ -10,6 +10,7 @@
 """
 import logging
 
+import pye3d
 from pye3d.detector_3d import Detector3D, CameraModel
 from pyglui import ui
 
@@ -19,6 +20,16 @@
 
 logger = logging.getLogger(__name__)
 
+version_installed = getattr(pye3d, "__version__", "0.0.1")
+version_supported = "0.0.1"
+
+if version_installed != version_supported:
+    logger.info(
+        f"Requires pye3d version {version_supported} "
+        f"(Installed: {version_installed})"
+    )
+    raise ImportError
+
 
 class Pye3DPlugin(PupilDetectorPlugin):
     uniqueness = "by_class"

pupil_src/shared_modules/pupil_detector_plugins/visualizer_pye3d.py pupil_src/shared_modules/pupil_detector_plugins/visualizer_pye3d/__init__.py

@@ -37,6 +37,8 @@
 from pyglui.cygl.utils import RGBA
 from visualizer import Visualizer
 
+from .eye import LeGrandEye
+
 
 class Eye_Visualizer(Visualizer):
     def __init__(self, g_pool, focal_length):