This is the core of recognition component for Penguin Statistics, including:
- screenshot recognition
- depot recognition (in development)
This recognizer is developed by C++ and providing front-end recognition service by WebAssembly technology. It can be also used in other scenarios to get the statistics of Arknights.
Head to releases to get the latest release of the WASM.
The WASMs here in releases are built automatically by GitHub Actions via Docker. Inspect them as you like at Dockerfile, GitHub Actions and GitHub Actions History.
To simplify the process of building WASM for the recognizer, we provide a builder Docker image for the one to build the WASM.
The Docker image is stored at GitHub Package recognizer-builder. To build your own WASM, you'd need to use a Dockerfile
to use the pre-built builder image as the base image, and build your WASM upon that. You can find the Dockerfile under this repository. After changing anything you'd want, you can build your own WASM by running the following command:
$ docker build .
This will build the WASM using the Dockerfile
(instead of using Dockerfile.builder
which is just used to build the builder). To extract the WASM and its corresponding JavaScript loader, you can run the following command:
$ docker cp <container-id>:/build/recognizer/build/dist/penguin-recognizer* ./build/penguin-recognizer*
Follow the following steps to build penguin-recognizer
manually, in which requires you to install and configure all build dependencies yourself.
- OS: Unix-like (Tested:
Ubuntu 20.04
) - Python, Cmake
Clone this repository and be ready to build!
$ git clone https://github.com/penguin-statistics/recognizer.git penguin-recognizer
$ cd penguin-recognizer
⚠ Note on
emsdk
versionCurrently,
penguin-recognizer
is built byEmscripten 1.39.0
, which is verified for supporting some old version Chromium browsers. According to the limitation ofOpenCV
, the latest supported version isEmscripten 2.0.10
, but it might not support some old version browsers.
# Get the emsdk repo
$ git clone https://github.com/emscripten-core/emsdk.git
# Enter that directory
$ cd emsdk
# Download and install the SDK tools.
$ ./emsdk install 1.39.0
# Make the "latest" SDK "active" for the current user. (writes .emscripten file)
$ ./emsdk activate 1.39.0
# Activate PATH and other environment variables in the current terminal
$ source ./emsdk_env.sh
# Check installation
$ emcc -v
ℹ️ Note on
OpenCV
versionCurrently,
penguin-recognizer
is built byOpenCV 4.5.1
.
Other verified versions are:
OpenCV 4.5.4
Tested by FlandiaYingman
OpenCV 4.5.5
(latest) Tested by KumoSiunaus
$ git clone --depth=1 -b 4.5.5 https://github.com/opencv/opencv.git opencv/sources
Before building OpenCV, some build settings have to be customized.
Open opencv/sources/platforms/js/build_js.py
by an editor, and change the following settings.
# In def get_cmake_cmd(self)
# Turn these settings to ON, which are necessary for building
-DWITH_JPEG=OFF -> ON
-DWITH_PNG=OFF -> ON
-DBUILD_opencv_imgcodecs=OFF -> ON
# (Optional) Turn these settings to OFF to accelerate building
-DWITH_QUIRC=ON -> OFF
-DBUILD_ZLIB=ON -> OFF
-DBUILD_opencv_calib3d=ON -> OFF
-DBUILD_opencv_dnn=ON -> OFF
-DBUILD_opencv_features2d=ON -> OFF
-DBUILD_opencv_flann=ON -> OFF
-DBUILD_opencv_photo=ON -> OFF
-DBUILD_EXAMPLES=ON -> OFF
-DBUILD_TESTS=ON -> OFF
-DBUILD_PERF_TESTS=ON -> OFF
# change directory to opencv
$ cd .../opencv
$ emcmake python ./sources/platforms/js/build_js.py build_wasm --build_wasm
$ cd build_wasm
$ emmake make
Now, the following files should exist
opencv/build_wasm/lib/libopencv_imgcodecs.a
opencv/build_wasm/3rdparty/lib/liblibjpeg-turbo.a
opencv/build_wasm/3rdparty/lib/liblibpng.a
If your
opencv
directory is at the same level as thepenguin-recognizer
directory, like so:. ├── opencv └── penguin-recognizer
then you can skip this step as it is unnecessary.
Open CMakeLists.txt
, modify the value of OPENCV_DIR
to the path to your opencv
directory.
set(OPENCV_DIR "/path/to/opencv")
# change directory to penguin-recognizer
$ cd .../penguin-recognizer
$ mkdir build && cd build
$ emcmake cmake ..
# Should print the following output if success:
# -- Configuring done
# -- Generating done
# -- Build files have been written to: .../penguin-recognizer/build
$ emmake make
# Should print the following output if success:
# Scanning dependencies of target penguin-recognizer
# [ 50%] Building CXX object CMakeFiles/penguin-recognizer.dir/src/recognizer_wasm.cpp.o
# [100%] Linking CXX executable penguin-recognizer.js
# [100%] Built target penguin-recognizer
Now, those files should exist:
penguin-recognizer/build/penguin-recognizer.js
penguin-recognizer/build/penguin-recognizer.wasm
In some situations, get wrong result of stage recognition. (e.g. WR-10 -> WR-1O)(Resolved in v4)In some situations, get wrong result of drop type recognition. (e.g. LMB(Resolved in v4)yellow
-> EXTRA_DROPgreen
)- In some situations, get wrong result of drop number recognition.
- add automatic fallback to fix
Known issues 1.
- add droptype order check to fix
Known issues 2.
- update drop number recognition strategy to fix
Known issues 3.
- depot recognition (in development)
- new wasm interface using emscripten::bind
- adapt to new interface (for now, CN server)
This project is made possible by the following contributors. Contributions are always welcome!