Japanese Learning Environment

Built using Laravel for Server API and React for the client side generated with create-react-app.

Japanese data comes from Electronic Dictionary Research and Development Group, and are used in conformance with the Group's licence. This site uses the JMdict, Kanjidic2, JMnedict, and Radkfile dictionary files. JLPT data comes from Jonathan Waller's JLPT Resources page.

Project features

• Laravel CRUD REST API endpoints for Articles, Lists, Roles, Users and Posts(forum).
• Likes, hashtags and comments relationships for Articles, Lists and Posts.
• Read, Search and save lists of Kanji(hierogliphs), Radicals(hierogpliphs formed parts of symbols), Words and sentences from Tatoeba community.
• Japanese language data formatted by reading from XML and extracting to CSV using plain PHP and MySQL scripts. Exported and being imported to Laravel via Laravel migrations CLI. Kanjis and words were matched against Jonathan Waller's JLPT Resources and assigned JLPT level accordingly.
• MySQL with Laravel's Eloquent ORM.
• Authentication with Laravel/passport.
• Text scanning algorithm to find kanjis and words used in user provided article.
• PDFs generation and downloading material with english meanings for Kanjis and Words based on saved Lists or Article content.
• Frontend data access using redux.
• Custom CSS stylings and boostrap with responsive templates.

Backend

• laravel-snappy - for PDF generating. PDFs generated using laravel's blade templates structure. wkhtmltopdf wkhtmltopdf is required in order for the laravel-snappy library to work.
• passport
• laravel helpers - more comfortable customized version of the original laravel helpers
• laravel/ui - scaffolding for the frontend of laravel API landing page. In this project React was used.

Frontend

• react-moment of moment for date formatting.
• axios for HTTP requesting.
• react-bootstrap for components bootstrap for overall styling.
• react-router-bootstrap for react-router and react-bootstrap integration.
• redux, redux-thunk and react-redux for data access.

Setup Docker

In /processor-api repository root run:

chmod +x .docker/entrypoint.sh

Run docker containers in detached mode:

docker-compose up -d --build

Test if Mysql initialized properly, by entering container via bash or docker desktop:

docker-compose exec db bash
mysql -u <DB_USERNAME> -p
prompted password: <DB_PASSWORD>

List tables, there should be all the common ones:

SHOW databases;
USE <DB_NAME>;
SHOW TABLES;

If tables are there, proceed with Japanese data migration. Data volume is big, so it may take over a minute and be cautions when rebuilding containers to avoid duplication.

Enter container via bash or docker desktop into laravel app:

# Enter container
docker-compose exec laravel-app bash
# Create japanese material tables
php artisan migrate --path=database/migrations/japanese-data

# Create new tables, generate encryption keys for Passport
php artisan passport:install

Necessary to fill-up the categories for the entities objects in "objecttemplates" table. Creates common and admin users Example custom lists and articles

php artisan db:seed
# seed single table
php aritsan db:seed --class=<ClassNameSeeder>

If by chance something seems cached or configs not updated try reset:

composer dump-autoload
php artisan config:clear
php artisan cache:clear

Take containers down and empty volumes for clean containers rebuild

docker-compose down -v
docker-compose up -d --build

Remember after clean rebuild to run migrations and seed once again.

Test API

Basic requests for testing purposes. Enter laravel api container and register/login:

curl -X POST http://nginx_webserver/api/login \
  -H "Content-Type: application/json" \
  -d '{"email": "[email protected]", "password": "test123"}'

# or save as a variable:
TOKEN=$(curl -s -X POST http://nginx_webserver/api/login \
  -H "Content-Type: application/json" \
  -d '{"email": "[email protected]", "password": "test123"}' \
  | jq -r '.accessToken')

Use variable:

curl -X GET http://nginx_webserver/api/articles \
  -H "Authorization: Bearer $TOKEN" \
  | jq

# or single article:
curl -X GET http://nginx_webserver/api/article/5 \
  -H "Authorization: Bearer $TOKEN" \
  | jq

Test PDF file on a local machine:

curl -X GET http://nginx_webserver/api/article/5/kanjis-pdf \
  -H "Authorization: Bearer $TOKEN" \
  -o kanjis-test.pdf
docker cp laravel_app:/var/www/html/kanjis-test.pdf C:\Users\USER-NAME\Downloads\

Setup Local

Required

• Fonts supporting Japanese language, for example: fonts-ipafont-gothic.
• Composer - PHP package manager
• Node. - Node js runtime for frontend assets of laravel.
• For PDF generating laravel-snappy to work requires wkhtmltopdf. Install it as a composer dependency or manually here into your machine.

Also, possibly paths for wkhtmltopdf will not work, uncomment variables in .env to use needed ones or modify them according needs.

Optional choices for faster setup:

• Xampp - web server Apache, PHP and MariaDB(MySQL).
• Laravel Herd - dev environment with all you need for laravel development.

Laravel API

In processor-api directory Install composer packages:

composer install

Change .env file settings for your database.

cp .env.example .env

Generate unique app key

php artisan key:generate

Database MySQL -

Requirements:

• Make sure your DB Collation is utf8mb4_general_ci for Japanese characters support.
• Must assure that MySQL config in my.ini has packets size adjusted max_allowed_packet=128M in order for migrations to enable importing bigger sized japanese files into the database.

If mysql config is correct, proceed steps below

# Create common tables
php artisan migrate --path=database/migrations/now

# Create japanese material tables
php artisan migrate --path=database/migrations/japanese-data

Add Passport clients

# Create new tables, generate encryption keys for Passport
php artisan passport:install

Seed initial data

# Necessary to fill-up the categories for the resources objects in "objecttemplates" table.
# Creates common and admin users
# Example custom lists
# Example articles
php artisan db:seed

API Documentation page is served on laravel side, to have it work install packages.

npm install
# to watch changes
npm run watch

React App

In client directory Install node modules:

npm install

Start react app

npm start

To Do List

• Make demo gifs to showcase main features.
• Add swagger for Laravel API, add models for swagger to use swagger UI for quick API usage
• Refactor single component to functional component using latest react hooks in a composition way to have example component.
• Strengthen authorization with more persistant implementation..
• Implement react-query for query-based approach of managing server-data facing for frontend and fixing issues like cache, refetch, cancel requests after unmount.
• Create nhk easy news scrapper to get each days news.
• re-vamp styling, especially for small UI elements like links, buttons with icons, screen spacings.
• Design a way to offload scanning algorithm from client to server.
• Create service worker to build queues for scanning algorithm of user texts to find kanjis and words.
• Create Github Actions for frontend.
• Create Github Actions for backend.
• Write E2E tests for frontend.
• [?] Client side PDF customization. Generate on Backend, customize on frontend.
• [?] Cache or Store pre-generated materials like PDFs.