Skip to content

Commit

Permalink
chore: update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
iPromKnight committed Jan 24, 2025
1 parent f42c2fa commit 10828ac
Show file tree
Hide file tree
Showing 4 changed files with 34 additions and 22 deletions.
3 changes: 2 additions & 1 deletion docs/Writerside/snippets/compose-file.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,10 @@ services:
condition: service_healthy

postgres:
image: postgres:17.1
image: postgres:17.2-alpine
container_name: postgres
restart: unless-stopped
shm_size: 2G
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_USER: postgres
Expand Down
6 changes: 4 additions & 2 deletions docs/Writerside/snippets/default-settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@
"Imdb": {
"EnableImportMatching": true,
"EnableEndpoint": true,
"MinimumScoreMatch": 0.85
"MinimumScoreMatch": 0.85,
"UseAllCores": false,
"NumberOfCores": 2
},
"Ingestion": {
"ZurgInstances": [],
Expand All @@ -47,4 +49,4 @@
"BatchSize": 5000
}
}
}
}
8 changes: 8 additions & 0 deletions docs/Writerside/topics/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,14 @@ _Default: `true`_
_The minimum score match for IMDB search results. Closer to 1 means a more exact match has to occur. A value between 1 and 0._
_Default: `0.85`_

**Imdb.UseAllCores**
_Should title matching use all available processor cores on the machine?_
_Default: `false`_

**Imdb.NumberOfCores**
_The number of processor cores to use for parallel operations during matching_
_Default: `2`_

### Ingestion Configuration
**Ingestion.ZurgInstances**
_A list of Zurg instances to scrape from._
Expand Down
39 changes: 20 additions & 19 deletions docs/Writerside/topics/Scraper.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ To run the scraper manually, you can use the following command:
docker compose stop zilean

docker run -it --rm \
-v ./settings.json:/app/data/settings.json \
-v ./settings.json:/app/data/settings.json \
--name zileanscraper \
ipromknight/zilean:latest \
/app/scraper <args>
Expand All @@ -28,41 +28,42 @@ This will run the scraper, and output the results to the console.

The following arguments are available when running the scraper:

| Argument | Description |
|---------------------|-------------------------------------------------------------------------------------------------------|
| `dmm-sync` | Run the DMM scraper to import DMM Hash Lists. |
| `generic-sync` | Run the Ingestion syncer to sync from Zilean and Zurg instances in your config. |
| `resync-imdb -s` | Force re-ingest IMDB metadata. |
| `resync-imdb -s -t` | Force re-ingest IMDB metadata, then try to update any entries in your database that do not have an id |
| Argument | Description |
|---------------------|------------------------------------------------------------------------------------------------------------|
| `dmm-sync` | Run the DMM scraper to import DMM Hash Lists. |
| `generic-sync` | Run the Ingestion syncer to sync from Zilean and Zurg instances in your config. |
| `resync-imdb -s` | Force re-ingest IMDB metadata. |
| `resync-imdb -s -t` | Force re-ingest IMDB metadata, then try to update any entries in your database that do not have an imdb id |
| `resync-imdb -t` | Try to update any entries in your database that do not have an imdb id |
| `resync-imdb -s -a` | Force re-ingest IMDB metadata, then Rematch / update any entries in your database with a new ImdbID |
| `resync-imdb -a` | Rematch / update any entries in your database with a new ImdbID |

## Examples

### Running the IMDB Resync with Title Update

```bash
docker run -it --rm \
-v ./settings.json:/app/data/settings.json \
--name zileanscraper \
ipromknight/zilean:latest \
docker exec -it zilean \
/app/scraper resync-imdb -s -t
```

### Running the IMDB Resync to rematch ALL Titles

```bash
docker exec -it zilean \
/app/scraper resync-imdb -a
```

### Running the DMM Sync

```bash
docker run -it --rm \
-v ./settings.json:/app/data/settings.json \
--name zileanscraper \
ipromknight/zilean:latest \
docker exec -it zilean \
/app/scraper dmm-sync
```

### Running the Generic Sync

```bash
docker run -it --rm \
-v ./settings.json:/app/data/settings.json \
--name zileanscraper \
ipromknight/zilean:latest \
docker exec -it zilean \
/app/scraper generic-sync
```

0 comments on commit 10828ac

Please sign in to comment.