Skip to content
This repository has been archived by the owner on Aug 8, 2024. It is now read-only.

Bryan-Jenks-Home-Lab/watchr

Repository files navigation

Watchr

Pull Requests

Watchr Is a file watching service for importing records into a Postgres database.

Watchr is organized through the usage of Pydantic models to govern the behavior of how a file get processed, the actions around its artifact cleanup, and other configuration.

This project is still rough and narrowly focused for my own personal use cases at the moment but in the future i may put forth more intention in making this more general and accessible to others.


Table of Contents



Features


Return To Top

<++>


Requirements


Return To Top

<++>


Recommended


Return To Top

<++>


Installation


Return To Top

<++>


Users


Return To Top

<++>

See Usage


Contributors


Return To Top

See CONTRIBUTING


Usage


Return To Top

<++>


Example


Return To Top

Pull image from ghcr.io

docker pull ghcr.io/bryan-jenks-home-lab/etl_strong_app:main

Run image

docker run -d \
    -e DB_CONNECTION_STRING='postgresql+psycopg2://<USER>:<PASSWORD>@<SERVER>:<PORT>/<DATABASE>' \
    -e EXPECTED_FILE=strong.csv \
    -e TARGET_TABLE=<DATABASE>.<SCHEMA>.<TABLE_NAME> \
    -e WATCH_PATH=/data/inbound \
    -e STAGING_PATH=/data/outbound \
    -e PROCESSED_PATH=/data/processed \
    -v /<PATH_ON_YOUR_MACHINE>/strong_app_import/inbound:/data/inbound \
    -v /<PATH_ON_YOUR_MACHINE>/strong_app_import/outbound:/data/outbound \
    -v /<PATH_ON_YOUR_MACHINE>/strong_app_import/processed:/data/processed \
    ghcr.io/bryan-jenks-home-lab/etl_strong_app:main

Documentation


Return To Top

<++>


Resources


Return To Top

<++>


Development


Return To Top

  1. python -m pip install poetry python-dotenv pre-commit
  2. pre-commit install
  3. pre-commit autoupdate

See Road Map


Security


Return To Top

See SECURITY


Future

Return To Top

See ROADMAP


History

Return To Top

See RELEASES


Community


Return To Top

<++>

See CODE OF CONDUCT


Credits


Return To Top

See AUTHORS


License


Return To Top

See LICENSE