Interval rsync

Source: https://github.com/txn2/irsync

Command line utility and Docker container for running rsync on an interval. rsync, synchronizes (one way) files between two sources, irsync wrapps rsync and runs it on a specified interval.

As a basic command-line utility, the only requirement is rsync. As a Docker container the conly requirement is Docker. The docker container size with all the requirements is only around 5 megs.

Background

I wrote up a little background on the motivation behind irsync on my imti.co blog, rSync Files on Interval.

Demo

Setup a quick demo using Dockers docker-compose command. Included with this project is a docker-compose.yaml with a simple client/server setup. In this composer configuration irsync is set to check the server every 30 seconds (after sync is complete. The server mounts the ./data/source directory and the client mounts the ./data/dest directory. Drop files in ./data/source and see them appear in ./data/dest.

Watch a 2 minute youtube video demoing the commands below

Setup and run demo (requires Docker):

# create a source and dest directories (mounted from the docker-compose)
mkdir -p ./data/source
mkdir -p ./data/dest

# make a couple of sample files
touch ./data/source/test1.txt
touch ./data/source/test2.txt

# get the docker-compose.yml
curl https://raw.githubusercontent.com/txn2/irsync/master/docker-compose.yml >docker-compose.yml

# run docker-compose in the background (-d flag)
docker-compose up -d

# view logs
docker-compose logs -f

# drop some more files in the ./data/source directory
# irsync is configured to check every 30 seconds in this demo.

#### Cleanup

# stop containers
docker-compose stop

# remove containers
docker-compose rm

Run on MacOS

Install

brew tap txn2/homebrew-tap
brew install irsync

Update (when new versions are released)

brew upgrade irsync

Run Prints the rsync version every 5 seconds.

irsync --irsync-interval-seconds=5 --version 

Adjust the interval to your requirement and add rsync flags as you normally would.

Run From Source

Example #1 Local with interval defaults

go run ./irsync.go -pvrt --exclude='custom' --exclude='special.txt' --delete ./data/source/ ./data/dest/

Run From Container

Example #1 Local

docker run --rm -v "$(pwd)"/data:/data txn2/irsync \
    -pvrt --delete /data/source/ /data/dest/

Example #2 Local Every 10 Seconds

docker run --rm -v "$(pwd)"/data:/data txn2/irsync \
    --irsync-interval-seconds=10 \
    -pvrt --delete /data/source/ /data/dest/

Example #3 Server to Local - Restart on Failure

docker run -d --name irsync-demo --restart on-failure \
    -v "$(pwd)"/data:/data txn2/irsync \
    --irsync-interval-seconds=10 \
    -pvrt --delete /data/source/ /data/dest/

Example#4: Custom container

Create Dockerfile:

FROM txn2/irsync:2.0.0
LABEL vendor="mk.imti.co"
LABEL co.imti.mk.source="https://github.com/txn2/irsync"

# if the rsync server requires a password
ENV RSYNC_PASSWORD=password

# exampe: keep local synchronized with server
# interval default: --irsync-interval-seconds=120
# activity timout default: --irsync-timeout-seconds=7200
CMD ["-pvrt", "--modify-window=30", "--delete", "--exclude='fun'", "rsync://[email protected]:873/data/", "/media"]

Build:

docker build -t custom-sync .

Run:

docker run -d --name custom-sync --restart on-failure \
    -v "$(pwd)"/data:/data custom-sync

Logs:

docker logs -f custom-sync

Environment Configuration

• RSYNC_PASSWORD=password Use environment variable RSYNC_PASSWORD to keep from being prompted for a password for servers requiring authentication.

Development

Building and Releasing

Interval rsync uses GORELEASER to build binaries and Docker containers.

Test Release Steps

Install GORELEASER with brew (MacOS):

brew install goreleaser/tap/goreleaser

Build without releasing:

goreleaser --skip-publish --rm-dist --skip-validate

Release Steps

• Commit latest changes
• Tag a version git tag -a v2.0 -m "Version 2.0"
• Push tag git push origin v2.0
• Run: GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist

Resources

• GORELEASER
• Docker
• rsync
• homebrew

License