Beat written in Go to retrieve flight prices from Skyscanner API and insert them into an elasticsearch cluster
$ mage build
$ ./flightbeat -e -d "*"
# input : flightbeat.yml (default)
# output: console (default)
The beat has to always be active on 1 node (out of 16). A periodic NodeJS job, leader.js makes sure that there is always a leader designated to run the beat. (and because a NodeJS job doesn't handle hard reboot, a cron job makes sure every hour that leader.js is up and running). In case of failure of the leader, the nodes perform a leader election to elect a new leader who will run the beat.
- install go
go get github.com/elastic/beats
go get github.com/magefile/mage
cd elastic/beats; mage GenerateCustomBeat
(fill everything explicitly, there is no default value)cd gforien/custombeat
- disable W10 App execution alias + copy python.exe to go/bin/python3.exe + disable W10 real-time protection (re-enable after)
make setup; make
This project is sampled from :