flightbeat ✈️⚡

Beat written in Go to retrieve flight prices from Skyscanner API and insert them into an elasticsearch cluster

👷 Build and run the beat

$ mage build
$ ./flightbeat -e -d "*"
# input : flightbeat.yml (default)
# output: console (default)

🚀 Deploy

The beat has to always be active on 1 node (out of 16). A periodic NodeJS job, leader.js makes sure that there is always a leader designated to run the beat. (and because a NodeJS job doesn't handle hard reboot, a cron job makes sure every hour that leader.js is up and running). In case of failure of the leader, the nodes perform a leader election to elect a new leader who will run the beat.

🔧 Generate your Beat from elastic/beats with mage (oct. 2020)

1. install go
2. go get github.com/elastic/beats
3. go get github.com/magefile/mage
4. cd elastic/beats; mage GenerateCustomBeat (fill everything explicitly, there is no default value)
5. cd gforien/custombeat
6. disable W10 App execution alias + copy python.exe to go/bin/python3.exe + disable W10 real-time protection (re-enable after)
7. make setup; make

📚 Sources

This project is sampled from :

• Beats developer guide
• Go by example
• HTTP requests in Go
• Biggest airports in the world
• Flight routes database

Gabriel FORIEN