- ETL
- ML/DL
- Poloniex (https://api-docs.poloniex.com/spot)
- Crypto price data
- Yahoo Finance (https://finance.yahoo.com/)
- Forex price data
- Stack index data
- Others
- Data from Poloniex via WebSocket
- Order book
- Market trade
- Crypto price for every minute
- Event streaming platform
- Kafka
- Streaming Data Processing Tools
- Kafka Consumer
- Flink
- Spark Streaming
- Cassandra
- MongoDB
- Main DB
- Hive built on HDFS
- ETL tools
- Spark
- Trino
- DBT(?)
- Druid for real-time analysis
- Trino for ad-hoc analysis
- Airflow