Amazon Kinesis is a platform on AWS to send your streaming data to. Kinesis makes it easy to load and analyze streaming data, and also providing the ability for you to build your own custom applications for your business needs.
-
Amazon Kinesis Data Streams is a scalable and durable real-time data streaming service that can continuously capture gigabytes of data per second from hundreds of thousands of sources.
image from A Cloud Guru- Stored for 24 hours up to a maximum of 7 days
Kinesis Streams Consists of Shards
- 5 transactions per second for reads, up to a maximum total data read rate of 2MB per second and up to 1,000 records per secind for writes, up to a maximum total data write of 1MB per second (including partition keys).
- The data capacity of your stream is a function of the number of shards that you specify for the stream. The total capacity of the stream is the sum of the capacities of its shards.
-
Amazon Kinesis Data Firehose is the easiest way to capture, transform, and load data streams into AWS data stores for near real-time analytics with existing business intelligence tools.
image from A Cloud Guru- Data is not persisted, you must do something with it
-
Amazon Kinesis Data Analytics is the easiest way to process data streams in real time with SQL or Java without having to learn new programming languages or processing frameworks.
image from A Cloud Guru- Can be used with Kinesis Streams and Kinesis Firehose
Streaming Data is data that is generated continuously by thousands of data sources, which typically send in the data records simultaneously, and in small sizes (order of Kilobytes).
Examples:
- Purchases from online stores (think amazon.com)
- Stock prices
- Game data (as the player plays)
- Social network data
- Geospatial data (think uber.com)
- iOT sensor data (think farm data streaming in from devices)
- What are the different kinds of Amazon Kinesis?
- Which Kinesis type offers data persistence of 24 hours by default but can be configured to be longer?
- Which Kinesis type requires you to analyze your data in real time and then find a place to store it?
- Which Kinesis type involves shards?
- True or false, Kinesis Analytics is used to analyze data streams for both Kinesis Streams and Kinesis Firehose.