Building a serverless real-time data consumer

Project Goals

I wanted to learn about how to consume data using Apache Kinsesis, and wanted to do so using a data stream with real world data rather than test or simulation data. I had some additional requirements nad things I wanted to learn along the way which are listed below in no particiular order.

1. Consume a real world data stream. Doing something useful or interesting with it is a secondary goal.
2. Build the consumer logic and infrastructure using SST where possible, falling back to CDK for resources that SST doesn’t support.

Steps

I found a page sharing some publicly available data streams here.