Skip to content

Web scraper using AWS Lambda, DynamoDB, EventBridge, Serverless Framework, and Puppeteer.

Notifications You must be signed in to change notification settings

edwinpgm/web-scraper-serverless

Repository files navigation

Web Scraper Serverless Example

Motivation

Learn more about serverless with AWS and create a functional project that allow me save information about the exchange rate PEN / USD using AWS Lambda, DynamoDB, EventBridge and Serverless Framework.

Prerequisites

Built using

Installation

Use yarn or npm for install dependencies.

yarn install

Then you should update the env.example.yml and change the name by env.yml. This file contains the sites from which we will extract the exchange rates.

Now we go to deploy our changes and create the infrastructure on AWS.

serverless deploy

Usage

You should go to Amazon EventBridge and create a rule for run the lambda function each certain time.

After that you could go to DynamoDB, select the table "rates" and you will see the extract data.

That's all!