Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add repeat option to batch method #34

Merged
merged 14 commits into from
Apr 28, 2023
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ target/

# These are backup files generated by rustfmt
**/*.rs.bk

# IDE
.idea/
69 changes: 34 additions & 35 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "rvp"
version = "0.1.2"
version = "0.1.3"
edition = "2021"
license = "MIT"
readme = "README.md"
Expand Down
111 changes: 108 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ RVP currently supports Intel Macs, M1 ARM Macs, and Linux. The tool has been tes

<summary>Unix (MacOs/Linux) manual install</summary>

This instruction works for both Linux and MacOS.
This instruction works for both Linux and macOS.

Download the latest release from the [releases page](https://github.com/samgozman/rvp/releases) for your platform.
For example, if you are using an Intel Mac, download the `rvp-x86_64-apple-darwin.tar.gz` file. For an M1 Mac, download the `rvp-aarch64-apple-darwin.tar.gz` file.
Expand Down Expand Up @@ -180,9 +180,114 @@ rvp batch --path ./weather.toml --params "israel/tel-aviv" "israel/jerusalem"

</details>

> `--params` option can be specified for each site in the config file. It simply replaces the `%%` placeholder in the URL. If you have **multiple resources** to parse, you can specify them as a **space-separated list**.
> `--params` option can be specified for each site in the config file. It simply replaces the `%%` placeholder in the URL.
> If you have **multiple resources** to parse, you can specify them as a **space-separated list**.

RVP's batch mode allows you to retrieve information from multiple sources and multiple values at once, making it a powerful tool for web scraping and data extraction.
#### Example 3: Parse stock information from multiple sources for the multiple stocks at once

Example config file: [stock.toml](examples/stock.toml)

```bash
rvp batch -p ./stock.toml --params AAPL MSFT -r --json
```

<details>

<summary>Output</summary>

```json
[
{
"name": "Name",
"value": "Apple Inc."
},
{
"name": "Market Cap",
"value": "2690.89B"
},
{
"name": "Price ($)",
"value": 168.29
},
{
"name": "Dividend ($)",
"value": 0.92
},
{
"name": "P/E",
"value": 28.61
},
{
"name": "% of Float Shorted",
"value": 0.73
},
{
"name": "Industry",
"value": "Computers/Consumer Electronics"
},
{
"name": "Sector",
"value": "Technology"
},
{
"name": "Put/Call Vol Ratio",
"value": 0.85
},
{
"name": "Put/Call OI Ratio ",
"value": 1.01
},
{
"name": "Name",
"value": "Microsoft Corporation"
},
{
"name": "Market Cap",
"value": "2271.87B"
},
{
"name": "Price ($)",
"value": 305.22
},
{
"name": "Dividend ($)",
"value": 2.72
},
{
"name": "P/E",
"value": 33.89
},
{
"name": "% of Float Shorted",
"value": 0.55
},
{
"name": "Industry",
"value": "Software"
},
{
"name": "Sector",
"value": "Technology"
},
{
"name": "Put/Call Vol Ratio",
"value": 0.82
},
{
"name": "Put/Call OI Ratio ",
"value": 1.0
}
]
```

</details>

> '-r, --repeat' flag is used to repeat the parameters for each resource.
> The number of parameters you provide will multiply the number of resources.
> So if you have 2 resources (r1, r2) and 2 parameters (p1, p2), you will get 4 results as:
> r1-p1, r2-p1, r1-p2, r2-p2

RVP batch mode allows you to retrieve information from multiple sources and multiple values at once, making it a powerful tool for web scraping and data extraction.

## Create config file

Expand Down
10 changes: 5 additions & 5 deletions examples/stock.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,27 @@ description = "Gather stock information from different sources"
url = "https://finviz.com/quote.ashx?t=%%"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table:nth-child(1) > tbody > tr > td > table.fullview-title > tbody > tr:nth-child(2) > td > a > b"
path = "body > div.content > div.ticker-wrapper tbody > tr > td > table.fullview-title > tbody > tr:nth-child(2) > td > a > b"
name = "Name"
parsed_type = "String"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(2) > td:nth-child(2) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(2) > td:nth-child(2) > b"
name = "Market Cap"
parsed_type = "String"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(11) > td:nth-child(12) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(11) > td:nth-child(12) > b"
name = "Price ($)"
parsed_type = "Number"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(7) > td:nth-child(2) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(7) > td:nth-child(2) > b"
name = "Dividend ($)"
parsed_type = "Number"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(1) > td:nth-child(4) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(1) > td:nth-child(4) > b"
name = "P/E"
parsed_type = "Number"

Expand Down
Loading