Skip to content

Commit

Permalink
Merge pull request #34 from samgozman/25-add-repeat-option-to-batch-m…
Browse files Browse the repository at this point in the history
…ethod

Add repeat option to batch method
  • Loading branch information
samgozman authored Apr 28, 2023
2 parents a44e41b + 9c57661 commit 2636a9d
Show file tree
Hide file tree
Showing 10 changed files with 245 additions and 67 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ target/

# These are backup files generated by rustfmt
**/*.rs.bk

# IDE
.idea/
69 changes: 34 additions & 35 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "rvp"
version = "0.1.2"
version = "0.1.3"
edition = "2021"
license = "MIT"
readme = "README.md"
Expand Down
111 changes: 108 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ RVP currently supports Intel Macs, M1 ARM Macs, and Linux. The tool has been tes

<summary>Unix (MacOs/Linux) manual install</summary>

This instruction works for both Linux and MacOS.
This instruction works for both Linux and macOS.

Download the latest release from the [releases page](https://github.com/samgozman/rvp/releases) for your platform.
For example, if you are using an Intel Mac, download the `rvp-x86_64-apple-darwin.tar.gz` file. For an M1 Mac, download the `rvp-aarch64-apple-darwin.tar.gz` file.
Expand Down Expand Up @@ -180,9 +180,114 @@ rvp batch --path ./weather.toml --params "israel/tel-aviv" "israel/jerusalem"

</details>

> `--params` option can be specified for each site in the config file. It simply replaces the `%%` placeholder in the URL. If you have **multiple resources** to parse, you can specify them as a **space-separated list**.
> `--params` option can be specified for each site in the config file. It simply replaces the `%%` placeholder in the URL.
> If you have **multiple resources** to parse, you can specify them as a **space-separated list**.
RVP's batch mode allows you to retrieve information from multiple sources and multiple values at once, making it a powerful tool for web scraping and data extraction.
#### Example 3: Parse stock information from multiple sources for the multiple stocks at once

Example config file: [stock.toml](examples/stock.toml)

```bash
rvp batch -p ./stock.toml --params AAPL MSFT -r --json
```

<details>

<summary>Output</summary>

```json
[
{
"name": "Name",
"value": "Apple Inc."
},
{
"name": "Market Cap",
"value": "2690.89B"
},
{
"name": "Price ($)",
"value": 168.29
},
{
"name": "Dividend ($)",
"value": 0.92
},
{
"name": "P/E",
"value": 28.61
},
{
"name": "% of Float Shorted",
"value": 0.73
},
{
"name": "Industry",
"value": "Computers/Consumer Electronics"
},
{
"name": "Sector",
"value": "Technology"
},
{
"name": "Put/Call Vol Ratio",
"value": 0.85
},
{
"name": "Put/Call OI Ratio ",
"value": 1.01
},
{
"name": "Name",
"value": "Microsoft Corporation"
},
{
"name": "Market Cap",
"value": "2271.87B"
},
{
"name": "Price ($)",
"value": 305.22
},
{
"name": "Dividend ($)",
"value": 2.72
},
{
"name": "P/E",
"value": 33.89
},
{
"name": "% of Float Shorted",
"value": 0.55
},
{
"name": "Industry",
"value": "Software"
},
{
"name": "Sector",
"value": "Technology"
},
{
"name": "Put/Call Vol Ratio",
"value": 0.82
},
{
"name": "Put/Call OI Ratio ",
"value": 1.0
}
]
```

</details>

> '-r, --repeat' flag is used to repeat the parameters for each resource.
> The number of parameters you provide will multiply the number of resources.
> So if you have 2 resources (r1, r2) and 2 parameters (p1, p2), you will get 4 results as:
> r1-p1, r2-p1, r1-p2, r2-p2
RVP batch mode allows you to retrieve information from multiple sources and multiple values at once, making it a powerful tool for web scraping and data extraction.

## Create config file

Expand Down
10 changes: 5 additions & 5 deletions examples/stock.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,27 @@ description = "Gather stock information from different sources"
url = "https://finviz.com/quote.ashx?t=%%"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table:nth-child(1) > tbody > tr > td > table.fullview-title > tbody > tr:nth-child(2) > td > a > b"
path = "body > div.content > div.ticker-wrapper tbody > tr > td > table.fullview-title > tbody > tr:nth-child(2) > td > a > b"
name = "Name"
parsed_type = "String"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(2) > td:nth-child(2) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(2) > td:nth-child(2) > b"
name = "Market Cap"
parsed_type = "String"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(11) > td:nth-child(12) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(11) > td:nth-child(12) > b"
name = "Price ($)"
parsed_type = "Number"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(7) > td:nth-child(2) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(7) > td:nth-child(2) > b"
name = "Dividend ($)"
parsed_type = "Number"

[[resources.selectors]]
path = "body > div:nth-child(10) > div > table.snapshot-table2 > tbody > tr:nth-child(1) > td:nth-child(4) > b"
path = "div.ticker-wrapper table:nth-child(1) div.snapshot-table-wrapper > table > tbody > tr:nth-child(1) > td:nth-child(4) > b"
name = "P/E"
parsed_type = "Number"

Expand Down
Loading

0 comments on commit 2636a9d

Please sign in to comment.