Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ben]Upstream cleaning #547

Merged
merged 11 commits into from
Jun 10, 2024
Merged

[ben]Upstream cleaning #547

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b3f176a
Merge branch 'Othergreengrasses-feature/test_column_order'
Othergreengrasses May 4, 2021
0f2945e
Merge branch 'master' of https://github.com/Othergreengrasses/wikipron
Othergreengrasses May 4, 2021
deca0e4
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Dec 18, 2022
b7c9def
Merge branch 'CUNY-CL:master' into master
Othergreengrasses May 4, 2023
c30493f
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Mar 8, 2024
8533f6c
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Mar 12, 2024
624194d
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Apr 9, 2024
6db7ce1
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Apr 12, 2024
386b9ad
Merge branch 'CUNY-CL:master' into master
Othergreengrasses Jun 10, 2024
da62d0d
[ben]upstream cleaning
Othergreengrasses Jun 10, 2024
71873ca
changelog updated
Othergreengrasses Jun 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Unreleased
- Repeats big scrape after \#523. (\#536)
- Fixes excessive line wrapping. (\#529)
- Big scrape for 2024. (\#514)
- Upstream cleaning for Bengali data. (\#547)

### Under `src/` and elsewhere

Expand Down
14 changes: 7 additions & 7 deletions data/scrape/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* Broad transcription files: 23
* Narrow transcription files: 22
* Scripts: 42
* Pronunciations: 4,311,273
* Pronunciations: 4,323,779


| Link | ISO 639-3 Code | ISO 639 Language Name | Wiktionary Language Name | Script | Dialect | Filtered | Narrow/Broad | # of entries |
Expand Down Expand Up @@ -56,13 +56,13 @@
| [TSV](tsv/bdq_latn_broad.tsv) | bdq | Bahnar | Bahnar | Latin | | False | Broad | 193 |
| [TSV](tsv/bel_cyrl_narrow.tsv) | bel | Belarusian | Belarusian | Cyrillic | | False | Narrow | 5,516 |
| [TSV](tsv/ben_beng_broad.tsv) | ben | Bengali | Bengali | Bengali | | False | Broad | 6,666 |
| [TSV](tsv/ben_beng_dhaka_broad.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | False | Broad | 4,784 |
| [TSV](tsv/ben_beng_dhaka_broad_filtered.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | True | Broad | 3,653 |
| [TSV](tsv/ben_beng_dhaka_narrow.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | False | Narrow | 4,093 |
| [TSV](tsv/ben_beng_dhaka_broad.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | False | Broad | 7,873 |
| [TSV](tsv/ben_beng_dhaka_broad_filtered.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | True | Broad | 6,405 |
| [TSV](tsv/ben_beng_dhaka_narrow.tsv) | ben | Bengali | Bengali | Bengali | Dhaka | False | Narrow | 7,382 |
| [TSV](tsv/ben_beng_narrow.tsv) | ben | Bengali | Bengali | Bengali | | False | Narrow | 5,933 |
| [TSV](tsv/ben_beng_rarh_broad.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | False | Broad | 4,903 |
| [TSV](tsv/ben_beng_rarh_broad_filtered.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | True | Broad | 3,879 |
| [TSV](tsv/ben_beng_rarh_narrow.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | False | Narrow | 4,224 |
| [TSV](tsv/ben_beng_rarh_broad.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | False | Broad | 7,873 |
| [TSV](tsv/ben_beng_rarh_broad_filtered.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | True | Broad | 6,426 |
| [TSV](tsv/ben_beng_rarh_narrow.tsv) | ben | Bengali | Bengali | Bengali | Rarh, Standard Bengali | False | Narrow | 7,382 |
| [TSV](tsv/bjb_latn_broad.tsv) | bjb | Banggarla | Barngarla | Latin | | False | Broad | 136 |
| [TSV](tsv/blt_tavt_narrow.tsv) | blt | Tai Dam | Tai Dam | Tai Viet | | False | Narrow | 239 |
| [TSV](tsv/bod_tibt_broad.tsv) | bod | Tibetan | Tibetan | Tibetan | | False | Broad | 2,699 |
Expand Down
12 changes: 6 additions & 6 deletions data/scrape/summary.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,13 @@ bcl_latn_narrow.tsv bcl Central Bikol Bikol Central Latin False Narrow 4936
bdq_latn_broad.tsv bdq Bahnar Bahnar Latin False Broad 193
bel_cyrl_narrow.tsv bel Belarusian Belarusian Cyrillic False Narrow 5516
ben_beng_broad.tsv ben Bengali Bengali Bengali False Broad 6666
ben_beng_dhaka_broad.tsv ben Bengali Bengali Bengali Dhaka False Broad 4784
ben_beng_dhaka_broad_filtered.tsv ben Bengali Bengali Bengali Dhaka True Broad 3653
ben_beng_dhaka_narrow.tsv ben Bengali Bengali Bengali Dhaka False Narrow 4093
ben_beng_dhaka_broad.tsv ben Bengali Bengali Bengali Dhaka False Broad 7873
ben_beng_dhaka_broad_filtered.tsv ben Bengali Bengali Bengali Dhaka True Broad 6405
ben_beng_dhaka_narrow.tsv ben Bengali Bengali Bengali Dhaka False Narrow 7382
ben_beng_narrow.tsv ben Bengali Bengali Bengali False Narrow 5933
ben_beng_rarh_broad.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali False Broad 4903
ben_beng_rarh_broad_filtered.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali True Broad 3879
ben_beng_rarh_narrow.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali False Narrow 4224
ben_beng_rarh_broad.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali False Broad 7873
ben_beng_rarh_broad_filtered.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali True Broad 6426
ben_beng_rarh_narrow.tsv ben Bengali Bengali Bengali Rarh, Standard Bengali False Narrow 7382
bjb_latn_broad.tsv bjb Banggarla Barngarla Latin False Broad 136
blt_tavt_narrow.tsv blt Tai Dam Tai Dam Tai Viet False Narrow 239
bod_tibt_broad.tsv bod Tibetan Tibetan Tibetan False Broad 2699
Expand Down
Loading