Changed Dropbox links to support direct download (#1422)

The suffix ?dl=1 is a Dropbox convention to support direct download without taking you to a landing page.
castorini · Nov 22, 2020 · 746447a · 746447a
1 parent 558ff2b
commit 746447a
Show file tree

Hide file tree

Showing 2 changed files with 88 additions and 82 deletions.
diff --git a/docs/experiments-covid-doc2query.md b/docs/experiments-covid-doc2query.md
@@ -14,22 +14,25 @@ These predicted queries are then appended to the original document and indexed a
 For CORD-19, these predictions were made using only article title and abstracts with T5 trained on MS MARCO passage date.
 These expansions were then appended to the abstract, full-text, and paragraph index conditions, as described on [this page](experiments-cord19.md).
 
+All the runs referenced on this page are stored in [this repo](https://git.uwaterloo.ca/jimmylin/covidex-trec-covid-runs).
+As an alternative to downloading each run separately, clone the repo and you'll have everything.
+
 ## Round 5
 
 These are runs that can be easily replicated with Anserini, from pre-built doc2query expanded CORD-19 indexes we have provided (version from 2020/07/16, the official corpus used in round 5).
 They were prepared _for_ round 5 (for participants who wish to have a baseline run to rerank); to provide a sense of effectiveness, we present evaluation results with the cumulative qrels from rounds 1, 2, 3, and 4 ([`qrels_covid_d4_j0.5-4.txt`](https://ir.nist.gov/covidSubmit/data/qrels-covid_d4_j0.5-4.txt) provided by NIST, stored in our repo as [`qrels.covid-round4-cumulative.txt`](../src/main/resources/topics-and-qrels/qrels.covid-round4-cumulative.txt)).
 
 |    | index     | field(s)                        | nDCG@10 | J@10 | R@1k | run file | checksum |
 |---:|:----------|:--------------------------------|--------:|-----:|-----:|:---------|----------|
-|  1 | abstract  | query+question                  | 0.4635 | 0.5300 | 0.4462 | [[download](https://www.dropbox.com/s/sa6abjrk1esxn38/expanded.anserini.covid-r5.abstract.qq.bm25.txt)]    | `9923233a31ac004f84b7d563baf6543c` |
-|  2 | abstract  | UDel qgen                       | 0.4548 | 0.5000 | 0.4527 | [[download](https://www.dropbox.com/s/t3s3oj9g0b1nphk/expanded.anserini.covid-r5.abstract.qdel.bm25.txt)]  | `e0c7a1879e5b1742045bba0f5293d558` |
-|  3 | full-text | query+question                  | 0.4450 | 0.6020 | 0.4473 | [[download](https://www.dropbox.com/s/utvw91nluzwm3ex/expanded.anserini.covid-r5.full-text.qq.bm25.txt)]   | `78aa7f481de91d22192163ed934d02ee` |
-|  4 | full-text | UDel qgen                       | 0.4817 | 0.6040 | 0.4711 | [[download](https://www.dropbox.com/s/xk2jyiwh5fjdwst/expanded.anserini.covid-r5.full-text.qdel.bm25.txt)] | `51cbae025bf90dadf8f26c5c31af9f66` |
-|  5 | paragraph | query+question                  | 0.4904 | 0.5820 | 0.5004 | [[download](https://www.dropbox.com/s/rjbyljcpziv31xx/expanded.anserini.covid-r5.paragraph.qq.bm25.txt)]   | `0b80444c8a737748ba9199ddf0795421` |
-|  6 | paragraph | UDel qgen                       | 0.4940 | 0.5700 | 0.5070 | [[download](https://www.dropbox.com/s/f4h2jhhla4o26wr/expanded.anserini.covid-r5.paragraph.qdel.bm25.txt)] | `2040b9a4759af722d50610f26989c328` |
-|  7 | -         | reciprocal rank fusion(1, 3, 5) | 0.4908 | 0.5880 | 0.5119 | [[download](https://www.dropbox.com/s/bj00pfwngi2j2g1/expanded.anserini.covid-r5.fusion1.txt)]             | `c0ffc7b1719f64d2f37ce99a9ef0413c` |
-|  8 | -         | reciprocal rank fusion(2, 4, 6) | 0.4846 | 0.5740 | 0.5218 | [[download](https://www.dropbox.com/s/f5ro0ex38gkvnqc/expanded.anserini.covid-r5.fusion2.txt)]             | `329f13267abf3f3d429a1593c1bd862f` |
-|  9 | abstract  | UDel qgen + RF                  | 0.6095 | 0.6320 | 0.5280 | [[download](https://www.dropbox.com/s/j6op32bcaszd1up/expanded.anserini.covid-r5.abstract.qdel.bm25%2Brm3Rf.txt)] | `a5e016c84d5547519ffbcf74c9a24fc8` |
+|  1 | abstract  | query+question                  | 0.4635 | 0.5300 | 0.4462 | [[download](https://www.dropbox.com/s/sa6abjrk1esxn38/expanded.anserini.covid-r5.abstract.qq.bm25.txt?dl=1)]    | `9923233a31ac004f84b7d563baf6543c` |
+|  2 | abstract  | UDel qgen                       | 0.4548 | 0.5000 | 0.4527 | [[download](https://www.dropbox.com/s/t3s3oj9g0b1nphk/expanded.anserini.covid-r5.abstract.qdel.bm25.txt?dl=1)]  | `e0c7a1879e5b1742045bba0f5293d558` |
+|  3 | full-text | query+question                  | 0.4450 | 0.6020 | 0.4473 | [[download](https://www.dropbox.com/s/utvw91nluzwm3ex/expanded.anserini.covid-r5.full-text.qq.bm25.txt?dl=1)]   | `78aa7f481de91d22192163ed934d02ee` |
+|  4 | full-text | UDel qgen                       | 0.4817 | 0.6040 | 0.4711 | [[download](https://www.dropbox.com/s/xk2jyiwh5fjdwst/expanded.anserini.covid-r5.full-text.qdel.bm25.txt?dl=1)] | `51cbae025bf90dadf8f26c5c31af9f66` |
+|  5 | paragraph | query+question                  | 0.4904 | 0.5820 | 0.5004 | [[download](https://www.dropbox.com/s/rjbyljcpziv31xx/expanded.anserini.covid-r5.paragraph.qq.bm25.txt?dl=1)]   | `0b80444c8a737748ba9199ddf0795421` |
+|  6 | paragraph | UDel qgen                       | 0.4940 | 0.5700 | 0.5070 | [[download](https://www.dropbox.com/s/f4h2jhhla4o26wr/expanded.anserini.covid-r5.paragraph.qdel.bm25.txt?dl=1)] | `2040b9a4759af722d50610f26989c328` |
+|  7 | -         | reciprocal rank fusion(1, 3, 5) | 0.4908 | 0.5880 | 0.5119 | [[download](https://www.dropbox.com/s/bj00pfwngi2j2g1/expanded.anserini.covid-r5.fusion1.txt?dl=1)]             | `c0ffc7b1719f64d2f37ce99a9ef0413c` |
+|  8 | -         | reciprocal rank fusion(2, 4, 6) | 0.4846 | 0.5740 | 0.5218 | [[download](https://www.dropbox.com/s/f5ro0ex38gkvnqc/expanded.anserini.covid-r5.fusion2.txt?dl=1)]             | `329f13267abf3f3d429a1593c1bd862f` |
+|  9 | abstract  | UDel qgen + RF                  | 0.6095 | 0.6320 | 0.5280 | [[download](https://www.dropbox.com/s/j6op32bcaszd1up/expanded.anserini.covid-r5.abstract.qdel.bm25%2Brm3Rf.txt?dl=1)] | `a5e016c84d5547519ffbcf74c9a24fc8` |
 
 **IMPORTANT NOTES!!!**
 
@@ -43,9 +46,9 @@ The final runs after removing judgments from 1, 2, 3, and 4 (cumulatively), are
 
 | runtag | run file | checksum |
 |:-------|:---------|:---------|
-| `r5.fusion1` = Row 7 | [[download](https://www.dropbox.com/s/5ke2c4x2z8de31h/expanded.anserini.final-r5.fusion1.txt)] | `2295216ed623d2621f00c294f7c389e1` |
-| `r5.fusion2` = Row 8 | [[download](https://www.dropbox.com/s/j1qdqr88cbsybae/expanded.anserini.final-r5.fusion2.txt)] | `a65fabe7b5b7bc4216be632296269ce6` |
-| `r5.rf` = Row 9      | [[download](https://www.dropbox.com/s/5bm4pdngh5bx3px/expanded.anserini.final-r5.rf.txt)]      | `24f0b75a25273b7b00d3e65065e98147` |
+| `r5.fusion1` = Row 7 | [[download](https://www.dropbox.com/s/5ke2c4x2z8de31h/expanded.anserini.final-r5.fusion1.txt?dl=1)] | `2295216ed623d2621f00c294f7c389e1` |
+| `r5.fusion2` = Row 8 | [[download](https://www.dropbox.com/s/j1qdqr88cbsybae/expanded.anserini.final-r5.fusion2.txt?dl=1)] | `a65fabe7b5b7bc4216be632296269ce6` |
+| `r5.rf` = Row 9      | [[download](https://www.dropbox.com/s/5bm4pdngh5bx3px/expanded.anserini.final-r5.rf.txt?dl=1)]      | `24f0b75a25273b7b00d3e65065e98147` |
 
 We have written scripts that automate the replication of these baselines:
 
@@ -65,9 +68,9 @@ The actual evaluated runs are (mirrored from URL above):
 
 | group | runtag | run file | checksum |
 |:------|:-------|:---------|:---------|
-| `anserini` | `r5.d2q.fusion1` (NIST post-processed) | [[download](https://www.dropbox.com/s/ojphpgilqs8xexc/expanded.anserini.final-r5.fusion1.post-processed.txt)] | `03ad001d94c772649e17f4d164d4b2e2` |
-| `anserini` | `r5.d2q.fusion2` (NIST post-processed) | [[download](https://www.dropbox.com/s/q7vx0l8n2u81s7z/expanded.anserini.final-r5.fusion2.post-processed.txt)] | `4137c93e76970616e0eff2803501cd08` |
-| `anserini` | `r5.d2q.rf` (NIST post-processed)      | [[download](https://www.dropbox.com/s/l4l1bbbi8msmrfh/expanded.anserini.final-r5.rf.post-processed.txt)]      | `3dfba85c0630865a7b581c4358cf4587` |
+| `anserini` | `r5.d2q.fusion1` (NIST post-processed) | [[download](https://www.dropbox.com/s/ojphpgilqs8xexc/expanded.anserini.final-r5.fusion1.post-processed.txt?dl=1)] | `03ad001d94c772649e17f4d164d4b2e2` |
+| `anserini` | `r5.d2q.fusion2` (NIST post-processed) | [[download](https://www.dropbox.com/s/q7vx0l8n2u81s7z/expanded.anserini.final-r5.fusion2.post-processed.txt?dl=1)] | `4137c93e76970616e0eff2803501cd08` |
+| `anserini` | `r5.d2q.rf` (NIST post-processed)      | [[download](https://www.dropbox.com/s/l4l1bbbi8msmrfh/expanded.anserini.final-r5.rf.post-processed.txt?dl=1)]      | `3dfba85c0630865a7b581c4358cf4587` |
 
 Effectiveness results (note that starting in Round 4, NIST changed from nDCG@10 to nDCG@20):
 
@@ -113,25 +116,25 @@ The runs below represent correspond to our [TREC-COVID baselines](experiments-co
 
 |    | index     | field(s)                        | run file | checksum |
 |---:|:----------|:--------------------------------|:---------|----------|
-|  1 | abstract  | query+question                  | [[download](https://www.dropbox.com/s/yxapvqec9o2ucon/expanded.anserini.covid-r4.abstract.qq.bm25.txt)]           | `d1d32cd6962c4e355a47e7f1fdfb0c74` |
-|  2 | abstract  | UDel qgen                       | [[download](https://www.dropbox.com/s/vnk3swwwfcncolk/expanded.anserini.covid-r4.abstract.qdel.bm25.txt)]         | `55ae93b92bae20ed64fc9f191c6ea667` |
-|  3 | full-text | query+question                  | [[download](https://www.dropbox.com/s/pkk3m90bv0rpxru/expanded.anserini.covid-r4.full-text.qq.bm25.txt)]          | `512e14c6d15eb36f7fc9c537281badd3` |
-|  4 | full-text | UDel qgen                       | [[download](https://www.dropbox.com/s/44hoa9xkf6tv0hq/expanded.anserini.covid-r4.full-text.qdel.bm25.txt)]        | `0901d7b083aa28afd431cf330fe7293c` |
-|  5 | paragraph | query+question                  | [[download](https://www.dropbox.com/s/z90xag7eh5pi53e/expanded.anserini.covid-r4.paragraph.qq.bm25.txt)]          | `f8512ba33d5cc79176d71424d05f81cb` |
-|  6 | paragraph | UDel qgen                       | [[download](https://www.dropbox.com/s/eno3z8pi7bnfy2p/expanded.anserini.covid-r4.paragraph.qdel.bm25.txt)]        | `123896c0af4cdbae471c21d2da7de1f7` |
-|  7 | -         | reciprocal rank fusion(1, 3, 5) | [[download](https://www.dropbox.com/s/zfbt15ivm37tolt/expanded.anserini.covid-r4.fusion1.txt)]                    | `77b619a2e6e87852b85d31637ceb6219` |
-|  8 | -         | reciprocal rank fusion(2, 4, 6) | [[download](https://www.dropbox.com/s/e7ki5e8jqi718bp/expanded.anserini.covid-r4.fusion2.txt)]                    | `1e7bb2a6e483d3629378c3107457b216` |
-|  9 | abstract  | UDel qgen + RF                  | [[download](https://www.dropbox.com/s/1uzy5ni33kvxq2o/expanded.anserini.covid-r4.abstract.qdel.bm25%2Brm3Rf.txt)] | `b6b1d949fff00e54b13e533e27455731` |
+|  1 | abstract  | query+question                  | [[download](https://www.dropbox.com/s/yxapvqec9o2ucon/expanded.anserini.covid-r4.abstract.qq.bm25.txt?dl=1)]           | `d1d32cd6962c4e355a47e7f1fdfb0c74` |
+|  2 | abstract  | UDel qgen                       | [[download](https://www.dropbox.com/s/vnk3swwwfcncolk/expanded.anserini.covid-r4.abstract.qdel.bm25.txt?dl=1)]         | `55ae93b92bae20ed64fc9f191c6ea667` |
+|  3 | full-text | query+question                  | [[download](https://www.dropbox.com/s/pkk3m90bv0rpxru/expanded.anserini.covid-r4.full-text.qq.bm25.txt?dl=1)]          | `512e14c6d15eb36f7fc9c537281badd3` |
+|  4 | full-text | UDel qgen                       | [[download](https://www.dropbox.com/s/44hoa9xkf6tv0hq/expanded.anserini.covid-r4.full-text.qdel.bm25.txt?dl=1)]        | `0901d7b083aa28afd431cf330fe7293c` |
+|  5 | paragraph | query+question                  | [[download](https://www.dropbox.com/s/z90xag7eh5pi53e/expanded.anserini.covid-r4.paragraph.qq.bm25.txt?dl=1)]          | `f8512ba33d5cc79176d71424d05f81cb` |
+|  6 | paragraph | UDel qgen                       | [[download](https://www.dropbox.com/s/eno3z8pi7bnfy2p/expanded.anserini.covid-r4.paragraph.qdel.bm25.txt?dl=1)]        | `123896c0af4cdbae471c21d2da7de1f7` |
+|  7 | -         | reciprocal rank fusion(1, 3, 5) | [[download](https://www.dropbox.com/s/zfbt15ivm37tolt/expanded.anserini.covid-r4.fusion1.txt?dl=1)]                    | `77b619a2e6e87852b85d31637ceb6219` |
+|  8 | -         | reciprocal rank fusion(2, 4, 6) | [[download](https://www.dropbox.com/s/e7ki5e8jqi718bp/expanded.anserini.covid-r4.fusion2.txt?dl=1)]                    | `1e7bb2a6e483d3629378c3107457b216` |
+|  9 | abstract  | UDel qgen + RF                  | [[download](https://www.dropbox.com/s/1uzy5ni33kvxq2o/expanded.anserini.covid-r4.abstract.qdel.bm25%2Brm3Rf.txt?dl=1)] | `b6b1d949fff00e54b13e533e27455731` |
 
 These runs are performed at [`539f7d`](https://github.com/castorini/anserini/commit/539f7d43a0183454a633f34aa20b46d2eeec1a19), 2020/07/24. Note that these runs were created _after_ the round 4 qrels became available, so this is a post-hoc simulation of "what would have happened".
 
 The final runs, after removing judgments from 1, 2, and 3 (cumulatively), are as follows:
 
 | runtag | run file | checksum |
 |:-------|:---------|:---------|
-| `r4.fusion1` = Row 7 | [[download](https://www.dropbox.com/s/mjgb5lz9ftty1w2/expanded.anserini.final-r4.fusion1.txt)] | `ae7513f68e2ca82d8b0efdd244082046` |
-| `r4.fusion2` = Row 8 | [[download](https://www.dropbox.com/s/5epunmkexqtupe6/expanded.anserini.final-r4.fusion2.txt)] | `590400c12b72ce8ed3b5af2f4c45f039` |
-| `r4.rf` = Row 9      | [[download](https://www.dropbox.com/s/kqbu3cui214ijyh/expanded.anserini.final-r4.rf.txt)]      | `b9e7bb80fd8dc97f93908d895fb07f7f` |
+| `r4.fusion1` = Row 7 | [[download](https://www.dropbox.com/s/mjgb5lz9ftty1w2/expanded.anserini.final-r4.fusion1.txt?dl=1)] | `ae7513f68e2ca82d8b0efdd244082046` |
+| `r4.fusion2` = Row 8 | [[download](https://www.dropbox.com/s/5epunmkexqtupe6/expanded.anserini.final-r4.fusion2.txt?dl=1)] | `590400c12b72ce8ed3b5af2f4c45f039` |
+| `r4.rf` = Row 9      | [[download](https://www.dropbox.com/s/kqbu3cui214ijyh/expanded.anserini.final-r4.rf.txt?dl=1)]      | `b9e7bb80fd8dc97f93908d895fb07f7f` |
 
 We have written scripts that automate the replication of these baselines: