-
Notifications
You must be signed in to change notification settings - Fork 50
/
GUM_academic_replication.rsd
98 lines (98 loc) · 24.6 KB
/
GUM_academic_replication.rsd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
1 Replication in Second Language Research : 5 _ _ pos1=NN|head_tok=replication|head_pos=NN|head_id=1|parent_pos=_|sid=1|stype=frag|len=6|head_func=root|edu_tense=None|heading=heading 73 organization-heading_r _ graphical-layout-
2 Narrative and Systematic Reviews and Recommendations for the Field 0 _ _ pos1=JJ|head_tok=review|head_pos=NNS|head_id=10|parent_pos=_|sid=2|stype=frag|len=9|head_func=root|edu_tense=None|heading=heading 1 elaboration-additional_r _ semantic-synonymy-3-5,14-15
3 Replication studies are considered by many to play a fundamental role in any scientific endeavor . 4 _ _ pos1=NN|head_tok=consider|head_pos=VVN|head_id=19|parent_pos=_|sid=3|stype=decl|len=16|head_func=root|edu_tense=None|para=para 73 context-background_r _ _
4 When using the same materials and procedures as a previous study , 0 _ _ pos1=WRB|head_tok=use|head_pos=VVG|head_id=33|parent_posVVP|sid=4|stype=decl|len=12|head_func=advcl|edu_tense=None|subord=RIGHT 5 contingency-condition_r _ dm-when-32
5 replication studies serve to test the reliability of the previous study ’s findings . 0 _ _ pos1=NN|head_tok=serve|head_pos=VVP|head_id=46|parent_pos=_|sid=4|stype=decl|len=14|head_func=root|edu_tense=Pres 3 explanation-justify_r _ _
6 When altering specific methodological or participant characteristics of a previous study , 0 _ _ pos1=WRB|head_tok=alter|head_pos=VVG|head_id=59|parent_posVVP|sid=5|stype=decl|len=12|head_func=advcl|edu_tense=None|subord=RIGHT 7 contingency-condition_r _ dm-when-58
7 they serve to test generalizability of the earlier findings under different conditions . 1 _ _ pos1=PP|head_tok=serve|head_pos=VVP|head_id=71|parent_pos=_|sid=5|stype=decl|len=13|head_func=root|edu_tense=Pres 5 joint-list_m _ semantic-lexical_chain-44,61
8 One indication of the importance of replication is found in the 50 or more calls for replication research in the field of second language 1 _ _ pos1=CD|head_tok=find|head_pos=VVN|head_id=91|parent_pos=_|sid=6|stype=decl|len=24|head_func=root|edu_tense=None|subord=RIGHT 3 explanation-evidence_r _ _
9 ( L2 ) 0 _ _ pos1=(|head_tok=L2|head_pos=NN|head_id=108|parent_posNN|sid=6|stype=decl|len=3|head_func=appos|edu_tense=None|subord=LEFT 8 restatement-partial_r _ semantic-synonymy-105-106,108;graphical-parentheses-107,109
10 research alone 1 _ _ pos1=NN|head_tok=alone|head_pos=RB|head_id=111|parent_posNN|sid=6|stype=decl|len=2|head_func=advmod|edu_tense=None|subord=LEFT 8 same-unit_m _ _
11 ( see references for 50 calls and commentaries in Appendix S1 in the Supporting Information online ) : 2 _ _ pos1=(|head_tok=see|head_pos=VV|head_id=113|parent_posNNS|sid=6|stype=decl|len=18|head_func=dep|edu_tense=None|subord=RIGHT 8 explanation-evidence_r _ lexical-indicative_word-113
12 from Santos ( 1989 ) through Polio and Gass 1997 to very recent proposals for specific replication studies , such as Vandergrift and Cross 2017 and even a book - length treatment 3 _ _ pos1=IN|head_tok=proposal|head_pos=NNS|head_id=143|parent_posNNS|sid=6|stype=decl|len=32|head_func=nmod|edu_tense=None|subord=LEFT|date 8 elaboration-additional_r _ semantic-repetition-89,146;graphical-colon-129;graphical-parentheses-162,166
13 ( Porte , 2012 ) . 0 _ _ pos1=(|head_tok=Porte|head_pos=NP|head_id=163|parent_posNN|sid=6|stype=decl|len=6|head_func=dep|edu_tense=None|subord=LEFT|date 12 explanation-evidence_r _ _
14 Beyond these calls , efforts 4 _ _ pos1=IN|head_tok=effort|head_pos=NNS|head_id=172|parent_posVVN|sid=7|stype=decl|len=5|head_func=nsubj|edu_tense=None|subord=RIGHT 8 joint-list_m _ dm-also-181
15 to actively promote and facilitate replication studies 0 _ _ pos1=TO|head_tok=promote|head_pos=VV|head_id=175|parent_posNNS|sid=7|stype=decl|len=7|head_func=acl|edu_tense=None|subord=LEFT 14 purpose-attribute_r _ syntactic-modified_head-172;syntactic-infinitival_clause-173
16 have also emerged . 1 _ _ pos1=VHP|head_tok=emerge|head_pos=VVN|head_id=182|parent_pos=_|sid=7|stype=decl|len=4|head_func=root|edu_tense=PresPerf 14 same-unit_m _ _
17 For example , the Instruments for Research into Second Languages 2 _ _ pos1=IN|head_tok=Instrument|head_pos=NPS|head_id=188|parent_posNN|sid=8|stype=decl|len=10|head_func=compound|edu_tense=None|subord=RIGHT 14 elaboration-additional_r 14:explanation-evidence:5:2:orphan-for example-184-185 semantic-repetition-178,222;graphical-parentheses-198,200,231,246
18 ( IRIS ) 0 _ _ pos1=(|head_tok=Iris|head_pos=NP|head_id=195|parent_posNPS|sid=8|stype=decl|len=3|head_func=appos|edu_tense=None|subord=LEFT 17 restatement-partial_r _ semantic-synonymy-188-193,195;graphical-parentheses-194,196
19 repository 1 _ _ pos1=NN|head_tok=repository|head_pos=NN|head_id=197|parent_posVVN|sid=8|stype=decl|len=1|head_func=nsubj:pass|edu_tense=None|subord=RIGHT 17 same-unit_m _ _
20 ( http://www.irisdatabase.org ) 2 _ _ pos1=(|head_tok=http://www.irisdatabase.org|head_pos=NP|head_id=199|parent_posNN|sid=8|stype=decl|len=3|head_func=appos|edu_tense=None|subord=LEFT 17 explanation-evidence_r _ _
21 was established in 2011 3 _ _ pos1=VBD|head_tok=establish|head_pos=VVN|head_id=202|parent_pos=_|sid=8|stype=decl|len=4|head_func=root|edu_tense=None|date 17 same-unit_m _ _
22 and holds , at the time of writing , over 3,800 materials 4 _ _ pos1=CC|head_tok=hold|head_pos=VVZ|head_id=206|parent_posVVN|sid=8|stype=decl|len=12|head_func=conj|edu_tense=Pres|subord=LEFT|date 17 joint-list_m _ dm-and-205
23 that can be used for replication , among other purposes , in L2 research 0 _ _ pos1=WDT|head_tok=use|head_pos=VVN|head_id=220|parent_posNNS|sid=8|stype=decl|len=14|head_func=acl:relcl|edu_tense=None|subord=LEFT 22 elaboration-attribute_r _ syntactic-modified_head-216;syntactic-relative_clause-217,220
24 ( Marsden & Mackey , 2014 ; Marsden , Mackey , & Plonsky , 2016 ) . 5 _ _ pos1=(|head_tok=Marsden|head_pos=NP|head_id=232|parent_posVVN|sid=8|stype=decl|len=17|head_func=dep|edu_tense=None|subord=LEFT|date 17 explanation-evidence_r _ _
25 The Open Science Framework 6 _ _ pos1=DT|head_tok=Framework|head_pos=NP|head_id=251|parent_posVVZ|sid=9|stype=decl|len=4|head_func=nsubj|edu_tense=None|subord=RIGHT 17 joint-list_m _ _
26 ( https://osf.io ) , 0 _ _ pos1=(|head_tok=https://osf.io|head_pos=NP|head_id=253|parent_posNP|sid=9|stype=decl|len=4|head_func=appos|edu_tense=None|subord=RIGHT 25 explanation-evidence_r _ _
27 also established in 2011 , 1 _ _ pos1=RB|head_tok=establish|head_pos=VVN|head_id=257|parent_posNP|sid=9|stype=decl|len=5|head_func=acl|edu_tense=None|subord=LEFT|date 25 elaboration-attribute_r _ syntactic-modified_head-251;syntactic-nominal_modifier-257
28 provides a web infrastructure 2 _ _ pos1=VVZ|head_tok=provide|head_pos=VVZ|head_id=261|parent_pos=_|sid=9|stype=decl|len=4|head_func=root|edu_tense=Pres 25 same-unit_m _ _
29 to facilitate collaboration 0 _ _ pos1=TO|head_tok=facilitate|head_pos=VV|head_id=266|parent_posVVZ|sid=9|stype=decl|len=3|head_func=advcl|edu_tense=None|subord=LEFT 28 purpose-goal_r _ syntactic-infinitival_clause-265
30 and has been used for large replication efforts in psychology 3 _ _ pos1=CC|head_tok=use|head_pos=VVN|head_id=271|parent_posVVZ|sid=9|stype=decl|len=10|head_func=conj|edu_tense=PresPerf|subord=LEFT 25 joint-list_m _ dm-and-268
31 ( e.g. , Open Science Collaboration , 2015 ) , 0 _ _ pos1=(|head_tok=Collaboration|head_pos=NP|head_id=283|parent_posNNS|sid=9|stype=decl|len=10|head_func=dep|edu_tense=None|subord=RIGHT|date 30 elaboration-additional_r _ graphical-parentheses-278,286;lexical-indicative_word-279
32 which continue to make waves in academia 1 _ _ pos1=WDT|head_tok=continue|head_pos=VVP|head_id=289|parent_posNNS|sid=9|stype=decl|len=7|head_func=acl:relcl|edu_tense=Pres|subord=LEFT 30 elaboration-attribute_r _ syntactic-modified_head-275;syntactic-relative_clause-288-289;graphical-parentheses-295,309,314,322
33 ( Laws , 2016 ; Lindsay , 2015 ; Martin & Clarke , 2017 ) 0 _ _ pos1=(|head_tok=Laws|head_pos=NP|head_id=296|parent_posNN|sid=9|stype=decl|len=15|head_func=dep|edu_tense=None|subord=LEFT|date 32 explanation-evidence_r _ _
34 and the general media 1 _ _ pos1=CC|head_tok=media|head_pos=NN|head_id=313|parent_posNN|sid=9|stype=decl|len=4|head_func=conj|edu_tense=None|subord=LEFT 32 same-unit_m _ _
35 ( Baker , 2015 ; Devlin , 2016 ) . 0 _ _ pos1=(|head_tok=Baker|head_pos=NP|head_id=315|parent_posNN|sid=9|stype=decl|len=10|head_func=dep|edu_tense=None|subord=LEFT|date 34 explanation-evidence_r _ _
36 In some fields , a flourishing metascience , 7 _ _ pos1=IN|head_tok=metascience|head_pos=NN|head_id=330|parent_posVVN|sid=10|stype=decl|len=8|head_func=nsubj|edu_tense=None|subord=RIGHT 17 joint-other_m _ _
37 that is , the scientific study of science 0 _ _ pos1=DT|head_tok=study|head_pos=NN|head_id=337|parent_posNN|sid=10|stype=decl|len=8|head_func=appos|edu_tense=None|subord=LEFT 36 restatement-partial_r _ semantic-synonymy-328-330,335-339;lexical-alternate_expression-332-333
38 ( see Munafò et al. , 2017 ) , 1 _ _ pos1=(|head_tok=see|head_pos=VV|head_id=341|parent_posNN|sid=10|stype=decl|len=9|head_func=parataxis|edu_tense=None|subord=LEFT|date 36 explanation-evidence_r _ lexical-indicative_word-341
39 has included syntheses 2 _ _ pos1=VHZ|head_tok=include|head_pos=VVN|head_id=350|parent_pos=_|sid=10|stype=decl|len=3|head_func=root|edu_tense=PresPerf 36 same-unit_m _ _
40 assessing the quantity and nature of replication efforts , for example , in education 0 _ _ pos1=VVG|head_tok=assess|head_pos=VVG|head_id=352|parent_posNNS|sid=10|stype=decl|len=14|head_func=acl|edu_tense=None|subord=LEFT 39 elaboration-attribute_r _ syntactic-modified_head-351;syntactic-nominal_modifier-352;graphical-parentheses-375,381
41 ( Makel & Plucker , 2014 0 _ _ pos1=(|head_tok=Makel|head_pos=NP|head_id=367|parent_posNN|sid=10|stype=decl|len=6|head_func=dep|edu_tense=None|subord=LEFT|date 40 explanation-evidence_r _ _
42 and in psychology 1 _ _ pos1=CC|head_tok=psychology|head_pos=NN|head_id=374|parent_posNN|sid=10|stype=decl|len=3|head_func=conj|edu_tense=None|subord=LEFT 40 same-unit_m _ _
43 ( Makel et al. , 2012 ) . 0 _ _ pos1=(|head_tok=Makel|head_pos=NP|head_id=376|parent_posNN|sid=10|stype=decl|len=8|head_func=dep|edu_tense=None|subord=LEFT|date 42 explanation-evidence_r _ _
44 The driving force behind this battery of calls , commentaries , infrastructure , and metascience is a perceived crisis in the state of replication research . 5 _ _ pos1=DT|head_tok=crisis|head_pos=NN|head_id=401|parent_pos=_|sid=11|stype=decl|len=26|head_func=root|edu_tense=Pres|para=para 8 causal-cause_r _ _
45 The severe concerns 2 _ _ pos1=DT|head_tok=concern|head_pos=NNS|head_id=411|parent_posVHP|sid=12|stype=decl|len=3|head_func=nsubj|edu_tense=None|subord=RIGHT 48 organization-preparation_r _ graphical-colon-421
46 underpinning the alleged crisis 0 _ _ pos1=VVG|head_tok=underpin|head_pos=VVG|head_id=412|parent_posNNS|sid=12|stype=decl|len=4|head_func=acl|edu_tense=None|subord=LEFT 45 elaboration-attribute_r _ syntactic-modified_head-411;syntactic-nominal_modifier-412
47 have several dimensions relating to : 1 _ _ pos1=VHP|head_tok=have|head_pos=VHP|head_id=416|parent_pos=_|sid=12|stype=decl|len=6|head_func=root|edu_tense=Pres|subord=RIGHT 45 same-unit_m _ _
48 (a) the ( small ) amount of published replication research ; 0 _ _ pos1=LS|head_tok=amount|head_pos=NN|head_id=427|parent_posVVG|sid=12|stype=decl|len=11|head_func=obl|edu_tense=None|subord=RIGHT 44 elaboration-additional_r _ semantic-synonymy-383-397,413-415;semantic-repetition-406-407,429-431;semantic-repetition-406,430
49 (b) the ( poor ) quality of replication research ; 0 _ _ pos1=LS|head_tok=quality|head_pos=NN|head_id=438|parent_posNN|sid=12|stype=decl|len=10|head_func=conj|edu_tense=None|subord=RIGHT 48 joint-list_m _ semantic-meronymy-423-431,434-441;graphical-items_in_sequence-433;graphical-semicolon-442
50 and (c) the ( lack of ) reproducibility , 0 _ _ pos1=CC|head_tok=lack|head_pos=NN|head_id=447|parent_posNN|sid=12|stype=decl|len=9|head_func=conj|edu_tense=None|subord=RIGHT 48 joint-list_m _ graphical-items_in_sequence-422,444;graphical-semicolon-432;semantic-meronymy-434-441,445-472;dm-and-443
51 which refers to the extent to which findings can ( not ) be reproduced in replication attempts 0 _ _ pos1=WDT|head_tok=refer|head_pos=VVZ|head_id=453|parent_posNN|sid=12|stype=decl|len=17|head_func=acl:relcl|edu_tense=Pres|subord=LEFT 50 elaboration-attribute_r _ syntactic-modified_head-447;syntactic-relative_clause-452-453,458,465
52 that have been undertaken . 0 _ _ pos1=WDT|head_tok=undertake|head_pos=VVN|head_id=472|parent_posNNS|sid=12|stype=decl|len=5|head_func=acl:relcl|edu_tense=PresPerf|subord=LEFT 51 elaboration-attribute_r _ syntactic-modified_head-468;syntactic-relative_clause-469,472
53 These concerns speak to the very core of science , 1 _ _ pos1=DT|head_tok=speak|head_pos=VVP|head_id=476|parent_pos=_|sid=13|stype=decl|len=10|head_func=root|edu_tense=Pres|subord=RIGHT 48 evaluation-comment_r _ lexical-indicative_word-479;lexical-indicative_word-485
54 raising fundamental questions about the validity and reliability of our work . 0 _ _ pos1=VVG|head_tok=raise|head_pos=VVG|head_id=484|parent_posVVP|sid=13|stype=decl|len=12|head_func=advcl|edu_tense=None|subord=LEFT 53 elaboration-additional_r _ semantic-lexical_chain-475,486;syntactic-present_participial_clause-484
55 Indeed , some commentators have called replication the “ gold standard ” of research evidence 1 _ _ pos1=RB|head_tok=call|head_pos=VVN|head_id=501|parent_pos=_|sid=14|stype=decl|len=15|head_func=root|edu_tense=PresPerf 44 adversative-antithesis_r 53:elaboration-additional:2:1:graphical-parentheses-511,525,535,544;orphan-indeed-496 _
56 ( Jasny , Chin , Chong , & Vignieri , 2011 , p. 1225 ) 0 _ _ pos1=(|head_tok=Jasny|head_pos=NP|head_id=512|parent_posNN|sid=14|stype=decl|len=15|head_func=dep|edu_tense=None|subord=LEFT|date 55 explanation-evidence_r _ _
57 and a “ linchpin of the scientific process ” 1 _ _ pos1=CC|head_tok=linchpin|head_pos=NN|head_id=529|parent_posNN|sid=14|stype=decl|len=9|head_func=conj|edu_tense=None|subord=LEFT 55 same-unit_m _ _
58 ( Let ’s replicate , 2006 , p. 330 ) . 0 _ _ pos1=(|head_tok=let|head_pos=VV|head_id=536|parent_posNN|sid=14|stype=decl|len=11|head_func=dep|edu_tense=None|subord=LEFT|date 57 explanation-evidence_r _ _
59 In the field of L2 research , 3 _ _ pos1=IN|head_tok=field|head_pos=NN|head_id=548|parent_posVV|sid=15|stype=decl|len=7|head_func=obl|edu_tense=None|subord=RIGHT|para=para 67 adversative-concession_r _ lexical-indicative_word-572;dm-however-584
60 given the importance of replication and the 50 calls for replication in L2 research 0 _ _ pos1=VVN|head_tok=give|head_pos=VVN|head_id=553|parent_posVV|sid=15|stype=decl|len=14|head_func=advcl|edu_tense=None|subord=RIGHT 59 context-circumstance_r _ dm-given-553;syntactic-past_participial_clause-553
61 that we identified , 0 _ _ pos1=WDT|head_tok=identify|head_pos=VVD|head_id=569|parent_posNNS|sid=15|stype=decl|len=4|head_func=acl:relcl|edu_tense=PastSimp|subord=LEFT 60 elaboration-attribute_r _ syntactic-modified_head-561;syntactic-relative_clause-567,569
62 one might expect a substantial number of published replication studies by now . 1 _ _ pos1=PP|head_tok=expect|head_pos=VV|head_id=573|parent_pos=_|sid=15|stype=decl|len=13|head_func=root|edu_tense=Modal 59 same-unit_m _ _
63 However , a perceived lack of prestige , excitement , and originality of replication plagues L2 research 2 _ _ pos1=RB|head_tok=plague|head_pos=VVZ|head_id=598|parent_pos=_|sid=16|stype=decl|len=17|head_func=root|edu_tense=Pres 67 causal-cause_r _ dm-and-643
64 ( Porte , 2012 ) , 0 _ _ pos1=(|head_tok=Porte|head_pos=NP|head_id=602|parent_posVVZ|sid=16|stype=decl|len=6|head_func=dep|edu_tense=None|subord=RIGHT|date 63 explanation-evidence_r _ _
65 as it does other disciplines 1 _ _ pos1=IN|head_tok=do|head_pos=VVZ|head_id=609|parent_posVVZ|sid=16|stype=decl|len=5|head_func=advcl|edu_tense=Pres|subord=LEFT 63 explanation-evidence_r 63:mode-manner:1:1:orphan-as-607 _
66 ( Berez - Kroeker et al. , 2017 ; Branco , Cohen , Vossen , Ide , & Calzolari , 2017 ; Chambers , 2017 ; Schmidt , 2009 ) , 0 _ _ pos1=(|head_tok=Berez|head_pos=NP|head_id=613|parent_posNNS|sid=16|stype=decl|len=31|head_func=dep|edu_tense=None|subord=RIGHT|date 65 explanation-evidence_r _ _
67 and these perceptions are thought to have caused , at least in part 3 _ _ pos1=CC|head_tok=think|head_pos=VVN|head_id=647|parent_posVVZ|sid=16|stype=decl|len=13|head_func=conj|edu_tense=None|subord=LEFT 73 context-background_r _ _
68 ( directly or indirectly ) , 0 _ _ pos1=(|head_tok=directly|head_pos=RB|head_id=657|parent_posVVN|sid=16|stype=decl|len=6|head_func=advmod|edu_tense=None|subord=RIGHT 67 elaboration-additional_r _ graphical-parentheses-656,660
69 alleged low rates and a poor quality of published replication studies . 1 _ _ pos1=VVN|head_tok=rate|head_pos=NNS|head_id=664|parent_posVVN|sid=16|stype=decl|len=12|head_func=obj|edu_tense=None|subord=LEFT 67 same-unit_m _ _
70 However , a systematic metascience on replication research has not yet been established in the field of L2 research , 4 _ _ pos1=RB|head_tok=establish|head_pos=VVN|head_id=686|parent_pos=_|sid=17|stype=decl|len=20|head_func=root|edu_tense=PresPerf|subord=RIGHT 67 adversative-contrast_m _ dm-however-674
71 leaving a poor understanding of the actual number and nature of replication studies 0 _ _ pos1=VVG|head_tok=leave|head_pos=VVG|head_id=694|parent_posVVN|sid=17|stype=decl|len=13|head_func=advcl|edu_tense=None|subord=LEFT 70 causal-result_r _ syntactic-present_participial_clause-694
72 that have been published . 0 _ _ pos1=WDT|head_tok=publish|head_pos=VVN|head_id=710|parent_posNNS|sid=17|stype=decl|len=5|head_func=acl:relcl|edu_tense=PresPerf|subord=LEFT 71 elaboration-attribute_r _ syntactic-modified_head-706;syntactic-relative_clause-707,710
73 The current study begins to address this gap through narrative and systematic reviews . 0 _ _ pos1=DT|head_tok=begin|head_pos=VVZ|head_id=715|parent_pos=_|sid=18|stype=decl|len=14|head_func=root|edu_tense=Pres|para=para 0 ROOT _ _
74 The narrative review considers challenges in replication research 0 _ _ pos1=DT|head_tok=consider|head_pos=VVZ|head_id=729|parent_pos=_|sid=19|stype=decl|len=8|head_func=root|edu_tense=Pres 73 elaboration-additional_r _ semantic-meronymy-721-724,726-728
75 and is largely informed by commentaries and metascience from psychology , 0 _ _ pos1=CC|head_tok=inform|head_pos=VVN|head_id=737|parent_posVVZ|sid=19|stype=decl|len=11|head_func=conj|edu_tense=None|subord=LEFT 74 joint-list_m _ dm-and-734
76 given that the cognitive and social subdomains of psychology are highly influential in L2 research , 0 _ _ pos1=VVN|head_tok=give|head_pos=VVN|head_id=745|parent_posVVN|sid=19|stype=decl|len=16|head_func=advcl|edu_tense=None|subord=RIGHT 75 explanation-justify_r _ dm-given-745
77 and also from education , another key sister discipline . 1 _ _ pos1=CC|head_tok=education|head_pos=NN|head_id=764|parent_posNN|sid=19|stype=decl|len=10|head_func=conj|edu_tense=None|subord=LEFT 75 same-unit_m _ _
78 The narrative review is organized around four broad themes : 1 _ _ pos1=DT|head_tok=organize|head_pos=VVN|head_id=775|parent_pos=_|sid=20|stype=decl|len=10|head_func=root|edu_tense=None|subord=RIGHT 74 elaboration-additional_r _ semantic-repetition-726-728,771-773;semantic-repetition-732-733,785-786;semantic-repetition-732,785
79 (a) the quantity of replication research , (b) the nature of replication research , (c) the relationship between initial and replication studies , and (d) the interpretation and extent of reproducibility of the findings of initial studies . 0 _ _ pos1=LS|head_tok=quantity|head_pos=NN|head_id=783|parent_posNNS|sid=20|stype=decl|len=38|head_func=appos|edu_tense=None|subord=LEFT 78 elaboration-additional_r _ numerical-same_count-777;semantic-meronymy-777-779,782-786;graphical-colon-780
80 To gain insight into these issues in the context of L2 research , 1 _ _ pos1=TO|head_tok=gain|head_pos=VV|head_id=820|parent_posVVZ|sid=21|stype=decl|len=13|head_func=advcl|edu_tense=None|subord=RIGHT 81 purpose-goal_r _ syntactic-infinitival_clause-819
81 the systematic review provides a synthesis of L2 studies in journal articles 2 _ _ pos1=DT|head_tok=provide|head_pos=VVZ|head_id=835|parent_pos=_|sid=21|stype=decl|len=12|head_func=root|edu_tense=Pres 74 joint-sequence_m _ _
82 that self - labeled as replications . 0 _ _ pos1=WDT|head_tok=label|head_pos=VVD|head_id=847|parent_posNNS|sid=21|stype=decl|len=7|head_func=acl:relcl|edu_tense=PastSimp|subord=LEFT 81 elaboration-attribute_r _ syntactic-modified_head-840;syntactic-relative_clause-844,847
83 The research questions and methods of the systematic review were largely determined by the narrative review 2 _ _ pos1=DT|head_tok=determine|head_pos=VVN|head_id=862|parent_pos=_|sid=22|stype=decl|len=16|head_func=root|edu_tense=None 81 elaboration-additional_r 84:adversative-concession:0:0:orphan-but-867 semantic-repetition-832-834,857-859;semantic-lexical_chain-840,852
84 but also emerged through the design and piloting of the coding instrument . 0 _ _ pos1=CC|head_tok=emerge|head_pos=VVD|head_id=869|parent_posVVN|sid=22|stype=decl|len=13|head_func=conj|edu_tense=PastSimp|subord=LEFT 83 joint-list_m _ dm-also-868
85 Finally , we offer further discussion and 16 recommendations for future replication work 2 _ _ pos1=RB|head_tok=offer|head_pos=VVP|head_id=883|parent_pos=_|sid=23|stype=decl|len=13|head_func=root|edu_tense=Pres 74 joint-sequence_m _ dm-finally-880;lexical-indicative_word-916;lexical-indicative_word-931;lexical-indicative_word-940;lexical-indicative_word-946;lexical-indicative_word-961
86 that draw on our narrative and systematic reviews and on our experience 0 _ _ pos1=WDT|head_tok=draw|head_pos=VVP|head_id=894|parent_posNNS|sid=23|stype=decl|len=12|head_func=acl:relcl|edu_tense=Pres|subord=LEFT 85 elaboration-attribute_r _ syntactic-modified_head-888;syntactic-relative_clause-893-894;syntactic-nominal_modifier-905-906
87 of carrying out multisite 0 _ _ pos1=IN|head_tok=multisite|head_pos=JJ|head_id=908|parent_posNNS|sid=23|stype=decl|len=4|head_func=amod|edu_tense=None|subord=RIGHT 86 elaboration-attribute_r _ syntactic-modified_head-904;graphical-parentheses-909,917,922,962
88 ( Morgan - Short et al. , 2018 ) 1 0 _ _ pos1=(|head_tok=Morgan|head_pos=NP|head_id=910|parent_posJJ|sid=23|stype=decl|len=10|head_func=dep|edu_tense=None|subord=LEFT|date 87 explanation-evidence_r _ _
89 and single site 1 _ _ pos1=CC|head_tok=site|head_pos=NN|head_id=921|parent_posJJ|sid=23|stype=decl|len=3|head_func=conj|edu_tense=None|subord=LEFT 87 same-unit_m _ _
90 ( Faretta - Stutenberg & Morgan - Short , 2011 ; Marsden , Williams , & Liu , 2013 ; McManus & Marsden , 2017 ; Morgan - Short , Heil , Botero - Moriaty , & Ebert , 2012 ) 0 _ _ pos1=(|head_tok=Faretta|head_pos=NP|head_id=923|parent_posNN|sid=23|stype=decl|len=41|head_func=dep|edu_tense=None|subord=LEFT|date 89 explanation-evidence_r _ _
91 replications . 1 _ _ pos1=NNS|head_tok=replication|head_pos=NNS|head_id=963|parent_posVVG|sid=23|stype=decl|len=2|head_func=obj|edu_tense=None|subord=LEFT 87 same-unit_m _ _
92 We start from the widely agreed premise 1 _ _ pos1=PP|head_tok=start|head_pos=VVP|head_id=966|parent_pos=_|sid=24|stype=decl|len=7|head_func=root|edu_tense=Pres 73 elaboration-additional_r 97:explanation-justify:1:1:orphan-thus-1018 _
93 that testing the reproducibility of findings should have an essential role in the testing and refinement of theory , at least for hypothesis - testing epistemologies 0 _ _ pos1=IN/that|head_tok=have|head_pos=VH|head_id=979|parent_posNN|sid=24|stype=decl|len=26|head_func=acl|edu_tense=Modal|subord=LEFT 92 elaboration-attribute_r _ syntactic-modified_head-971;syntactic-nominal_modifier-972,979
94 that seek to ascertain generalizability 0 _ _ pos1=WDT|head_tok=seek|head_pos=VVP|head_id=999|parent_posNNS|sid=24|stype=decl|len=5|head_func=acl:relcl|edu_tense=Pres|subord=LEFT 93 elaboration-attribute_r _ syntactic-modified_head-997;syntactic-relative_clause-998-999
95 and for other epistemologies 1 _ _ pos1=CC|head_tok=epistemology|head_pos=NNS|head_id=1006|parent_posNNS|sid=24|stype=decl|len=4|head_func=conj|edu_tense=None|subord=LEFT 93 same-unit_m _ _
96 in which constructs are deemed to be definable and observable . 0 _ _ pos1=IN|head_tok=deem|head_pos=VVN|head_id=1011|parent_posNNS|sid=24|stype=decl|len=11|head_func=acl:relcl|edu_tense=None|subord=LEFT 95 elaboration-attribute_r _ syntactic-modified_head-1006;syntactic-relative_clause-1008,1011
97 Thus , our overall aim is to provide conceptual clarification and an empirical base for future discussion and production of replication studies , with a view 2 _ _ pos1=RB|head_tok=provide|head_pos=VV|head_id=1025|parent_pos=_|sid=25|stype=decl|len=26|head_func=root|edu_tense=None 73 purpose-goal_r _ lexical-indicative_word-1022
98 to improving the amount and quality of L2 replication research . 0 _ _ pos1=IN|head_tok=improve|head_pos=VVG|head_id=1045|parent_posNN|sid=25|stype=decl|len=11|head_func=acl|edu_tense=None|subord=LEFT 97 elaboration-attribute_r _ syntactic-modified_head-1043;syntactic-nominal_modifier-1044-1045