&AF>?OyHaI}*Wf`u> zS_njK7?-1blZu>eTdX$Ib-KRv}#ESU%<^z-uUsrfvk=zD1#zB};tu+aJ3$YUTG}9jgp)$rNXIYOd z8@789GEj `<|ed c|sW`P-BXK<4ApL__s zDa2JGC4 BqM2oG)&k-B6Z3JJL<5xN8s_h|c17H}3jL(M>DgK^gu~c|I(T zmw@jb6;}0*#^fdFQS$hg$&rC(#*8_`LRXnppY#zkoA3zbxf(QcD(Uk&mzZ}`*u1}| zH^u4Krk#u@;$6#GtP(4>`?{{wUo#pXL|-%2JY@2>Wp(oPaC?_#cIx?Hf%n@RqNGgs zcX(S()B{!%^bT}iWsn+%a9hJzmX=!$$Aa^`gbUMTk_uLiycY!t7oUHh#+g6DS`0sW z381ygV & z&%~=V$*G)A<0D=h@`*VW3yE{HT#+h3bIbiLj0xO0Yz3mOXJ=psplnfOqO?<+MZ6 z@g?G@x} SWc)33vMG(uPxp=Tskph`;Jt=i9;XPnOEr9VRht0%x^5@{@#YXtJjpA4?&&$K-! zfK5C)T=IW?ZFG!yn^gHBzdKzZVDaMRZ-$NXU;F5BLMIc2Pe$Pg!%yQXqxAWimOWY7 zGDy{B@)blj`})&F-IOm!618UV^u~imc*`RHmw_{mz;FTLNf9#aD&9@@=O$!IOFOxT z@so=Q9K}f!cr$gTa*vV+_-8rbGk8_jEHYTwiXB8et4GAh(w|LBSdu-F!oJtM{Ujr~ zy8F$#Udmol`Q?RwSR_<5F|Xe~`hFMCCPVs_1nR(fC@Z4-6XCxVZPXNu&lere_mfRr zkIZLZI$FVUnujStPfSj5oxS^Aq2W?jKidO@AV+GgEp9oK=DfC?V^$2g#^aL}&DKN9VWuur+Dmq(BcScruIm~V zJ>l5$gqn`ZmaQP0G$FMj{qU&hKPA*<+#CJx;4SPSee8}iQbYYDW U%CKpRwszEM0!1gw(9U!`yGxmTCjc4gW>AN5wfhjW{G=&ks760NwdH?dGR`}U z!!%JfSi#3Vc>3*I!in!&cLI2mLgQ;-Wj`piCVNwT>A}+v=XOZs%MQ mkc#b@I0x>Z|E!^$XqDTaq@-<&VMtRsH*TMoUiaR3 X&|8qVC?Y@M%Md|a+W {_E&CvM5gli zSug+IzcUG=!Rq1L#;-v7-m>qP3ugxUTfdNRj^m@m^}dG4r0`LVu03Dlu|YnF-92M@ z&uErb^L(`@?hGyv(D5#$NknN q=f(l&A83AD>yH zq!#uwo=Vb_X12`&*1^71y=pa~1AU(dT0~LbN&YhKyW;l7-CM|H9w%Hq*NYXp?L)T^ ztC)Z4mo4w6^$`36mwx-V@y1XAzRG-TM!1kvQMxS0_|E_U@K_)_c*1)0pP7NZoGh*C z7ghPAJQ5UcIuV95P>X|fDDl1re?@^+A1oBZD#ZAAXt4Qrw&2M;O7ORz|NjrV#gRPj WD0^IMg7bH%rKY5 Date: Mon, 16 Dec 2019 14:13:19 -0500 Subject: [PATCH 11/26] Document composer version (#5858) Adds an explicit minimum version for composer based on discussion in #5845. There was some concern on the mailing list that an issue during the install was caused by an old composer version. It makes sense to be a little more specific about it in our documentation. Resolves #5845 --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a69c4f4c902..10d45e02224 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Deploy and log in with username *admin* and the password that's set up during de * Apache **2.4** or higher * MySQL >= 5.7 (or MariaDB >= 10.3) * PHP 7.2 or higher - * [Composer](https://getcomposer.org/) + * [Composer](https://getcomposer.org/) 1.4 or higher * NodeJS 8.0 or higher * NPM * make From e83257f773b0461b759e2b293a4e1b5e8c614ea9 Mon Sep 17 00:00:00 2001 From: Rida Abou-Haidar Date: Tue, 17 Dec 2019 09:28:17 -0500 Subject: [PATCH 12/26] [WIKI] Study Parameter - Identifiers (#4469) Add document describing how to configure the ID generation for the study. --- .../01 - Study Variables/01 - Identifiers.md | 88 +++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/01 - Identifiers.md diff --git a/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/01 - Identifiers.md b/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/01 - Identifiers.md new file mode 100644 index 00000000000..50a81ac7630 --- /dev/null +++ b/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/01 - Identifiers.md @@ -0,0 +1,88 @@ +# Identifiers + +## Overview +By default LORIS provides 3 different identifiers for each candidate: + +- **CandID**, also known as the **DCCID**, is a unique randomized 6-digit numeric ID (e.g. '436792') assigned automatically by LORIS upon the candidate's registration. The CandID is not configurable. +- **PSCID** (Project Study Center ID) is a unique configurable ID. It can be set up to be either manually entered when registering a candidate or automatically generated and usually contains the site or project abbreviation followed by sequential or randomized characters (e.g. 'MTL0006'), but its exact format is customizable. +- **ExternalID** is a unique configurable ID. It can be set up to be either manually entered when registering a candidate or automatically generated and it is generally completely de-identified (no site or project information incorporated) to be used for data dissemination. + +The format and the generation of the **PSCID** and **ExternalID** must be configured by an admin by editing the `config.xml` file. + +## Configuration + +### Configuration from the front end +_not yet available_ + +### Configuration from SQL +_not yet available_ + +### Configuration from the config.xml file + +Both the format and the generation of PSCIDs can be configured by an administrator in the `config.xml` file. These settings are applied to any and all new candidates. + +PSCIDs can be created for new subjects in one of 3 ways: *sequentially generated*, *manually entered*, or *randomly generated*. + +1. ***sequential*** generates PSCIDs sequentially for each new candiddate registered. **(default)** + + ```xml + + + ``` + > Example PSCID generated: MTL1234 + > Where the site's alias is MTL + +2. ***manual*** asks the user to enter the PSCID when registering a new candidate. + + ```xml +sequential ++ ++ + + + ``` + > Example PSCID accepted: A1 + +3. ***random*** generates PSCIDs with a random numerical value for each new participant registered. + + ```xml +user ++ ++ + + ``` + > Example PSCID generated: PREFIX3994 + + Options for the `type` element of the `random ++ +PREFIX ++ ` tag are: + - `siteAbbrev`: A string value that will be used as a dynamic prefix. Value drawn from the `Alias` + field of the `psc` table in the database. + - `projectAbbrev`: A string value that will be used as a dynamic prefix. Value drawn from the `Alias` + field of the `Project` table in the database. + - `static`: A string value that will be used as a fixed prefix. Value defined in the `config.xml` file. + - `numeric`: An integer value generated dynamically in accordance to the generation method defined. + - `alphanumeric`: An alphanumeric string value generated dynamically in accordance to the generation method defined. + - `alpha`: An alphabetic string value generated dynamically in accordance to the generation method defined. + + > Note: The last 3 types above (`numeric`,`alphanumeric`,`alpha`) can be associated with + a `length` attribute. The length defaults to `4` when not specified. + + > Note: The last 3 types above (`numeric`,`alphanumeric`,`alpha`) can be associated with + minimum `min` and maximum `max` values for sequentially and randomly generated PSCIDs. + By default sequence will start at the lowest possible values (i.e.: 0000, AAAA). + +## Interaction With LORIS + + The **PSCID** and **CandID** are used throughout LORIS including when uploading files and media linked to a candidate. The **CandID** is mainly used internally to link data across the database. + + The **PSCID** is dependent on the list of sites in the `psc` table of the database, more specifically the `Alias` column that is used for the `siteAbbrev` type in the generation of the ID. Please refer to the [Sites Parameter Setup](03 - Sites.md) page for more details. + + The **PSCID** is also dependent on the list of projects in the `Project` table of the database, more specifically the `Alias` column that is used for the `projectAbbrev` type in the generation of the ID. Please refer to the [Projects Parameter Setup](02 - Projects.md) page for more details. From 8480d7678bd1e3ddacb381c97d1c02e22c4f662c Mon Sep 17 00:00:00 2001 From: Rida Abou-Haidar Date: Tue, 17 Dec 2019 09:59:35 -0500 Subject: [PATCH 13/26] [WIKI] Study Parameters - Subprojects (#5852) Add documentation on how to define/configure subprojects to replace documentation in the wiki. --- .../01 - Study Variables/04 - Subprojects.md | 57 +++++++++++++++++++ docs/wiki/99 - Developers/SQL Dictionary.md | 34 +++++++++++ 2 files changed, 91 insertions(+) create mode 100644 docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/04 - Subprojects.md diff --git a/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/04 - Subprojects.md b/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/04 - Subprojects.md new file mode 100644 index 00000000000..f17f2a9d2e8 --- /dev/null +++ b/docs/wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/04 - Subprojects.md @@ -0,0 +1,57 @@ +# Subprojects + +## Overview +**Subprojects** are defined in the front-end by the Configuration module, and are +stored in the `subproject` table of the database. +The **Subproject** plays an important role in determining the instrument battery +that will be assigned to a candidate at a timepoint. The subproject can be used as +a variable when determining which instruments are populated -- this allows for +subproject-specific batteries to be defined in the `test_battery` table. + +## Adding Subproject Options + +### Front End (Recommended) +Subprojects are defined in the Configuration module, which can be found in LORIS +under the **Admin** menu tab. Click on _To configure study subprojects click here_ +link at the top of the page. Refer to the help section of the module for further +instructions on how to add or modify subprojects. + +### SQL +Subprojects can be added directly in SQL using the following command. + +```sql +INSERT INTO subproject (title) VALUES('SCI'); +``` + + +### API + _not yet available. See [API documentation](../../../API/) for latest additions_ + + +## Interaction With LORIS + +### Projects +**Subprojects** must be associated to at least one Project in order to be able to +create timepoints for candidates. This association should be defined directly on the +front end through the Configuration module. + +> Note: the only way to view a list of all the projects affiliated to a subproject +is via the MySQL back-end. + +Sometimes it's useful to add project-subproject affiliations directly in the MySQL +back-end, for example when adding datasets to your LORIS. The following MySQL +statement is provided as an example for linking already-defined subprojects with an +existing project: + + ```sql + INSERT INTO project_subproject_rel + SELECT + p.ProjectID, + s.SubprojectID + FROM + Project p, + subproject s + WHERE + p.Name = "%PROJECT_NAME%" + AND s.title IN("%SUBPROJECT_1%", "%SUBPROJECT_2%", "%SUBPROJECT_3%"); + ``` diff --git a/docs/wiki/99 - Developers/SQL Dictionary.md b/docs/wiki/99 - Developers/SQL Dictionary.md index f045f9d3611..adee6149b52 100644 --- a/docs/wiki/99 - Developers/SQL Dictionary.md +++ b/docs/wiki/99 - Developers/SQL Dictionary.md @@ -47,3 +47,37 @@ | `OptimumMaxDays ` | Candidate's maximum age in days for visit to be flagged as _within **optimal** parameters of the study_ | The only effect of this field is a YES/NO flag showing up on the instrument_list page| | `WindowMidpointDays `| Candidate's ideal age in days for a visit | | + +- Table: `Project` + + *This table stores the list of projects configured for the study.* + + | Field | Description | Notes | + |:--------------------:|:---------------------------------------------:|:--------------------------------------------------------------------------------------------:| + | `ProjectID` | Identifier of the project | Avoid setting this field explicitly when inserting data, it auto increments. | + | `Name` | Full name of the project | | + | `recruitmentTarget` | Expected number of candidates to be recruited | | + + +- Table: `subproject` + + *This table stores the list of subprojects configured for the study.* + + | Field | Description | Notes | + |:------------------:|:---------------------------------------------:|:----------------------------------------------------------------------------:| + | `SubprojectID` | Identifier of the subproject | Avoid setting this field explicitly when inserting data, it auto increments. | + | `title` | Name of the subproject | | + | `useEDC` | Use the Expected date Of Confinement | | + | `WindowDifference` | | Deprecated | + | `RecruitmentTarget`| Expected number of candidates to be recruited | | + + +- Table: `project_subproject_rel` + + *This table stores the association of projects with subprojects* + + | Field | Description | Notes | + |:-------------------------:|:------------------------------------------------:|:----------------------------------------------------------------------------:| + | `ProjectSubprojectRelID ` | Identifier of the project-subproject relation | Avoid setting this field explicitly when inserting data, it auto increments. Other tables in the database require this field as a foreign key reference to point to a specific project-subproject tuple. | + | `ProjectID ` | Project identifier | | + | `SubprojectID ` | Subproject identifier | | From 80f7f52f7a8c6f84a618da36668bb13f61e09e02 Mon Sep 17 00:00:00 2001 From: Ling Ma Date: Tue, 17 Dec 2019 13:07:34 -0500 Subject: [PATCH 14/26] [Survey Accounts] Fix formatting of error message (#5855) Use the full name of the instrument in the survey accounts error message, not the short name. Fixes #5774. --- modules/survey_accounts/php/addsurvey.class.inc | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/modules/survey_accounts/php/addsurvey.class.inc b/modules/survey_accounts/php/addsurvey.class.inc index b9bffb8c062..a76b60ec74d 100644 --- a/modules/survey_accounts/php/addsurvey.class.inc +++ b/modules/survey_accounts/php/addsurvey.class.inc @@ -110,9 +110,13 @@ class AddSurvey extends \NDB_Form $reminder = " already exists for given candidate for visit "; foreach ($instrument_list as $instrument) { if ($values['Test_name'] == $instrument['Test_name']) { + $instrument_instance = \NDB_BVL_Instrument::factory( + $instrument['Test_name'] + ); return array( - 'Test_name' => "Instrument ". $values['Test_name']. - $reminder. $values['VL'], + 'Test_name' => "Instrument ". + $instrument_instance->getFullName(). + $reminder. $values['VL'], ); } } From 94e10bd02abffa04ffb73116cbb51bd590d8226c Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Tue, 17 Dec 2019 13:24:56 -0500 Subject: [PATCH 15/26] [ReadTheDocs] Add help style guide to navigation bar (#5862) Adds the help style guide to the Read The Docs sidebar. --- mkdocs.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/mkdocs.yml b/mkdocs.yml index ccb779829a0..7cd5c3f797b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,4 +24,5 @@ nav: - Developers: - 'Overview': 'wiki/99 - Developers/README.md' - 'SQL Dictionary': 'wiki/99 - Developers/SQL Dictionary.md' + - 'Style Guide (for help text)': 'HelpStyleGuide.md' theme: readthedocs From 88c48d72f569b0f1e54584bd538d4e4f22447014 Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Tue, 17 Dec 2019 14:57:33 -0500 Subject: [PATCH 16/26] Move config files for static analysis tools to test/ directory (#5871) Moves the config files for PHPCS and PHPMD to the test/ folder instead of the docs/ folder. These files aren't documents so they shouldn't be in this folder. --- {docs => test}/LorisCS.xml | 0 {docs => test}/LorisPHPMD.xml | 0 {docs => test}/SrcCS.xml | 0 test/run-php-linter.sh | 6 +++--- 4 files changed, 3 insertions(+), 3 deletions(-) rename {docs => test}/LorisCS.xml (100%) rename {docs => test}/LorisPHPMD.xml (100%) rename {docs => test}/SrcCS.xml (100%) diff --git a/docs/LorisCS.xml b/test/LorisCS.xml similarity index 100% rename from docs/LorisCS.xml rename to test/LorisCS.xml diff --git a/docs/LorisPHPMD.xml b/test/LorisPHPMD.xml similarity index 100% rename from docs/LorisPHPMD.xml rename to test/LorisPHPMD.xml diff --git a/docs/SrcCS.xml b/test/SrcCS.xml similarity index 100% rename from docs/SrcCS.xml rename to test/SrcCS.xml diff --git a/test/run-php-linter.sh b/test/run-php-linter.sh index e98d5692615..cf479c90ccc 100755 --- a/test/run-php-linter.sh +++ b/test/run-php-linter.sh @@ -9,12 +9,12 @@ find docs modules htdocs php src -name '*.class.inc' -print0 -o -name '*.php' -p # php/ # htdocs/ # modules/ -vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php,inc php/ htdocs/ modules/ || exit $?; +vendor/bin/phpcs --standard=test/LorisCS.xml --extensions=php,inc php/ htdocs/ modules/ || exit $?; # Run PHPCS on some scripts -- fixing the files format later # vendor/bin/phpcs --standard=docs/LorisCS.xml tools/CouchDB_Confirm_Integrity.php # Run PHPCS on src/ directory using a different ruleset conforming to PSR2. -vendor/bin/phpcs --standard=docs/SrcCS.xml --extensions=php/php src/ || exit $?; +vendor/bin/phpcs --standard=test/SrcCS.xml --extensions=php/php src/ || exit $?; -vendor/bin/phpmd php/libraries text docs/LorisPHPMD.xml || exit $?; +vendor/bin/phpmd php/libraries text 'test/LorisPHPMD.xml' || exit $?; From 4722dffe8902f7df210d9240007295a67ce41e8d Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Tue, 17 Dec 2019 15:06:21 -0500 Subject: [PATCH 17/26] Update CONTRIBUTING.md with new branch information (#5403) Remove references to old major/minor/bugfix branch scheme and update text to refer to our new branch system post-22 release. Fixes #5392 --- CONTRIBUTING.md | 35 ++++++----------------------------- 1 file changed, 6 insertions(+), 29 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d0db6a32a7a..d302daa074a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,35 +17,12 @@ some of the factors we'll consider when reviewing your code. ## Development branches -Please create a fork of the LORIS repository on your own GitHub account and -push changes to local branches rather than pushing new branches directly -to our repository. - -You should base your pull requests on one of the following branches -depending on the kind of change you are making: - -#### Bug Fixes - - Branch: `bugfix` - - Content: Generally these changes do not require SQL scripts - and are concise with the sole objective to correct a single problem - in the code. - -#### Minor Changes and Small Features - - Branch: `minor` - - Content: Features affecting self-contained components such - as modules. Additions to Libraries, API, or modules that do not change - any function signatures. - -#### Major Changes, Non Backwards-Compatible Changes and Large Features - - Branch: `major` - - Content: Any change modifying a function signature in a - library class. Features require extensive LORIS-wide testing. New - complex systems and features spanning across multiple modules and - libraries. Deprecated functions clean-up. - -For more information about making well-organized pull requests, -please read our in-depth Wiki page, -["Contributing to the Code"](https://github.com/aces/Loris/wiki/Contributing-to-the-Code). +For the most part, changes to the codebase should be sent to the `master` +branch, which is the default. + +Small bug fixes for a given version of LORIS should be sent to the branch named +for that version. For example, fixing a bug in LORIS version 22 should be sent +to the `22.0-release` branch. ## Pull Request Title and Description From 0e2a1515f86368c6dec92c104fd5dfa7a5772192 Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Tue, 17 Dec 2019 15:06:55 -0500 Subject: [PATCH 18/26] Cleanup pull request template [ci skip] (#5763) Remove reference to Redmine. --- .github/PULL_REQUEST_TEMPLATE.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index c4cc0e9bd1f..04924afc125 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -5,6 +5,6 @@ 1. -#### Links to related tickets (GitHub, Redmine, ...) +#### Link(s) to related issue(s) -* +* Resolves # (Reference the issue this fixes, if any.) From 718f97ca21191fc71f827509a58b0de95e8fbbe2 Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Tue, 17 Dec 2019 15:08:27 -0500 Subject: [PATCH 19/26] [Utility] Deprecate getAssociativeSiteList() (#5780) - Replaces contents of getSiteList() with getAssociativeSiteList() because that code optionally allows for the exclusion of DCC data. - Changes getAssociativeSiteList() to getSiteList() in the codebase. - getAssociativeSiteList() now throws a DeprecationException for any projects that may be using it. --- .../api/php/endpoints/candidates.class.inc | 2 +- .../ajax/get_recruitment_bar_data.php | 4 +- .../ajax/get_recruitment_line_data.php | 2 +- .../ajax/get_recruitment_pie_data.php | 2 +- modules/dashboard/ajax/get_scan_line_data.php | 2 +- modules/issue_tracker/ajax/EditIssue.php | 2 +- php/libraries/Utility.class.inc | 43 ++++++++----------- test/unittests/UtilityTest.php | 10 +---- 8 files changed, 25 insertions(+), 42 deletions(-) diff --git a/modules/api/php/endpoints/candidates.class.inc b/modules/api/php/endpoints/candidates.class.inc index 607763af072..d4105495f60 100644 --- a/modules/api/php/endpoints/candidates.class.inc +++ b/modules/api/php/endpoints/candidates.class.inc @@ -198,7 +198,7 @@ class Candidates extends Endpoint implements \LORIS\Middleware\ETagCalculator $centerid = array_search( $data['Candidate']['Site'], - \Utility::getAssociativeSiteList() + \Utility::getSiteList() ); $pscid = $data['Candidate']['PSCID'] ?? null; diff --git a/modules/dashboard/ajax/get_recruitment_bar_data.php b/modules/dashboard/ajax/get_recruitment_bar_data.php index 0756e166502..fa727e3bca7 100644 --- a/modules/dashboard/ajax/get_recruitment_bar_data.php +++ b/modules/dashboard/ajax/get_recruitment_bar_data.php @@ -3,7 +3,7 @@ * This file is used by the Dashboard to get the data for * the recruitment bar chart via AJAX * - * PHP version 5 + * PHP version 7 * * @category Main * @package Loris @@ -17,7 +17,7 @@ $DB = Database::singleton(); $sexData = array(); -$list_of_sites = Utility::getAssociativeSiteList(true, false); +$list_of_sites = Utility::getSiteList(true, false); foreach ($list_of_sites as $siteID => $siteName) { $sexData['labels'][] = $siteName; diff --git a/modules/dashboard/ajax/get_recruitment_line_data.php b/modules/dashboard/ajax/get_recruitment_line_data.php index 7920f81035c..1d1161d934f 100644 --- a/modules/dashboard/ajax/get_recruitment_line_data.php +++ b/modules/dashboard/ajax/get_recruitment_line_data.php @@ -30,7 +30,7 @@ $recruitmentData['labels'] = createChartLabels($recruitmentStartDate, $recruitmentEndDate); -$list_of_sites = Utility::getAssociativeSiteList(true, false); +$list_of_sites = Utility::getSiteList(true, false); foreach ($list_of_sites as $siteID => $siteName) { $recruitmentData['datasets'][] = array( diff --git a/modules/dashboard/ajax/get_recruitment_pie_data.php b/modules/dashboard/ajax/get_recruitment_pie_data.php index 04d772f25eb..d1a1cb34812 100644 --- a/modules/dashboard/ajax/get_recruitment_pie_data.php +++ b/modules/dashboard/ajax/get_recruitment_pie_data.php @@ -17,7 +17,7 @@ $DB = Database::singleton(); $recruitmentBySiteData = array(); -$list_of_sites = Utility::getAssociativeSiteList(true, false); +$list_of_sites = Utility::getSiteList(true, false); foreach ($list_of_sites as $siteID => $siteName) { diff --git a/modules/dashboard/ajax/get_scan_line_data.php b/modules/dashboard/ajax/get_scan_line_data.php index 665c3e96724..90e1f8ed0ca 100644 --- a/modules/dashboard/ajax/get_scan_line_data.php +++ b/modules/dashboard/ajax/get_scan_line_data.php @@ -42,7 +42,7 @@ ); $scanData['labels'] = createLineChartLabels($scanStartDate, $scanEndDate); -$list_of_sites = Utility::getAssociativeSiteList(true, false); +$list_of_sites = Utility::getSiteList(true, false); foreach ($list_of_sites as $siteID => $siteName) { $scanData['datasets'][] = array( "name" => $siteName, diff --git a/modules/issue_tracker/ajax/EditIssue.php b/modules/issue_tracker/ajax/EditIssue.php index 468c3afe091..faafd3e5f29 100644 --- a/modules/issue_tracker/ajax/EditIssue.php +++ b/modules/issue_tracker/ajax/EditIssue.php @@ -564,7 +564,7 @@ function getIssueFields() //get field options if ($user->hasPermission('access_all_profiles')) { // get the list of study sites - to be replaced by the Site object - $sites = Utility::getAssociativeSiteList(); + $sites = Utility::getSiteList(); } else { // allow only to view own site data $sites = $user->getStudySites(); diff --git a/php/libraries/Utility.class.inc b/php/libraries/Utility.class.inc index 633ca0634b4..7c4419329d4 100644 --- a/php/libraries/Utility.class.inc +++ b/php/libraries/Utility.class.inc @@ -119,14 +119,16 @@ class Utility /** * Returns a list of sites in the database * - * @param bool $study_site If true only return sites that are - * study sites according to the psc - * table + * @param boolean $study_site if true only return study sites from psc + * table + * @param boolean $DCC Whether the DCC should be included or not * * @return array an associative array("center ID" => "site name") */ - static function getSiteList(bool $study_site = true): array - { + static function getSiteList( + bool $study_site = true, + bool $DCC = true + ): array { $factory = NDB_Factory::singleton(); $DB = $factory->database(); @@ -135,6 +137,10 @@ class Utility if ($study_site) { $query .= "WHERE Study_site='Y'"; } + if (!$DCC) { + $query .= " AND CenterID <> 1"; + } + $result = $DB->pselect($query, array()); // fix the array @@ -142,7 +148,6 @@ class Utility foreach ($result as $row) { $list[$row["CenterID"]] = $row["Name"]; } - natcasesort($list); return $list; } @@ -158,31 +163,17 @@ class Utility * Note that even though CenterID is numeric, the array * should be interpreted as an associative array since the keys * refer to the centerID, not the array index. + * + * @deprecated */ static function getAssociativeSiteList( bool $study_site = true, bool $DCC = true ): array { - $factory = NDB_Factory::singleton(); - $DB = $factory->database(); - - // get the list of study sites - to be replaced by the Site object - $query = "SELECT CenterID, Name FROM psc "; - if ($study_site) { - $query .= "WHERE Study_site='Y'"; - } - if (!$DCC) { - $query .= " AND CenterID <> 1"; - } - - $result = $DB->pselect($query, array()); - - // fix the array - $list = array(); - foreach ($result as $row) { - $list[$row["CenterID"]] = $row["Name"]; - } - return $list; + throw new \DeprecatedException( + ' This function is deprecated. Please use \Utility::getSiteList()' + . ' instead.' + ); } /** diff --git a/test/unittests/UtilityTest.php b/test/unittests/UtilityTest.php index 7e2733490ca..176f1d48229 100644 --- a/test/unittests/UtilityTest.php +++ b/test/unittests/UtilityTest.php @@ -403,10 +403,9 @@ public function testGetVisitList() * TODO Potential edge cases: test with the study_site and DCC booleans as false * * @covers Utility::getSiteList() - * @covers Utility::getAssociativeSiteList * @return void */ - public function testGetSiteListAndGetAssociativeSiteList() + public function testGetSiteList() { $this->_dbMock->expects($this->any()) ->method('pselect') @@ -419,13 +418,6 @@ public function testGetSiteListAndGetAssociativeSiteList() '2' => 'site2'), Utility::getSiteList() ); - - $this->assertEquals( - array( - '1' => 'site1', - '2' => 'site2'), - Utility::getAssociativeSiteList() - ); } /** From dcb2a47eb9080966b36a679b27067740dbb97f2e Mon Sep 17 00:00:00 2001 From: Rida Abou-Haidar Date: Tue, 17 Dec 2019 15:54:06 -0500 Subject: [PATCH 20/26] Add CHANGELOG file (#5840) Add template changelog file. --- CHANGELOG.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000000..e2a2d1811df --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,16 @@ +## CHANGELOG + +- ***When adding content to this document, make sure to create a section for each module +if the changes only impact a single module and that section does not already exist in +the document. When changes affect the entire software, make sure to add them in the +core section.*** + +- ***When possible please provide the number of the pull request(s) containing the +changes in the following format: PR #1234*** + +### VERSION + +#### Core + +#### Modules +##### module1 From 98d99c22f91f7bd224957b8f3ac10cfa7af4317e Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Thu, 19 Dec 2019 09:59:46 -0500 Subject: [PATCH 21/26] Remove reference to "major" branch in Unit Test Guide (#5877) Remove references to delete branch from documentation. --- test/UnitTestGuide.md | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/test/UnitTestGuide.md b/test/UnitTestGuide.md index 7824a4b96ec..bcfb21b8513 100644 --- a/test/UnitTestGuide.md +++ b/test/UnitTestGuide.md @@ -20,9 +20,6 @@ Note that integration tests are run by Travis via GitHub and out of the scope of ## Setup -**A note on Branching:** LORIS test development should be done on the _major_ branch by convention. However links in this guide sometimes point to the branch holding the latest release (_master)_. - - ### **Setting up your Test Dev Environment** A very similar set-up guide can be found in the [README.md](https://github.com/aces/Loris/blob/master/test/README.md) in the test directory. @@ -57,7 +54,7 @@ Run the command `npm run tests:unit` to execute all unit tests. The first time t ### **How to Run Tests** -To run all unit tests under [test/unittests](https://github.com/aces/Loris/tree/major/test/unittests), use the command below. (Run this from the LORIS root directory and NOT inside the test directory.) +To run all unit tests under [test/unittests](https://github.com/aces/Loris/tree/master/test/unittests), use the command below. (Run this from the LORIS root directory and NOT inside the test directory.) `npm run tests:unit` @@ -123,19 +120,19 @@ If any tests produce a failure or error, a big red error message will appear in `"tests:unit": "./test/dockerized-unit-tests.sh"` -So, the script runs the contents of [test/dockerized-unit-tests.sh](https://github.com/aces/Loris/blob/major/test/dockerized-unit-tests.sh). If we take a look at this file, it runs the unit tests using docker-compose and vendor/bin/phpunit. It specifies which tests to run with this line: +So, the script runs the contents of [test/dockerized-unit-tests.sh](https://github.com/aces/Loris/blob/master/test/dockerized-unit-tests.sh). If we take a look at this file, it runs the unit tests using docker-compose and vendor/bin/phpunit. It specifies which tests to run with this line: `--configuration test/phpunit.xml --testsuite LorisUnitTests $*` -The list of tests to run is defined in [test/phpunit.xml](https://github.com/aces/Loris/blob/major/test/phpunit.xml) under the “LorisUnitTests” testsuite section. If you look at this testsuite block, you can see that it refers to every file in the `test/unittests/` directory! +The list of tests to run is defined in [test/phpunit.xml](https://github.com/aces/Loris/blob/master/test/phpunit.xml) under the “LorisUnitTests” testsuite section. If you look at this testsuite block, you can see that it refers to every file in the `test/unittests/` directory! ### **Troubleshooting** **General Errors:** -If the _major_ branch has been updated on the Loris repo, and your test-dev environment is now out of sync (branch and/or database) you will see seemingly unrelated errors like: +If the remote branch has been updated on the LORIS repository and your test-dev environment is now out of sync (branch and/or database) you will see seemingly unrelated errors like: Example A: @@ -150,9 +147,9 @@ ERROR: Service 'unit-tests' failed to build: The command '/bin/sh -c apt-get upd **How to fix:** -You will need to update your environment to the major branch: +You will need to update your environment to include the latest changes from LORIS. -Rebase your branch and update your database. It is convenient to reload a backup of your database and then run the patches needed to update it to _major_. **Only** reload a backup of your database if nothing else is working! +Rebase your branch and update your database. It is convenient to reload a backup of your database and then run the patches needed to update it. **Only** reload a backup of your database if nothing else is working! After rebasing, you will need to run these commands, which resets your docker-compose environment. @@ -278,7 +275,7 @@ Here is an example of this, taken from [Loris_PHPUnit_Database_TestCase.php](htt [PHPUnit documentation](https://phpunit.readthedocs.io/en/8.2/writing-tests-for-phpunit.html#data-providers) -Example Implementation: [test/unittests/UtilityTest.php::testCalculateAgeFormat](https://github.com/aces/Loris/blob/major/test/unittests/UtilityTest.php#L200) +Example Implementation: [test/unittests/UtilityTest.php::testCalculateAgeFormat](https://github.com/aces/Loris/blob/master/test/unittests/UtilityTest.php#L200) Data providers are used to provide an array of different inputs to a test. @@ -432,7 +429,7 @@ This will make testing a lot easier. See issues #[4989](https://github.com/aces/ **1. With the ‘pselect’-style database method** - Example implementation: [test/unittests/UtilityTest.php](https://github.com/aces/Loris/blob/major/test/unittests/UtilityTest.php) -- Not including the first 2 tests! + Example implementation: [test/unittests/UtilityTest.php](https://github.com/aces/Loris/blob/master/test/unittests/UtilityTest.php) -- Not including the first 2 tests! @@ -519,7 +516,7 @@ public function testExample() **2. With the ‘setFakeTableData’ database method** -Example implementation: [test/unittests/UserTest.php](https://github.com/aces/Loris/blob/major/test/unittests/UserTest.php) +Example implementation: [test/unittests/UserTest.php](https://github.com/aces/Loris/blob/master/test/unittests/UserTest.php) **This is not a ‘pure’ unit test because it does not use a mock database! Instead, you are essentially creating a database object and inputting fake tables into it, which means that these tests usually take longer to execute.** @@ -620,7 +617,7 @@ $this->_dbMock->setFakeTableData( **Important:** -**The table should only be added once**. So, setFakeTableData should never be included in the setUp method because you will get a “Table already exists” error after the first test is run. If you create some helper method, like “_setUpFakeTables” that adds tables that will be used in every test, **it should still only be run once**, like in the first test. See [test/unittests/UserTest.php](https://github.com/aces/Loris/blob/major/test/unittests/UserTest.php) on the major branch for an example. +**The table should only be added once**. So, setFakeTableData should never be included in the setUp method because you will get a “Table already exists” error after the first test is run. If you create some helper method, like “_setUpFakeTables” that adds tables that will be used in every test, **it should still only be run once**, like in the first test. See [test/unittests/UserTest.php](https://github.com/aces/Loris/blob/master/test/unittests/UserTest.php) for an example. Once the tables that the query uses are added, you can test the method as normal, and the query should run on the “fake” database you’ve created! From 994554dcbd25001bc708adb20b8965d160520b6b Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Thu, 19 Dec 2019 10:00:54 -0500 Subject: [PATCH 22/26] [Core] Update package-lock file (#5879) Switch repositories to HTTPS --- package-lock.json | 51 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/package-lock.json b/package-lock.json index f8610765697..a52dbeb74c2 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1828,7 +1828,7 @@ }, "buffer": { "version": "4.9.1", - "resolved": "http://registry.npmjs.org/buffer/-/buffer-4.9.1.tgz", + "resolved": "https://registry.npmjs.org/buffer/-/buffer-4.9.1.tgz", "integrity": "sha1-bRu2AbB6TvztlwlBMgkwJ8lbwpg=", "dev": true, "requires": { @@ -3535,7 +3535,8 @@ "ansi-regex": { "version": "2.1.1", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "aproba": { "version": "1.2.0", @@ -3556,12 +3557,14 @@ "balanced-match": { "version": "1.0.0", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "brace-expansion": { "version": "1.1.11", "bundled": true, "dev": true, + "optional": true, "requires": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" @@ -3576,17 +3579,20 @@ "code-point-at": { "version": "1.1.0", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "concat-map": { "version": "0.0.1", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "console-control-strings": { "version": "1.1.0", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "core-util-is": { "version": "1.0.2", @@ -3703,7 +3709,8 @@ "inherits": { "version": "2.0.3", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "ini": { "version": "1.3.5", @@ -3715,6 +3722,7 @@ "version": "1.0.0", "bundled": true, "dev": true, + "optional": true, "requires": { "number-is-nan": "^1.0.0" } @@ -3729,6 +3737,7 @@ "version": "3.0.4", "bundled": true, "dev": true, + "optional": true, "requires": { "brace-expansion": "^1.1.7" } @@ -3736,12 +3745,14 @@ "minimist": { "version": "0.0.8", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "minipass": { "version": "2.3.5", "bundled": true, "dev": true, + "optional": true, "requires": { "safe-buffer": "^5.1.2", "yallist": "^3.0.0" @@ -3760,6 +3771,7 @@ "version": "0.5.1", "bundled": true, "dev": true, + "optional": true, "requires": { "minimist": "0.0.8" } @@ -3840,7 +3852,8 @@ "number-is-nan": { "version": "1.0.1", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "object-assign": { "version": "4.1.1", @@ -3852,6 +3865,7 @@ "version": "1.4.0", "bundled": true, "dev": true, + "optional": true, "requires": { "wrappy": "1" } @@ -3937,7 +3951,8 @@ "safe-buffer": { "version": "5.1.2", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "safer-buffer": { "version": "2.1.2", @@ -3973,6 +3988,7 @@ "version": "1.0.2", "bundled": true, "dev": true, + "optional": true, "requires": { "code-point-at": "^1.0.0", "is-fullwidth-code-point": "^1.0.0", @@ -3992,6 +4008,7 @@ "version": "3.0.1", "bundled": true, "dev": true, + "optional": true, "requires": { "ansi-regex": "^2.0.0" } @@ -4035,12 +4052,14 @@ "wrappy": { "version": "1.0.2", "bundled": true, - "dev": true + "dev": true, + "optional": true }, "yallist": { "version": "3.0.3", "bundled": true, - "dev": true + "dev": true, + "optional": true } } }, @@ -4870,7 +4889,7 @@ }, "minimist": { "version": "0.0.8", - "resolved": "http://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz", "integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0=", "dev": true }, @@ -4897,7 +4916,7 @@ }, "mkdirp": { "version": "0.5.1", - "resolved": "http://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz", + "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz", "integrity": "sha1-MAV0OOrGz3+MR2fzhkjWaX11yQM=", "dev": true, "requires": { @@ -5048,7 +5067,7 @@ }, "readable-stream": { "version": "2.3.6", - "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.6.tgz", + "resolved": "http://registry.npmjs.org/readable-stream/-/readable-stream-2.3.6.tgz", "integrity": "sha512-tQtKA9WIAhBF3+VLAseyMqZeBjW0AHJoxOtYqSUZNJxauErmLbVm2FW1y+J/YA9dUrAC39ITejlZWhVIwawkKw==", "dev": true, "requires": { @@ -6623,7 +6642,7 @@ }, "through": { "version": "2.3.8", - "resolved": "http://registry.npmjs.org/through/-/through-2.3.8.tgz", + "resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz", "integrity": "sha1-DdTJ/6q8NXlgsbckEV1+Doai4fU=", "dev": true }, From 939f6770157ff329cca5b7b8aa3401a733bd1889 Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Thu, 19 Dec 2019 10:01:52 -0500 Subject: [PATCH 23/26] Move CentOS install instructions to ReadTheDocs (#5872) Moves the README for CentOS into the new documentation structure. --- .../01 - LORIS Install/CentOS/README.CentOS7.md | 0 .../01 - LORIS Install/CentOS/README.md | 1 - 2 files changed, 1 deletion(-) rename README.CentOS7.md => docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.CentOS7.md (100%) delete mode 100644 docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.md diff --git a/README.CentOS7.md b/docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.CentOS7.md similarity index 100% rename from README.CentOS7.md rename to docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.CentOS7.md diff --git a/docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.md b/docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.md deleted file mode 100644 index 7811fb32de0..00000000000 --- a/docs/wiki/00 - SERVER INSTALL AND CONFIGURATION/01 - LORIS Install/CentOS/README.md +++ /dev/null @@ -1 +0,0 @@ -CentOS specific instructions \ No newline at end of file From 1702589b7e03e05df83c7d86ce6695ba32d38158 Mon Sep 17 00:00:00 2001 From: John Saigle <4022790+johnsaigle@users.noreply.github.com> Date: Fri, 20 Dec 2019 11:03:11 -0500 Subject: [PATCH 24/26] [ReadTheDocs] Add Subprojects & Timepoints to nav bar (#5865) --- mkdocs.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mkdocs.yml b/mkdocs.yml index 7cd5c3f797b..9eab792d9b0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -14,6 +14,8 @@ nav: - 'Introduction': 'wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/00 - Introduction to Study Variables.md' - 'Projects': 'wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/02 - Projects.md' - 'Sites': 'wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/03 - Sites.md' + - 'Subprojects': 'wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/04 - Subprojects.md' + - 'Timepoints': 'wiki/01 - STUDY PARAMETERS SETUP/01 - Study Variables/05 - Timepoints.md' - Clinical Instruments: 'wiki/01 - STUDY PARAMETERS SETUP/02 - Clinical Instruments/README.md' - Modules: 'wiki/01 - STUDY PARAMETERS SETUP/03 - Loris Modules/README.md' From 6c290e4039eed2d666908f05603958e67876c342 Mon Sep 17 00:00:00 2001 From: Rida Abou-Haidar Date: Fri, 20 Dec 2019 12:34:47 -0500 Subject: [PATCH 25/26] [data_release/document_repository] Make upload directory configurable (#5815) This gives the option to the users to configure where the default upload directory of data_release and document_repository modules will be. this also removes the burden on LORIS to ensure these locations are readable/writeable and follows the standard established by the other modules uploading data to loris. --- SQL/0000-00-03-ConfigTables.sql | 2 ++ ...-29-Add_upload_directory_configuration.sql | 9 ++++++++ modules/data_release/README.md | 9 +------- modules/data_release/ajax/FileUpload.php | 21 ++++++++--------- modules/data_release/ajax/GetFile.php | 10 +++++--- .../document_repository/php/files.class.inc | 23 ++++++++++++++----- php/libraries/Utility.class.inc | 12 ++++++++++ raisinbread/RB_files/RB_Config.sql | 2 ++ raisinbread/RB_files/RB_ConfigSettings.sql | 2 ++ raisinbread/migration.md | 2 ++ tools/install.sh | 11 --------- 11 files changed, 64 insertions(+), 39 deletions(-) create mode 100644 SQL/New_patches/2019-11-29-Add_upload_directory_configuration.sql diff --git a/SQL/0000-00-03-ConfigTables.sql b/SQL/0000-00-03-ConfigTables.sql index 8f341181b3e..e2d69cf173e 100644 --- a/SQL/0000-00-03-ConfigTables.sql +++ b/SQL/0000-00-03-ConfigTables.sql @@ -75,6 +75,8 @@ INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'publication_uploads', 'Path to uploaded publications', 1, 0, 'web_path', ID, 'Publications', 10 FROM ConfigSettings WHERE Name="paths"; INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'publication_deletions', 'Path to deleted publications', 1, 0, 'web_path', ID, 'Deleted Publications', 11 FROM ConfigSettings WHERE Name="paths"; INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'MINCToolsPath', 'Path to the MINC tools', 1, 0, 'web_path', ID, 'Path to the MINC tools', 12 FROM ConfigSettings WHERE Name="paths"; +INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'documentRepositoryPath', 'Path to uploaded document repository files', 1, 0, 'web_path', ID, 'Document Repository Upload Path', 13 FROM ConfigSettings WHERE Name="paths"; +INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'dataReleasePath', 'Path to uploaded data release files', 1, 0, 'web_path', ID, 'Data release Upload Path', 14 FROM ConfigSettings WHERE Name="paths"; INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, Label, OrderNumber) VALUES ('gui', 'Settings related to the overall display of LORIS', 1, 0, 'GUI', 3); diff --git a/SQL/New_patches/2019-11-29-Add_upload_directory_configuration.sql b/SQL/New_patches/2019-11-29-Add_upload_directory_configuration.sql new file mode 100644 index 00000000000..f056928a409 --- /dev/null +++ b/SQL/New_patches/2019-11-29-Add_upload_directory_configuration.sql @@ -0,0 +1,9 @@ +INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'documentRepositoryPath', 'Path to uploaded document repository files', 1, 0, 'text', cs1.ID, 'Document Repository Upload Path', MAX(cs2.OrderNumber)+1 FROM ConfigSettings cs1 JOIN ConfigSettings cs2 WHERE cs1.Name="paths" AND cs2.parent=cs1.ID; +INSERT INTO ConfigSettings (Name, Description, Visible, AllowMultiple, DataType, Parent, Label, OrderNumber) SELECT 'dataReleasePath', 'Path to uploaded data release files', 1, 0, 'text', cs1.ID, 'Data Release Upload Path', MAX(cs2.OrderNumber)+1 FROM ConfigSettings cs1 JOIN ConfigSettings cs2 WHERE cs1.Name="paths" AND cs2.parent=cs1.ID; + +-- For backwards compatibility, check the previous base and default to same folder as previous setting +SELECT Value INTO @base FROM Config c JOIN ConfigSettings cs ON cs.ID=c.ConfigID WHERE cs.Name="base"; + +INSERT INTO Config (ConfigID, Value) SELECT ID, CONCAT(@base,"modules/document_repository/user_uploads/") FROM ConfigSettings WHERE Name="documentRepositoryPath"; +INSERT INTO Config (ConfigID, Value) SELECT ID, CONCAT(@base,"modules/data_release/user_uploads/") FROM ConfigSettings WHERE Name="dataReleasePath"; + diff --git a/modules/data_release/README.md b/modules/data_release/README.md index b5927a21a02..9678ea2e209 100644 --- a/modules/data_release/README.md +++ b/modules/data_release/README.md @@ -44,15 +44,8 @@ Note: At the moment, the only way to remove a user's permission to a specific ## Configurations -- Data release uploads are stored under the - `modules/data_release/user_uploads directory`, which can easily be symlinked - to another location if necessary. Note that this directory needs to be - writable by your web server. +- `dataReleasePath` designates the target location for release file uploads. ## Other notes: -- Uploads are stored under the `modules/data_release/user_uploads` directory which -can easily be symlinked to another location if necessary, ensure that it can be written -to by your web server. -- Remove permissions by deleting rows in the data_release_permissions table. - Upload date will automatically be added during file upload. diff --git a/modules/data_release/ajax/FileUpload.php b/modules/data_release/ajax/FileUpload.php index 908718401ec..68a94df3f09 100644 --- a/modules/data_release/ajax/FileUpload.php +++ b/modules/data_release/ajax/FileUpload.php @@ -12,8 +12,10 @@ * @link https://github.com/aces/Loris */ -$DB = \Database::singleton(); -$user = \User::singleton(); +$factory = \NDB_Factory::singleton(); +$DB = $factory->database(); +$user = $factory->user(); +$config = $factory->config(); if ($_POST['action'] == 'upload' && $user->hasPermission("data_release_upload") @@ -21,27 +23,26 @@ $fileName = $_FILES["file"]["name"]; $version = $_POST['version']; $upload_date = date('Y-m-d'); - $base_path = __DIR__ . "/../user_uploads/"; - $factory = NDB_Factory::singleton(); $settings = $factory->settings(); $baseURL = $settings->getBaseURL(); + $path = \Utility::appendForwardSlash($config->getSetting('dataReleasePath')); - if (!file_exists(__DIR__ . "/../user_uploads/")) { + if (!file_exists($path)) { error_log( - "ERROR: File upload failed. Default user_uploads" + "ERROR: File upload failed. Upload" . " directory not found." ); header("HTTP/1.1 500 Internal Server Error"); - } elseif (!is_writable(__DIR__ . "/../user_uploads/")) { + } elseif (!is_writable($path)) { error_log( - "File upload failed. Default user_uploads directory" + "File upload failed. Upload directory" . " does not appear to be writeable." ); header("HTTP/1.1 500 Internal Server Error"); } else { - $target_path = $base_path . $fileName; + $target_path = $path . $fileName; if (move_uploaded_file($_FILES["file"]["tmp_name"], $target_path)) { $DB->insert( 'data_release', @@ -82,5 +83,3 @@ header("HTTP/1.1 400 Bad Request"); echo "There was an error uploading the file"; } - - diff --git a/modules/data_release/ajax/GetFile.php b/modules/data_release/ajax/GetFile.php index cc29403c89b..28bc800653a 100644 --- a/modules/data_release/ajax/GetFile.php +++ b/modules/data_release/ajax/GetFile.php @@ -12,7 +12,12 @@ * @link https://github.com/aces/Loris */ -$user =& User::singleton(); + +$factory = \NDB_Factory::singleton(); +$db = $factory->database(); +$user = $factory->user(); +$config = $factory->config(); +$path = \Utility::appendForwardSlash($config->getSetting('dataReleasePath')); $File = $_GET['File']; // Make sure that the user isn't trying to break out of the $path by @@ -23,13 +28,12 @@ header("HTTP/1.1 400 Bad Request"); exit(4); } -$FullPath = __DIR__ . "/../user_uploads/$File"; +$FullPath = $path . $File; if (!file_exists($FullPath)) { error_log("ERROR: File $FullPath does not exist"); header("HTTP/1.1 404 Not Found"); exit(5); } -$db =& Database::singleton(); $fileID = $db->pselectOne( "SELECT ID FROM data_release WHERE " . "file_name=:fn", diff --git a/modules/document_repository/php/files.class.inc b/modules/document_repository/php/files.class.inc index 5204a565dc0..6c5f42df096 100644 --- a/modules/document_repository/php/files.class.inc +++ b/modules/document_repository/php/files.class.inc @@ -67,6 +67,10 @@ class Files extends \NDB_Page */ public function handle(ServerRequestInterface $request) : ResponseInterface { + $config = \NDB_Factory::singleton()->config(); + $path = \Utility::appendForwardSlash( + $config->getSetting("documentRepositoryPath") + ); switch ($request->getMethod()) { case "POST": if ($this->uploadDocFile($request)) { @@ -107,7 +111,7 @@ class Files extends \NDB_Page $name = \User::singleton()->getUsername(); $record = urldecode(basename($request->getUri()->getPath())); if (!is_numeric($record)) { - $file = __DIR__ . "/../user_uploads/$name/$record"; + $file = $path. "$name/$record"; return (new \LORIS\Http\Response()) ->withHeader('Content-Type', 'application/octet-stream') ->withHeader( @@ -171,14 +175,18 @@ class Files extends \NDB_Page */ function deleteFile($rid): void { + $factory = \NDB_Factory::singleton(); + $DB = $factory->database(); + $user = $factory->user(); + $config = $factory->config(); + $path = \Utility::appendForwardSlash( + $config->getSetting("documentRepositoryPath") + ); // create Database object - $DB = \Database::singleton(); - $user = \User::singleton(); $Notifier = new \NDB_Notifier( "document_repository", "delete" ); - $factory = \NDB_Factory::singleton(); $baseURL = $factory->settings()->getBaseURL(); $fileName = $DB->pselectOne( "SELECT File_name FROM document_repository @@ -205,7 +213,7 @@ class Files extends \NDB_Page $Notifier->notify($msg_data); } - $path = __DIR__ . "/../user_uploads/$dataDir"; + $path = $path.$dataDir; if (file_exists($path)) { unlink($path); @@ -349,6 +357,9 @@ class Files extends \NDB_Page $baseURL = $factory->settings()->getBaseURL(); $config = $factory->config(); $base = $config->getSetting('base'); + $path = \Utility::appendForwardSlash( + $config->getSetting("documentRepositoryPath") + ); $name = \User::singleton()->getUsername(); $DB = \Database::singleton(); $category = $req['category']; // required @@ -368,7 +379,7 @@ class Files extends \NDB_Page $fileSize = $uploadedFile->getSize(); $fileName = $uploadedFile->getClientFileName(); $fileType = pathinfo($fileName, PATHINFO_EXTENSION); - $uploadPath = "$base/modules/document_repository/user_uploads/$name/"; + $uploadPath = $path.$name."/"; // $category is a string representation of an ID, and so should be at // least equal to zero. if (intval($category) < 0) { diff --git a/php/libraries/Utility.class.inc b/php/libraries/Utility.class.inc index 7c4419329d4..f86fb94f02e 100644 --- a/php/libraries/Utility.class.inc +++ b/php/libraries/Utility.class.inc @@ -996,5 +996,17 @@ class Utility return $scan_types; } + + /** + * Append a forward slash to a path if it doesn't already exist + * + * @param string $path path to which the slash should be appended + * + * @return string + */ + static function appendForwardSlash(string $path) : string + { + return rtrim($path, '/\\') . '/'; + } } diff --git a/raisinbread/RB_files/RB_Config.sql b/raisinbread/RB_files/RB_Config.sql index 4cad972980e..f62cdcc71d6 100644 --- a/raisinbread/RB_files/RB_Config.sql +++ b/raisinbread/RB_files/RB_Config.sql @@ -94,5 +94,7 @@ INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (97,70,'/data-raisinbrea INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (98,93,'V1'); INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (99,101,''); INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (102,19,'false'); +INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (103,102,'/data/document_repository_uploads/'); +INSERT INTO `Config` (`ID`, `ConfigID`, `Value`) VALUES (104,103,'/data/data_release_uploads/'); UNLOCK TABLES; SET FOREIGN_KEY_CHECKS=1; diff --git a/raisinbread/RB_files/RB_ConfigSettings.sql b/raisinbread/RB_files/RB_ConfigSettings.sql index 5eadfcc5df5..b398d819b02 100644 --- a/raisinbread/RB_files/RB_ConfigSettings.sql +++ b/raisinbread/RB_files/RB_ConfigSettings.sql @@ -98,5 +98,7 @@ INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMult INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMultiple`, `DataType`, `Parent`, `Label`, `OrderNumber`) VALUES (99,'usePwnedPasswordsAPI','Whether to query the Have I Been Pwned password API on password changes to prevent the usage of common and breached passwords',1,0,'boolean',1,'Enable \"Pwned Password\" check',22); INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMultiple`, `DataType`, `Parent`, `Label`, `OrderNumber`) VALUES (100,'EnvironmentFile','Name of the environment file that need to be sourced for the imaging pipeline',1,0,'text',69,'Name of the environment file',20); INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMultiple`, `DataType`, `Parent`, `Label`, `OrderNumber`) VALUES (101,'MINCToolsPath','Path to the MINC tools',1,0,'web_path',26,'Path to the MINC tools',12); +INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMultiple`, `DataType`, `Parent`, `Label`, `OrderNumber`) VALUES (102,'documentRepositoryPath','Path to uploaded document repository files',1,0,'text',26,'Document Repository Upload Path',13); +INSERT INTO `ConfigSettings` (`ID`, `Name`, `Description`, `Visible`, `AllowMultiple`, `DataType`, `Parent`, `Label`, `OrderNumber`) VALUES (103,'dataReleasePath','Path to uploaded data release files',1,0,'text',26,'Data Release Upload Path',14); UNLOCK TABLES; SET FOREIGN_KEY_CHECKS=1; diff --git a/raisinbread/migration.md b/raisinbread/migration.md index 0dabaf770e1..5fb3f154c62 100644 --- a/raisinbread/migration.md +++ b/raisinbread/migration.md @@ -55,5 +55,7 @@ 2019-11-25-Default_value_for_session_submitted.sql # NEW +2019-10-09_move_MINCToolsPath_configuration_to_Config_tables.sql +2019-11-29-Add_upload_directory_configuration.sql # CLEAN-UP \ No newline at end of file diff --git a/tools/install.sh b/tools/install.sh index 875abe353cf..3c8aa755791 100755 --- a/tools/install.sh +++ b/tools/install.sh @@ -117,7 +117,6 @@ mkdir -p ../smarty/templates_c # Setting 770 permissions for templates_c chmod 770 ../smarty/templates_c -# Changing group to 'www-data' or 'apache' to give permission to create directories in Document Repository module # Detecting distribution if ! os_distro=$(hostnamectl 2>/dev/null) then @@ -130,20 +129,12 @@ debian=("Debian" "Ubuntu") redhat=("Red" "CentOS" "Fedora" "Oracle") if [[ " ${debian[*]} " =~ " $os_distro " ]]; then - mkdir -p ../modules/document_repository/user_uploads - mkdir -p ../modules/data_release/user_uploads - sudo chown www-data.www-data ../modules/document_repository/user_uploads - sudo chown www-data.www-data ../modules/data_release/user_uploads sudo chown www-data.www-data ../smarty/templates_c # Make Apache the group for project directory, so that the web based install # can write the config.xml file. sudo chgrp www-data ../project sudo chmod 770 ../project elif [[ " ${redhat[*]} " =~ " $os_distro " ]]; then - mkdir -p ../modules/document_repository/user_uploads - mkdir -p ../modules/data_release/user_uploads - sudo chown apache.apache ../modules/document_repository/user_uploads - sudo chown apache.apache ../modules/data_release/user_uploads sudo chown apache.apache ../smarty/templates_c # Make Apache the group for project directory, so that the web based install # can write the config.xml file. @@ -152,8 +143,6 @@ elif [[ " ${redhat[*]} " =~ " $os_distro " ]]; then else echo "$os_distro Linux distribution detected. We currently do not support this. " echo "Please manually change subdirectory ownership and permissions to ensure the web server can read *and write* in the following: " - echo "../modules/data_release/user_uploads " - echo "../modules/document_repository/user_uploads " echo "../smarty/templates_c " echo "" fi From 79c05e6f89ccd3bcb21921c206e93e41f5bed5d4 Mon Sep 17 00:00:00 2001 From: Dave MacFarlane Date: Fri, 20 Dec 2019 12:37:15 -0500 Subject: [PATCH 26/26] [Dashboard] Make welcome message more enthusiastic. (#5878) "Welcome, $name." doesn't sound very welcoming. "Welcome, $name!" sounds more welcoming. --- modules/dashboard/templates/form_dashboard.tpl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/dashboard/templates/form_dashboard.tpl b/modules/dashboard/templates/form_dashboard.tpl index 009d33f98ca..e1ad5f33898 100644 --- a/modules/dashboard/templates/form_dashboard.tpl +++ b/modules/dashboard/templates/form_dashboard.tpl @@ -6,7 +6,7 @@ -Welcome, {$username}.
+Welcome, {$username}!
Last login: {$last_login}
{if !is_null($project_description)}{$project_description}