-
Notifications
You must be signed in to change notification settings - Fork 1
feat: Implement taxonomy sample data generation #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
yusuf-musleh
merged 10 commits into
main
from
yusuf-musleh/implement-generate-taxonomy-data
Oct 15, 2023
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
4b10229
feat: Add creating sample orgs, courses + populate
yusuf-musleh d0c27cf
feat: Add sample taxonomies generation for orgs
yusuf-musleh f226060
feat: Add sample tag generation for taxonomies
yusuf-musleh d6709e2
feat: Add tags togenerated courses + components
yusuf-musleh 62b95ab
feat: Add Open Canada Taxonomy data as sample
yusuf-musleh e6cfeaa
feat: Add LightCast Taxonomy data + refactor
yusuf-musleh d0ace90
docs: Add README.md
yusuf-musleh 32431b3
feat: Remove hardcoded path + Update run command
yusuf-musleh 506f2cb
docs: Add License information + note on USER_EMAIL
yusuf-musleh f7e51b0
feat: Make LightCast/Canada Skill Taxonomy all org
yusuf-musleh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Taxonomy Sample Data | ||
|
|
||
| The purpose of this repo is to generate Sample and Real World Taxonomy Data that can be used when testing and developing on the edx-platform. | ||
|
|
||
| Running this script will do the following: | ||
|
|
||
| 1. 2 Test Organizations will be created | ||
| 1. For each organization created, a sample course is created that is populated with a course export that contains a variety of section/subsections/units and components | ||
| 1. For each of these created organizations the following taxonomies will be created: | ||
| 1. a disabled taxonomy with 10 Tags | ||
| 1. an enabled flat taxonomy with 5000 Tags | ||
| 1. an enabled hierarchical taxonomy with three levels and 4^x tags per level (4 root tags, each with 16 child tags, each with 64 grandchild tags) | ||
| 1. a small enabled taxonomy with 2 levels with 2 Tags each | ||
| 1. A multi org Taxonomy is created and enabled/used by both orgs | ||
| 1. (Optional) A 4 level Taxonomy containing data obtained from [Open Canada Taxonomy](https://open.canada.ca/data/en/dataset/6093c709-2a0d-4c23-867e-27987a79212c/resource/0a120b15-9708-4d8a-8af2-2431c4540c0b) | ||
| 1. (Optional) A 3 level Taxonomy containing data obtained from [LightCast Skills Taxonomy](https://docs.google.com/spreadsheets/d/1DA3JfpBE5Krc0daImuu5Y0nsH93PEfdrWRrEa-sR-6k/edit#gid=1319222368) | ||
| 1. Once the Taxonomies and their Tags have been created, the script will Tag each of the courses along with all the components they contain with 1 of each Tag from the the above | ||
|
|
||
| **Note:** This script is designed to be idempotent. Meaning that the end state is the same every time you run it. So if you make modifications to the sample courses on Studio or the Taxonomy data in the shell and run this script again, it will reset all your changes. | ||
|
|
||
|
|
||
| ### Getting Started | ||
|
|
||
| 1. To begin, clone this repo inside a directory so it can be accessed from within the devstack (eg: `/edx/src/`) | ||
| 1. Set the `TAXONOMY_SAMPLE_PATH` variable inside the `generate.py` file to point to the cloned repo inside your devstack/tutor environment, eg: `/edx/src/taxonomy-sample-data` | ||
| 1. Set the `USER_EMAIL` variable in `generate.py` to the user the courses/units will be created under. It defaults to `edx@example.com`. | ||
| 1. (Optional) If you would not like to include taxonomy data from real world examples, such as: | ||
| - [Open Canada Taxonomy](https://open.canada.ca/data/en/dataset/6093c709-2a0d-4c23-867e-27987a79212c/resource/0a120b15-9708-4d8a-8af2-2431c4540c0b) | ||
| - [LightCast Skills Taxonomy](https://docs.google.com/spreadsheets/d/1DA3JfpBE5Krc0daImuu5Y0nsH93PEfdrWRrEa-sR-6k/edit#gid=1319222368) | ||
|
|
||
| Then set the following flags to `False` in `generate.py` accordingly: | ||
|
|
||
| ```py | ||
| IMPORT_OPEN_CANADA_TAXONOMY = True | ||
| IMPORT_LIGHTCAST_SKILLS_TAXONOMY = True | ||
| ``` | ||
|
|
||
| 1. To run the script, enter the LMS shell (`make lms-shell`) and run the following command: | ||
| ```sh | ||
| python manage.py cms shell < /path/to/taxonomy-sample-data/generate.py | ||
| ``` | ||
|
|
||
| ### License Information | ||
|
|
||
| This includes a copy of the "[Skills and Competencies Taxonomy Data](https://open.canada.ca/data/en/dataset/6093c709-2a0d-4c23-867e-27987a79212c)" from Employment and Social Development Canada, which is under the [Open Government Licence - Canada](https://open.canada.ca/en/open-government-licence-canada). | ||
|
|
||
| This includes a copy of the [Lightcast Open Skills Taxonomy](https://lightcast.io/open-skills), which is "free for individual and not-for-profit use" and is available under the [Lightcast Open Skills Terms of Use](https://lightcast.io/downloads/lightcast-open-skills-license-agreement.pdf). | ||
Binary file not shown.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.