Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion lms/envs/production.py
Original file line number Diff line number Diff line change
Expand Up @@ -445,10 +445,18 @@ def get_env_setting(setting):

##### Custom Courses for EdX #####
if FEATURES['CUSTOM_COURSES_EDX']:
INSTALLED_APPS += ['lms.djangoapps.ccx', 'openedx.core.djangoapps.ccxcon.apps.CCXConnectorConfig']
INSTALLED_APPS += [
'lms.djangoapps.ccx',
'openedx.core.djangoapps.ccxcon.apps.CCXConnectorConfig',
'cms.djangoapps.contentstore.apps.ContentstoreConfig',
'openedx.core.djangoapps.content.search',
'openedx.core.djangoapps.content_staging',
Comment on lines +451 to +453

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @Anas12091101. It makes sense that CCX outline creation is failing because it's happening in the LMS, whereas outline regeneration signal handler is part of the CMS. But, adding CMS apps to the LMS is not a viable solution--we need to keep separation between these two systems. Enabling these apps to fix this bug will over time lead to a dependency cascade of CMS that need to be turned on in the LMS.

I'm not sure yet what the right solution is. @ormsbee , do you have thoughts?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked at what CCX does in a while, but my initial inclination is to actually add CCX to CMS, and stop the cross-process thing where we emit course_published on the CMS side to spawn celery workers on the LMS side. It's been the cause of a number of annoying-to-track down bugs in things like block transformers, particularly since a lot of dev/test runs celery in-process.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Anas12091101 Can you elaborate on what it means that CCX is published in LMS? Is there code you can point to? Towards Dave's point, maybe we can adjust this publishing to occur in the CMS process.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kdmccormick, the course publish signals are fired from
https://github.com/openedx/edx-platform/blob/master/lms/djangoapps/ccx/views.py#L333-L336

These are the logs printed after I clicked the save button:

2025-07-23 15:56:31 2025-07-23 10:56:31,148 INFO 58 [tracking] [user 200050] [ip 172.21.0.1] logger.py:41 - {"name": "/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/save_ccx", "context": {"course_id": "ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2", "course_user_tags": {}, "user_id": 200050, "path": "/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/save_ccx", "org_id": "UAI_SOURCE", "enterprise_uuid": ""}, "username": "anas_", "session": "f863d7d21d0bf519e5b474419dd7d8ac", "ip": "172.21.0.1", "agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36", "host": "local.openedx.io:8000", "referer": "http://local.openedx.io:8000/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/ccx_coach", "accept_language": "en-US,en;q=0.9", "event": "{\"GET\": {}, \"POST\": {}}", "time": "2025-07-23T10:56:31.147584+00:00", "event_type": "/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/save_ccx", "event_source": "server", "page": null}
2025-07-23 15:56:31 2025-07-23 10:56:31,266 INFO 58 [openedx.core.djangoapps.cors_csrf.helpers] [user 200050] [ip 172.21.0.1] helpers.py:64 - Origin 'http://local.openedx.io:8000' was not in `CORS_ORIGIN_WHITELIST`; full referer was 'http://local.openedx.io:8000/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/ccx_coach' and requested host was 'local.openedx.io:8000'; CORS_ORIGIN_ALLOW_ALL=False
2025-07-23 15:56:31 2025-07-23 10:56:31,407 INFO 58 [openedx.core.djangoapps.content.course_overviews.models] [user 200050] [ip 172.21.0.1] models.py:299 - Attempting to load CourseOverview for course ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2 from modulestore.
2025-07-23 15:56:31 2025-07-23 10:56:31,423 INFO 58 [openedx.core.djangoapps.content.course_overviews.models] [user 200050] [ip 172.21.0.1] models.py:197 - Updating course overview for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2.
2025-07-23 15:56:31 2025-07-23 10:56:31,707 INFO 58 [openedx.core.djangoapps.content.block_structure.models] [user 200050] [ip 172.21.0.1] models.py:260 - BlockStructure: Deleted 1 out of total 6 files in store; data_usage_key: ccx-block-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2+type@course+block@course, num_to_keep: 5.
2025-07-23 15:56:31 2025-07-23 10:56:31,708 INFO 58 [openedx.core.djangoapps.content.block_structure.models] [user 200050] [ip 172.21.0.1] models.py:296 - BlockStructure: Updated in store <class 'django.core.files.storage.filesystem.FileSystemStorage'> at /openedx/media; data_usage_key: ccx-block-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2+type@course+block@course, data_version: 687f4901712771d041005ce0, data_edit_timestamp: 2025-07-22 08:16:57.009000+00:00, transformers_schema_version: UbgW6dMpqAfez+wDHqiK7iIZOMU=, block_structure_schema_version: 2, size: 2678
2025-07-23 15:56:31 2025-07-23 10:56:31,711 INFO 58 [openedx.core.djangoapps.content.block_structure.store] [user 200050] [ip 172.21.0.1] store.py:144 - BlockStructure: Added to cache; data_usage_key: ccx-block-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2+type@course+block@course, data_version: 687f4901712771d041005ce0, data_edit_timestamp: 2025-07-22 08:16:57.009000+00:00, transformers_schema_version: UbgW6dMpqAfez+wDHqiK7iIZOMU=, block_structure_schema_version: 2, size: 0.00MB
2025-07-23 15:56:31 2025-07-23 10:56:31,828 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task openedx.core.djangoapps.content.block_structure.tasks.update_course_in_cache_v2[c32d00b4-402c-4e22-b399-224f9a502814] succeeded in 0.3449637449812144s: None
2025-07-23 15:56:31 2025-07-23 10:56:31,836 INFO 58 [openedx.core.djangoapps.course_date_signals.handlers] [user 200050] [ip 172.21.0.1] handlers.py:125 - Extracting course dates for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:31 2025-07-23 10:56:31,852 INFO 58 [openedx.core.djangoapps.course_date_signals.handlers] [user 200050] [ip 172.21.0.1] handlers.py:159 - Extracting dates from 23 items in ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:31 2025-07-23 10:56:31,858 INFO 58 [edx_when.api] [user 200050] [ip 172.21.0.1] api.py:94 - Setting date for CCXBlockUsageLocator(CCXLocator('UAI_SOURCE', 'CS11011', '2019_t5', None, None, '2'), 'course', 'course'), start, datetime.datetime(2025, 1, 1, 0, 0, tzinfo=tzlocal())
2025-07-23 15:56:31 2025-07-23 10:56:31,867 INFO 58 [edx_when.api] [user 200050] [ip 172.21.0.1] api.py:94 - Setting date for CCXBlockUsageLocator(CCXLocator('UAI_SOURCE', 'CS11011', '2019_t5', None, None, '2'), 'course', 'course'), end, datetime.datetime(2029, 7, 24, 0, 0, tzinfo=<bson.tz_util.FixedOffset object at 0x7fc21ccaedd0>)
2025-07-23 15:56:31 2025-07-23 10:56:31,873 INFO 58 [edx.celery.task] [user 200050] [ip 172.21.0.1] tasks.py:162 - Starting XBlockCaches update for course_key: ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:31 2025-07-23 10:56:31,900 INFO 58 [edx.celery.task] [user 200050] [ip 172.21.0.1] tasks.py:164 - Ending XBlockCaches update for course_key: ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:31 2025-07-23 10:56:31,901 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task openedx.core.djangoapps.bookmarks.tasks.update_xblocks_cache[3d39dfaf-1dba-4569-a721-819cfefcc2dd] succeeded in 0.02816107898252085s: None
2025-07-23 15:56:31 2025-07-23 10:56:31,903 INFO 58 [openedx.core.djangoapps.course_apps.tasks] [user 200050] [ip 172.21.0.1] tasks.py:41 - Caching course apps status for course with id: ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:31 2025-07-23 10:56:31,986 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task openedx.core.djangoapps.course_apps.tasks.update_course_apps_status[799074af-0a65-4130-a92e-3e00764475f0] succeeded in 0.08319871698040515s: None
2025-07-23 15:56:31 2025-07-23 10:56:31,996 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task lms.djangoapps.discussion.tasks.update_discussions_map[f5ee4b73-3dd4-48ca-94a6-1e1f43641c88] succeeded in 0.008286842989036813s: None
2025-07-23 15:56:31 2025-07-23 10:56:31,998 INFO 58 [celery_utils.logged_task] [user 200050] [ip 172.21.0.1] logged_task.py:25 - Task lms.djangoapps.discussion.tasks.update_discussions_map[f5ee4b73-3dd4-48ca-94a6-1e1f43641c88] submitted with arguments [{'course_id': 'ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2'}], None
2025-07-23 15:56:32 2025-07-23 10:56:32,006 INFO 58 [openedx.core.djangoapps.ccxcon.tasks] [user 200050] [ip 172.21.0.1] tasks.py:31 - Course update to CCXCon returned no errors. Course key: ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,007 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task openedx.core.djangoapps.ccxcon.tasks.update_ccxcon[41c8b36b-1965-4ded-bb39-db1302b9331c] succeeded in 0.007804680994013324s: None
2025-07-23 15:56:32 2025-07-23 10:56:32,038 INFO 58 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user 200050] [ip 172.21.0.1] outlines.py:394 - Replacing CourseOutline for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2 (version 687f4901712771d041005ce0, 3 sequences)
2025-07-23 15:56:32 2025-07-23 10:56:32,054 INFO 58 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user 200050] [ip 172.21.0.1] outlines.py:440 - Found CourseContext for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2, updating...
2025-07-23 15:56:32 2025-07-23 10:56:32,152 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task cms.djangoapps.contentstore.tasks.update_outline_from_modulestore_task[fb96b07e-1cac-4b26-8b08-b400afe364e6] succeeded in 0.14330179599346593s: None
2025-07-23 15:56:32 2025-07-23 10:56:32,153 INFO 58 [openedx.core.djangoapps.discussions.tasks] [user 200050] [ip 172.21.0.1] tasks.py:54 - Updating discussion settings for course: ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,164 INFO 58 [openedx.core.djangoapps.discussions.handlers] [user 200050] [ip 172.21.0.1] handlers.py:56 - Updating existing discussion topic links for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,169 INFO 58 [openedx.core.djangoapps.discussions.handlers] [user 200050] [ip 172.21.0.1] handlers.py:79 - Creating new discussion topic links for ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,173 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task openedx.core.djangoapps.discussions.tasks.update_discussions_settings_from_course_task[d091e50d-d118-4555-8272-8a482c2fd07e] succeeded in 0.01974512101151049s: None
2025-07-23 15:56:32 2025-07-23 10:56:32,174 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function _listen_for_course_publish at 0x7fc22f18af20>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,174 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function _listen_for_course_publish at 0x7fc22d251bc0>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,175 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function _listen_for_course_publish at 0x7fc22d2528e0>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,176 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function update_block_structure_on_course_publish at 0x7fc22d252fc0>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,176 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function extract_dates at 0x7fc22d075620>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,177 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function trigger_update_xblocks_cache_task at 0x7fc22cf36a20>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,178 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function update_course_apps at 0x7fc22cf37420>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,179 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function update_discussions_on_course_publish at 0x7fc22cf81800>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,179 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function course_published_handler at 0x7fc22ce332e0>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,180 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function _listen_for_course_publish at 0x7fc22ce400e0>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,181 INFO 58 [lms.djangoapps.ccx.views] [user 200050] [ip 172.21.0.1] views.py:338 - Signal fired when course is published. Receiver: <function listen_for_course_publish at 0x7fc22ce40900>. Response: None
2025-07-23 15:56:32 2025-07-23 10:56:32,195 INFO 58 [cms.djangoapps.contentstore.tasks] [user 200050] [ip 172.21.0.1] tasks.py:285 - Attempting to register exams for course ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,267 INFO 58 [cms.djangoapps.contentstore.tasks] [user 200050] [ip 172.21.0.1] tasks.py:291 - Successfully registered exams for course ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,268 INFO 58 [cms.djangoapps.contentstore.tasks] [user 200050] [ip 172.21.0.1] tasks.py:296 - Publishing course ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2
2025-07-23 15:56:32 2025-07-23 10:56:32,275 INFO 58 [celery.app.trace] [user 200050] [ip 172.21.0.1] trace.py:128 - Task cms.djangoapps.contentstore.tasks.update_special_exams_and_publish[3b62001f-a7be-4a49-bdcc-140839981ba4] succeeded in 0.08135622600093484s: None
2025-07-23 15:56:33 [23/Jul/2025 10:56:33] "POST /courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/save_ccx HTTP/1.1" 200 2404
2025-07-23 15:56:33 2025-07-23 10:56:33,760 INFO 58 [tracking] [user 200050] [ip 172.21.0.1] logger.py:41 - {"name": "/__debug__/history_sidebar/", "context": {"user_id": 200050, "path": "/__debug__/history_sidebar/", "course_id": "", "org_id": "", "enterprise_uuid": ""}, "username": "anas_", "session": "f863d7d21d0bf519e5b474419dd7d8ac", "ip": "172.21.0.1", "agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36", "host": "local.openedx.io:8000", "referer": "http://local.openedx.io:8000/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/ccx_coach", "accept_language": "en-US,en;q=0.9", "event": "{\"GET\": {\"store_id\": [\"1c2a75525c1142209a5f345f4ead65f4\"]}, \"POST\": {}}", "time": "2025-07-23T10:56:33.760040+00:00", "event_type": "/__debug__/history_sidebar/", "event_source": "server", "page": null}
2025-07-23 15:56:33 2025-07-23 10:56:33,786 INFO 58 [openedx.core.djangoapps.cors_csrf.helpers] [user 200050] [ip 172.21.0.1] helpers.py:64 - Origin 'http://local.openedx.io:8000' was not in `CORS_ORIGIN_WHITELIST`; full referer was 'http://local.openedx.io:8000/courses/ccx-v1:UAI_SOURCE+CS11011+2019_t5+ccx@2/ccx_coach' and requested host was 'local.openedx.io:8000'; CORS_ORIGIN_ALLOW_ALL=False
2025-07-23 15:56:33 [23/Jul/2025 10:56:33] "GET /__debug__/history_sidebar/?store_id=1c2a75525c1142209a5f345f4ead65f4 HTTP/1.1" 200 7183

I also found this conversation regarding why ccx was not originally implemented in cms.
https://openedx.slack.com/archives/C0F584CH0/p1467920837000185

]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to understand this better.

First of all, you're only adding 3 new apps here, right?

  • ContentstoreConfig
  • search
  • content-staging

How is it that these aren't running already? Why does the CCX feature need them turned on?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that a CCX course is created from LMS. All the above apps are already defined in CMS. The course publishing signals are not triggered in LMS until we define cms.djangoapps.contentstore.apps.ContentstoreConfig in our INSTALLED_APPS in LMS. The last 2 apps (openedx.core.djangoapps.content.search and openedx.core.djangoapps.content_staging) are required to be defined as they are being used in the code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you determine that these were the minimal required apps to add from CMS?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We needed to ensure that signals were triggered when creating a CCX course. When I added ContentstoreConfig to INSTALLED_APPS, I ran into this error:
RuntimeError: Model class openedx.core.djangoapps.content.search.models.SearchAccess doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

After some debugging, I found that ContentstoreConfig depends on the search app, which wasn't listed in INSTALLED_APPS for LMS. Adding search fixed the issue, and the course outline started publishing.

However, I then saw another error in the logs when the course publish signals were triggered:
RuntimeError: Model class openedx.core.djangoapps.content_staging.models.StagedContent doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

Adding content_staging to INSTALLED_APPS resolved this, and everything worked without errors.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We needed to ensure that signals were triggered when creating a CCX course.

Tying this into my other comment, is it possible to add a test for triggering signals?


MODULESTORE_FIELD_OVERRIDE_PROVIDERS += (
'lms.djangoapps.ccx.overrides.CustomCoursesForEdxOverrideProvider',
)
COURSE_IMPORT_EXPORT_STORAGE = DEFAULT_FILE_STORAGE

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We included cms.djangoapps.contentstore.apps.ContentstoreConfig in the INSTALLED_APPS of the LMS to ensure the app is properly loaded and our course publish signals are triggered properly. The app relies on the COURSE_IMPORT_EXPORT_STORAGE setting, which must be defined in the LMS configuration.

Failure to define this setting results in an error, as demonstrated in this related build log.


FIELD_OVERRIDE_PROVIDERS = tuple(FIELD_OVERRIDE_PROVIDERS)

Expand Down
Loading