This repository has been archived by the owner on Oct 11, 2023. It is now read-only.
LIGHT Refactor Phase 1 - New Data Model + Distributed Deploy #291
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
small intro fix (actor_name)
Pre alpha fixes2
fixed scroll issues in side bar and main page
on_use events are totally broken
add missing missions
More backend and event fixes pre-launch
Mission entry
Merged
* Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB
* Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Addressing comments, clarifying code
* Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou
JackUrb
changed the title
* Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix
* Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Enums for model types
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Creating LIGHT's ModelServer * Undo server change * tornado simplicity * Handling for inline candidate models * But regular models should also work without this * Enums for model types
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Creating LIGHT's ModelServer * Undo server change * tornado simplicity * Handling for inline candidate models * But regular models should also work without this * Async... all of the things... * Async the server too * clearing up async server tests * Correct async mock * internalize init_world * clean up tornado usage * small GameInstance bug * small GameInstance bug * Some deploy fixes * Moving safety model to async part * Some safety fixes * test fixes * Enums for model types * Checking for non-list to convert first
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Creating LIGHT's ModelServer * Undo server change * tornado simplicity * Handling for inline candidate models * But regular models should also work without this * Async... all of the things... * Async the server too * clearing up async server tests * Correct async mock * internalize init_world * clean up tornado usage * small GameInstance bug * small GameInstance bug * Some deploy fixes * now using aws as a storage backend * Moving safety model to async part * Some safety fixes * test fixes * silly elif fix
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Creating LIGHT's ModelServer * Undo server change * tornado simplicity * Handling for inline candidate models * But regular models should also work without this * Async... all of the things... * Async the server too * clearing up async server tests * Correct async mock * internalize init_world * clean up tornado usage * small GameInstance bug * small GameInstance bug * Some deploy fixes * now using aws as a storage backend * Moving safety model to async part * Some safety fixes * test fixes * silly elif fix * Taking bug-fixes from stable server * Model server changes too * Skip another web test, works in prod, refactor incoming * Enums for model types * Checking for non-list to convert first * Slightly more clarity
* Some initial transitions over to model pool * Moving initialization code out from where it occurred * Wiring more of the system together * Adding opt for reranked generative * Filling out UserDB * Abstract enforce get first * Clearer argument - num_turns * Using enums in DB * Initial episode data model * Update content loggers to use episode formatting * Updating tables to work with testing * Fixing some test changes * Fixing small warnings that were noise during tests * Moving default log path * Test fix * Correcting math thanks to Kurt * Updating env DB classes to SQLAlchemy * Name keys and Elems coded * Adding arbitrary node attributes * First complete pass of EnvDB * Mypy fixings * Fixing agents * Writing some tests * Finishing tests for object and room creates and queries * Edge testing * Arbitrary attributes testing * Quests and testing * And finally, DBGraph tests * fixing episode change * TODO function * final mypy fixes * DBID testing * a -> either a or an depending on aeiou * adding WorldConfig to hold complex configuration vars * Moving episode_db into relevant GraphBuilders * Game launches, but not logging * Local BaseDB, now saving episodes * Missing files * deleting miscommit * test fix * Migrating to UserDB * No more LIGHTDatabase in TornadoServer * Fixing tests * Works after testing locally * Updated messaging for unimplemented * Upgrading OneRoomGraphBuilder to ModelPool * Completing (almost) the rest of Modelpool references * Works without loading models in play_map * Model pool actually works * Safety working as well * removing prints * Fixing some tests, skipping starspace * Runs on server too * Creating LIGHT's ModelServer * Undo server change * tornado simplicity * Handling for inline candidate models * But regular models should also work without this * Async... all of the things... * Async the server too * clearing up async server tests * Correct async mock * internalize init_world * clean up tornado usage * small GameInstance bug * small GameInstance bug * Some deploy fixes * now using aws as a storage backend * Moving safety model to async part * Some safety fixes * test fixes * silly elif fix * Taking bug-fixes from stable server * Model server changes too * Skip another web test, works in prod, refactor incoming * Ensure we're not using FB user data * Methods for scrubbing the datasets * Deleting player data and related graph info * Fixing bugs, adding tests * Environment exporting * Export episode DB too * 60 != 90 * Addressing comments
12 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Master PR to track progress against the phase 1 of the light refactor, which roughly coincides with parts of #296. These changes are critical for consolidating the LIGHT codebase and having it ready for both research and production use.
Much of this will be handled in follow-up PRs that merge in. The current plan:
New Data Model
InteractionLoggers)EnvDBand associated testing)EpisodeDBin main game path #297 (Port main game to write to Interactions DB when new episodes are created)UserDBas main game identity storage #298 (Port main game to use users DB for users)Distributed deploy
ModelPool#300 (Model Pool completion)asyncioall over LIGHT #304 (No longer synchronous)Elements that are part of Phase 2/3 that won't be tracked here.