Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of Prompt Template seems convoluted and ill-defined #19

Open
tanweer-mahdi opened this issue Mar 19, 2025 · 1 comment
Open
Assignees

Comments

@tanweer-mahdi
Copy link

tanweer-mahdi commented Mar 19, 2025

One of the critical aspect of experiment tracking is creating different version of an artefact and track/evaluate their impact.. With the current design, I can see possibilities of bad UX:

1. Letting user to create new version of Prompt Template by changing every attribute

Whenever a user create a new version of an asset, they are allowed to change every asset attribute including Name and Description. To track an asset, the user is only left with an uuid - which is not pleasantly readable string

Proposal: User cannot change the name of the Name of an asset across different versions. Name and assed ID should be immutable. Description should be at the discretion of the user

2. User explicitly pass Asset ID while calling the create_or_update method

After creating an asset, it is up to user to properly pass the ID in subsequent version creation. A mismatch in it can lead to pretty bad experiment design.

Proposal: Let's handle it via a state variable and wrap the create_or_update method inside a class.

3. Asset definition in README.md is in conflict with the method name create_or_update

Minor issue - but I think should not use the name create_or_update because we explicitly mention in the doc that there is no concept of update in the assets, rather versions. If we go with proposal (2) - then we can have 2 explicit method: create_new_template and update_template_version

@tanweer-mahdi
Copy link
Author

@ra1han confirming that we agree on:

  1. Create abstraction on the creation of ID
  2. Streamline asset creation/update via some stateful structure (a Python class mainly)
  3. Validations on name of the asset

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant