feat: Add PromptTemplate type #5787

anticorrelator · 2024-12-19T16:29:35Z

resolves #5771

axiomofjoy · 2024-12-19T18:16:45Z

src/phoenix/server/api/types/PromptVersion.py

@@ -29,7 +31,7 @@ class PromptVersion(Node):
    description: str
    template_type: PromptTemplateType
    template_format: PromptTemplateFormat
-    template: JSON
+    template: PromptTemplateVersion


template_version?

This is the wrong type

axiomofjoy · 2024-12-19T18:21:42Z

app/schema.graphql

@@ -1086,6 +1086,11 @@ type JSONInvocationParameter implements InvocationParameterBase {
  defaultValue: JSON
 }

+type JSONPromptMessageGQL {


Can we remove GQL from the type names in the GraphQL schema?

axiomofjoy · 2024-12-19T18:23:01Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+    AI = "ai"  # E.g. the assistant. Normalize to AI for consistency.
+
+
+class PromptStringTemplate(BaseModel):


I see the appeal of keeping the Pydantic models next to their corresponding GraphQL types, but will these models also be needed for the REST API?

Yeah they will - let's de-couple

@axiomofjoy I will say decoupling these into different files might lead to some weird circular import stuff

You can reduce the circular imports by just having the conversion code be functional in its own file and not be attached to classes

axiomofjoy · 2024-12-19T18:24:23Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+        return TextPromptMessageGQL.from_model(self)
+
+
+class JSONPromptMessage(BaseModel):


Do we need both to and from methods?

maybe eventually but we're feeling it out

axiomofjoy · 2024-12-19T18:25:02Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+
+class PromptMessagesTemplateV1(BaseModel):
+    _version: str = "messages-v1"
+    template: list[Union[TextPromptMessage, JSONPromptMessage]]


message_templates or messages?

ChatTemplateV1?

I think it's probably a good idea for the different template types to conform to the same interface for now

This should just be a union no?

wait no, I think it's list[Union[...]] since there can be multiple messages

they key should be messages that's what's confusing here

ah I see, I changed it to template so all of our template wrappers have the same interface which I'm kind of partial to (the wrapper for the plain string template just has a template key)

It shouldn't leak into the graphQL type

app/schema.graphql

mikeldking · 2024-12-19T20:54:43Z

app/schema.graphql

+
+type PromptMessagesTemplateV1GQL {
+  version: String!
+  template: [TextPromptMessageGQLJSONPromptMessageGQL!]!


I'm not following why this is a list?

mikeldking · 2024-12-19T20:55:22Z

app/schema.graphql

 type PromptVersion implements Node {
  """The Globally Unique ID of this object"""
  id: GlobalID!
  user: String
  description: String!
  templateType: PromptTemplateType!
  templateFormat: PromptTemplateFormat!
-  template: JSON!
+  template: PromptTemplateVersion!


The union type should be here

mikeldking · 2024-12-20T18:37:14Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+import strawberry
+from strawberry.scalars import JSON
+
+from phoenix.server.api.helpers.prompthub.models import (


Suggested change

from phoenix.server.api.helpers.prompthub.models import (

from phoenix.server.api.helpers.prompts.models import (

axiomofjoy · 2024-12-20T18:37:28Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+PromptTemplate = strawberry.union(
+    "PromptTemplateVersion", (PromptStringTemplate, PromptChatTemplateV1)
+)


Maybe just name PromptTemplateVersion to match the underlying GraphQL type.

axiomofjoy · 2024-12-20T18:39:56Z

src/phoenix/server/api/types/PromptVersionTemplate.py

+@strawberry.type
+class JSONPromptMessage:
+    role: PromptMessageRole
+    content: JSON


Is this for tool call messages, images, etc.?

axiomofjoy · 2024-12-20T18:41:22Z

app/schema.graphql

+  content: String!
+}
+
+union TextPromptMessageJSONPromptMessage = TextPromptMessage | JSONPromptMessage


Can we name this union type as PromptMessage?

mikeldking · 2024-12-20T18:47:47Z

app/schema.graphql

@@ -1377,6 +1382,11 @@ type Prompt implements Node {
  promptVersions(first: Int = 50, last: Int, after: String, before: String): PromptVersionConnection!
 }

+type PromptChatTemplateV1 {


Suggested change

type PromptChatTemplateV1 {

type PromptChatTemplate {

mikeldking · 2024-12-20T18:47:56Z

app/schema.graphql

@@ -1377,6 +1382,11 @@ type Prompt implements Node {
  promptVersions(first: Int = 50, last: Int, after: String, before: String): PromptVersionConnection!
 }

+type PromptChatTemplateV1 {
+  version: String!


Suggested change

version: String!

__version: String!

mikeldking · 2024-12-20T18:49:05Z

app/schema.graphql

@@ -1377,6 +1382,11 @@ type Prompt implements Node {
  promptVersions(first: Int = 50, last: Int, after: String, before: String): PromptVersionConnection!
 }

+type PromptChatTemplateV1 {
+  version: String!
+  messages: [TextPromptMessageJSONPromptMessage!]!


Suggested change

messages: [TextPromptMessageJSONPromptMessage!]!

messages: [PromptTemplateMessages!]!

mikeldking · 2024-12-20T18:49:58Z

app/schema.graphql

@@ -1395,6 +1405,12 @@ type PromptEdge {
  node: Prompt!
 }

+enum PromptMessageRole {
+  USER
+  SYSTEM


add a description that this includes the openAI developer role

mikeldking · 2024-12-20T18:50:16Z

app/schema.graphql

@@ -1395,6 +1405,12 @@ type PromptEdge {
  node: Prompt!
 }

+enum PromptMessageRole {
+  USER


Missing tool

* Flesh out PromptVersionTemplate type * Return GraphQL objects with the correct type * Use new types in node query * Decouple pydantic models and gql types * Rebuild gql schema * Rework model names * Update gql schema * Propagate name into schema * Incorporate feedback * Update schema * adjust UI to new schema * cleanup * Remove `hub` naming and clean up type annotations --------- Co-authored-by: Mikyo King <[email protected]>

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Dec 19, 2024

anticorrelator changed the base branch from main to prompts December 19, 2024 16:29

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Dec 19, 2024

axiomofjoy reviewed Dec 19, 2024

View reviewed changes

mikeldking reviewed Dec 19, 2024

View reviewed changes

anticorrelator added 5 commits December 20, 2024 11:29

Flesh out PromptVersionTemplate type

005a183

Return GraphQL objects with the correct type

10c0675

Use new types in node query

a5cae16

Decouple pydantic models and gql types

31387c6

Rebuild gql schema

1154306

anticorrelator force-pushed the dustin/add-promptversiontemplate-type branch from 7a90962 to 1154306 Compare December 20, 2024 16:51

anticorrelator added 2 commits December 20, 2024 13:24

Rework model names

728a9fa

Update gql schema

d98a482

mikeldking reviewed Dec 20, 2024

View reviewed changes

Propagate name into schema

f66f44b

axiomofjoy approved these changes Dec 20, 2024

View reviewed changes

mikeldking reviewed Dec 20, 2024

View reviewed changes

app/schema.graphql

@@ -1395,6 +1405,12 @@ type PromptEdge {

node: Prompt!

}

enum PromptMessageRole {

USER

Copy link

Contributor

mikeldking Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing tool

anticorrelator added 2 commits December 20, 2024 14:24

Incorporate feedback

4d7092d

Update schema

30ce59a

mikeldking changed the title ~~feat: Add PromptVersionTemplate type~~ feat: Add PromptTemplate type Dec 20, 2024

adjust UI to new schema

120a23f

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Dec 20, 2024

mikeldking and others added 2 commits December 20, 2024 15:40

cleanup

9e6cc2b

Remove hub naming and clean up type annotations

3ebafc4

anticorrelator merged commit f3fe933 into prompts Dec 20, 2024
49 checks passed

anticorrelator deleted the dustin/add-promptversiontemplate-type branch December 20, 2024 22:58

axiomofjoy mentioned this pull request Jan 7, 2025

feat(prompts): tool call definitions #5922

Merged

mikeldking mentioned this pull request Jan 27, 2025

feat(prompts): add open in playground action button on the table #6185

Merged

This was referenced Jan 29, 2025

fix(prompts): output schema #6194

Merged

fix(prompts): normalized tools #6220

Merged

mikeldking mentioned this pull request Feb 19, 2025

chore(main): release arize-phoenix 8.0.0 #6460

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add PromptTemplate type #5787

feat: Add PromptTemplate type #5787

anticorrelator commented Dec 19, 2024

axiomofjoy Dec 19, 2024

mikeldking Dec 19, 2024

axiomofjoy Dec 19, 2024

mikeldking Dec 19, 2024

axiomofjoy Dec 19, 2024

mikeldking Dec 19, 2024

anticorrelator Dec 19, 2024

mikeldking Dec 19, 2024

axiomofjoy Dec 19, 2024

anticorrelator Dec 19, 2024

axiomofjoy Dec 19, 2024

axiomofjoy Dec 19, 2024

anticorrelator Dec 19, 2024

mikeldking Dec 19, 2024

anticorrelator Dec 19, 2024

anticorrelator Dec 19, 2024

mikeldking Dec 19, 2024

anticorrelator Dec 19, 2024

mikeldking Dec 19, 2024

mikeldking Dec 19, 2024

mikeldking Dec 19, 2024

mikeldking Dec 20, 2024

axiomofjoy Dec 20, 2024

axiomofjoy Dec 20, 2024

axiomofjoy Dec 20, 2024 •

edited

Loading

mikeldking Dec 20, 2024

mikeldking Dec 20, 2024

mikeldking Dec 20, 2024

mikeldking Dec 20, 2024

mikeldking Dec 20, 2024

		AI = "ai" # E.g. the assistant. Normalize to AI for consistency.


		class PromptStringTemplate(BaseModel):

		return TextPromptMessageGQL.from_model(self)


		class JSONPromptMessage(BaseModel):

	from phoenix.server.api.helpers.prompthub.models import (
	from phoenix.server.api.helpers.prompts.models import (

	messages: [TextPromptMessageJSONPromptMessage!]!
	messages: [PromptTemplateMessages!]!

feat: Add PromptTemplate type #5787

feat: Add PromptTemplate type #5787

Conversation

anticorrelator commented Dec 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

axiomofjoy Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

axiomofjoy Dec 20, 2024 •

edited

Loading