[sdk] 1/n Standardize Input Schema - Input Attachment Data

[Ankush-lastmile]

[rfc][sdk] 1/n Standardize Input Schema - Input Attachment Data

Starting point of standardizing Input Schema for prompts

This diff introduces the type AttachmentDataWithStringValue to python and typescript sdk which will allow attachments to have structure as well as validation in python (pydantic validation).

testplan

[Ankush-lastmile]

This type and code is pretty much a duplicate of

class OutputDataWithStringValue(BaseModel):
    """
    This represents the output content that is storied as a string, but we use
    both the `kind` field here and the `mime_type` in ExecuteResult to convert
    the string into the output format we want.
    """

    kind: Literal["file_uri", "base64"]
    value: str

Should this be merged into one type and abstracted? My take on that is no because That would reduce the explicitness and readability of this addition, but unsure what's right here.

[rholinshead]

Nice, LGTM, just some comments. Would you be able to hook this up to the editor in a subsequent PR so that we are using the correct types there? I wonder if we can just remove the attachments stuff from PromptInputObjectSchema now with this? There's technically support for attachment metadata there but it could also just be set with the JSON toggle

[Ankush-lastmile]

Would you be able to hook this up to the editor in a subsequent PR so that we are using the correct types there? I wonder if we can just remove the attachments stuff from PromptInputObjectSchema now with this? There's technically support for attachment metadata there but it could also just be set with the JSON toggle

Looking into this, I have a couple of questions; will sync offline

[saqadri]

One change required, otherwise looks good. This doesn't actually update the model parsers it seems to actually use this new type yet, coming on top of this I assume?

[saqadri]

Suggested change

	data: Union[AttachmentDataWithStringValue, Any]
	data: Union[AttachmentDataWithStringValue, str, Any]

Technically, an attachment can also be a string, just like ExecuteResult.data: data: Union[OutputDataWithValue, str, Any]

[Ankush-lastmile]

discussed offline: updating to include str to be explicit, since we know some models like gemini take in multiple parts or stablediffusion has a 'negative prompt` which is just as much as input. We can iterate as we go about how this looks.

Typescript is defined as JSONValue so no need to add explicit string; Already supported.

[Ankush-lastmile]

This doesn't actually update the model parsers it seems to actually use this new type yet, coming on top of this I assume?

Yes; those diffs will come on top