Is it desirable to have LLM interactions be deterministic?

[cproctor]

If so, we may want to consider providing a SEED directive. Like INCLUDE, SEED X would only be allowed at the top of a story.

Reasons to consider deterministic LLM interactions:

• It would make it much easier to debug, test, and resolve abusive use of LLMs.
• We could cache requests to the LLM, drastically reducing the backend cost of supporting LLMs.

Notes/Issues

• How do we handle INCLUDE statements which import stories with different seeds? The simplest approach would be to say that only the top-level story's SEED is active.
• SEEDwill not affect random functions within the story (e.g. random, random_integer, random_gaussian). Perhaps we should consider a different name?
• A malicious user could evade the cache by interpolating random numbers into the prompt. This would be pretty easy to spot.
• Is there a use case for exposing the seed to users? Alternatively, we could always make requests deterministic and cached, and hide this.