max tokens is always 3071

[Dual-0]

Hello,

no matter which model I use e.g. gpt-4-32k-0613 or the new gpt-4-1106-preview with 128k context window I always get:

Mon, 06 Nov 2023 21:44:24 GMT [ERROR] [OpenAI-API Error: Error: Prompt is too long. Max token count is 3071, but prompt is 3539 tokens long.]
TypeError: Cannot read properties of undefined (reading 'response')
     at CommandHandler.onMessage (file:///opt/matrix_gpt_bot/3.1.4/dist/handlers.js:132:86)
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

thanks in advance

[jame25]

gpt-4-1106-preview is working perfectly for me with v3.1.4, I did however also update npm to latest version.

[Dual-0]

I think we need the options maxContextTokens and maxPromptTokens from use-client.js in clientOptions
My test went well with:

    maxContextTokens: 128000,
    maxPromptTokens: 32000,

gpt-4-1106-preview is working perfectly for me with v3.1.4, I did however also update npm to latest version.

you have made an input with over 3071 tokens?

[Dual-0]

Thanks @max298

I raised maxContextTokens: 128000. I don't use maxPromptTokens at the moment so it falls back to default value. Works good so far. Any advice for maxPromptTokens?