Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: requestUrl() and server-sent events #165

Open
ofalvai opened this issue May 15, 2024 · 4 comments
Open

Bug: requestUrl() and server-sent events #165

ofalvai opened this issue May 15, 2024 · 4 comments

Comments

@ofalvai
Copy link

ofalvai commented May 15, 2024

I'm using Obsidian's requestUrl() helper to bypass CORS restrictions. The API I'm calling is Anthropic's standard chat API that streams AI response chunks using server-sent events. I'm only focusing on desktop only for now, so I assume requestUrl() calls a Node.js or Electron networking API.

I noticed that even though I have a working API request, the timing of events doesn't feel right.

Given this code (which I tried to make as short as possible, sorry!):

console.log("Before requestUrl")
const response = await requestUrl({
	url: "https://api.anthropic.com/v1/messages",
	method: "POST",
	contentType: "application/json",
	headers: {
		"anthropic-version": "2023-06-01",
		"x-api-key": this.apiKey,
		"accept": "text/event-stream"
	},
	body: JSON.stringify({
		stream: true,
		model: this.model,
		system: "Dummy system message",
		messages: [{
			role: "user",
			content: "Dummy user message",
		}],
	}),
})
console.log("After requestUrl")

// Creating a Web API Response because `events()` expects a Response object
const nativeResponse = new Response(response.arrayBuffer, {
	headers: response.headers,
	status: response.status,
})
console.log("After nativeResponse")

// `events()` is a thin wrapper around SSEs: https://github.com/lukeed/fetch-event-stream
const stream = events(nativeResponse)
console.log("After events")

for await (const event of stream) {
	console.log("Stream event", event)
}

...what I see is that await requestUrl() blocks until the last server-sent event is received. In practice, streaming the response chunks doesn't feel any different than using the non-streaming API because the entire response appears at once (after a long delay).

I don't think this is a problem with the fetch-event-stream wrapper because the blocking happens earlier, between the log lines Before requestUrl and After requestUrl. The remaining log lines appear instantly and at the same time.

I'm not sure this is a bug in itself, maybe I'm doing something wrong here? I would appreciate any help or guidance about how to use requestUrl() or what it's doing behind the scenes.
Thank you!

@ofalvai
Copy link
Author

ofalvai commented May 15, 2024

Note: I know there are other Obsidian plugins out there using the Anthropic API with streaming, so I checked one. This one solves the CORS issue in a different way, by running a proxy server with Node.js: https://github.com/logancyang/obsidian-copilot/blob/master/src/proxyServer.ts#L18

@joethei
Copy link
Collaborator

joethei commented May 15, 2024

requestUrl does not support response streaming.

@ofalvai
Copy link
Author

ofalvai commented May 16, 2024

Can you give me some pointers how requestUrl() behaves in the desktop Obsidian app (my plugin is desktop only). What makes it work this way, and is there any way for me to work around it? I assume this function delegates to one of the Node HTTP APIs, can I rely on that as long as I'm only targeting the desktop app?

@joethei
Copy link
Collaborator

joethei commented May 16, 2024

It's being delegated to the net.request Electron API.
The reasons requestUrl does not support streaming is mobile, it's a lot more complicated to do that with Capacitor.
If your plugin is "desktopOnly" then you can use the Node API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants