Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Externalize Vendor Code #3393

Draft
wants to merge 163 commits into
base: develop
Choose a base branch
from
Draft

Externalize Vendor Code #3393

wants to merge 163 commits into from

Conversation

lalalune
Copy link
Member

@lalalune lalalune commented Feb 9, 2025

This PR has many core breaking changes

The main change is that vendor code has been removed from core. For any third party vendor plugins, interfaces are available to register plugins to handle nearly everything (or at least there will be soon)

  • Model providers have been moved into plugins and out of core
  • Some services have been turned into function handlers that call model providers
  • Some plugins are being developed here until they are ready to be migrated to their own repos

Other than that, many things were moved, removed, consolidated or simplified. The primary goal of this branch is to enable the project to scale and reach more users. Also, switched from pnpm to bun

file: fs.createReadStream(audioFile.path),
model: "whisper-1",
});
const transcription = await runtime.useModel(ModelClass.TRANSCRIPTION, fs.createReadStream(audioFile.path));

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
.

Copilot Autofix AI about 5 hours ago

To fix the problem, we need to ensure that the file path derived from user input is validated and sanitized before being used. We can achieve this by resolving the file path to a known safe directory and ensuring it does not escape this directory. This can be done using path.resolve and fs.realpathSync to normalize the path and check it against the safe directory.

  1. Define a safe root directory where uploaded files are stored.
  2. Resolve the audioFile.path to ensure it is within the safe directory.
  3. If the resolved path is not within the safe directory, return an error response.
Suggested changeset 1
packages/agent/src/server.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/packages/agent/src/server.ts b/packages/agent/src/server.ts
--- a/packages/agent/src/server.ts
+++ b/packages/agent/src/server.ts
@@ -93,3 +93,9 @@
 
-                const transcription = await runtime.useModel(ModelClass.TRANSCRIPTION, fs.createReadStream(audioFile.path));
+                const safeRoot = path.join(process.cwd(), "/data/uploads");
+                const resolvedPath = fs.realpathSync(path.resolve(safeRoot, audioFile.path));
+                if (!resolvedPath.startsWith(safeRoot)) {
+                    res.status(403).send("Invalid file path");
+                    return;
+                }
+                const transcription = await runtime.useModel(ModelClass.TRANSCRIPTION, fs.createReadStream(resolvedPath));
 
EOF
@@ -93,3 +93,9 @@

const transcription = await runtime.useModel(ModelClass.TRANSCRIPTION, fs.createReadStream(audioFile.path));
const safeRoot = path.join(process.cwd(), "/data/uploads");
const resolvedPath = fs.realpathSync(path.resolve(safeRoot, audioFile.path));
if (!resolvedPath.startsWith(safeRoot)) {
res.status(403).send("Invalid file path");
return;
}
const transcription = await runtime.useModel(ModelClass.TRANSCRIPTION, fs.createReadStream(resolvedPath));

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants