Skip to content

feat: mcp tests added #1399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Pratham-Mishra04 wants to merge 6 commits into from
Closed

feat: mcp tests added #1399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1c04269
feat: mcp tests added
Pratham-Mishra04 Jan 12, 2026
e3b6ca5
feat: add oauth support to mcp
Pratham-Mishra04 Jan 23, 2026
cec3f0d
mcp auth server setup
akshaydeo Jan 23, 2026
7318932
code mode compact signatures
akshaydeo Jan 26, 2026
12425fa
mcp flow refactoring
akshaydeo Jan 27, 2026
b9c19ef
mcp tool groups
akshaydeo Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,14 @@ transports/bifrost-http/logs/
transports/bifrost-http/tmp/
node_modules
/dist
**/dist
**/tmp/
temp*/
!examples/mcps/temperature/
tmp/
tmp-*
private
**/bin

# Go workspaces (local only)
go.work
Expand Down
123 changes: 123 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -731,6 +731,129 @@ test-governance: install-gotestsum $(if $(DEBUG),install-delve) ## Run governanc
exit 1; \
fi

setup-mcp-tests: ## Build all MCP test servers in examples/mcps/
@echo "$(GREEN)Building MCP test servers...$(NC)"
@for mcp_dir in examples/mcps/*/; do \
if [ -d "$$mcp_dir" ] && [ -f "$$mcp_dir/go.mod" ]; then \
mcp_name=$$(basename $$mcp_dir); \
echo "$(CYAN)Building $$mcp_name...$(NC)"; \
mkdir -p "$$mcp_dir/bin"; \
cd "$$mcp_dir" && GOWORK=off go build -o bin/$$mcp_name . && cd - > /dev/null && echo "$(GREEN) ✓ $$mcp_name$(NC)"; \
fi; \
done
@echo ""
@echo "$(GREEN)✓ All MCP test servers built$(NC)"
Comment on lines +734 to +745
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Failing MCP server builds can be masked.
The loop returns the status of the last iteration, so an earlier go build failure can still print success and exit 0. Track failures and exit non-zero to avoid running tests with missing binaries.

🔧 Proposed fix 
 setup-mcp-tests: ## Build all MCP test servers in examples/mcps/
 	`@echo` "$(GREEN)Building MCP test servers...$(NC)"
-	`@for` mcp_dir in examples/mcps/*/; do \
+	`@FAILED`=0; \
+	for mcp_dir in examples/mcps/*/; do \
 		if [ -d "$$mcp_dir" ] && [ -f "$$mcp_dir/go.mod" ]; then \
 			mcp_name=$$(basename $$mcp_dir); \
 			echo "$(CYAN)Building $$mcp_name...$(NC)"; \
 			mkdir -p "$$mcp_dir/bin"; \
-			cd "$$mcp_dir" && GOWORK=off go build -o bin/$$mcp_name . && cd - > /dev/null && echo "$(GREEN)  ✓ $$mcp_name$(NC)"; \
+			if cd "$$mcp_dir" && GOWORK=off go build -o bin/$$mcp_name .; then \
+				echo "$(GREEN)  ✓ $$mcp_name$(NC)"; \
+			else \
+				FAILED=1; \
+			fi; \
+			cd - > /dev/null; \
 		fi; \
-	done
+	done; \
+	if [ $$FAILED -ne 0 ]; then \
+		echo "$(RED)✗ One or more MCP test servers failed to build$(NC)"; \
+		exit 1; \
+	fi
 	`@echo` ""
 	`@echo` "$(GREEN)✓ All MCP test servers built$(NC)"
🤖 Prompt for AI Agents 
In `@Makefile` around lines 734 - 745, The setup-mcp-tests target currently masks
failures because the loop returns the last iteration status; modify the target
to track any failed go build by introducing a failure flag (e.g., FAILED=0)
before the for-loop, check the exit status after each GOWORK=off go build for
"$$mcp_dir" and set FAILED=1 and print an error if a build fails, and after the
loop test FAILED and exit non-zero (exit 1) when any build failed; reference the
Makefile target name setup-mcp-tests and the variables mcp_dir / mcp_name and
the GOWORK=off go build invocation to locate where to add the flag, checks, and
final conditional exit.


test-mcp: install-gotestsum ## Run MCP tests by file type (Usage: make test-mcp TYPE=connection [TESTCASE=TestName] [PATTERN=substring])
@echo "$(GREEN)Running MCP tests...$(NC)"
@mkdir -p $(TEST_REPORTS_DIR)
@if [ -n "$(PATTERN)" ] && [ -n "$(TESTCASE)" ]; then \
echo "$(RED)Error: PATTERN and TESTCASE are mutually exclusive$(NC)"; \
echo "$(YELLOW)Use PATTERN for substring matching or TESTCASE for exact match$(NC)"; \
exit 1; \
fi
@if [ ! -d "core/internal/mcptests" ]; then \
echo "$(RED)Error: MCP tests directory not found$(NC)"; \
exit 1; \
fi
@TEST_FAILED=0; \
REPORT_FILE=""; \
if [ -f .env ]; then \
echo "$(YELLOW)Loading environment variables from .env...$(NC)"; \
set -a; . ./.env; set +a; \
fi; \
if [ -n "$(TYPE)" ]; then \
TYPE_CLEAN=$$(echo $(TYPE) | sed 's/_test\.go$$//'); \
TEST_FILE="core/internal/mcptests/$${TYPE_CLEAN}_test.go"; \
if [ ! -f "$$TEST_FILE" ]; then \
echo "$(RED)Error: Test file '$$TEST_FILE' not found$(NC)"; \
echo "$(YELLOW)Available test types:$(NC)"; \
ls -1 core/internal/mcptests/*_test.go 2>/dev/null | sed 's|core/internal/mcptests/||' | sed 's|_test\.go$$||' | sed 's/^/ - /'; \
exit 1; \
fi; \
TEST_PATTERN=$$(grep -h "^func Test" $$TEST_FILE 2>/dev/null | sed 's/func \(Test[^(]*\).*/\1/' | paste -sd '|' - || echo "^Test"); \
if [ -n "$(TESTCASE)" ]; then \
echo "$(CYAN)Running $(TYPE) test: $(TESTCASE)...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(TESTCASE).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run "^$(TESTCASE)$$" . || TEST_FAILED=1; \
elif [ -n "$(PATTERN)" ]; then \
echo "$(CYAN)Running $(TYPE) tests matching '$(PATTERN)'...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(PATTERN).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
Comment on lines +775 to +785
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Sanitize TESTCASE/PATTERN before using them in report filenames.
Subtests (e.g., TestX/Sub) or patterns with / can create invalid paths and cause gotestsum to fail writing the report. Normalize to a safe filename and apply similarly for PATTERN branches (including the all-types path).

🔧 Proposed fix 
-			REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(TESTCASE).xml"; \
+			SAFE_TESTCASE=$$(echo "$(TESTCASE)" | sed 's|/|_|g; s|[^A-Za-z0-9_.-]|_|g'); \
+			REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$${SAFE_TESTCASE}.xml"; \
...
-			REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(PATTERN).xml"; \
+			SAFE_PATTERN=$$(echo "$(PATTERN)" | sed 's|/|_|g; s|[^A-Za-z0-9_.-]|_|g'); \
+			REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$${SAFE_PATTERN}.xml"; \
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if [ -n "$(TESTCASE)" ]; then \
echo "$(CYAN)Running $(TYPE) test: $(TESTCASE)...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(TESTCASE).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run "^$(TESTCASE)$$" . || TEST_FAILED=1; \
elif [ -n "$(PATTERN)" ]; then \
echo "$(CYAN)Running $(TYPE) tests matching '$(PATTERN)'...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$(PATTERN).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
if [ -n "$(TESTCASE)" ]; then \
echo "$(CYAN)Running $(TYPE) test: $(TESTCASE)...$(NC)"; \
SAFE_TESTCASE=$$(echo "$(TESTCASE)" | sed 's|/|_|g; s|[^A-Za-z0-9_.-]|_|g'); \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$${SAFE_TESTCASE}.xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run "^$(TESTCASE)$$" . || TEST_FAILED=1; \
elif [ -n "$(PATTERN)" ]; then \
echo "$(CYAN)Running $(TYPE) tests matching '$(PATTERN)'...$(NC)"; \
SAFE_PATTERN=$$(echo "$(PATTERN)" | sed 's|/|_|g; s|[^A-Za-z0-9_.-]|_|g'); \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE)-$${SAFE_PATTERN}.xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
🤖 Prompt for AI Agents 
In `@Makefile` around lines 775 - 785, Sanitize TESTCASE and PATTERN before
embedding them into REPORT_FILE: ensure any '/' or other filesystem-special
characters in the TESTCASE/PATTERN variables are normalized (for example replace
'/' with '_' and remove or escape characters that could make an invalid
filename) and use those sanitized variables when constructing REPORT_FILE;
update the branches that set REPORT_FILE (both the TESTCASE and PATTERN
branches, and the all-types path) to reference the sanitized names (e.g.,
SANITIZED_TESTCASE / SANITIZED_PATTERN) so gotestsum writes a safe report path.

--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run ".*$(PATTERN).*" . || TEST_FAILED=1; \
else \
echo "$(CYAN)Running all $(TYPE) tests (pattern: $$TEST_PATTERN)...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-$(TYPE).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run "$$TEST_PATTERN" . || TEST_FAILED=1; \
fi; \
cd ../../..; \
if [ -z "$$CI" ] && [ -z "$$GITHUB_ACTIONS" ] && [ -z "$$GITLAB_CI" ] && [ -z "$$CIRCLECI" ] && [ -z "$$JENKINS_HOME" ]; then \
if which junit-viewer > /dev/null 2>&1; then \
echo "$(YELLOW)Generating HTML report...$(NC)"; \
junit-viewer --results=$$REPORT_FILE --save=$${REPORT_FILE%.xml}.html 2>/dev/null || true; \
echo ""; \
echo "$(CYAN)HTML report: $${REPORT_FILE%.xml}.html$(NC)"; \
echo "$(CYAN)Open with: open $${REPORT_FILE%.xml}.html$(NC)"; \
else \
echo ""; \
echo "$(CYAN)JUnit XML report: $$REPORT_FILE$(NC)"; \
fi; \
else \
echo ""; \
echo "$(CYAN)JUnit XML report: $$REPORT_FILE$(NC)"; \
fi; \
else \
if [ -n "$(TESTCASE)" ]; then \
echo "$(CYAN)Running test case: $(TESTCASE) across all MCP tests...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-all-$(TESTCASE).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run "^$(TESTCASE)$$" || TEST_FAILED=1; \
elif [ -n "$(PATTERN)" ]; then \
echo "$(CYAN)Running tests matching '$(PATTERN)' across all MCP tests...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-all-$(PATTERN).xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v -run ".*$(PATTERN).*" || TEST_FAILED=1; \
else \
echo "$(CYAN)Running all MCP tests...$(NC)"; \
REPORT_FILE="$(TEST_REPORTS_DIR)/mcp-all.xml"; \
cd core/internal/mcptests && GOWORK=off gotestsum \
--format=$(GOTESTSUM_FORMAT) \
--junitfile=../../../$$REPORT_FILE \
-- -v || TEST_FAILED=1; \
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fi; \
cd ../../..; \
if [ -z "$$CI" ] && [ -z "$$GITHUB_ACTIONS" ] && [ -z "$$GITLAB_CI" ] && [ -z "$$CIRCLECI" ] && [ -z "$$JENKINS_HOME" ]; then \
if which junit-viewer > /dev/null 2>&1; then \
echo "$(YELLOW)Generating HTML report...$(NC)"; \
junit-viewer --results=$$REPORT_FILE --save=$${REPORT_FILE%.xml}.html 2>/dev/null || true; \
echo ""; \
echo "$(CYAN)HTML report: $${REPORT_FILE%.xml}.html$(NC)"; \
echo "$(CYAN)Open with: open $${REPORT_FILE%.xml}.html$(NC)"; \
else \
echo ""; \
echo "$(CYAN)JUnit XML report: $$REPORT_FILE$(NC)"; \
fi; \
else \
echo ""; \
echo "$(CYAN)JUnit XML report: $$REPORT_FILE$(NC)"; \
fi; \
fi; \
if [ $$TEST_FAILED -eq 1 ]; then \
exit 1; \
fi

test-all: test-core test-plugins test ## Run all tests
@echo ""
@echo "$(GREEN)═══════════════════════════════════════════════════════════$(NC)"
Expand Down
Loading