Skip to content

Add tests for acipolicygen #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
DomAyre wants to merge 19 commits into from
Closed

Add tests for acipolicygen #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
acc900e
chore: Disable aks-mcp by default, offer --aks-mcp flag to enable it …
nilo19 Sep 18, 2025
b0d843e
[Release] Update index.json for extension [ aks-agent-1.0.0b3 ]
azclibot Sep 18, 2025
d169483
Add blue green upgrade support for aks-preview (#8999)
CustardTart32 Sep 18, 2025
b06cbbb
Feature/aks acns performance (#9136)
santhoshmprabhu Sep 18, 2025
9f6efdf
[Release] Update index.json for extension [ aks-preview-18.0.0b38 ]
azclibot Sep 18, 2025
d4d58b1
[connectedk8s] Update extension CLI to v1.10.9 (#9177)
bavneetsingh16 Sep 19, 2025
e0c6357
[Release] Update index.json for extension [ connectedk8s-1.10.9 ]
azclibot Sep 19, 2025
3e96534
[Communication] Add deliveryReport and Tag parameters to sms send com…
phermanov-msft Sep 22, 2025
9fdce07
[Release] Update index.json for extension [ communication-1.14.0 ]
azclibot Sep 22, 2025
8357d6f
[amlfs] Add az amlfs auto-import commands (#9134)
Aman-Jain-14 Sep 22, 2025
5618fed
Create role assignment for MSI when enable_vnet_integration is true (…
MuhammadAliFleet Sep 22, 2025
de8f0f3
[Release] Update index.json for extension [ amlfs-1.2.0 ]
azclibot Sep 22, 2025
1c11305
Managed Network Fabric - Removing the `externalnetwork update-bfd-adm…
idanielsteven Sep 22, 2025
beed460
[Release] Update index.json for extension [ managednetworkfabric-8.1.1 ]
azclibot Sep 22, 2025
5d81a3c
Update the layer hashes for image which has changed and pull specific…
DomAyre Sep 22, 2025
cd5decb
{CI} Sync resourceManagement.yml according To ADO Wiki Page - Service…
azclibot Sep 22, 2025
98bb702
Add tests for acipolicygen on arm template
DomAyre Sep 18, 2025
b184394
Add missing sha based references
DomAyre Sep 19, 2025
667b777
Remove helper script
DomAyre Sep 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@

/src/ip-group/ @necusjz @kairu-ms @jsntcy

/src/connectedk8s/ @bavneetsingh16 @deeksha345 @anagg929 @atchutbarli
/src/connectedk8s/ @bavneetsingh16 @deeksha345 @anagg929 @atchutbarli @bgriddaluru

/src/storagesync/ @jsntcy

Expand Down
9 changes: 9 additions & 0 deletions .github/policies/resourceManagement.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1976,6 +1976,9 @@ configuration:
mentionees:
- ambhatna
- deepthiprasad
- akning-ms
- junsu-msft
- coffeemug
replyTemplate: Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc ${mentionees}.
assignMentionees: False
- if:
Expand Down Expand Up @@ -2316,6 +2319,9 @@ configuration:
mentionees:
- ambhatna
- deepthiprasad
- akning-ms
- junsu-msft
- coffeemug
replyTemplate: Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc ${mentionees}.
assignMentionees: False
- if:
Expand All @@ -2329,6 +2335,9 @@ configuration:
- daeunyim
- rajsell
- deepthiprasad
- akning-ms
- junsu-msft
- coffeemug
replyTemplate: Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc ${mentionees}.
assignMentionees: False
- if:
Expand Down
7 changes: 6 additions & 1 deletion src/aks-agent/HISTORY.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,21 @@ To release a new version, please select a new version number (usually plus 1 to

Pending
+++++++

1.0.0b3
+++++++
* Disable aks-mcp by default, offer --aks-mcp flag to enable it.
* Don't print version check at bottom toolbar


1.0.0b2
+++++++

* Add MCP integration for `az aks agent` with aks-mcp binary management and local server lifecycle (download, version validation, start/stop, health checks).
* Introduce dual-mode operation: MCP mode (enhanced) and Traditional mode (built-in toolsets), with mode-specific system prompts.
* Implement smart toolset refresh strategy with persisted mode state to avoid unnecessary refresh on repeated runs.
* Add `--no-aks-mcp` flag to force Traditional mode when desired.
* Add `az aks agent status` command to display MCP binary availability/version, server health, and overall mode/readiness.
* Add `az aks agent --status` command to display MCP binary availability/version, server health, and overall mode/readiness.
* Add structured error handling with user-friendly messages and actionable suggestions for MCP/binary/server/config errors.
* Port and adapt comprehensive unit tests covering binary manager, MCP manager, configuration generation/validation, status models/collection, error handling, user feedback, parameters, smart refresh, MCP integration, and status command.

Expand Down
10 changes: 10 additions & 0 deletions src/aks-agent/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Key capabilities
- Configurable via a JSON/YAML config file provided with --config-file.
- Control echo and tool output visibility with --no-echo-request and --show-tool-output.
- Refresh the available toolsets with --refresh-toolsets.
- Stay in traditional toolset mode by default, or opt in to aks-mcp integration with ``--aks-mcp`` when you need the enhanced capabilities.

Prerequisites
-------------
Expand Down Expand Up @@ -63,6 +64,15 @@ Run in non-interactive batch mode

az aks agent "Diagnose networking issues" --no-interactive --max-steps 15 --model azure/my-gpt4.1-deployment

Opt in to MCP mode
------------------

Traditional toolsets remain the default. Enable the aks-mcp integration when you want the enhanced toolsets by passing ``--aks-mcp``. You can return to traditional mode on a subsequent run with ``--no-aks-mcp``.

.. code-block:: bash

az aks agent --aks-mcp "Check node health with MCP" --name MyManagedCluster --resource-group MyResourceGroup --model azure/my-gpt4.1-deployment

Using a configuration file
--------------------------

Expand Down
4 changes: 2 additions & 2 deletions src/aks-agent/azext_aks_agent/_help.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@
- name: --status
type: bool
short-summary: Show AKS agent configuration and status information.
- name: --no-aks-mcp
- name: --aks-mcp
type: bool
short-summary: Disable AKS MCP integration and use traditional toolsets.
short-summary: Enable AKS MCP integration for enhanced capabilities. Traditional mode is the default.

examples:
- name: Ask about pod issues in the cluster with Azure OpenAI
Expand Down
17 changes: 13 additions & 4 deletions src/aks-agent/azext_aks_agent/_params.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import os.path

from azure.cli.core.api import get_config_dir
from azure.cli.core.commands.parameters import get_three_state_flag

from azext_aks_agent._consts import CONST_AGENT_CONFIG_FILE_NAME

Expand Down Expand Up @@ -86,8 +87,16 @@ def load_arguments(self, _):
help="Show AKS agent configuration and status information.",
)
c.argument(
"no_aks_mcp",
options_list=["--no-aks-mcp"],
help="Disable AKS MCP integration and use traditional toolsets.",
action="store_true",
"use_aks_mcp",
options_list=["--aks-mcp"],
default=False,
arg_type=get_three_state_flag(
positive_label="Enable AKS MCP integration",
negative_label="Disable AKS MCP integration",
),
help=(
"Enable AKS MCP integration for enhanced capabilities. "
"Traditional mode is the default. Use --aks-mcp to enable MCP mode, or "
"--no-aks-mcp to explicitly disable it."
),
)
10 changes: 5 additions & 5 deletions src/aks-agent/azext_aks_agent/agent/agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ def aks_agent(
no_echo_request,
show_tool_output,
refresh_toolsets,
no_aks_mcp=False,
use_aks_mcp=False,
):
"""
Interact with the AKS agent using a prompt or piped input.
Expand All @@ -147,8 +147,8 @@ def aks_agent(
:type show_tool_output: bool
:param refresh_toolsets: Refresh the toolsets status.
:type refresh_toolsets: bool
:param no_aks_mcp: Disable AKS MCP integration and use traditional toolsets.
:type no_aks_mcp: bool
:param use_aks_mcp: Enable AKS MCP integration and use enhanced toolsets.
:type use_aks_mcp: bool
"""

with CLITelemetryClient():
Expand Down Expand Up @@ -177,7 +177,7 @@ def aks_agent(
interactive = False

# Determine MCP mode and smart refresh logic
use_aks_mcp = not no_aks_mcp
use_aks_mcp = bool(use_aks_mcp)
current_mode = "mcp" if use_aks_mcp else "traditional"
smart_refresh = _should_refresh_toolsets(current_mode, refresh_toolsets)

Expand Down Expand Up @@ -293,7 +293,7 @@ def _initialize_mcp_manager(verbose: bool = False):
[
"Ensure all required dependencies are installed",
"Try reinstalling the aks-preview extension",
"Use --no-aks-mcp flag to disable MCP integration"
"Use --aks-mcp flag to enable MCP integration"
]
)

Expand Down
2 changes: 1 addition & 1 deletion src/aks-agent/azext_aks_agent/agent/binary_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ async def download_binary(
"PLATFORM_UNSUPPORTED",
[
"Check if your platform is supported by the aks-mcp project",
"Try using --no-aks-mcp to disable MCP integration",
"Run without --aks-mcp to stay in traditional mode",
f"Manually install the binary from {self.GITHUB_RELEASES_URL} if available"
]
)
Expand Down
2 changes: 1 addition & 1 deletion src/aks-agent/azext_aks_agent/agent/error_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ class MCPError(AgentError):

def __init__(self, message: str, error_code: str = None, suggestions: List[str] = None):
default_suggestions = [
"Try running with --no-aks-mcp to use traditional mode",
"Try running without --aks-mcp to stay in traditional mode",
"Check your internet connection for MCP binary download",
"Verify that port 8003 is available for MCP server"
]
Expand Down
2 changes: 1 addition & 1 deletion src/aks-agent/azext_aks_agent/agent/mcp_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ async def start_server(self) -> bool:
"The MCP binary should be automatically downloaded",
"Check your internet connection for binary download",
"Try running the command again to retry download",
"Use --no-aks-mcp to disable MCP integration"
"Run without --aks-mcp to stay in traditional mode"
]
)

Expand Down
2 changes: 1 addition & 1 deletion src/aks-agent/azext_aks_agent/agent/status_models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ def get_recommendations(self) -> List[str]:
# Traditional mode specific recommendations
elif self.config.is_traditional_mode:
if self.mcp_binary.ready:
recommendations.append("Consider using MCP mode for enhanced capabilities (remove --no-aks-mcp)")
recommendations.append("Consider using MCP mode for enhanced capabilities (run with --aks-mcp)")

# General recommendations
if not self.config.config_valid:
Expand Down
6 changes: 3 additions & 3 deletions src/aks-agent/azext_aks_agent/custom.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def aks_agent(
show_tool_output=False,
refresh_toolsets=False,
status=False,
no_aks_mcp=False,
use_aks_mcp=False,
):
# If only status is requested, display and return early
if status:
Expand All @@ -47,7 +47,7 @@ def aks_agent(
no_echo_request,
show_tool_output,
refresh_toolsets,
no_aks_mcp,
use_aks_mcp=use_aks_mcp,
)


Expand Down Expand Up @@ -237,7 +237,7 @@ def _get_recommendations(status_info):
if binary_info.get("available"):
recommendations.append(
"Consider using MCP mode for enhanced capabilities by running 'az aks agent' "
"(remove --no-aks-mcp if used)"
"(run again with --aks-mcp to switch modes)"
)
else:
recommendations.append("✅ AKS agent is ready in traditional mode")
Expand Down
3 changes: 1 addition & 2 deletions src/aks-agent/azext_aks_agent/tests/latest/test_aks_agent_error_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def test_mcp_error_inheritance(self):

assert error.error_code == "MCP"
assert len(error.suggestions) > 0
assert any("--no-aks-mcp" in suggestion for suggestion in error.suggestions)
assert any("--aks-mcp" in suggestion for suggestion in error.suggestions)
assert any("internet connection" in suggestion.lower() for suggestion in error.suggestions)

def test_binary_error_inheritance(self):
Expand Down Expand Up @@ -221,4 +221,3 @@ def test_error_codes_uniqueness(self):

assert binary_download.error_code == "BINARY_DOWNLOAD"
assert binary_validation.error_code == "BINARY_VALIDATION"

36 changes: 28 additions & 8 deletions src/aks-agent/azext_aks_agent/tests/latest/test_aks_agent_mcp_integration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ def test_error_handler_functionality(self):
formatted_message = AgentErrorHandler.format_error_message(mcp_error)
assert "AKS Agent Error (MCP_SETUP)" in formatted_message
assert "Suggestions:" in formatted_message
assert "Try running with --no-aks-mcp" in formatted_message
assert "Try running without --aks-mcp" in formatted_message

def test_setup_traditional_mode_config_loading(self):
"""Test traditional mode setup with actual config loading."""
Expand Down Expand Up @@ -257,12 +257,22 @@ def test_aks_agent_calls_sync_implementation(self, mock_stdin):
# Mock stdin to avoid pytest capture issues
mock_stdin.isatty.return_value = True # No piped input

# Call the function with no_aks_mcp=True to avoid MCP setup
# Call the function with use_aks_mcp=False to avoid MCP setup
try:
aks_agent(
self.mock_cmd, "rg", "cluster", "test prompt", self.test_model,
self.test_api_key, self.test_max_steps, self.test_config_file,
False, False, False, False, True # no_aks_mcp=True
self.mock_cmd,
"rg",
"cluster",
"test prompt",
self.test_model,
self.test_api_key,
self.test_max_steps,
self.test_config_file,
False,
False,
False,
False,
use_aks_mcp=False,
)
except Exception as e:
# Expected to fail due to missing Holmes dependencies in test environment
Expand All @@ -283,9 +293,19 @@ def test_python_version_check(self, mock_stdin):

with pytest.raises(CLIError) as exc_info:
aks_agent(
self.mock_cmd, "rg", "cluster", "test prompt", self.test_model,
self.test_api_key, self.test_max_steps, self.test_config_file,
False, False, False, False, True # no_aks_mcp=True
self.mock_cmd,
"rg",
"cluster",
"test prompt",
self.test_model,
self.test_api_key,
self.test_max_steps,
self.test_config_file,
False,
False,
False,
False,
use_aks_mcp=False,
)

assert "upgrade the python version to 3.10" in str(exc_info.value)
27 changes: 13 additions & 14 deletions src/aks-agent/azext_aks_agent/tests/latest/test_aks_agent_parameters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,44 +13,44 @@
class TestAksAgentParameters(unittest.TestCase):
"""Test AKS Agent parameter parsing and functionality."""

def test_agent_function_signature_includes_no_aks_mcp(self):
"""Test that the agent function includes no_aks_mcp parameter."""
def test_agent_function_signature_includes_use_aks_mcp(self):
"""Test that the agent function includes use_aks_mcp parameter."""
import inspect
from azext_aks_agent.agent.agent import aks_agent

# Get function signature
sig = inspect.signature(aks_agent)

# Verify no_aks_mcp parameter exists
self.assertIn("no_aks_mcp", sig.parameters)
# Verify use_aks_mcp parameter exists
self.assertIn("use_aks_mcp", sig.parameters)

# Verify default value is False
param = sig.parameters["no_aks_mcp"]
param = sig.parameters["use_aks_mcp"]
self.assertEqual(param.default, False)

def test_custom_function_signature_includes_no_aks_mcp(self):
"""Test that the custom.py function includes no_aks_mcp parameter."""
def test_custom_function_signature_includes_use_aks_mcp(self):
"""Test that the custom.py function includes use_aks_mcp parameter."""
import inspect
from azext_aks_agent.custom import aks_agent

# Get function signature
sig = inspect.signature(aks_agent)

# Verify no_aks_mcp parameter exists
self.assertIn("no_aks_mcp", sig.parameters)
# Verify use_aks_mcp parameter exists
self.assertIn("use_aks_mcp", sig.parameters)

# Verify default value is False
param = sig.parameters["no_aks_mcp"]
param = sig.parameters["use_aks_mcp"]
self.assertEqual(param.default, False)

def test_parameter_boolean_type(self):
"""Test that no_aks_mcp parameter behaves as a boolean flag."""
"""Test that use_aks_mcp parameter behaves as a boolean flag."""
# Test default value behavior
import inspect
from azext_aks_agent.agent.agent import aks_agent

sig = inspect.signature(aks_agent)
param = sig.parameters["no_aks_mcp"]
param = sig.parameters["use_aks_mcp"]

# Should have a default value of False
self.assertIsInstance(param.default, bool)
Expand All @@ -63,10 +63,9 @@ def test_parameter_docstring_updated(self):
# Check if docstring mentions the new parameter
docstring = aks_agent.__doc__
self.assertIsNotNone(docstring, "Function should have a docstring")
self.assertIn("no_aks_mcp", docstring, "Docstring should mention no_aks_mcp parameter")
self.assertIn("use_aks_mcp", docstring, "Docstring should mention use_aks_mcp parameter")
self.assertIn("MCP", docstring, "Docstring should mention MCP")


if __name__ == '__main__':
unittest.main()

2 changes: 1 addition & 1 deletion src/aks-agent/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

from setuptools import find_packages, setup

VERSION = "1.0.0b2"
VERSION = "1.0.0b3"

CLASSIFIERS = [
"Development Status :: 4 - Beta",
Expand Down
7 changes: 7 additions & 0 deletions src/aks-preview/HISTORY.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ To release a new version, please select a new version number (usually plus 1 to
Pending
+++++++
* Add machine command `az aks machine add` to add a machine to an existing machine pool.
* Add blue-green upgrade strategy support for AKS node pools:
- `az aks nodepool add/update/upgrade`: Add `--upgrade-strategy` parameter to switch between rolling and blue-green nodepool upgrades.
- `az aks nodepool add/update/upgrade`: Add `--drain-batch-size`, `--drain-timeout-bg`, `--batch-soak-duration`, `--final-soak-duration` parameters to configure blue-green upgrade settings.

18.0.0b38
+++++++
* Add option `--acns-datapath-acceleration-mode <None|BpfVeth>` to `az aks create/update`

18.0.0b37
+++++++
Expand Down
Loading