Skip to content

fix(anthropic): temp disable beta apis instrumentation #3258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
galkleinman merged 2 commits into from
Aug 14, 2025
Merged

fix(anthropic): temp disable beta apis instrumentation #3258

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b144213
temp(anthropic): disable beta apis instrumentation
galkleinman Aug 14, 2025
1728094
disable related tests
galkleinman Aug 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 68 additions & 56 deletions ...entelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,32 +81,32 @@
"method": "stream",
"span_name": "anthropic.chat",
},
# Beta API methods (regular Anthropic SDK)
{
"package": "anthropic.resources.beta.messages.messages",
"object": "Messages",
"method": "create",
"span_name": "anthropic.chat",
},
{
"package": "anthropic.resources.beta.messages.messages",
"object": "Messages",
"method": "stream",
"span_name": "anthropic.chat",
},
# Beta API methods (Bedrock SDK)
{
"package": "anthropic.lib.bedrock._beta_messages",
"object": "Messages",
"method": "create",
"span_name": "anthropic.chat",
},
{
"package": "anthropic.lib.bedrock._beta_messages",
"object": "Messages",
"method": "stream",
"span_name": "anthropic.chat",
},
# # Beta API methods (regular Anthropic SDK)
Copy link
Copy Markdown
Contributor

@ellipsis-dev ellipsis-dev Bot Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temporarily disabled beta API instrumentation by commenting out blocks. Consider using a feature flag or config option to conditionally enable beta instrumentation instead of leaving commented-out code.

# {
# "package": "anthropic.resources.beta.messages.messages",
# "object": "Messages",
# "method": "create",
# "span_name": "anthropic.chat",
# },
# {
# "package": "anthropic.resources.beta.messages.messages",
# "object": "Messages",
# "method": "stream",
# "span_name": "anthropic.chat",
# },
# # Beta API methods (Bedrock SDK)
# {
# "package": "anthropic.lib.bedrock._beta_messages",
# "object": "Messages",
# "method": "create",
# "span_name": "anthropic.chat",
# },
# {
# "package": "anthropic.lib.bedrock._beta_messages",
# "object": "Messages",
# "method": "stream",
# "span_name": "anthropic.chat",
# },
]
Comment on lines +84 to 110
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Sanity-check: ensure no remaining references to now-disabled beta wrappers

Since beta wrappers are disabled, verify there are no left-over tests or code paths expecting those spans.

Run this script to locate potential references to disabled beta APIs across the repo:

Also applies to: 125-151

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find beta messages and bedrock beta wrapper usages
rg -n -A2 -B2 'resources\.beta\.messages|lib\.bedrock\._beta_messages' || true

# Find bedrock async client usages to inspect related tests
rg -n -A3 -B3 'AsyncAnthropicBedrock|with_raw_response' || true

Length of output: 24853

Remove stale beta wrapper references

The tests and commented‐out code for beta wrappers remain in the repo even though beta spans are now disabled. Please clean these up:

• packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/init.py
– Lines 84–110 and 125–151: commented‐out beta API entries for both regular and async methods.
→ Consider deleting these blocks if they’re no longer needed.

• packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py
– Lines 132–147: test_async_anthropic_bedrock_beta_with_raw_response still invokes async_anthropic_bedrock_client.beta.messages.with_raw_response.create.
→ Remove or adapt this test to reflect that beta wrappers have been disabled.

🤖 Prompt for AI Agents 
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py
lines 84–110 and 125–151: the file contains stale, commented-out beta API
wrapper entries for regular and async methods; delete these commented blocks to
remove dead code and keep the file clean.
packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py
lines 132–147: the test calls
async_anthropic_bedrock_client.beta.messages.with_raw_response.create which
relies on disabled beta wrappers; remove or modify this test to call the
supported non-beta API (or delete the test if no equivalent exists) and update
any assertions accordingly.


WRAPPED_AMETHODS = [
Expand All @@ -122,32 +122,32 @@
"method": "create",
"span_name": "anthropic.chat",
},
# Beta API async methods (regular Anthropic SDK)
{
"package": "anthropic.resources.beta.messages.messages",
"object": "AsyncMessages",
"method": "create",
"span_name": "anthropic.chat",
},
{
"package": "anthropic.resources.beta.messages.messages",
"object": "AsyncMessages",
"method": "stream",
"span_name": "anthropic.chat",
},
# Beta API async methods (Bedrock SDK)
{
"package": "anthropic.lib.bedrock._beta_messages",
"object": "AsyncMessages",
"method": "create",
"span_name": "anthropic.chat",
},
{
"package": "anthropic.lib.bedrock._beta_messages",
"object": "AsyncMessages",
"method": "stream",
"span_name": "anthropic.chat",
},
# # Beta API async methods (regular Anthropic SDK)
# {
# "package": "anthropic.resources.beta.messages.messages",
# "object": "AsyncMessages",
# "method": "create",
# "span_name": "anthropic.chat",
# },
# {
# "package": "anthropic.resources.beta.messages.messages",
# "object": "AsyncMessages",
# "method": "stream",
# "span_name": "anthropic.chat",
# },
# # Beta API async methods (Bedrock SDK)
# {
# "package": "anthropic.lib.bedrock._beta_messages",
# "object": "AsyncMessages",
# "method": "create",
# "span_name": "anthropic.chat",
# },
# {
# "package": "anthropic.lib.bedrock._beta_messages",
# "object": "AsyncMessages",
# "method": "stream",
# "span_name": "anthropic.chat",
# },
]


Expand Down Expand Up @@ -183,6 +183,7 @@ async def _aset_token_usage(
choice_counter: Counter = None,
):
from opentelemetry.instrumentation.anthropic.utils import _aextract_response_data

response = await _aextract_response_data(response)

if usage := response.get("usage"):
Expand Down Expand Up @@ -276,6 +277,7 @@ def _set_token_usage(
choice_counter: Counter = None,
):
from opentelemetry.instrumentation.anthropic.utils import _extract_response_data

response = _extract_response_data(response)

if usage := response.get("usage"):
Expand Down Expand Up @@ -443,7 +445,10 @@ async def _ahandle_response(span: Span, event_logger: Optional[EventLogger], res
else:
if not span.is_recording():
return
from opentelemetry.instrumentation.anthropic.span_utils import aset_response_attributes
from opentelemetry.instrumentation.anthropic.span_utils import (
aset_response_attributes,
)

await aset_response_attributes(span, response)


Expand Down Expand Up @@ -669,7 +674,10 @@ async def _awrap(
kwargs,
)
elif response:
from opentelemetry.instrumentation.anthropic.utils import ashared_metrics_attributes
from opentelemetry.instrumentation.anthropic.utils import (
ashared_metrics_attributes,
)

metric_attributes = await ashared_metrics_attributes(response)

if duration_histogram:
Expand Down Expand Up @@ -774,9 +782,13 @@ def _instrument(self, **kwargs):
wrapped_method,
),
)
logger.debug(f"Successfully wrapped {wrap_package}.{wrap_object}.{wrap_method}")
logger.debug(
f"Successfully wrapped {wrap_package}.{wrap_object}.{wrap_method}"
)
except Exception as e:
logger.debug(f"Failed to wrap {wrap_package}.{wrap_object}.{wrap_method}: {e}")
logger.debug(
f"Failed to wrap {wrap_package}.{wrap_object}.{wrap_method}: {e}"
)
pass # that's ok, we don't want to fail if some methods do not exist

for wrapped_method in WRAPPED_AMETHODS:
Expand Down
55 changes: 40 additions & 15 deletions packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,19 @@ def async_anthropic_bedrock_client(instrument_legacy):
return AsyncAnthropicBedrock(
aws_region=aws_region,
aws_access_key=aws_access_key,
aws_secret_key=aws_secret_key
aws_secret_key=aws_secret_key,
)


@pytest.mark.skip
@pytest.mark.asyncio
@pytest.mark.vcr
async def test_async_anthropic_bedrock_with_raw_response(
instrument_legacy, async_anthropic_bedrock_client, span_exporter, log_exporter, reader
instrument_legacy,
async_anthropic_bedrock_client,
span_exporter,
log_exporter,
reader,
):
"""Test that AsyncAnthropicBedrock with_raw_response.create generates spans"""
response = await async_anthropic_bedrock_client.messages.with_raw_response.create(
Expand All @@ -53,7 +58,11 @@ async def test_async_anthropic_bedrock_with_raw_response(
)
assert (anthropic_span.attributes[f"{SpanAttributes.LLM_PROMPTS}.0.role"]) == "user"
# For raw response, content is accessed differently
response_content = response.parse().content[0].text if hasattr(response, 'parse') else response.content[0].text
response_content = (
response.parse().content[0].text
if hasattr(response, "parse")
else response.content[0].text
)
assert (
anthropic_span.attributes.get(f"{SpanAttributes.LLM_COMPLETIONS}.0.content")
== response_content
Expand All @@ -71,10 +80,15 @@ async def test_async_anthropic_bedrock_with_raw_response(
)


@pytest.mark.skip
@pytest.mark.asyncio
@pytest.mark.vcr
async def test_async_anthropic_bedrock_regular_create(
instrument_legacy, async_anthropic_bedrock_client, span_exporter, log_exporter, reader
instrument_legacy,
async_anthropic_bedrock_client,
span_exporter,
log_exporter,
reader,
):
"""Test that regular AsyncAnthropicBedrock create works (for comparison)"""
response = await async_anthropic_bedrock_client.messages.create(
Expand Down Expand Up @@ -115,21 +129,28 @@ async def test_async_anthropic_bedrock_regular_create(
)


@pytest.mark.skip
@pytest.mark.asyncio
@pytest.mark.vcr
async def test_async_anthropic_bedrock_beta_with_raw_response(
instrument_legacy, async_anthropic_bedrock_client, span_exporter, log_exporter, reader
instrument_legacy,
async_anthropic_bedrock_client,
span_exporter,
log_exporter,
reader,
):
"""Test that AsyncAnthropicBedrock beta.messages.with_raw_response.create generates spans"""
response = await async_anthropic_bedrock_client.beta.messages.with_raw_response.create(
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Tell me a joke about OpenTelemetry",
}
],
model="anthropic.claude-3-haiku-20240307-v1:0",
response = (
await async_anthropic_bedrock_client.beta.messages.with_raw_response.create(
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Tell me a joke about OpenTelemetry",
}
],
model="anthropic.claude-3-haiku-20240307-v1:0",
)
)

spans = span_exporter.get_finished_spans()
Expand All @@ -143,7 +164,11 @@ async def test_async_anthropic_bedrock_beta_with_raw_response(
)
assert (anthropic_span.attributes[f"{SpanAttributes.LLM_PROMPTS}.0.role"]) == "user"
# For raw response, content is accessed differently
response_content = response.parse().content[0].text if hasattr(response, 'parse') else response.content[0].text
response_content = (
response.parse().content[0].text
if hasattr(response, "parse")
else response.content[0].text
)
assert (
anthropic_span.attributes.get(f"{SpanAttributes.LLM_COMPLETIONS}.0.content")
== response_content
Expand Down