Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Bug Report: Errors in metrics in the instrumentation of bedrock #2138

Open
1 task done
naghost-dev opened this issue Oct 15, 2024 · 2 comments · May be fixed by #2139
Open
1 task done

🐛 Bug Report: Errors in metrics in the instrumentation of bedrock #2138

naghost-dev opened this issue Oct 15, 2024 · 2 comments · May be fixed by #2139
Labels
bug Something isn't working

Comments

@naghost-dev
Copy link

Which component is this bug for?

Bedrock Instrumentation

📜 Description

Currently, the instrumentation of bedrock, the responses are not being handled properly, for example the response from amazon.titan-embed-image-v1 when not returning the 'results' field returns the error sum(int(result.get("tokenCount")) for result in response_body.get("results")), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable

👟 Reproduction steps

  1. Execute the model invocation: amazon.titan-embed-image-v1

👍 Expected behavior

It should correctly return the information in the trace spans and metrics

👎 Actual Behavior with Screenshots

It currently returns an exception
image

🤖 Python Version

3.11

📃 Provide any additional context for the Bug.

No response

👀 Have you spent some time to check if this bug has been raised before?

  • I checked and didn't find similar issue

Are you willing to submit PR?

Yes I am willing to submit a PR!

Copy link

dosubot bot commented Oct 15, 2024

Hey there, @naghost-dev! 👋 I'm here to help you with bug fixes, answer your questions, and guide you on becoming a contributor. Let's work together to resolve this issue.

The error you're encountering is due to the response_body.get("results") returning None, which leads to a TypeError when trying to iterate over it. To handle this, you can modify the _set_amazon_span_attributes function to check if response_body.get("results") is None before attempting to iterate over it. Here's a possible solution:

def _set_amazon_span_attributes(span, request_body, response_body, metric_params):
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_TYPE, LLMRequestTypeValues.COMPLETION.value
    )
    config = request_body.get("textGenerationConfig", {})
    _set_span_attribute(span, SpanAttributes.LLM_REQUEST_TOP_P, config.get("topP"))
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_TEMPERATURE, config.get("temperature")
    )
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_MAX_TOKENS, config.get("maxTokenCount")
    )

    results = response_body.get("results")
    if results is not None:
        _record_usage_to_span(
            span,
            response_body.get("inputTextTokenCount"),
            sum(int(result.get("tokenCount")) for result in results),
            metric_params,
        )

    if should_send_prompts():
        _set_span_attribute(
            span, f"{SpanAttributes.LLM_PROMPTS}.0.user", request_body.get("inputText")
        )

        if results is not None:
            for i, result in enumerate(results):
                _set_span_attribute(
                    span,
                    f"{SpanAttributes.LLM_COMPLETIONS}.{i}.content",
                    result.get("outputText"),
                )

This modification checks if results is None before iterating over it, preventing the TypeError. You can implement this change and test it to ensure it resolves the issue.

To continue talking to Dosu, mention @dosu.

@dosubot dosubot bot added the bug Something isn't working label Oct 15, 2024
@naghost-dev
Copy link
Author

naghost-dev commented Oct 15, 2024

PR#2139

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant