Detailed Upgrade Guidance for 0.56 Release

[jdisanti]

Generated smithy-rs clients have undergone an architectural overhaul that replaces the tower Service trait (referred to as the “middleware” implementation) with a new orchestration system that will be referred to as the “orchestrator” for the rest of this post. Users that implement middleware via tower Service or any of our MapRequest layers will experience significant breaking changes that will require refactoring to resolve.

This new implementation brings a much easier client creation API, and a less error-prone API for customizing the client that has more extension points and encourages best practices. It also brings smithy-rs generated clients closer to how generated clients work in other code generators (e.g., smithy-kotlin, smithy-swift, and others in the future) so that customizations written for one language’s Smithy clients can be easily ported to another language. In the long term, the overall high-level client architecture will be the same across the board for Smithy clients in all languages.

At the highest level, the orchestrator can be described by its parts:

• Config is used to configure the client. This hasn’t really changed much from the middleware implementation.
• Runtime Components are configurable implementations that the orchestrator will use to resolve endpoints, identities and auth schemes, make requests, and more.
• Runtime Plugins can be registered in the config to change runtime components and config.
• Interceptors are hook points into the request lifecycle that can read and mutate the input, request, response, and output. These are registered in config.

When a request is made, the config is augmented by the runtime plugins, and the runtime components for that request are established by the runtime plugins. The list of interceptors is retrieved from config, and then the orchestrator goes through a fixed request execution pipeline with various hook points for customization with interceptors and runtime components.

There isn’t a ton of documentation on how this all works under the hood yet, but we do have the following docs, as well as the docs of the aws-smithy-runtime-api and aws-smithy-runtime crates, which cover the finer details.

• RFC-0034: Smithy Orchestrator
• What is the ‘orchestrator’ and why does it exist?
• Identity and Auth in Clients
• aws-smithy-runtime-api docs (note: it may take a few days for docs.rs to update these after release)
• aws-smithy-runtime docs

Middleware deprecation

The release of 0.56 makes the new orchestrator the default client implementation. The middleware implementation is still around and can be opted into if necessary, but this his highly discouraged as it will be completely removed and unsupported in the next minor (non-patch) smithy-rs release. If you find that your app’s not behaving as expected after the upgrade, you have a few options:

• Submit a bug report or feature request. This is the best way to help improve the new orchestrator-based client and we’d like to hear what you have to say. Get in touch with us early so that we can make sure the next release meets your requirements.
• Downgrade to an older version. We’ve done our best to ensure that this isn’t necessary, but nobody’s perfect. If you’ve already tried the two approaches listed above, then you can still downgrade to v0.55.0.

Interceptors aren’t middleware

But they’re pretty close. Here are the key differences:

• Interceptors don’t require implementers to define complex generic bounds.
• Interceptors can’t redirect control flow in the same ways that middlewares can. While interceptors can return errors, they can’t stop other interceptors from running. Nor can an interceptor return a response early – i.e. middlewares that can “short-circuit” a call with a success response won’t be possible to implement as an interceptor.

Interceptors are a generic extension point with the goal of supporting “anything someone should be able to do” as opposed to “anything anyone might want to do.” Interceptors allow users to “inject” logic into specific stages of the request/response lifecycle. These stages are known as “hooks”.

Hooks are either read-only (read) or read/write (modify). The current list of hooks is as follows:

1. Read Before Execution - Called before anything happens. This is the first thing the SDK calls during operation execution.
2. Modify Before Serialization - Called before the input given by the user is serialized into an HTTP request. Allows modifying the input.
3. Read Before Serialization - The last thing the orchestrator calls before serializing the input into an HTTP request.
4. Read After Serialization - The first thing the orchestrator calls after serializing the input into an HTTP request.
5. (Retry Loop)
   1. Modify Before Retry Loop - The last thing the orchestrator calls before entering the retry loop. Allows modifying the HTTP request.
   2. Read Before Attempt - The first thing the orchestrator calls “inside” of the retry loop.
   3. Modify Before Signing - Before the HTTP request is signed. Allows modifying the HTTP request.
   4. Read Before Signing - The last thing the orchestrator calls before signing the HTTP request.
   5. Read After Signing - The first thing the orchestrator calls after signing the HTTP request.
   6. Modify Before Transmit - Before the HTTP request is sent to the service. Allows modifying the HTTP request.
   7. Read Before Transmit - The last thing the orchestrator calls before sending the HTTP request.
   8. Read After Transmit - The last thing the orchestrator calls after receiving the HTTP response.
   9. Modify Before Deserialization - Before the HTTP response is deserialized. Allows modifying the HTTP response.
   10. Read Before Deserialization - The last thing the orchestrator calls before deserializing the HTTP response into an output.
   11. Read After Deserialization - The last thing the orchestrator calls after deserializing the HTTP response into an output.
   12. Modify Before Attempt Completion - Before the retry loop ends. Allows modifying the output.
   13. Read After Attempt - The last thing the orchestrator calls “inside” of the retry loop.
6. Modify Before Execution Completion - Before the execution ends. Allows modifying the output.
7. Read After Execution - The final hook in the request/response lifecycle.

A hook will always run all applicable interceptors, even if one fails. If one or more interceptors run by a hook fail, orchestrator execution will skip ahead. The hook that is skipped to depends on whether the retry loop was entered. The Modify Before Attempt Completion and Read After Attempt hooks will always run if an interceptor fails during the retry loop. The Modify Before Execution Completion and Read After Execution hooks will always run.

If you’re relying on custom middlewares, you’ll need to convert them to the interceptor pattern in order to upgrade the SDK. Let’s look at two of the middlewares the SDK team had to convert as an example.

What hook(s) should I implement for my interceptor?

While it’s possible for a single interceptor to implement all of the above hooks at once, implementing one is enough for most use-cases. The question, then, is “which one should I implement?” When making this decision, here are a few things to consider:

1. Do you need to access the interceptor context (input, request, response, or output)?
   1. Your interceptor will have to run in a hook where the necessary context exists. For example, You can’t access the HTTP request until after it’s been serialized by the orchestrator. The Read After Serialization hook is the earliest time that that’s possible.
2. Do you need to modify the interceptor context or just read it?
   1. Modifying the context (or the config bag) can only be done in a Modify_* hook.
3. Should your interceptor run once for each attempt (including retries) or just once for all attempts?
   1. Hooks occurring outside of the retry loop will only be run once. Hooks inside the retry loop will run once for each attempt (the initial request as well as retry attempts.)
4. Do you want the interceptor to always run, no matter what?
   1. Interceptors that must always run should implement the Modify Before Execution Completion or Read After Execution hook. If the retry loop is entered, then the Modify Before Attempt Completion and Read After Attempt hooks will be run; However, if the orchestrator encounters an error before entering the retry loop, then it will skip to the Modify Before Execution Completion hook, so they may not always run.
5. If you need to grab a value from the ConfigBag, when is that value set?
   1. Your interceptor will have to run in the next hook after the value is set.
6. Does your interceptor insert headers into the request?
   1. Implement the Modify Before Signing hook (or an earlier hook) so that the inserted headers will be included when signing the request. Otherwise, the service will respond with a signature error.

NOTE: The most common hook implemented by our internal interceptors is Modify Before Signing. The second most common hook is Modify Before Transmit*.
*

What if my middleware modifies or replaces the request body?

Middleware that completely replace the HTTP request body will not translate to interceptors well. You can use an interceptor to std::mem::swap the HTTP body out with another one (and SdkBody can wrap any arbitrary streaming body implementation), but unless you replace the body with an in-memory variant of SdkBody, you will lose the ability to retry. Note: you would have potentially lost the ability to retry in middleware also, depending on how the middleware was implemented.

If you need to replace the HTTP body with one that is streaming as opposed to in-memory, you should do it as a custom HttpConnector instead of trying to do it in an interceptor.

Setting interceptors

All necessary interceptors are set by default, but users may want to define and set their own. Interceptors can be added one-at-a-time or in a list:

#[tokio::main]
async fn main() {
    let sdk_config = aws_config::load_from_env();

    let conf = aws_sdk_s3::config::Builder::from(&sdk_config)
        // Option #1:
        // Add interceptors by repeatedly calling the ``interceptor`` method
        .interceptor(YourInterceptor::new())
        .build();
    let client = aws_sdk_s3::Client::from_conf(conf);
}

The above method works by taking ownership of self. The two methods below take a mutable reference to self.

#[tokio::main]
async fn main() {
    let sdk_config = aws_config::load_from_env();

    let mut conf_builder = aws_sdk_s3::config::Builder::from(&sdk_config);
     // Option #2:
    // Add interceptors by repeatedly calling the `add_interceptor` method.
    // You'll need to wrap each one in a `SharedInterceptor`.
    conf_builder.push_interceptor(SharedInterceptor::new(YourInterceptor::new()));
    // Option #3:
    // Set all of your interceptors at once by passing in an iterator/vec.
    // You'll need to wrap each one in a `SharedInterceptor`.
    // **This will unset interceptors set using methods #1 or #2**
    conf_builder.set_interceptors(vec![
        SharedInterceptor::new(YourInterceptor::new()),
        SharedInterceptor::new(YourInterceptor::new()),
    ]);
    let conf = conf_builder.build();
    let client = aws_sdk_s3::Client::from_conf(conf);
}

Example #1: Updating the recursion detection middleware

The recursion detection middleware is simple, but has a very important job. It detects when an SDK is being run in a Lambda and inserts a special tracking header so that services handling the request can avoid infinitely recursive lambda invocations.

Both the middleware and interceptors implementations rely on the same inner function:

fn augument_request<B>(req: &mut http::Request<B>, env: &Env) {
    // Do nothing if a trace header was already set
    if req.headers().contains_key(TRACE_ID_HEADER) {
        return;
    }
     // Check to see if we're running in a lambda and then insert the trace ID
    // into the request.
    if let (Ok(_function_name), Ok(trace_id)) =
        (env.get(env::LAMBDA_FUNCTION_NAME), env.get(TRACE_ID))
    {
        req.headers_mut()
            .insert(TRACE_ID_HEADER, encode_header(trace_id.as_bytes()));
    }
}

The middleware is defined as a struct that implements the MapRequest helper trait:

#[derive(Default, Debug, Clone)]
pub struct RecursionDetectionStage {
    env: Env,
}

impl RecursionDetectionStage {
    /// Creates a new `RecursionDetectionStage`
    pub fn new() -> Self {
        Self::default()
    }
}

impl MapRequest for RecursionDetectionStage {
    type Error = std::convert::Infallible;

    fn name(&self) -> &ˈstatic str {
        "recursion_detection"
    }

    fn apply(&self, request: Request) -> Result<Request, Self::Error> {
        request.augment(|mut req, _conf| {
            augument_request(&mut req, &self.env);
            Ok(req)
        })
    }
}

When converted to the new interceptor pattern, it looks like this:

#[derive(Debug, Default)]
pub struct RecursionDetectionInterceptor {
    env: Env,
}

impl RecursionDetectionInterceptor {
    /// Creates a new `RecursionDetectionInterceptor`
    pub fn new() -> Self {
        Self::default()
    }
}

impl Interceptor for RecursionDetectionInterceptor {
    fn modify_before_signing(
        &self,
        context: &mut BeforeTransmitInterceptorContextMut<ˈ_>,
        _cfg: &mut ConfigBag,
    ) -> Result<(), BoxError> {
        let request = context.request_mut();
        augment_request(request, &self.env);
        Ok(())
    }
}

Not so different, right? The most significant decision when converting simple middleware into interceptors is what hook(s) to implement. In this case, we chose the modify_before_signing hook, inserting the trace ID header right before signing the request. That way, the configured signer may choose to sign it.

Example #2: Reconnecting when retrying 50X errors

By default, the AWS SDK for Rust won’t re-use a connection if it’s retrying a server error. This helps avoid sending another request to a server that’s struggling to respond, instead allowing another server to handle it. The way this is implemented is a bit complex but that’s what makes it a good example.

The PoisonService middleware is responsible for “poisoning” connections to servers that have returned 50X errors. When a connection is “poisoned” that signals to the underlying HTTP client that it shouldn’t be re-used for future requests.

Because the PoisonService is a bit more complex, it implements the tower::Service trait instead of the MapRequest helper trait:

pub(crate) struct PoisonLayer<S> {
    inner: PhantomData<S>,
    mode: ReconnectMode,
}

impl<S> PoisonLayer<S> {
    pub(crate) fn new(mode: ReconnectMode) -> Self {
        Self {
            inner: Default::default(),
            mode,
        }
    }
}

impl<S> Clone for PoisonLayer<S> {
    fn clone(&self) -> Self {
        Self {
            inner: Default::default(),
            mode: self.mode,
        }
    }
}

impl<S> tower::Layer<S> for PoisonLayer<S> {
    type Service = PoisonService<S>;

    fn layer(&self, inner: S) -> Self::Service {
        PoisonService {
            inner,
            mode: self.mode,
        }
    }
}

#[derive(Clone)]
pub(crate) struct PoisonService<S> {
    inner: S,
    mode: ReconnectMode,
}

impl<H, R, S, O, E> tower::Service<Operation<H, R>> for PoisonService<S>
where
    R: ClassifyRetry<SdkSuccess<O>, SdkError<E>>,
    S: tower::Service<Operation<H, R>, Response = SdkSuccess<O>, Error = SdkError<E>>,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = PoisonServiceFuture<S::Future, R>;

    fn poll_ready(&mut self, cx: &mut Context<ˈ_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }

    fn call(&mut self, mut req: Operation<H, R>) -> Self::Future {
        let classifier = req.retry_classifier().clone();
        // Create a special struct that will "capture" a connection when a request
        // is set. If the response that comes back is a 50X error, we'll call the 
        // .poison() method on the captured connection before retrying the request.
        let capture_smithy_connection = CaptureSmithyConnection::new();
        // The HTTP Client which manages connections will react to seeing this
        // in the property bag by inserting a reference to the connection into
        // the capture_smithy_connection struct.
        req.properties_mut()
            .insert(capture_smithy_connection.clone());
        PoisonServiceFuture {
            inner: self.inner.call(req),
            conn: capture_smithy_connection,
            mode: self.mode,
            classifier,
        }
    }
}

pin_project! {
    pub struct PoisonServiceFuture<F, R> {
        #[pin]
        inner: F,
        classifier: R,
        conn: CaptureSmithyConnection,
        mode: ReconnectMode
    }
}

impl<F, R, T, E> Future for PoisonServiceFuture<F, R>
where
    F: Future<Output = Result<SdkSuccess<T>, SdkError<E>>>,
    R: ClassifyRetry<SdkSuccess<T>, SdkError<E>>,
{
    type Output = F::Output;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<ˈ_>) -> Poll<Self::Output> {
        let this = self.project();
        match this.inner.poll(cx) {
            Poll::Ready(resp) => {
                let retry_kind = this.classifier.classify_retry(resp.as_ref());
                // This is where we check the response and poison the connection
                // if necessary.
                if this.mode == &ReconnectMode::ReconnectOnTransientError
                    && retry_kind == RetryKind::Error(ErrorKind::TransientError)
                {
                    if let Some(smithy_conn) = this.conn.get() {
                        tracing::info!("poisoning connection: {:?}", smithy_conn);
                        smithy_conn.poison();
                    } else {
                        tracing::trace!("No smithy connection found! The underlying HTTP connection never set a connection.");
                    }
                }
                Poll::Ready(resp)
            }
            Poll::Pending => Poll::Pending,
        }
    }
}

When converted to an interceptor, we end up with this:

#[derive(Debug, Default)]
pub struct ConnectionPoisoningInterceptor {}

impl ConnectionPoisoningInterceptor {
    pub fn new() -> Self {
        Self::default()
    }
}

impl Interceptor for ConnectionPoisoningInterceptor {
    fn modify_before_transmit(
        &self,
        context: &mut BeforeTransmitInterceptorContextMut<ˈ_>,
        cfg: &mut ConfigBag,
    ) -> Result<(), BoxError> {
    
        let capture_smithy_connection = CaptureSmithyConnectionWrapper::new();
        // In this case, the HTTP client can't see into interceptor state, so we
        // have to store the capture_smithy_connection struct into the request's
        // .extensions() property bag.
        context
            .request_mut()
            .extensions_mut()
            .insert(capture_smithy_connection.clone_inner());
        cfg.interceptor_state().store_put(capture_smithy_connection);

        Ok(())
    }

    fn modify_before_deserialization(
        &self,
        context: &mut BeforeDeserializationInterceptorContextMut<ˈ_>,
        cfg: &mut ConfigBag,
    ) -> Result<(), BoxError> {
        let reconnect_mode = cfg
            .load::<RetryConfig>()
            .map(RetryConfig::reconnect_mode)
            .unwrap_or(ReconnectMode::ReconnectOnTransientError);
        let captured_connection = cfg.load::<CaptureSmithyConnectionWrapper>().cloned();
        let retry_classifiers = cfg.retry_classifiers();

        let error_is_transient = retry_classifiers
            .classify_retry(context.into_inner())
            .map(|reason| reason == RetryReason::Error(ErrorKind::TransientError))
            .unwrap_or_default();
        let connection_poisoning_is_enabled =
            reconnect_mode == ReconnectMode::ReconnectOnTransientError;

        if error_is_transient && connection_poisoning_is_enabled {
            debug!("received a transient error, poisoning the connection...");

            // After we get a response and it's a 50X error, we poison the captured
            // connection.
            if let Some(captured_connection) = captured_connection.and_then(|conn| conn.get()) {
                captured_connection.poison();
                debug!("the connection was poisoned")
            } else {
                error!(
                    "unable to poison the connection because no connection was found! The underlying HTTP connector never set a connection."
                );
            }
        }

        Ok(())
    }
}

Summary

We hope this new method of extending the request/response lifecycle will be easy to use, but if you have trouble upgrading your middlewares, please file an issue and we’ll do our best to advise you.

Config hasn’t changed much

While the internals of SDK configuration has changed, we’ve put effort into ensuring the user experience remains the same. SdkConfig and service specific configs will have all the same fields they did before, as well as a few additions:

• push_interceptor: Add a SharedInterceptor that runs for specific hooks in the request/response lifecycle. Generally speaking, interceptors set to run on a hook are executed according to the pre-defined priority in a non-deterministic order. However, the default set of interceptors provided by the SDK are guaranteed to run before interceptors set by the user, ensuring the user always has the final say.
  • A SharedInterceptor is a wrapper type for structs implementing Interceptor.
• time_source: Set the time source for a client. The time source is used to provide the current time. By default, this will use the host machine’s system time. If you’re running on WASM, you’ll have to set your own time source depending on your specific runtime.
  • Setting a custom time source is useful for some kinds of tests.

Generic clients are simpler to use

If you’re generating a generic smithy client, client configuration and construction works just like it does for SDK clients, but with less defaults:

#[tokio::main]
async fn main () {
    let config = generic_smithy_service::Config::builder()
        // To resolve a specific endpoint, pass a `&'str`.
        .endpoint_resolver("https://localhost:13734")
        // A sleep implementation is needed for retries and timeouts.
        .sleep_impl(SharedAsyncSleep::new(default_async_sleep().unwrap()))
        // A source of time is also needed.
        .time_source(SharedTimeSource::new(SystemTimeSource::new()))
        // Retry configuration works the same for generic and SDK clients
        .retry_config(RetryConfig::standard())
        // Timeout configuration works the same for generic and SDK clients
        .timeout_config(TimeoutConfig::builder()
            .operation_timeout(Duration::from_secs(30))
            .operation_attempt_timeout(Duration::from_secs(10))
            .connect_timeout(Duration::from_secs(3))
            .read_timeout(Duration::from_secs(3))
            .build()
        )
        .build();
    // Clients are created from a Config.
    let client = generic_smithy_service::client::Client::from_conf(config);

    let res = client.get_something()
        // Operation parameter methods would be called here
        .send()
        .await
        .expect("request is successful");

    println!("{res:?}");
}

CustomizableOperation::map_operation has been removed

A customizable operation allows a user to modify the current operation before it is dispatched. map_request and mutate_request on CustomizableOperation can be used in the orchestrator in the exactly the same way as before. However, map_operation will no longer be supported since the orchestrator does not use the aws_smithy_http::operation::Operation struct.

The following examples demonstrate how to upgrade existing code that uses map_operation in order to achieve the same functionality in the orchestrator.

Example #1: Inserting an item into a property bag

Previously map_operation allowed users to put an item into aws_smithy_http::property_bag::PropertyBag like so:

let sdk_config: aws_types::sdk_config::SdkConfig = /* create an SdkConfig */;
let s3_client = aws_sdk_s3::client::Client::new(&sdk_config);
s3_client
    .get_object()
    .customize()
    .await?
    .map_operation(|mut op| {
        op.properties_mut()
            .insert(SigningService::from_static(/* a string for signing */));
        Ok::<_, Infallible>(op)
    })
    .unwrap()
    .send()
    .await;

This will no longer compile in the orchestrator because Operation will cease to exist. You can upgrade the above code as follows (or any use case that inserts an item into the PropertyBag for that matter):

// Somewhere in your crate
use aws_smithy_runtime_api::box_error::BoxError;
use aws_smithy_runtime_api::client::interceptors::Interceptor;
use aws_smithy_runtime_api::client::interceptors::context::BeforeTransmitInterceptorContextMut;
use aws_smithy_types::config_bag::ConfigBag;

#[derive(Debug)]
struct CustomSigningInterceptor;

impl Interceptor for CustomSigningInterceptor {
     fn modify_before_signing(
        &self,
        // The context allows access to inputs, requests, responses, and outputs
        // depending on the hook. We don't need to access those in this example.
        _context: &mut BeforeTransmitInterceptorContextMut<ˈ_>,
        cfg: &mut ConfigBag,
    ) -> Result<(), BoxError> {
        cfg
            .interceptor_state()
            .store_put(SigningService::from_static(/* a string for signing */));
    }
}

// Use `CustomSigningInterceptor` as follows

let sdk_config: aws_types::sdk_config::SdkConfig = /* create an SdkConfig */;
let s3_client = aws_sdk_s3::client::Client::new(&sdk_config);
s3_client
    .get_object()
    .customize()
    .await?
    .interceptor(CustomSigningInterceptor)
    .send()
    .await;

Just remember that items to be stored using store_put need to implement the Storable trait (here is an example of how a type can be made Storable).
Furthermore, the Interceptor trait defines several methods, and which one to use depends on the existing code to upgrade (the example above happens to use modify_before_signing). Refer back to What hook(s) should I implement for my interceptor? for more details.

Example #2: Extracting information from aws_smithy_http::operation::Metadata

map_operation used to allow users to extract fields in a Metadata, which is a struct attached to an operation that identifies the API being called. It can be useful to obtain a service name and an operation name for a use case such as metrics collection. For instance,

let sdk_config: aws_types::sdk_config::SdkConfig = /* create an SdkConfig */;
let s3_client = aws_sdk_s3::client::Client::new(&sdk_config);
s3_client
    .get_object()
    .customize()
    .await?
    .map_operation(|operation| {
        operation.try_clone().map(|operation| {
            let (_, parts) = operation.into_request_response();
            parts.metadata.map(|metadata| {
                let service_name = metadata.service().to_string().to_uppercase();
                let operation_name = metadata.name().to_string();
                /*
                 * do something with `service_name` and `operation_name`
                 */
            })
        });

        Ok(operation)
    })?
    .send()
    .await;

This piece of code can be upgraded as follows. You need to define your interceptor that extracts a Metadata out of ConfigBag:

// Somewhere in your crate
use aws_smithy_runtime_api::box_error::BoxError;
use aws_smithy_runtime_api::client::interceptors::Interceptor;
use aws_smithy_runtime_api::client::interceptors::context::BeforeTransmitInterceptorContextMut;
use aws_smithy_types::config_bag::ConfigBag;

#[derive(Debug)]
struct ServiceOperationNamesInterceptor;

impl Interceptor for ServiceOperationNamesInterceptor {
      // We show `modify_before_signing` for the purpose of this example.
     // Choose a different trait method to implement depending on the use case.
     fn modify_before_signing(
        &self,
        _context: &mut BeforeTransmitInterceptorContextMut<ˈ_>,
        cfg: &mut ConfigBag,
    ) -> Result<(), BoxError> {
        let metadata = cfg.load::<aws_smithy_http::operation::Metadata>().expect("metadata should exist");
        let service_name = metadata.service().to_string();
        let operation_name = metadata.name().to_string();
        /*
         * do something with `service_name` and `operation_name`
         */ 
    }
}

// Use `ServiceOperationNamesInterceptor` as follows

let sdk_config: aws_types::sdk_config::SdkConfig = /* create an SdkConfig */;
let s3_client = aws_sdk_s3::client::Client::new(&sdk_config);
s3_client
    .get_object()
    .customize()
    .await?
    .interceptor(ServiceOperationNamesInterceptor)
    .send()
    .await;