Signing OkHttp requests with the AWS V4 signing algorithm

A lot of companies nowadays depend on services provided by Amazon. Babbel is no exception. Calling these services through Http usually requires the requests to be signed. Although Amazon provides a vast amount of libraries that handle this for you, sometimes you need to use your own. Maybe the level of customization you’re seeking is not possible with the provided libraries. Maybe you want to add a feature that uses other 3rd party libraries that are incompatible with the ones provided by Amazon. Or maybe you simply want to avoid adding an entire library just to use a very small part of it. In this blog post, I’ll introduce an open source library we’ve built here at Babbel that signs OkHttp requests.

---

Some years back we’ve decided to start using the Amazon API Gateway to service our frontend apps. In the Android app, we’ve started by using the Amazon Android SDK. At the time, the API gateway SDK was quite new and some of the functionality we needed was missing. We needed to set our own user agent, but this was overridden by the SDK. We required to intercept some requests and manipulate them, but the SDK design didn’t allow to do this in an easy way. Perhaps all of this is already possible to do with the Amazon libraries (in fact Babbel contributed with a PR to prevent the User Agent from being overridden), but it’s long since we’ve decided on another approach. Quite early in our integration of the Amazon API Gateway we’ve decided to drop the API gateway SDK and went with Retrofit.

However, not all dependencies to the Amazon SDK were removed. We’ve kept a dependency on the core SDK so we could use the signer written by Amazon.

This was not the ideal scenario. If you know Retrofit, you know it depends on OkHttp. This means that every request will be an OkHttp request. By contrast, the Amazon signer uses requests from the Amazon SDK, which are different than the ones used in OkHttp. Here’s the current signer signature for reference (comments are stripped for simplicity):

package com.amazonaws.auth;

import com.amazonaws.Request;

public interface Signer {
    public void sign(Request<?> request, AWSCredentials credentials);
}

This lead to an in-between stage where we were copying the OkHttp request into an Amazon request so we could pass it to the Amazon signer. Granted that this glue-code was not the most efficient one, it still worked and enabled us to forget about maintaining the signing code.

Unfortunately, recently we’ve tried to add support for Android Pie and discovered that the signer is incompatible with this Android version. The code was trying to get a logger using the Apache Commons Logging facility and this is incompatible with Android Pie. At the time of writing this post, Amazon has fixed the issue as you can see here. However, while adding support in our app for Android Pie we had no fix yet.

We were faced with 2 choices:

1. Fix it ourselves and submit a PR to Amazon for feedback and get it possibly merged
2. Implement the signing ourselves completely

Since we had already acknowledged that the glue-code copying the OkHttp request into an Amazon request wasn’t the best one and taking into account that the signing algorithm is well known, we’ve decided to build our own library – okhttp-aws-signer. Moreover, given that tests are provided (along with a lot of other resources) by Amazon, it becomes quite simple to implement the signing algorithm.

How does it work?

The idea is to create a signer object with the region of your service and the name of the service you want to execute.

val signer = OkHttpAwsV4Signer("eu-west-1", "execute-API")

From there one can use the signer to sign the OkHttp requests.

signer.sign(request, accessKeyId, secretAccessKey)

It’s important to note that the particular implementation of the algorithm requires the host and x-amz-date header to be present in the request. Therefore, it might be good to guarantee to have them. Moreover, the date must be formatted with the pattern yyyyMMdd'T'HHmmss'Z'.

val newRequest = request.newBuilder()
    .addHeader("host", request.url().host())
    .addHeader("x-amz-date", SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'", Locale.US).format(Date()))
    .build()

signer.sign(newRequest, accessKeyId, secretAccessKey)

How can we integrate this with Retrofit?

In the end, having the signer is already a great step to remove the only dependency we had on Amazon. However, we still need to have it integrated with Retrofit. We’ve chosen to build an interceptor to sign all requests. Here’s how it looks like:

class AwsSigingInterceptor(private val signer: OkHttpAwsV4Signer) : Interceptor {
    private val dateFormat: ThreadLocal<SimpleDateFormat>

    init {
        dateFormat = object : ThreadLocal<SimpleDateFormat>() {
            override fun initialValue(): SimpleDateFormat {
                val localFormat = SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'", Locale.US)
                localFormat.timeZone = TimeZone.getTimeZone("UTC")
                return localFormat
            }
        }
    }

    override fun intercept(chain: Chain): Response =
        chain.run {
            val request = request()

            val newRequest = request.newBuilder()
                .addHeader("x-amz-date", dateFormat.get().format(clock.now()))
                .addHeader("host", request.url().host())
                .build()

            val signed = signer.sign(newRequest, "<accessKeyId>", "<secretAccessKey>")

            proceed(signed)
        }
}

The interceptor above simply creates a new request with the needed headers and signs it as described in the section above. Because it can be called from multiple threads, we secure the SimpleDateFormat with a ThreadLocal object. This is a well-known issue with SimpleDateFormat in Java that you can read about here.

After the request is signed, we can simply proceed with it through the chain of interceptors. It’s worth noting that here we are hard-coding the credentials for the signing. However, these can be dynamically retrieved. If a request doesn’t need signing we can simply pass null to these credentials and the signer will simply return the same request.

Once we initialize the Retrofit instance, we can set it up with this interceptor.

val signingInterceptor = AwsSigingInterceptor(OkHttpAwsV4Signer("<region>", "<service>"))

val retrofit = Retrofit.Builder()
    .client(OkHttpClient.Builder()
        .addInterceptor(signingInterceptor)
        .build())
    // ...
    .build()

Limitations

As you might have already noticed, the signer is created for a specific region and service. Right now, the only way to sign requests going to different services or regions would be to have multiple signer instances. This can be fixed by making the sign method accept both the region and the service when signing requests.

The signer only supports the version 4 of the algorithm and requires certain information to be present in the headers. Also, it always uses the same hashing algorithm and isn’t flexible enough to change it on demand. This is not a limitation of the signing algorithm provided by Amazon, it’s a limitation of the code. The hashing methods are hardcoded in the source. Eventually, they can be made configurable and the signing process should still work.

Summary

This post introduces a library to sign OkHttp requests with the Amazon Signing Algorithm. We’ve felt the need to implement this ourselves so we wouldn’t be depending on the entire Amazon SDK when we needed simply the signing part.

Even though the library has some limitations, it is being used in our main Android Application and has proven to suit our needs. Now, while making it open source we expect to help others and improve it by removing the mentioned limitations when needed.

Every contribution is welcomed.