Securing your API - The 5 levels of "it wasn't me"

It is non-controversial that APIs are prolific and popular. Sometimes these are also neglected entrypoints into your systems.

When you're designing an API you have to think about who the user is, the information they care about, etc. You also have to think about how bad actors may use and/or abuse your API.

For a bad actor, the possibilities are endless really... things like DoS, escalation of privileges, lateral movements, etc.

The cherry on top is when consumers of the API make mistakes but blame it on bad actors, or you! In this post we will look at 5 different ways you can protect your API, and refute statements like "it wasn't me". It will also become apparent that the more sophisticated and secure a solution is, the more complex and cumbersome it is to interact with your API.

Backend

As a companion resource, all the example requests in this blog post are performed against a Go backend server. You can find the source code here.

Public

When your API is public, it is effectively a free-for-all. In this scenario anyone can call your API. Word of advice, at the very least you may want to add some throttling.

$ curl -sSL -XGET localhost:8080/v1/public
{"content":"Hi there to the public"}

Nothing fancy or crazy, you also have absolutely no way of knowing who actually made the request!

Private

If you want only certain individuals to interact with your API, you can gate it with some sort of key. This prevents any random requests from making it through, requiring the possession of a shared secret.

If each client has its own shared secret, it becomes trivial to monitor which client is making which calls. This mechanism also provides a simple way to revoke access, if needed.

$ curl -sSL -XPOST \
	-H "X-API-Key: MY_KEY" \
    localhost:8080/v1/private
{"content":"Hi there in private"}

A payload may be sent with the request too. And although we know which client made the call, the client could claim that the payload was forged or corrupt. Maybe through a man-in-the-middle attack or simply it got corrupted in-flight.

Checksum

Thankfully, data integrity is a well understood problem. Checksums are a mechanism through which a hash is produced for a given piece of content.

The receiver of the data can independently calculate the very same hash, given the content it received, and assert both values are the same.

In order for checksums to be robust, calculating them should be trivial, but the opposite direction, hash-to-content should be hard, such that a bad actor cannot inject a malicious payload that happens to produce the same hash (second pre-image resistance). It is no surprise then when we see cryptographic hash functions such as SHA-256 being used for this purpose!

$ export CONTENT="secret"
$ export CONTENT_CHECKSUM=$(echo -n "$CONTENT" | sha256sum | cut -d' ' -f 1)
$ echo -n "$CONTENT" | curl -sSL -XPOST \
	-H "X-API-Key: MY_KEY" \
    -H "X-Checksum: $CONTENT_CHECKSUM" \
    -d @- \
    localhost:8080/v1/checksum
{"content":"verified"}

// With corrupt content
$ echo -n "$CONTENT"_WRONG | curl -sSL -XPOST \
	-H "X-API-Key: MY_KEY" \
    -H "X-Checksum: $CONTENT_CHECKSUM" \
    -d @- \
    localhost:8080/v1/checksum
{"content":"invalid checksum"}

This solution does mitigate data corruption, but it doesn't really solve for the man-in-the-middle attack. It's possible that a bad actor may be able to change both the request payload and the companion checksum, making the request seem legitimate, i.e. the consumer may still claim "it wasn't me"!

Tamper proof

If we suspect a bad actor can stand between the consumer and the provider of an API, then we can leverage the fact that both these parties share a secret, the API Key! The API key usually is generated by the API provider.

Keyed-hash message authentication codes, or HMACs, provide a way to both guarantee integrity and authenticity, since it requires the use of such a shared secret.

$ export CONTENT="secret"
$ export API_KEY="MY_KEY"
$ export CONTENT_HMAC=$(echo -n "$CONTENT" | hmac256 $API_KEY)
$ echo -n "$CONTENT" | curl -sSL -XPOST \
	-H "X-API-Key: $API_KEY" \
    -H "X-HMAC: $CONTENT_HMAC" \
    -d @- \
    localhost:8080/v1/tamperproof
{"content":"verified"}

// With corrupt content
$ echo -n "$CONTENT"_WRONG | curl -sSL -XPOST \
	-H "X-API-Key: $API_KEY" \
    -H "X-HMAC: $CONTENT_HMAC" \
    -d @- \
    localhost:8080/v1/tamperproof
{"content":"invalid HMAC"}

// With different API Key
$ echo -n "$CONTENT"_WRONG | curl -sSL -XPOST \
	-H "X-API-Key: DIFFERENT_KEY" \
    -H "X-HMAC: $CONTENT_HMAC" \
    -d @- \
    localhost:8080/v1/tamperproof
{"content":"invalid HMAC"}

Things are definitely getting more complex aren't they?

With HMACs we mitigate some problems, and its security is closely tied to the secrecy of the shared secret. Because it's shared, it means that multiple parties can be compromised in order to extract it. The client may still claim "it wasn't me", arguing that the API keys were obtained from the API provider somehow.

Non-repudiation

As we enter the final stage, we have to cover non-repudiation.

As an API provider, you have to generate and persist the API keys on your side (hopefully a hashed version of it right?), and so does the client. If the key was compromised, you have no definitive way of knowing which side was the culprit, leaving you with an unproductive blame game to play.

If a digital signature is required, that means that as an API provider, you only have access to the public side of the key material. Cryptographically there is no way for anyone else to digitally sign a request. By definition, the security of that key material is the responsibility of the caller, and the cryptographic trail moots any repudiation attempts!

There are different ways to achieve this, from using client-side certificates, to explicit private-public key pairs (which are pretty much the same thing). The most important part is that no only does the caller have to store and use these keys, similar to what they'd do with an API key, they also have to generate and rotate key material, which is a lot more complex.

More expensive attacks

It gets better! As mentioned, you have to persist client API keys such that you can authenticate requests. That makes your storage system a juicy target, holding a lot of potentially profitable keys!

Thinking from the perspective of a bad actor, they can invest a lot of resources compromising a single system, with high yield potential.

With signed API requests, the dynamics drastically change. Because each client holds and protects their own private keys, any attacker would have to individually target each of them. It becomes a much more expensive attack, hopefully to the point of it not being worth it.

Why and why not

There are several articles around the dichotomy between security and convenience (Forbes, Auth0).

Properly managing private keys is not an easy challenge to solve! Do your clients have the expertise and/or need to properly implement key management on their side?

Is the data you're protecting that valuable? Your TODO list is of great importance, but do you think someone would invest $100k (random number) to get unauthorized access to it?

Proven strategy

If you believe this to be a security layer that makes sense for your business, you should be confident about it, because this has been used for a long time, and it's a battle tested solution!

AWS uses it on their APIs, so does Coinbase, and many more.

Remember, use best practices!

$ export CONTENT="secret"
$ export API_KEY="MY_KEY"
$ export PRIVATE_KEY=$(openssl ecparam -genkey -name secp384r1 -noout)
$ export PUBLIC_KEY=$(openssl ec -in <(echo "$PRIVATE_KEY") -pubout | head -n -1 | tail -n +2 | base64 -d | xxd -p -c 256)
$ export SIGNATURE=$(echo -n $CONTENT | sha256sum - | cut -d' ' -f 1 | xxd -r -p | openssl pkeyutl -sign -inkey <(echo "$PRIVATE_KEY") | xxd -p -c 256)
$ echo -n "$CONTENT" | curl -sSL -XPOST \
	-H "X-API-Key: $API_KEY" \
    -H "X-Public-Key: $PUBLIC_KEY" \
    -H "X-Signature: $SIGNATURE" \
    -d @- \
    localhost:8080/v1/nonrepudiation
{"content":"verified"}

// With corrupt content
$ echo -n "$CONTENT"_WRONG | curl -sSL -XPOST \
	-H "X-API-Key: $API_KEY" \
    -H "X-Public-Key: $PUBLIC_KEY" \
    -H "X-Signature: $SIGNATURE" \
    -d @- \
    localhost:8080/v1/nonrepudiation
{"content":"invalid signature"}

// With different signature
$ export PRIVATE_KEY2=$(openssl ecparam -genkey -name secp384r1 -noout)
$ export SIGNATURE2=$(echo -n $CONTENT | sha256sum - | cut -d' ' -f 1 | xxd -r -p | openssl pkeyutl -sign -inkey <(echo "$PRIVATE_KEY2") | xxd -p -c 256)
$ echo -n "$CONTENT" | curl -sSL -XPOST \
	-H "X-API-Key: $API_KEY" \
    -H "X-Public-Key: $PUBLIC_KEY" \
    -H "X-Signature: $SIGNATURE2" \
    -d @- \
    localhost:8080/v1/nonrepudiation
{"content":"invalid signature"}

By placing the onus of private-public key management on the consumer side, we achieve a phenomenal level of security, including non-repudiation properties, at the cost of a much more complex system.

Conclusion

There is no "one solution fits all". When you're developing your APIs, assess who the consumers are, what the data is and how valuable it is.

In this article we just covered some of the possible mechanisms to employ in your API. There are many others that have been left out for the sake of brevity (maybe content for a part 2).

As always, choose the right tool for the job! I hope this helped you get a better understanding of the tools and strategies available to you. I would love to hear from you, and what you're doing to protect your APIs!