A good API can make your web service dramatically more useful. Web APIs are the connection point between your service and your mobile app or Progressive Web App (PWA). APIs also allow your users and third-party developers to build features you didn’t consider and automate tasks that would otherwise be tedious.
One of the first concerns of an API is authentication: verifying which user is making a particular request. API tokens are a typical solution to that problem.
The following guidelines can help make sure issuing API tokens goes smoothly for you and your clients.
Tell The Client When The Token Expires
When you return a new token to a client, include a datetime that tells them when the token expires.
Allow Early Renewal
Ensure that clients can receive a fresh API token at any time.
For example, if your tokens are good for two hours, a client should be able to request one at noon which expires at 2pm. At 1:45pm, they should be able to request a new token that expires at 3:45pm.
Without these renewal capabilities, a number of issues can arise:
- Since clients can’t renew early—and will lose API access if they renew late—they must renew at exactly the expiration time. Any downtime in the token issuing service or any network glitches which happen at that moment will cause them to lose API access.
- Even if they can get a token at the right moment, they will probably make some failed requests while they’re in the process of getting it.
- If they have multiple machines or processes using the token, you’ll get a “thundering herd” of requests for the new token when the expiration time arrives.
By contrast, if they can renew early, clients can tolerate a little downtime in the token issuing service—since they can try to renew early and repeat until they succeed. This ensures they always have a valid API token and enables them to spread out requests for fresh tokens (so that you don’t get a pile of requests at once). You can even encourage this by adding a little random padding to your token expiration times.
Let Old Tokens Continue To Work
If a client gets a token at noon which expires at 2pm, and gets a fresh one at 1:45pm which expires at 3:45pm, the one issued at noon should still continue to work until its expiration.
The reason is simple: if getting a new token invalidates the old one, any requests that have been started with the old token will fail.
Use HTTPS
Like any web request with sensitive information (arguably all of them), requests to your API should use HTTPS. Failure to use HTTPS will mean that users’ API tokens can be stolen and their accounts hijacked.
Proper use of HTTPS will keep the entire request and response a secret between your API and the user who is calling it.
Return a Different Token Value Every Time
Any time you include a secret value in a response, you should ensure that it changes in every response. Otherwise, if you’re using HTTPS (see above) and compression (which saves bandwidth), you’ll be open to a BREACH attack.
If you issue a token to a user at noon, and another at 12:01pm, and another at 12:02pm, etc., every token should be different, and all should continue to work until their stated expiration time.
Enable Forced Expiration
It’s important to enable forced expiration of tokens to revoke API access as needed. Part of your validity check when an API request comes in needs to involve reviewing some server-side state and asking, “has this token been revoked?”
You could keep token revocation state anywhere you like, but storing it as part of a user record is the simplest option.
Detect Bogus Tokens Without A Database Hit
Even though you have to store revocation somewhere, checking your database is relatively expensive. If you get a bunch of requests with tokens which are either nonsense or are expired, you want to avoid having to do a database query to confirm.
It’s much better if you can determine that a token is one you really issued—which hasn’t expired, and which belongs to User X—and only then pay the cost of a database lookup to see if it’s been revoked.
Bonus: if possible, apply a per-user rate limit without consulting the main database, assuming you keep usage stats in memory, in :ets, in Redis, or something similar.
Require Tokens In A Header
You should require the API token to be sent in an HTTP header.
This allows it to be sent the same way with GET requests (where you might be tempted to use URL parameters) as with POST requests (where you might be tempted to put it in the request body).
A header like Authorization: Bearer <token> is the typical way of doing this.
Handling API Tokens in Phoenix
The following plan incorporates all of these elements in Elixir Phoenix.
When issuing a token for the first time, store a raw_token on a user’s database row.
Since the raw token is useless unless it’s signed, its value doesn’t matter; it could be 1 if you like.
The point is to have something you can change.
To generate a signed token, put the user’s ID and their raw token together in a data structure and sign it with Phoenix.Token, which uses your endpoint’s configured secret_key_base.
For instance, Phoenix.Token.sign(MyApp.Endpoint, "user salt", [id: 123, raw_token: 1]) will produce a long, random-looking string value.
Phoenix.Token.sign/3 adds a hidden datetime value, which comes into play when you verify the value later.
Do note what the Phoenix.Token docs say about security here:
The data stored in the token is signed to prevent tampering but not encrypted. This means it is safe to store identification information (such as user IDs) but should not be used to store confidential information (such as credit card numbers).
Any time you get another request for a token, you’ll do the exact same thing; you’ll sign the same raw value, but the output will be different because of the datetime that’s included automatically.
When you receive a request with a token header, verify the token with Phoenix.Token.
Use a max_age to enforce that the token can’t be too old.
For instance, Phoenix.Token.verify(MyApp.Endpoint, "user salt", token, max_age: 28800) will verify any token issued in the last eight hours.
This means you can issue as many signed tokens as you like; each one remains valid until it reaches the max_age you specify.
If Phoenix.Token.verify/4 returns {:ok, [id: 123, raw_token: 1]}, you know that this was an API token you signed and that it isn’t outdated; all attempts to guess API tokens have now been screened out.
And, with the user ID in hand, you can also rate limit per user. For example, you can use ex_rated which tracks usage in an :ets table.
You can then check whether the token is currently valid by looking up the user record with that ID and verifing that the raw token value is still the current one for that user. If so, the request is good. If not, either the owner of the token is mistakenly using an old one, or else their token has been stolen and someone else is attempting to use it. Here you might want to record the attempted access, including the user id, token value, and IP address.
To expire a token after issuing it, change the raw token value stored on a user row. Once you’ve done that, any tokens based on the old raw value will fail the “user lookup” step. If you like, you could keep a record of changes to the token value; this would give more context for your records of failed access attempts, so that you could say “this request was made with a token that had been changed by an administrator the day before.”
If you need to revoke a user’s API access completely, delete their raw token value and ensure that your token issuing code won’t sign a nil value.
That’s it!
A final word of advice from the perspective of encapsulation: verifying the token and looking up the user are should be done in plugs as the request comes in.
If successful, the plug should store the user in conn.assigns; otherwise, it should send a failure response.
This way, only the plugs need to know about your authentication logic; everything that touches the request afterward can safely assume that it has access to the current API user.
DockYard is a digital product agency offering custom software, mobile, and web application development consulting. We provide exceptional professional services in strategy, user experience, design, and full stack engineering using Ember.js, React.js, Ruby, and Elixir. With a nationwide staff, we’ve got consultants in key markets across the United States, including San Francisco, Los Angeles, Denver, Chicago, Austin, New York, and Boston.
