Update 5/12/2016: Watch Stormpath CTO Les Hazlewood’s presentation on REST + JSON API design best practices.

And don’t forget, Stormpath now secures authentication to your API – without code!

Create, Update and HTTP Idempotence

For developers building REST-based APIs, there is a great deal of misinformation and some understandable confusion about when to use HTTP PUT and when to use HTTP POST. Some say POST should be used to create a resource, and PUT to modify one. Others that PUT should be used to create and POST to modify one. Neither is quite right.

Often, developers think of each HTTP method as a 1:1 relationship with CRUD operations:

CRUD       HTTP
Create      POST
Read         GET
Update     PUT
Delete      DELETE

This can be true, with GET and DELETE specifically, but when it comes to which HTTP methods should be associated with create and update, the answer comes down to idempotency.

Idempotency

Idempotence is an important concept in the HTTP specification that states idempotent HTTP requests will result in the same state on the server no matter how many times that same request is executed. GET, HEAD, PUT, and DELETE all have this attribute, but POST does not.

To help illustrate idempotency, we will use an Account collection (“/accounts) and for brevity we will say that each account resource has three properties called givenName, surname, and status.

Let’s say you submit an update request using the HTTP PUT method. In the body, you set givenName a value of “John” and surname a value of “Smith”. Later, you submit another request HTTP PUT request, this time setting givenName to “Johnny”. Is this idempotent? No. Why? Because other requests might have changed the server state of the account resource in between our two requests. For instance, between the two requests, status could have been changed to “blocked.” Our example request cannot guarantee that the state of the account on the server is identical when it is repeated.

Request:

Possible account state after our two requests (due to side effects from other requests):

or

To quote Dino Chiesa, “PUT implies putting a resource – completely replacing whatever is available at the given URL with a different thing.” With PUT requests, you MUST send all the available properties/values, not just the ones you want to change. If we were to send the status of “disabled” in addition to the givenName and surname, the call would be idempotent and eliminate the side effects. Idempotency is a fundamental property of the HTTP specification and must be adhered to to guarantee web interoperability and scale.

Finally, we should point out that HTTP idempotency only applies to server state – not client state.  For example, a client could send a server-idempotent request successfully, and then send that same exact server-idempotent request immediately again and experience an error (e.g. perhaps due to a constraint violation in the server) and this is totally ‘legal’ for HTTP.  As long as the requests result in the same identical state on the server, HTTP idempotency is maintained.

HTTP POST vs HTTP PUT

Now that idempotency is clear, which method should you use when performing create and update operations? The following is a quick reference for when it is appropriate to use each method.

Creates

Use POST to create resources when you do not know the resource identifier. With POST creates, it is best practice to return the status of “201 Created” and the location of the newly created resource, since its location was unknown at the time of submission.  This allows the client to access the new resource later if they need to.

Response:

Use PUT when you allow the client to specify the resource identifier of the newly created resource.  But remember, since PUT is idempotent, you must send all possible values.

Updates

You can use POST to send either all available values or just a subset of available values:

If you want to use PUT to update a resource, it must be a full resource update; you MUST send all attribute values in a PUT request to guarantee idempotency.

Use PUT when you want or need to send all available values in order to follow idempotency requirements, for instance in the case of a full resource update. I

You can also use POST to send all values as well, and the server state could be the same as a full PUT – it’s just not required to be by the HTTP specification. Note that idempotency has a strong correlation to being cacheable by HTTP caching servers, and therefore POST requests are generally not cached. If you are ok with this caching side effect, you can use POST for both full and partial updates.

POST is currently the only non-idempotent method. The HTTP specification is very generic about it, and basically states it to be a ‘server processing directive’. This means it is ‘safe’ to do whatever you want during a POST request.

Finally, we should note that there is another method that has not been finalized for the HTTP specification yet, called PATCH. PATCH is meant to be used as an alternative to POST for partial updates. However, since POST can already simply handle partial updates, there doesn’t seem to be a strong movement for PATCH to be finalized and approved quickly by the HTTP specification committee. If approved, PATCH would join POST as the only other non-idempotent HTTP method.

  • gtrevg

    One detail I’m trying to nail down is what you do in the case of read only fields, such as “dateModified”. A client could never modify those values, but if you were to do a GET on that resource you’d get a different value for “dateModified” after having done a PUT with the full set of fields. Doing multiple PUTs with the full set would be fine as it would be idempotent for all the writable fields.

    What’s the best way of solving this? Does that mean that any object that has a date modified or version number or eTag or something generated by the server that you would not be able to use PUT?

    Thanks

  • Rafi

    Agreed. When I first read this page I had to scratch my head, read the Wikipedia page on Idempotency, then talk to a few colleagues.

    When I explain idempotency to my students, I ask them to think about the side-effects of the call getting stuck in a loop and being called multiple times instead of just once. Its not that after the call the state on the server is identical to after the last call – its about the side effects. The “dateModified” question is indeed a case of a side-effect that would turn make some PUT implementations non-idempotent. Of course, you could implement yours to not update the field if the value would be the same, thereby removing the side-effect and making the PUT idempotent again.

  • Jon Clawley

    I think there is some confusion here arising from the actual implementation of PUT that you may be using.

    Assuming that the PUT method has been implemented as an overwrite of the existing resource (why would it not be?) then by definition it will be idempotent, regardless if all values are sent or not.
    The completeness of the values is a distraction. It is as irrelevant as the actual values of the present attributes.
    Otherwise the only logical case is that once a resource has been PUT it can never be modified at all with another PUT?

    Modification of attribute values is logically no different to their presence/absence, unless your implementation of PUT attempts a merge with the existing resource, which would I agree make it idempotent in the case of incomplete data.

    But why implement PUT as a merge going against the semantics of it completely?

    • Julien Heller

      Late to the party, but I think you can see how incomplete PUTs aren’t idempotent if you imagine this example:

      Resource A current state:
      { “foo”: “bar”, “hello”: “world” }

      PUT resource A
      { “foo”: “baz” }
      PUT resource A
      { “hello”: “kitty” }
      PUT resource A
      { “foo”: “baz” }

      Requests 1 and 3 were identical, however they resulted in a different state for resource A.

      • Jon Clawley

        Your example illustrates the point well I think, the PUT semantics implemented are for a merge of the data in the body with the existing resource data (an “incomplete” PUT as you describe).

        I would argue that PUT is really designed for a replace semantics and if that constraint is stuck with, it guarantees idempotency of the PUT by definition.

        POST seems more appropriate for a merge semantics to me.

  • Simon Boddy

    I’m getting increasingly excited about all this. Maybe it was all the time I was whipped, kept in a dark room and forced to write a payments web service. The thing is, you can’t build a reliable API on stateless, sessionless TCP/HTTP with just these methods. Reliable in the common sense of the term. Clients will keep placing the same request until they get a response they accept as definitive. Sometimes requests go missing. Sometimes requests are processed and responses go missing (Typically, requests succeed but the client has given up waiting and assumes failure). Safe methods are uncontentious, but you won’t do much with just safe methods. POST is in the specification as non-idempotent, but a genuinely non-idempotent unsafe request is deeply problematic. Conversely, an idempotent request is not much good if it only covers “server state”. The client needs to know the outcome of her request, and if the first response goes missing, she will keep requesting. If these subsequent requests produce no further effect on the server, but can’t resend the original result (confirmation) to the client, then we have an endless loop of misunderstanding. Reliable messaging solutions have been around forever but are widely regarded as not worth the complexity. A lightweight solution that stood in front of a web service, looked after uniqueness, and replayed answers to already-seen requests would help, but it would need to be able to distinguish definitive errors from provisional ones in the underlying web service, requiring deep application knowledge. The whole REST thing seems to be based on the presumption that HTTP is “reliable enough” but I don’t think you’ll be seeing payments apis built like this.

  • PurpleJello

    Based on the definitions of POST AND PUT its obvious this whole browser HTTP header system was originally designed around some primitive form of web page “STATE” or session as follows:

    PUT always resets STATE of the same web page on the server to something new, and for all users on that page by posting all new values regardless of PUTS or POSTS by other users prior. Its therefore STATELESS
    POST follows web page STATE, modifying it based on its own POSTS or POSTS of by others (partial or full), OR accepts new state based on a PUT by another user. Therefore its STATEFUL

    Very few implementations online are stateless now. So why use PUT? Notice POST always modifies the servers current page STATE, but never resets it. With a full page POST it can however. A “shared state” by all POSTers is implied. But PUT always reset sor overrides the servers page state, resetting it to a new STATE or creating a new page altogether with its own STATE.

    Its makes sense now why they created PUT, and yet also why its not implemented in modern HTTP browsers (only GET and POST are used). POST can do the same thing as PUT based on server implementation.