Skip to content

draft.md

Latest commit

 

History

History
263 lines (147 loc) · 12.3 KB

draft.md

File metadata and controls

263 lines (147 loc) · 12.3 KB

coding: utf-8

title: The Idempotency HTTP Header Field docname: draft-idempotency-header-00 category: std ipr: trust200902 stand_alone: yes pi: [toc, tocindent, sortrefs, symrefs, strict, compact, comments, inline]

author:

ins: J. Jena
name: Jayadeba Jena
email: jjena@paypal.com
uri: https://github.com/jayadeba

normative:

informative:

--- abstract

The HTTP Idempotency request header field can be used to carry idempotency key in order to make non-idempotent HTTP methods such as POST or PATCH fault-tolerant.

--- middle

Introduction

Idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. It does not matter if the operation is called only once, or 10s of times over. The result SHOULD be the same.

Idempotency is important in building a fault-tolerant HTTP API. An HTTP request method is considered idempotent if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request. {{!RFC7231}} defines methods OPTIONS, HEAD, GET, PUT and DELETE as idempotent. However, POST and PATCH methods are NOT idempotent.

Suppose a client on HTTP API wants to create or update a resource using POST method. Since POST is NOT an idempotent method, calling it multiple times can result in duplication or wrong updates. What would happen if you sent out the POST request to the server, but you get a timeout? Is the resource actually created or updated? Does the timeout happened during sending of the request to the server, or while receiving the response on the client? Can we safely retry again, or do we need to figure out first what has happened with the resource? If POST was an idempotent method, we would not have to answer such questions. We could safely resend a request until we actually get a response back from the server.

For many use cases in HTTP API, creation of duplicate records is a severe problem from business perspective. For example, in Fintech industry, duplicate records for requests involving any kind of payment transaction on a financial account MUST NOT be allowed. In other cases, processing of duplicate webhooks due to retries is not warranted.

Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 {{!RFC2119}} {{!RFC8174}} when, and only when, they appear in all capitals, as shown here.

This specification uses the Augmented Backus-Naur Form (ABNF) notation of {{!RFC5234}} and includes, by reference, the IMF-fixdate rule as defined in Section 7.1.1.1 of {{!RFC7231}}.

The term "resource" is to be interpreted as defined in Section 2 of {{!RFC7231}}, that is identified by an URI.

The Idempotency HTTP Request Header Field

An idempotency key is a unique value generated by the client which the server uses to recognize subsequent retries of the same request. The Idempotency-Key HTTP request header field carries this key.

Syntax

The Idempotency-Key request header field describes

Idempotency-Key       = idempotency-key-value

idempotency-key-value = opaque-value
opaque-value          = DQUOTE *idempotencyvalue DQUOTE
idempotencyvalue      = %x21 / %x23-7E / obs-text
       ; VCHAR except double quotes, plus obs-text

Clients MUST NOT include more than one Idempotency-Key header field in the same request.

The following example shows an idempotency key using UUID version 4 scheme:

Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324"

Uniqueness of Idempotency Key

The idempotency key that is supplied as part of every POST request MUST be unique and can not be reused with another request with a different request payload.

How to make the key unique is up to the client and it's agreed protocol with the server. It is RECOMMENDED that UUID or a similar random identifier be used as the idempotency key.

Resource Behavior

How an idempotency key is made unique key is determined by the client. We RECOMMEND using UUID or similar random string with enough entropy to avoid collisions.

The resource SHOULD save the resulting HTTP status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed. Subsequent requests with the same key MUST return the same result, including errors.

It is also recommended that the resource implements the idempotency keys to be time-based and, thus, be able to purge or delete a key upon its expiry. Resource SHOULD publish expiration policy related documentation.

Error Scenarios

If the Idempotency-Key request header is missing for a documented idempotent operation requiring this header, the resource MUST reply with an HTTP 400 status code with body containing a link pointing to the relevant documentation. Alternately, using the HTTP header Link, client could be informed about the error too as shown below.

HTTP/1.1 400 Bad Request
Link: <https://developer.example.com/idempotency>;
  rel="describedby"; type="text/html"

If there is an attempt to reuse an idempotency key with a different request payload, the resource MUST reply with an HTTP 422 status code with body containing a link pointing to the relevant documentation. Using the HTTP header Link, client could be informed about the error as following.

HTTP/1.1 422 Unprocessable Entity
Link: <https://developer.example.com/idempotency>;
rel="describedby"; type="text/html"

For other errors, the resource MUST return the appropriate status code and error message.

IANA Considerations

The Idempotency-Key HTTP Request Header Field

The Idempotency-Key request header should be added to the permanent registry of message header fields (see {{!RFC3864}}), taking into account the guidelines given by HTTP/1.1 {{!RFC7231}}.

Header Field Name: Idempotency-Key

Applicable Protocol: Hypertext Transfer Protocol (HTTP)

Status: Standard

Author: Jayadeba Jena, <jjean@paypal.com>
        Sanjay Dalal <sanjay.dalal@cal.berkeley.edu>

Change controller: IETF

Specification document: this specification,
            Section 2 "The Idempotency HTTP Request Header Field"

Implementation Status

Note to RFC Editor: Please remove this section before publication.

This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in {{?RFC7942}}. The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist.

According to RFC 7942, "this will allow reviewers and working groups to assign due consideration to documents that have the benefit of running code, which may serve as evidence of valuable experimentation and feedback that have made the implemented protocols more mature. It is up to the individual working groups to use this information as they see fit".

Organization: Stripe

Organization: Adyen

Organization: Dwolla

Organization: Interledger

Organization: WorldPay

Organization: Yandex

Implementing the Concept

This is a list of implementations that implement the general concept, but do so using different mechanisms:

Organization: Django

Organization: PayPal

Organization: RazorPay

Organization: Twilio

Organization: OpenBanking

Organization: Square

Organization: Google Standard Payments

Security Considerations

This section is meant to inform developers, information providers, and users of known security concerns specific to the HTTP conditional request mechanisms. More general security considerations are addressed in HTTP "Message Syntax and Routing" {{!RFC7230}} and "Semantics and Content" {{!RFC7231}}.

The validators defined by this specification are not intended to ensure the validity of a representation, guard against malicious changes, or detect man-in-the-middle attacks. At best, they enable more efficient ...? when all participants are behaving nicely. At worst, the conditions will fail and the client will receive a response that is no more harmful than an HTTP exchange without conditional requests.

An entity-tag can be abused in ways that create ? risks.

Examples

The first example shows an idempotency-key header field with key value using UUID version 4 scheme:

Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324"

Second example shows an idempotency-key header field with key value using some random string generator:

Idempotency-Key: "clkyoesmbgybucifusbbtdsbohtyuuwz"

--- back

Acknowledgments

The authors would like to thank Mark Nottingham for his support for this Internet Draft. We would like to acknowledge that this draft is inspired by Idempotency related patterns described in API documentation of PayPal and Stripe as well as Internet Draft on POST Once Exactly authored by Mark Nottingham.

The authors take all responsibility for errors and omissions.

Appendix

Appendix A. Imported ABNF

The following core rules are included by reference, as defined in Appendix B.1 of [RFC5234]: ALPHA (letters), CR (carriage return), CRLF (CR LF), CTL (controls), DIGIT (decimal 0-9), DQUOTE (double quote), HEXDIG (hexadecimal 0-9/A-F/a-f), LF (line feed), OCTET (any 8-bit sequence of data), SP (space), and VCHAR (any visible US-ASCII character).

The rules below are defined in [RFC7230]:

 obs-text      = <obs-text, see [RFC7230], Section 3.2.6>