lua-resty-http
Lua HTTP client cosocket driver for OpenResty / ngx_lua.
Status
Production ready.
API
- new
- connect
- set_proxy_options
- set_timeout
- set_timeouts
- set_keepalive
- get_reused_times
- close
- request
- request_uri
- request_pipeline
- parse_uri
- get_client_body_reader
- Response
Deprecated
These methods may be removed in future versions.
Single-shot request
local httpc = require("resty.http").new()
-- Single-shot requests use the `request_uri` interface.
local res, err = httpc:request_uri("http://example.com/helloworld", {
method = "POST",
body = "a=1&b=2",
headers = {
["Content-Type"] = "application/x-www-form-urlencoded",
},
})
if not res then
ngx.log(ngx.ERR, "request failed: ", err)
return
end
-- At this point, the entire request / response is complete and the connection
-- will be closed or back on the connection pool.
-- The `res` table contains the expeected `status`, `headers` and `body` fields.
local status = res.status
local length = res.headers["Content-Length"]
local body = res.body
Streamed request
local httpc = require("resty.http").new()
-- First establish a connection
local ok, err, ssl_session = httpc:connect({
scheme = "https",
host = "127.0.0.1",
port = 8080,
})
if not ok then
ngx.log(ngx.ERR, "connection failed: ", err)
return
end
-- Then send using `request`, supplying a path and `Host` header instead of a
-- full URI.
local res, err = httpc:request({
path = "/helloworld",
headers = {
["Host"] = "example.com",
},
})
if not res then
ngx.log(ngx.ERR, "request failed: ", err)
return
end
-- At this point, the status and headers will be available to use in the `res`
-- table, but the body and any trailers will still be on the wire.
-- We can use the `body_reader` iterator, to stream the body according to our
-- desired buffer size.
local reader = res.body_reader
local buffer_size = 8192
repeat
local buffer, err = reader(buffer_size)
if err then
ngx.log(ngx.ERR, err)
break
end
if buffer then
-- process
end
until not buffer
local ok, err = httpc:set_keepalive()
if not ok then
ngx.say("failed to set keepalive: ", err)
return
end
-- At this point, the connection will either be safely back in the pool, or closed.
````
# Connection
## new
`syntax: httpc, err = http.new()`
Creates the HTTP connection object. In case of failures, returns `nil` and a string describing the error.
## connect
`syntax: ok, err, ssl_session = httpc:connect(options)`
Attempts to connect to the web server while incorporating the following activities:
- TCP connection
- SSL handshake
- HTTP proxy configuration
In doing so it will create a distinct connection pool name that is safe to use with SSL and / or proxy based connections, and as such this syntax is strongly recommended over the original (now deprecated) [TCP only connection syntax](#TCP-only-connect).
The options table has the following fields:
* `scheme`: scheme to use, or nil for unix domain socket
* `host`: target host, or path to a unix domain socket
* `port`: port on target host, will default to `80` or `443` based on the scheme
* `pool`: custom connection pool name. Option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsockconnect), except that the default will become a pool name constructed using the SSL / proxy properties, which is important for safe connection reuse. When in doubt, leave it blank!
* `pool_size`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsockconnect)
* `backlog`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsockconnect)
* `proxy_opts`: sub-table, defaults to the global proxy options set, see [set\_proxy\_options](#set_proxy_options).
* `ssl_reused_session`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsocksslhandshake)
* `ssl_verify`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsocksslhandshake), except that it defaults to `true`.
* `ssl_server_name`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsocksslhandshake)
* `ssl_send_status_req`: option as per [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsocksslhandshake)
* `ssl_client_cert`: will be passed to `tcpsock:setclientcert`. Requires `ngx_lua_http_module` >= v0.10.23.
* `ssl_client_priv_key`: as above.
## set\_timeout
`syntax: httpc:set_timeout(time)`
Sets the socket timeout (in ms) for subsequent operations. See [set\_timeouts](#set_timeouts) below for a more declarative approach.
## set\_timeouts
`syntax: httpc:set_timeouts(connect_timeout, send_timeout, read_timeout)`
Sets the connect timeout threshold, send timeout threshold, and read timeout threshold, respectively, in milliseconds, for subsequent socket operations (connect, send, receive, and iterators returned from receiveuntil).
## set\_keepalive
`syntax: ok, err = httpc:set_keepalive(max_idle_timeout, pool_size)`
Either places the current connection into the pool for future reuse, or closes the connection. Calling this instead of [close](#close) is "safe" in that it will conditionally close depending on the type of request. Specifically, a `1.0` request without `Connection: Keep-Alive` will be closed, as will a `1.1` request with `Connection: Close`.
In case of success, returns `1`. In case of errors, returns `nil, err`. In the case where the connection is conditionally closed as described above, returns `2` and the error string `connection must be closed`, so as to distinguish from unexpected errors.
See [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsocksetkeepalive) for parameter documentation.
## set\_proxy\_options
`syntax: httpc:set_proxy_options(opts)`
Configure an HTTP proxy to be used with this client instance. The `opts` table expects the following fields:
* `http_proxy`: an URI to a proxy server to be used with HTTP requests
* `http_proxy_authorization`: a default `Proxy-Authorization` header value to be used with `http_proxy`, e.g. `Basic ZGVtbzp0ZXN0`, which will be overriden if the `Proxy-Authorization` request header is present.
* `https_proxy`: an URI to a proxy server to be used with HTTPS requests
* `https_proxy_authorization`: as `http_proxy_authorization` but for use with `https_proxy` (since with HTTPS the authorisation is done when connecting, this one cannot be overridden by passing the `Proxy-Authorization` request header).
* `no_proxy`: a comma separated list of hosts that should not be proxied.
Note that this method has no effect when using the deprecated [TCP only connect](#TCP-only-connect) connection syntax.
## get\_reused\_times
`syntax: times, err = httpc:get_reused_times()`
See [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsockgetreusedtimes).
## close
`syntax: ok, err = httpc:close()`
See [OpenResty docs](https://github.com/openresty/lua-nginx-module#tcpsockclose).
# Requesting
## request
`syntax: res, err = httpc:request(params)`
Sends an HTTP request over an already established connection. Returns a `res` table or `nil` and an error message.
The `params` table expects the following fields:
* `version`: The HTTP version number. Defaults to `1.1`.
* `method`: The HTTP method string. Defaults to `GET`.
* `path`: The path string. Defaults to `/`.
* `query`: The query string, presented as either a literal string or Lua table..
* `headers`: A table of request headers.
* `body`: The request body as a string, a table of strings, or an iterator function yielding strings until nil when exhausted. Note that you must specify a `Content-Length` for the request body, or specify `Transfer-Encoding: chunked` and have your function implement the encoding. See also: [get\_client\_body\_reader](#get_client_body_reader)).
When the request is successful, `res` will contain the following fields:
* `status`: The status code.
* `reason`: The status reason phrase.
* `headers`: A table of headers. Multiple headers with the same field name will be presented as a table of values.
* `has_body`: A boolean flag indicating if there is a body to be read.
* `body_reader`: An iterator function for reading the body in a streaming fashion.
* `read_body`: A method to read the entire body into a string.
* `read_trailers`: A method to merge any trailers underneath the headers, after reading the body.
If the response has a body, then before the same connection can be used for another request, you must read the body using `read_body` or `body_reader`.
## request\_pipeline
`syntax: responses, err = httpc:request_pipeline(params)`
This method works as per the [request](#request) method above, but `params` is instead a nested table of parameter tables. Each request is sent in order, and `responses` is returned as a table of response handles. For example:
```lua
local responses = httpc:request_pipeline({
{ path = "/b" },
{ path = "/c" },
{ path = "/d" },
})
for _, r in ipairs(responses) do
if not r.status then
ngx.log(ngx.ERR, "socket read error")
break
end
ngx.say(r.status)
ngx.say(r:read_body())
end
Due to the nature of pipelining, no responses are actually read until you attempt to use the response fields (status / headers etc). And since the responses are read off in order, you must read the entire body (and any trailers if you have them), before attempting to read the next response.
Note this doesn’t preclude the use of the streaming response body reader. Responses can still be streamed, so long as the entire body is streamed before attempting to access the next response.
Be sure to test at least one field (such as status) before trying to use the others, in case a socket read error has occurred.
Response
res.body_reader
The body_reader
iterator can be used to stream the response body in chunk sizes of your choosing, as follows:
local reader = res.body_reader
local buffer_size = 8192
repeat
local buffer, err = reader(buffer_size)
if err then
ngx.log(ngx.ERR, err)
break
end
if buffer then
-- process
end
until not buffer
If the reader is called with no arguments, the behaviour depends on the type of connection. If the response is encoded as chunked, then the iterator will return the chunks as they arrive. If not, it will simply return the entire body.
Note that the size provided is actually a maximum size. So in the chunked transfer case, you may get buffers smaller than the size you ask, as a remainder of the actual encoded chunks.
res:read_body
syntax: body, err = res:read_body()
Reads the entire body into a local string.
res:read_trailers
syntax: res:read_trailers()
This merges any trailers underneath the res.headers
table itself. Must be called after reading the body.
Utility
parse_uri
syntax: local scheme, host, port, path, query? = unpack(httpc:parse_uri(uri, query_in_path?))
This is a convenience function allowing one to more easily use the generic interface, when the input data is a URI.
As of version 0.10
, the optional query_in_path
parameter was added, which specifies whether the querystring is to be included in the path
return value, or separately as its own return value. This defaults to true
in order to maintain backwards compatibility. When set to false
, path
will only include the path, and query
will contain the URI args, not including the ?
delimiter.
get_client_body_reader
syntax: reader, err = httpc:get_client_body_reader(chunksize?, sock?)
Returns an iterator function which can be used to read the downstream client request body in a streaming fashion. You may also specify an optional default chunksize (default is 65536
), or an already established socket in
place of the client request.
Example:
local req_reader = httpc:get_client_body_reader()
local buffer_size = 8192
repeat
local buffer, err = req_reader(buffer_size)
if err then
ngx.log(ngx.ERR, err)
break
end
if buffer then
-- process
end
until not buffer
This iterator can also be used as the value for the body field in request params, allowing one to stream the request body into a proxied upstream request.
local client_body_reader, err = httpc:get_client_body_reader()
local res, err = httpc:request({
path = "/helloworld",
body = client_body_reader,
})
Deprecated
These features remain for backwards compatability, but may be removed in future releases.
TCP only connect
The following versions of the connect
method signature are deprecated in favour of the single table
argument documented above.
syntax: ok, err = httpc:connect(host, port, options_table?)
syntax: ok, err = httpc:connect("unix:/path/to/unix.sock", options_table?)
NOTE: the default pool name will only incorporate IP and port information so is unsafe to use in case of SSL and/or Proxy connections. Specify your own pool or, better still, do not use these signatures.
ssl_handshake
syntax: session, err = httpc:ssl_handshake(session, host, verify)
Calling this method manually is no longer necessary since it is incorporated within connect. It is retained for now for compatibility with users of the older TCP only connect syntax.
See OpenResty docs.
proxy_request / proxy_response
These two convenience methods were intended simply to demonstrate a common use case of implementing reverse proxying, and the author regrets their inclusion in the module. Users are encouraged to roll their own rather than depend on these functions, which may be removed in a subsequent release.
proxy_request
syntax: local res, err = httpc:proxy_request(request_body_chunk_size?)
Performs a request using the current client request arguments, effectively proxying to the connected upstream. The request body will be read in a streaming fashion, according to request_body_chunk_size
(see documentation on the client body reader below).
proxy_response
syntax: httpc:proxy_response(res, chunksize?)
Sets the current response based on the given res
. Ensures that hop-by-hop headers are not sent downstream, and will read the response according to chunksize
(see documentation on the body reader above).
Author
James Hurst james@pintsized.co.uk, with contributions from @hamishforbes, @Tieske, @bungle et al.