jsx

index

quickstart
description
data types
exports
callback exports
- Module:init/1
- Module:handle_event/2
acknowledgements

quickstart

to add to a rebar3 project

Add to rebar.config

...
{erl_opts, [debug_info]}.
{deps, [
       ...
       {jsx, "~> 3.0"}
]}.
...

to build the library and run tests

$ rebar3 compile
$ rebar3 eunit

to convert a utf8 binary containing a json string into an erlang term

1> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>, []).
#{<<"awesome">> => true,<<"library">> => <<"jsx">>}
2> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>, [{return_maps, false}]).
[{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]
3> jsx:decode(<<"[\"a\",\"list\",\"of\",\"words\"]">>).
[<<"a">>, <<"list">>, <<"of">>, <<"words">>]

to convert an erlang term into a utf8 binary containing a json string

1> jsx:encode(#{<<"library">> => <<"jsx">>, <<"awesome">> => true}).
<<"{\"awesome\":true,\"library\":\"jsx\"}">>
2> jsx:encode([{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]).
<<"{\"library\": \"jsx\", \"awesome\": true}">>
3> jsx:encode([<<"a">>, <<"list">>, <<"of">>, <<"words">>]).
<<"[\"a\",\"list\",\"of\",\"words\"]">>

to check if a binary or a term is valid json

1> jsx:is_json(<<"[\"this is json\"]">>).
true
2> jsx:is_json("[\"this is not\"]").
false
3> jsx:is_term([<<"this is a term">>]).
true
4> jsx:is_term([this, is, not]).
false

to minify some json

1> jsx:minify(<<"{
  \"a list\": [
    1,
    2,
    3
  ]
}">>).
<<"{\"a list\":[1,2,3]}">>

to prettify some json

1> jsx:prettify(<<"{\"a list\":[1,2,3]}">>).
<<"{
  \"a list\": [
    1,
    2,
    3
  ]
}">>

incomplete input

jsx can handle incomplete json texts. if the option stream is passed to the decoder or parser and if a partial json text is parsed, rather than returning a term from your callback handler, jsx returns {incomplete, F} where F is a function with an identical API to the anonymous fun returned from decoder/3, encoder/3 or parser/3. it retains the internal state of the parser at the point where input was exhausted. this allows you to parse as you stream json over a socket or file descriptor, or to parse large json texts without needing to keep them entirely in memory

however, it is important to recognize that jsx is conservative by default. jsx will not consider the parsing complete even when input is exhausted and the json text is unambiguously incomplete. to end parsing call the incomplete function with the argument end_stream (or end_json) like:

1> {incomplete, F} = jsx:decode(<<"[">>, [stream]).
{incomplete,#Fun<jsx_decoder.1.122947756>}
2> F(end_stream).  % can also be `F(end_json)`
** exception error: bad argument
3> {incomplete, G} = F(<<"]">>).
{incomplete,#Fun<jsx_decoder.1.122947756>}
4> G(end_stream).  % can also be `G(end_json)`
[]

data types

`json_term()`

json_term() = [json_term()]
    | [{binary() | atom() | integer(), json_term()}]
    | #{} % map of any size, not just the empty map
    | true
    | false
    | null
    | integer()
    | float()
    | binary()
    | atom()
		| datetime()

the erlang representation of json. binaries should be utf8 encoded, or close at least

`json_text()`

json_text() = binary()

a utf8 encoded binary containing a json string

`event()`

event() = start_object
    | end_object
    | start_array
    | end_array
    | {key, binary()}
    | {string, binary()}
    | {integer, integer()}
    | {float, float()}
    | {literal, true}
    | {literal, false}
    | {literal, null}
    | end_json

the subset of token() emitted by the decoder and encoder to handlers

`option()`

option() = dirty_strings
    | escaped_forward_slashes
    | escaped_strings
    | repeat_keys
    | stream
    | strict
    | {strict, [strict_option()]}
    | return_tail
    | uescape
    | unescaped_jsonp

strict_option() = comments
    | trailing_commas
    | utf8
    | single_quotes
    | escapes

jsx functions all take a common set of options. not all flags have meaning in all contexts, but they are always valid options. functions may have additional options beyond these. see individual function documentation for details

dirty_strings

json escaping is lossy; it mutates the json string and repeated application can result in unwanted behaviour. if your strings are already escaped (or you’d like to force invalid strings into “json” you monster) use this flag to bypass escaping. this can also be used to read in really invalid json strings. everything between unescaped quotes are passed as is to the resulting string term. note that this takes precedence over any other options
escaped_forward_slashes

json strings are escaped according to the json spec. this means forward slashes (solidus) are only escaped when this flag is present. otherwise they are left unescaped. you may want to use this if you are embedding json directly into a html or xml document
escaped_strings

by default both the encoder and decoder return strings as utf8 binaries appropriate for use in erlang. escape sequences that were present in decoded terms are converted into the appropriate codepoint while encoded terms are unaltered. this flag escapes strings as if for output in json, removing control codes and problematic codepoints and replacing them with the appropriate escapes
stream

see incomplete input
strict

as mentioned earlier, jsx is pragmatic. if you’re more of a json purist or you’re really into bdsm stricter adherence to the spec is possible. the following restrictions are available
- comments
  
  comments are disabled and result in a badarg error
- trailing_commas
  
  trailing commas in an object or list result in badarg errors
- utf8
  
  invalid codepoints and malformed unicode result in badarg errors
- single_quotes
  
  only keys and strings delimited by double quotes (u+0022) are allowed. the single quote (u+0027) results in a badarg error
- escapes
  
  escape sequences not adhering to the json spec result in a badarg error
- control_codes
  
  control codes in strings result in badarg errors
any combination of these can be passed to jsx by using {strict, [strict_option()]}. strict is equivalent to {strict, [comments, trailing_commas, utf8, single_quotes, escapes, control_codes]}
return_tail

upon reaching the end of a valid json term in an input stream return the term and any remaining bytes in the input stream as {with_tail, term(), binary()} where the second member of the tuple is the json term and the third is any remaining bytes. note that leading whitespace will be stripped from the tail
uescape

escape all codepoints outside the ascii range for 7 bit clean output. note this escaping takes place even if no other string escaping is requested (via escaped_strings)
unescaped_jsonp

javascript interpreters treat the codepoints u+2028 and u+2029 as significant whitespace. json strings that contain either of these codepoints will be parsed incorrectly by some javascript interpreters. by default, these codepoints are escaped (to \u2028 and \u2029, respectively) to retain compatibility. this option simply removes that escaping

exports

`encoder/3`, `decoder/3` & `parser/3`

decoder(Module, Args, Opts) -> Fun((JSONText) -> any())
encoder(Module, Args, Opts) -> Fun((JSONTerm) -> any())
parser(Module, Args, Opts) -> Fun((Tokens) -> any())

  Module = atom()
  Args = any()
  Opts = [option()]
  JSONText = json_text()
  JSONTerm = json_term()
  Tokens = event() | [event()]

jsx is a json compiler with interleaved tokenizing, syntactic analysis and semantic analysis stages. included are two tokenizers; one that handles json texts (decoder/3) and one that handles erlang terms (encoder/3). there is also an entry point to the syntactic analysis stage for use with user-defined tokenizers (parser/3)

all three functions return an anonymous function that takes the appropriate type of input and returns the result of performing semantic analysis, the tuple {incomplete, F} where F is a new anonymous function (see incomplete input) or a badarg error exception if syntactic analysis fails

Module is the name of the callback module

Args is any term that will be passed to Module:init/1 prior to syntactic analysis to produce an initial state

Opts are detailed here

check out callback module documentation for details of the callback module interface

`decode/1,2`

decode(JSON) -> Term
decode(JSON, Opts) -> Term

  JSON = json_text()
  Term = json_term()
  Opts = [option() | labels | {labels, Label} | return_maps]
    Label = binary | atom | existing_atom | attempt_atom
    F = fun((any()) -> any())

decode parses a json text (a utf8 encoded binary) and produces an erlang term

the option labels controls how keys are converted from json to erlang terms. binary (the default behavior) does no conversion beyond normal escaping. atom converts keys to erlang atoms and results in a badarg error if the keys fall outside the range of erlang atoms. existing_atom is identical to atom except it will not add new atoms to the atom table and will result in a badarg error if the atom does not exist. attempt_atom will convert keys to atoms when they exist, and leave them as binary otherwise

the option {return_maps, false} will return objects as proplists instead of maps.

raises a badarg error exception if input is not valid json

`encode/1,2`

encode(Term) -> JSON
encode(Term, Opts) -> JSON

  Term = json_term()
  JSON = json_text()
  Opts = [option() | space | {space, N} | indent | {indent, N}]
    N = pos_integer()

encode converts an erlang term into json text (a utf8 encoded binary)

the option {space, N} inserts N spaces after every comma and colon in your json output. space is an alias for {space, 1}. the default is {space, 0}

the option {indent, N} inserts a newline and N spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. indent is an alias for {indent, 1}. the default is {indent, 0}

raises a badarg error exception if input is not a valid erlang representation of json

`format/1,2`

format(JSON) -> JSON
format(JSON, Opts) -> JSON

  JSON = json_text()
  Opts = [option() | space | {space, N} | indent | {indent, N} | {newline, LF}]
     N = pos_integer()
    LF = binary()

format parses a json text (a utf8 encoded binary) and produces a new json text according to the format rules specified by Opts

the option {space, N} inserts N spaces after every comma and colon in your json output. space is an alias for {space, 1}. the default is {space, 0}

the option {newline, LF} defines a custom newline symbol(s). the default is {newline, <<$\n>>}

raises a badarg error exception if input is not valid json

`minify/1`

minify(JSON) -> JSON

  JSON = json_text()

minify parses a json text (a utf8 encoded binary) and produces a new json text stripped of whitespace

raises a badarg error exception if input is not valid json

`prettify/1`

prettify(JSON) -> JSON

  JSON = json_text()

prettify parses a json text (a utf8 encoded binary) and produces a new json text equivalent to format(JSON, [{space, 1}, {indent, 2}])

raises a badarg error exception if input is not valid json

`is_json/1,2`

is_json(MaybeJSON) -> true | false
is_json(MaybeJSON, Opts) -> true | false

  MaybeJSON = any()
  Opts = options()

returns true if input is a valid json text, false if not

what exactly constitutes valid json may be altered

`is_term/1,2`

is_term(MaybeJSON) -> true | false
is_term(MaybeJSON, Opts) -> true | false

  MaybeJSON = any()
  Opts = options()

returns true if input is a valid erlang representation of json, false if not

what exactly constitutes valid json may be altered via options

callback exports

the following functions should be exported from a jsx callback module

`Module:init/1`

Module:init(Args) -> InitialState

  Args = any()
  InitialState = any()

whenever any of encoder/3, decoder/3 or parser/3 are called, this function is called with the Args argument provided in the calling function to obtain InitialState

`Module:handle_event/2`

Module:handle_event(Event, State) -> NewState

  Event = [event()]
  State = any()
  NewState = any()

semantic analysis is performed by repeatedly calling handle_event/2 with a stream of events emitted by the tokenizer and the current state. the new state returned is used as the input to the next call to handle_event/2. the following events must be handled:

start_object

the start of a json object
’{key, binary()}’

the key of an entry in a json object
end_object

the end of a json object
start_array

the start of a json array
end_array

the end of a json array
{string, binary()}

a json string. it will usually be a utf8 encoded binary. see the options for possible exceptions. note that keys are also json strings
{integer, integer()}

an erlang integer (bignum)
{float, float()}

an erlang float
{literal, true}

the atom true
{literal, false}

the atom false
{literal, null}

the atom null
end_json

this event is emitted when syntactic analysis is completed. you should do any cleanup and return the result of your semantic analysis

index

quickstart

to add to a rebar3 project

to build the library and run tests

to convert a utf8 binary containing a json string into an erlang term

to convert an erlang term into a utf8 binary containing a json string

to check if a binary or a term is valid json

to minify some json

to prettify some json

incomplete input

data types

json_term()

json_text()

event()

option()

exports

encoder/3, decoder/3 & parser/3

decode/1,2

encode/1,2

format/1,2

minify/1

prettify/1

is_json/1,2

is_term/1,2

callback exports

Module:init/1

Module:handle_event/2

Articles

`json_term()`

`json_text()`

`event()`

`option()`

`encoder/3`, `decoder/3` & `parser/3`

`decode/1,2`

`encode/1,2`

`format/1,2`

`minify/1`

`prettify/1`

`is_json/1,2`

`is_term/1,2`

`Module:init/1`

`Module:handle_event/2`