Valid message
A valid message is an object (1) not missing any required fields and (2) exclusively composed of JS types understood by the wire format writer.
There are two possible types of valid messages and the encoder is able to work with both of these for convenience:
- Message instances (explicit instances of message classes with default values on their prototype) naturally satisfy the requirements of a valid message and
- Plain JavaScript objects that just so happen to be composed in a way satisfying the requirements of a valid message as well.
In a nutshell, the wire format writer understands the following types:
Field type | Expected JS type (create, encode) | Conversion (fromObject) |
---|---|---|
s-/u-/int32 s-/fixed32 |
number (32 bit integer) |
value | 0 if signedvalue >>> 0 if unsigned |
s-/u-/int64 s-/fixed64 |
Long -like (optimal)number (53 bit integer) |
Long.fromValue(value) with long.jsparseInt(value, 10) otherwise |
float double |
number |
Number(value) |
bool | boolean |
Boolean(value) |
string | string |
String(value) |
bytes | Uint8Array (optimal)Buffer (optimal under node)Array.<number> (8 bit integers) |
base64.decode(value) if a string Object with non-zero .length is assumed to be buffer-like |
enum | number (32 bit integer) |
Looks up the numeric id if a string |
message | Valid message | Message.fromObject(value) |
repeated T | Array<T> |
Copy |
map |
Object<K,V> |
Copy |
- Explicit
undefined
andnull
are considered as not set if the field is optional. - Maps are objects where the key is the string representation of the respective value or an 8 characters long hash string for
Long
-likes.
Toolset
With that in mind and again for performance reasons, each message class provides a distinct set of methods with each method doing just one thing. This avoids unnecessary assertions / redundant operations where performance is a concern but also forces a user to perform verification (of plain JavaScript objects that might just so happen to be a valid message) explicitly where necessary - for example when dealing with user input.
Note that Message
below refers to any message class.
- Message.verify(message:
Object
):null|string
verifies that a plain JavaScript object satisfies the requirements of a valid message and thus can be encoded without issues. Instead of throwing, it returns the error message as a string, if any.
var payload = "invalid (not an object)";
var err = AwesomeMessage.verify(payload);
if (err)
throw Error(err);
- Message.encode(message:
Message|Object
[, writer:Writer
]):Writer
encodes a message instance or valid plain JavaScript object. This method does not implicitly verify the message and it’s up to the user to make sure that the payload is a valid message.
var buffer = AwesomeMessage.encode(message).finish();
Message.encodeDelimited(message:
Message|Object
[, writer:Writer
]):Writer
works likeMessage.encode
but additionally prepends the length of the message as a varint.Message.decode(reader:
Reader|Uint8Array
):Message
decodes a buffer to a message instance. If required fields are missing, it throws autil.ProtocolError
with aninstance
property set to the so far decoded message. If the wire format is invalid, it throws anError
.
try {
var decodedMessage = AwesomeMessage.decode(buffer);
} catch (e) {
if (e instanceof protobuf.util.ProtocolError) {
// e.instance holds the so far decoded message with missing required fields
} else {
// wire format is invalid
}
}
Message.decodeDelimited(reader:
Reader|Uint8Array
):Message
works likeMessage.decode
but additionally reads the length of the message prepended as a varint.Message.create(properties:
Object
):Message
creates a new message instance from a set of properties that satisfy the requirements of a valid message. Where applicable, it is recommended to preferMessage.create
overMessage.fromObject
because it doesn’t perform possibly redundant conversion.
var message = AwesomeMessage.create({ awesomeField: "AwesomeString" });
- Message.fromObject(object:
Object
):Message
converts any non-valid plain JavaScript object to a message instance using the conversion steps outlined within the table above.
var message = AwesomeMessage.fromObject({ awesomeField: 42 });
// converts awesomeField to a string
- Message.toObject(message:
Message
[, options:ConversionOptions
]):Object
converts a message instance to an arbitrary plain JavaScript object for interoperability with other libraries or storage. The resulting plain JavaScript object might still satisfy the requirements of a valid message depending on the actual conversion options specified, but most of the time it does not.
var object = AwesomeMessage.toObject(message, {
enums: String, // enums as string names
longs: String, // longs as strings (requires long.js)
bytes: String, // bytes as base64 encoded strings
defaults: true, // includes default values
arrays: true, // populates empty arrays (repeated fields) even if defaults=false
objects: true, // populates empty objects (map fields) even if defaults=false
oneofs: true // includes virtual oneof fields set to the present field's name
});
For reference, the following diagram aims to display relationships between the different methods and the concept of a valid message:
In other words:
verify
indicates that callingcreate
orencode
directly on the plain object will [result in a valid message respectively] succeed.fromObject
, on the other hand, does conversion from a broader range of plain objects to create valid messages. (ref)
Examples
Using .proto files
It is possible to load existing .proto files using the full library, which parses and compiles the definitions to ready to use (reflection-based) message classes:
// awesome.proto
package awesomepackage;
syntax = "proto3";
message AwesomeMessage {
string awesome_field = 1; // becomes awesomeField
}
protobuf.load("awesome.proto", function(err, root) {
if (err)
throw err;
// Obtain a message type
var AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage");
// Exemplary payload
var payload = { awesomeField: "AwesomeString" };
// Verify the payload if necessary (i.e. when possibly incomplete or invalid)
var errMsg = AwesomeMessage.verify(payload);
if (errMsg)
throw Error(errMsg);
// Create a new message
var message = AwesomeMessage.create(payload); // or use .fromObject if conversion is necessary
// Encode a message to an Uint8Array (browser) or Buffer (node)
var buffer = AwesomeMessage.encode(message).finish();
// ... do something with buffer
// Decode an Uint8Array (browser) or Buffer (node) to a message
var message = AwesomeMessage.decode(buffer);
// ... do something with message
// If the application uses length-delimited buffers, there is also encodeDelimited and decodeDelimited.
// Maybe convert the message back to a plain object
var object = AwesomeMessage.toObject(message, {
longs: String,
enums: String,
bytes: String,
// see ConversionOptions
});
});
Additionally, promise syntax can be used by omitting the callback, if preferred:
protobuf.load("awesome.proto")
.then(function(root) {
...
});
Using JSON descriptors
The library utilizes JSON descriptors that are equivalent to a .proto definition. For example, the following is identical to the .proto definition seen above:
// awesome.json
{
"nested": {
"awesomepackage": {
"nested": {
"AwesomeMessage": {
"fields": {
"awesomeField": {
"type": "string",
"id": 1
}
}
}
}
}
}
}
JSON descriptors closely resemble the internal reflection structure:
Type (T) | Extends | Type-specific properties |
---|---|---|
ReflectionObject | options | |
Namespace | ReflectionObject | nested |
Root | Namespace | nested |
Type | Namespace | fields |
Enum | ReflectionObject | values |
Field | ReflectionObject | rule, type, id |
MapField | Field | keyType |
OneOf | ReflectionObject | oneof (array of field names) |
Service | Namespace | methods |
Method | ReflectionObject | type, requestType, responseType, requestStream, responseStream |
- Bold properties are required. Italic types are abstract.
T.fromJSON(name, json)
creates the respective reflection object from a JSON descriptorT#toJSON()
creates a JSON descriptor from the respective reflection object (its name is used as the key within the parent)
Exclusively using JSON descriptors instead of .proto files enables the use of just the light library (the parser isn’t required in this case).
A JSON descriptor can either be loaded the usual way:
protobuf.load("awesome.json", function(err, root) {
if (err) throw err;
// Continue at "Obtain a message type" above
});
Or it can be loaded inline:
var jsonDescriptor = require("./awesome.json"); // exemplary for node
var root = protobuf.Root.fromJSON(jsonDescriptor);
// Continue at "Obtain a message type" above
Using custom classes
Message classes can also be extended with custom functionality and it is also possible to register a custom constructor with a reflected message type:
...
// Define a custom constructor
function AwesomeMessage(properties) {
// custom initialization code
...
}
// Register the custom constructor with its reflected type (*)
root.lookupType("awesomepackage.AwesomeMessage").ctor = AwesomeMessage;
// Define custom functionality
AwesomeMessage.customStaticMethod = function() { ... };
AwesomeMessage.prototype.customInstanceMethod = function() { ... };
// Continue at "Create a new message" above
(*) Besides referencing its reflected type through AwesomeMessage.$type
and AwesomeMesage#$type
, the respective custom class is automatically populated with:
AwesomeMessage.create
AwesomeMessage.encode
andAwesomeMessage.encodeDelimited
AwesomeMessage.decode
andAwesomeMessage.decodeDelimited
AwesomeMessage.verify
AwesomeMessage.fromObject
,AwesomeMessage.toObject
andAwesomeMessage#toJSON
Afterwards, decoded messages of this type are instanceof AwesomeMessage
.
Alternatively, it is also possible to reuse and extend the internal constructor if custom initialization code is not required:
...
// Reuse the internal constructor
var AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage").ctor;
// Define custom functionality
AwesomeMessage.customStaticMethod = function() { ... };
AwesomeMessage.prototype.customInstanceMethod = function() { ... };
// Continue at "Create a new message" above
Using the JS API
The API shown above works pretty much the same with TypeScript. However, because everything is typed, accessing fields on instances of dynamically generated message classes requires either using bracket-notation (i.e. message["awesomeField"]
) or explicit casts. Alternatively, it is possible to use a typings file generated for its static counterpart.
import { load } from "protobufjs"; // respectively "./node_modules/protobufjs"
load("awesome.proto", function(err, root) {
if (err)
throw err;
// example code
const AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage");
let message = AwesomeMessage.create({ awesomeField: "hello" });
console.log(`message = ${JSON.stringify(message)}`);
let buffer = AwesomeMessage.encode(message).finish();
console.log(`buffer = ${Array.prototype.toString.call(buffer)}`);
let decoded = AwesomeMessage.decode(buffer);
console.log(`decoded = ${JSON.stringify(decoded)}`);
});
Using generated static code
If you generated static code to bundle.js
using the CLI and its type definitions to bundle.d.ts
, then you can just do:
import { AwesomeMessage } from "./bundle.js";
// example code
let message = AwesomeMessage.create({ awesomeField: "hello" });
let buffer = AwesomeMessage.encode(message).finish();
let decoded = AwesomeMessage.decode(buffer);
Protocol Buffers
protobuf.js
- API Documentation
- CHANGELOG
- Frequently asked questions on our wiki