XUtils

Protoc Gen Elm

Generate Protobuf En/Decoders from .proto files

Elm
https://github.com/andreasewering/protoc-gen-elm

Explanations about the generated code

In general, the generated code tries to be close to what the code looks like in other languages while still being ideomatic Elm code. Elm’s concept of “Only one solution to solve” a problem has several consequences here.

General

  • Protobufs messages are product types, enums and oneofs are union types.
  • Each message and enum generates encode[name] and decode[name] functions, which integrate seamlessly with elm-protocol-buffers
  • Each message and enum generates a default[name] function, which sets the defaults as seen in the table above
  • enums and oneofs generate seperate modules, to avoid naming collisions.
  • oneofs come in two forms, one where every constructor includes a generic type and one where all types are applied. These are needed for use inside of other messages (you will see why in the section “Module Nesting”)

Recursive Data Types

For ease of construction, protoc-gen-elm prefers to generate type aliases instead of nominal types. Type aliases have one downside though: they cannot be recursive. Otherwise, the Elm compiler would have to do infinite work to expand the type. So if you have a recursive type like this:

message Rec {
  repeated Rec rec = 1;
}

we generate

type alias Rec = { rec : List Rec_ }

type Rec_ = Rec_ Rec

and corresponding wrapRec and unwrapRec functions.

gRPC

If your .proto file includes a service declaration, an Elm module will be generated based on package and the services name.

This file:

package some_package

service SomeService {}

will generate a Proto/SomePackage/SomeService.elm module.

The code that needs to be generated inside is actually rather small. A gRPC call just needs

  • the package name
  • the method name
  • the service name
  • references to the en/decoder functions

The rest of the work is done by the elm-grpc package. It provides functions to convert the generated Grpc.Rpc instances into Cmds and Tasks, as well as setting the usual Http Request fields (headers, timeout, tracker etc.)

Live Example

To run a minimal live example in your browser, follow the instructions in /example/grpc/README.md. For a more advanced/realistic example, look at /example/tonic_vite/README.md.

Well-known types

If you want to use protobufs well-known-types, you need to install the pre-built package elm-protoc-types or include the paths to the proto files in the compilation.

Example: If this is your proto file test.proto which uses the well-known type Timestamp,

import "google/protobuf/timestamp.proto";

message TestMessage {
  google.protobuf.Timestamp timestamp = 1;
}

the protoc invocation will need to include the path to the well-known types .proto file.

protoc --elm_out=. test.proto /usr/local/include/google/protobuf/timestamp.proto

Limitations

Development

Note: Currently, this project won’t run on Windows (WSL works) because of shell scripts/executable js files.

Execute npm install, npm run build and npm test and you should be good to go. You will need protoc installed and on your PATH.

  • The plugin logic is written in Elm itself. To be executable via node, there is a index.js wrapper. It converts the incoming bytes to base64, because there currently is no way to directly send the type Bytes through a port.
  • Main.elm essentially wires up the binding to JS: A request is received through a port, gets decoded, processed and then sent through another port.
  • For decoding the protoc request, it uses “itself”, meaning that upgrading protoc versions should be done by running the plugin against the new include files from protoc to generate the new encoders/decoders (use the upgrade.sh script).
  • A Mapper converts the request into a convenient internal structure
  • A Generator then uses this internal structure to build an Elm AST which is then pretty-printed to a file.

Run build.sh to build the elm code into index.min.js (which is imported by the entrypoint index.js).

To analyse the protoc requests, there are debug.js, DebugMain and build_debug.sh files. Run build_debug.sh, then use debug.js in place of index.js when running protoc. This should dump the deserialized request into debug.log. You can then put this into the Elm repl for example or use it as input for tests.

Articles

  • coming soon...