XUtils

config

Configuration library supporting Java properties, JSON or its human optimized superset HOCON.

Java
https://github.com/lightbend/config

Configuration library for JVM languages.

Maven Central Build Status

Essential Information

Binary Releases

Typesafe Config is compatible with Java 8 and above.

You can find published releases on Maven Central.

<dependency>
    <groupId>com.typesafe</groupId>
    <artifactId>config</artifactId>
    <version>1.4.3</version>
</dependency>

sbt dependency:

libraryDependencies += "com.typesafe" % "config" % "1.4.3"

Link for direct download if you don’t use a dependency manager:

Release Notes

Please see NEWS.md in this directory, https://github.com/lightbend/config/blob/main/NEWS.md

API docs

Build

The build uses sbt and the tests are written in Scala; however, the library itself is plain Java and the published jar has no Scala dependency.

Using the Library

API Example

import com.typesafe.config.ConfigFactory

Config conf = ConfigFactory.load();
int bar1 = conf.getInt("foo.bar");
Config foo = conf.getConfig("foo");
int bar2 = foo.getInt("bar");

Longer Examples

See the examples in the examples/ directory.

You can run these from the sbt console with the commands project config-simple-app-java and then run.

In brief, as shown in the examples:

  • libraries should use a Config instance provided by the app, if any, and use ConfigFactory.load() if no special Config is provided. Libraries should put their defaults in a reference.conf on the classpath.
  • apps can create a Config however they want (ConfigFactory.load() is easiest and least-surprising), then provide it to their libraries. A Config can be created with the parser methods in ConfigFactory or built up from any file format or data source you like with the methods in ConfigValueFactory.

Immutability

Objects are immutable, so methods on Config which transform the configuration return a new Config. Other types such as ConfigParseOptions, ConfigResolveOptions, ConfigObject, etc. are also immutable. See the API docs for details of course.

Schemas and Validation

There isn’t a schema language or anything like that. However, two suggested tools are:

  • use the checkValid() method
  • access your config through a Settings class with a field for each setting, and instantiate it on startup (immediately throwing an exception if any settings are missing)

In Scala, a Settings class might look like:

class Settings(config: Config) {

    // validate vs. reference.conf
    config.checkValid(ConfigFactory.defaultReference(), "simple-lib")

    // non-lazy fields, we want all exceptions at construct time
    val foo = config.getString("simple-lib.foo")
    val bar = config.getInt("simple-lib.bar")
}

See the examples/ directory for a full compilable program using this pattern.

Standard behavior

The convenience method ConfigFactory.load() loads the following (first-listed are higher priority):

  • system properties
  • application.conf (all resources on classpath with this name)
  • application.json (all resources on classpath with this name)
  • application.properties (all resources on classpath with this name)
  • reference.conf (all resources on classpath with this name)

The idea is that libraries and frameworks should ship with a reference.conf in their jar. Applications should provide an application.conf, or if they want to create multiple configurations in a single JVM, they could use ConfigFactory.load("myapp") to load their own myapp.conf.

Libraries and frameworks should default to ConfigFactory.load() if the application does not provide a custom Config object. This way, libraries will see configuration from application.conf and users can configure the whole app, with its libraries, in a single application.conf file.

Libraries and frameworks should also allow the application to provide a custom Config object to be used instead of the default, in case the application needs multiple configurations in one JVM or wants to load extra config files from somewhere. The library examples in examples/ show how to accept a custom config while defaulting to ConfigFactory.load().

For applications using application.{conf,json,properties}, system properties can be used to force a different config source (e.g. from command line -Dconfig.file=path/to/config-file):

  • config.resource specifies a resource name - not a basename, i.e. application.conf not application
  • config.file specifies a filesystem path, again it should include the extension, not be a basename
  • config.url specifies a URL

Note: you need to pass -Dconfig.file=path/to/config-file before the jar itself, e.g. java -Dconfig.file=path/to/config-file.conf -jar path/to/jar-file.jar. Same applies for -Dconfig.resource=config-file.conf

These system properties specify a replacement for application.{conf,json,properties}, not an addition. They only affect apps using the default ConfigFactory.load() configuration. In the replacement config file, you can use include "application" to include the original default config file; after the include statement you could go on to override certain settings.

If you set config.resource, config.file, or config.url on-the-fly from inside your program (for example with System.setProperty()), be aware that ConfigFactory has some internal caches and may not see new values for system properties. Use ConfigFactory.invalidateCaches() to force-reload system properties.

Note about resolving substitutions in reference.conf and application.conf

The substitution syntax ${foo.bar} will be resolved twice. First, all the reference.conf files are merged and then the result gets resolved. Second, all the application.conf are layered over the unresolved reference.conf and the result of that gets resolved again.

The implication of this is that the reference.conf stack has to be self-contained; you can’t leave an undefined value ${foo.bar} to be provided by application.conf. It is however possible to override a variable that reference.conf refers to, as long as reference.conf also defines that variable itself.

Merging config trees

Any two Config objects can be merged with an associative operation called withFallback, like merged = firstConfig.withFallback(secondConfig).

The withFallback operation is used inside the library to merge duplicate keys in the same file and to merge multiple files. ConfigFactory.load() uses it to stack system properties over application.conf over reference.conf.

You can also use withFallback to merge in some hardcoded values, or to “lift” a subtree up to the root of the configuration; say you have something like:

foo=42
dev.foo=57
prod.foo=10

Then you could code something like:

Config devConfig = originalConfig
                     .getConfig("dev")
                     .withFallback(originalConfig)

There are lots of ways to use withFallback.

How to handle defaults

Many other configuration APIs allow you to provide a default to the getter methods, like this:

boolean getBoolean(String path, boolean fallback)

Here, if the path has no setting, the fallback would be returned. An API could also return null for unset values, so you would check for null:

// returns null on unset, check for null and fall back
Boolean getBoolean(String path)

The methods on the Config interface do NOT do this, for two major reasons:

  1. If you use a config setting in two places, the default fallback value gets cut-and-pasted and typically out of sync. This can result in Very Evil Bugs.
  2. If the getter returns null (or None, in Scala) then every time you get a setting you have to write handling code for null/None and that code will almost always just throw an exception. Perhaps more commonly, people forget to check for null at all, so missing settings result in NullPointerException.

For most situations, failure to have a setting is simply a bug to fix (in either code or the deployment environment). Therefore, if a setting is unset, by default the getters on the Config interface throw an exception.

If you want to allow a setting to be missing from application.conf in a particular case, then here are some options:

  1. Set it in a reference.conf included in your library or application jar, so there’s a default value.
  2. Use the Config.hasPath() method to check in advance whether the path exists (rather than checking for null/None after as you might in other APIs).
  3. Catch and handle ConfigException.Missing. NOTE: using an exception for control flow like this is much slower than using Config.hasPath(); the JVM has to do a lot of work to throw an exception.
  4. In your initialization code, generate a Config with your defaults in it (using something like ConfigFactory.parseMap()) then fold that default config into your loaded config using withFallback(), and use the combined config in your program. “Inlining” your reference config in the code like this is probably less convenient than using a reference.conf file, but there may be reasons to do it.
  5. Use Config.root() to get the ConfigObject for the Config; ConfigObject implements java.util.Map<String,?> and the get() method on Map returns null for missing keys. See the API docs for more detail on Config vs. ConfigObject.
  6. Set the setting to null in reference.conf, then use Config.getIsNull and Config.hasPathOrNull to handle null in a special way while still throwing an exception if the setting is entirely absent.

The recommended path (for most cases, in most apps) is that you require all settings to be present in either reference.conf or application.conf and allow ConfigException.Missing to be thrown if they are not. That’s the design intent of the Config API design.

Consider the “Settings class” pattern with checkValid() to verify that you have all settings when you initialize the app. See the Schemas and Validation section of this README for more details on this pattern.

If you do need a setting to be optional: checking hasPath() in advance should be the same amount of code (in Java) as checking for null afterward, without the risk of NullPointerException when you forget. In Scala, you could write an enrichment class like this to use the idiomatic Option syntax:

implicit class RichConfig(val underlying: Config) extends AnyVal {
  def getOptionalBoolean(path: String): Option[Boolean] = if (underlying.hasPath(path)) {
     Some(underlying.getBoolean(path))
  } else {
     None
  }
}

Since this library is a Java library it doesn’t come with that out of the box, of course.

It is understood that sometimes defaults in code make sense. For example, if your configuration lets users invent new sections, you may not have all paths up front and may be unable to set up defaults in reference.conf for dynamic paths. The design intent of Config isn’t to prohibit inline defaults, but simply to recognize that it seems to be the 10% case (rather than the 90% case). Even in cases where dynamic defaults are needed, you may find that using withFallback() to build a complete nothing-missing Config in one central place in your code keeps things tidy.

Whatever you do, please remember not to cut-and-paste default values into multiple places in your code. You have been warned! :-)

Understanding Config and ConfigObject

To read and modify configuration, you’ll use the Config interface. A Config looks at a JSON-equivalent data structure as a one-level map from paths to values. So if your JSON looks like this:

  "foo" : {
    "bar" : 42
    "baz" : 43
  }

Using the Config interface, you could write conf.getInt("foo.bar"). The foo.bar string is called a path expression (HOCON.md has the syntax details for these expressions). Iterating over this Config, you would get two entries; "foo.bar" : 42 and "foo.baz" : 43. When iterating a Config you will not find nested Config (because everything gets flattened into one level).

When looking at a JSON tree as a Config, null values are treated as if they were missing. Iterating over a Config will skip null values.

You can also look at a Config in the way most JSON APIs would, through the ConfigObject interface. This interface represents an object node in the JSON tree. ConfigObject instances come in multi-level trees, and the keys do not have any syntax (they are just strings, not path expressions). Iterating over the above example as a ConfigObject, you would get one entry "foo" : { "bar" : 42, "baz" : 43 }, where the value at "foo" is another nested ConfigObject.

In ConfigObject, null values are visible (distinct from missing values), just as they are in JSON.

ConfigObject is a subtype of ConfigValue, where the other subtypes are the other JSON types (list, string, number, boolean, null).

Config and ConfigObject are two ways to look at the same internal data structure, and you can convert between them for free using Config.root() and ConfigObject.toConfig().

ConfigBeanFactory

As of version 1.3.0, if you have a Java object that follows JavaBean conventions (zero-args constructor, getters and setters), you can automatically initialize it from a Config.

Use ConfigBeanFactory.create(config.getConfig("subtree-that-matches-bean"), MyBean.class) to do this.

Creating a bean from a Config automatically validates that the config matches the bean’s implied schema. Bean fields can be primitive types, typed lists such as List<Integer>, java.time.Duration, ConfigMemorySize, or even a raw Config, ConfigObject, or ConfigValue (if you’d like to deal with a particular value manually).

Using HOCON, the JSON Superset

The JSON superset is called “Human-Optimized Config Object Notation” or HOCON, and files use the suffix .conf. See HOCON.md in this directory for more detail.

After processing a .conf file, the result is always just a JSON tree that you could have written (less conveniently) in JSON.

Examples of HOCON

All of these are valid HOCON.

Start with valid JSON:

{
    "foo" : {
        "bar" : 10,
        "baz" : 12
    }
}

Drop root braces:

"foo" : {
    "bar" : 10,
    "baz" : 12
}

Drop quotes:

foo : {
    bar : 10,
    baz : 12
}

Use = and omit it before {:

foo {
    bar = 10,
    baz = 12
}

Remove commas:

foo {
    bar = 10
    baz = 12
}

Use dotted notation for unquoted keys:

foo.bar=10
foo.baz=12

Put the dotted-notation fields on a single line:

foo.bar=10, foo.baz=12

The syntax is well-defined (including handling of whitespace and escaping). But it handles many reasonable ways you might want to format the file.

Note that while you can write HOCON that looks a lot like a Java properties file (and many properties files will parse as HOCON), the details of escaping, whitespace handling, comments, and so forth are more like JSON. The spec (see HOCON.md in this directory) has some more detailed notes on this topic.

Uses of Substitutions

The ${foo.bar} substitution feature lets you avoid cut-and-paste in some nice ways.

Factor out common values

This is the obvious use,

standard-timeout = 10ms
foo.timeout = ${standard-timeout}
bar.timeout = ${standard-timeout}

Inheritance

If you duplicate a field with an object value, then the objects are merged with last-one-wins. So:

foo = { a : 42, c : 5 }
foo = { b : 43, c : 6 }

means the same as:

foo = { a : 42, b : 43, c : 6 }

You can take advantage of this for “inheritance”:

data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic}
data-center-east = { name = "east" }
data-center-west = ${data-center-generic}
data-center-west = { name = "west", cluster-size = 8 }

Using include statements you could split this across multiple files, too.

If you put two objects next to each other (close brace of the first on the same line with open brace of the second), they are merged, so a shorter way to write the above “inheritance” example would be:

data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic} { name = "east" }
data-center-west = ${data-center-generic} { name = "west", cluster-size = 8 }

Set array values outside configuration files

Setting the value of array items from java properties or environment variables require specifying the index in the array for the value. So, while in HOCON you can set multiple values into an array or append to an array:

## HOCON
items = ["a", "b"]
items += "c"

Using java properties you specify the exact position:

-Ditems.0="a" -Ditems.1="b"

as well as with environment variables:

export CONFIG_FORCE_items_0=a
export CONFIG_FORCE_items_1=b

Miscellaneous Notes

Debugging Your Configuration

If you have trouble with your configuration, some useful tips.

  • Set the Java system property -Dconfig.trace=loads to get output on stderr describing each file that is loaded. Note: this feature is not included in the older version in Play/Akka 2.0.
  • Use myConfig.root().render() to get a Config as a string with comments showing where each value came from. This string can be printed out on console or logged to a file etc.
  • If you see errors like com.typesafe.config.ConfigException$Missing: No configuration setting found for key foo, and you’re sure that key is defined in your config file, they might appear e.g. when you’re loading configuration from a thread that’s not the JVM’s main thread. Try passing the ClassLoader in manually - e.g. with ConfigFactory.load(getClass().getClassLoader()) or setting the context class loader. If you don’t pass one, Lightbend Config uses the calling thread’s contextClassLoader, and in some cases, it may not have your configuration files in its classpath, so loading the config on that thread can yield unexpected, erroneous results.

Other APIs (Wrappers, Ports and Utilities)

This may not be comprehensive - if you’d like to add mention of your wrapper, just send a pull request for this README. We would love to know what you’re doing with this library or with the HOCON format.

Guice integration

Java (yep!) wrappers for the Java library

Scala wrappers for the Java library

Clojure wrappers for the Java library

Kotlin wrappers for the Java library

Ruby port

Puppet module

Python port

C++ port

JavaScript port

C# port

Rust port

Go port

Erlang port

Linting tool

Online playground

Maintenance notes

Articles

  • coming soon...