Hazelcast

---

What is Hazelcast

The world’s leading companies trust Hazelcast to modernize applications and take instant action on data in motion to create new revenue streams, mitigate risk, and operate more efficiently. Businesses use Hazelcast’s unified real-time data platform to process streaming data, enrich it with historical context and take instant action with standard or ML/AI-driven automation - before it is stored in a database or data lake.

Hazelcast is named in the Gartner Market Guide to Event Stream Processing and a leader in the GigaOm Radar Report for Streaming Data Platforms. To join our community of CXOs, architects and developers at brands such as Lowe’s, HSBC, JPMorgan Chase, Volvo, New York Life, and others, visit hazelcast.com.

When to use Hazelcast

Hazelcast provides a platform that can handle multiple types of workloads for building real-time applications.

• Stateful data processing over streaming data or data at rest
• Querying streaming and batch data sources directly using SQL
• Ingesting data through a library of connectors and serving it using low-latency SQL queries
• Pushing updates to applications on events
• Low-latency queue-based or pub-sub messaging
  
• Fast access to contextual and transactional data via caching patterns such as read/write-through and write-behind
• Distributed coordination for microservices
• Replicating data from one region to another or between data centers in the same region

Key Features

• Stateful and fault-tolerant data processing and querying over data streams and data at rest using SQL or dataflow API
• A comprehensive library of connectors such as Kafka, Hadoop, S3, RDBMS, JMS and many more
• Distributed messaging using pub-sub and queues
• Distributed, partitioned, queryable key-value store with event listeners, which can also be used to store contextual data for enriching event streams with low latency
• A production-ready Raft-implementation which allows lineralizable (CP) concurrency primitives such as distributed locks.
• Tight integration for deploying machine learning models with Python to a data processing pipeline
• Cloud-native, run everywhere architecture
• Zero-downtime operations with rolling upgrades
• At-least-once and exactly-once processing guarantees for stream processing pipelines
• Data replication between data centers and geographic regions using WAN
• Microsecond performance for key-value point lookups and pub-sub
• Unique data processing architecture results in 99.99% latency of under 10ms for streaming queries with millions of events per second.
• Client libraries in Java, Python, Node.js, .NET, C++ and Go

Data Processing using SQL

Jet also powers the SQL engine in Hazelcast which can execute both streaming and batch queries. Internally, all SQL queries are converted to Jet jobs.

CREATE MAPPING trades (
    id BIGINT,
    ticker VARCHAR,
    price DECIMAL,
    amount BIGINT)
TYPE Kafka
OPTIONS (
    'valueFormat' = 'json',
    'bootstrap.servers' = 'kafka:9092'
);
SELECT ticker, ROUND(price * 100) AS price_cents, amount
  FROM trades
  WHERE price * amount > 100;
+------------+----------------------+-------------------+
|ticker      |           price_cents|             amount|
+------------+----------------------+-------------------+
|EFGH        |                  1400|                 20|

Messaging

Hazelcast provides lightweight options for adding messaging to your application. The two main constructs for messaging are topics and queues.

Topics

Topics provide a publish-subscribe pattern where each message is fanned out to multiple subscribers. See the examples below in Java and Python:

var hz = Hazelcast.bootstrappedInstance();
ITopic<String> topic = hz.getTopic("my_topic");
topic.addMessageListener(msg -> System.out.println(msg));
topic.publish("message");

topic = client.get_topic("my_topic")

def handle_message(msg):
    print("Received message %s"  % msg.message)
topic.add_listener(on_message=handle_message)
topic.publish("my-message")

For examples in other languages, please refer to the docs.

Queues

Queues provide FIFO-semantics and you can add items from one client and remove from another. See the examples below in Java and Python:

var client = Hazelcast.newHazelcastClient();
IQueue<String> queue = client.getQueue("my_queue");
queue.put("new-item")

import hazelcast

client = hazelcast.HazelcastClient()
q = client.get_queue("my_queue")
my_item = q.take().result()
print("Received item %s" % my_item)

For examples in other languages, please refer to the docs.

Get Started

Follow the Getting Started Guide to install and start using Hazelcast.

Documentation

Read the documentation for in-depth details about how to install Hazelcast and an overview of the features.

Get Help

You can use Slack for getting help with Hazelcast

Building From Source

Building Hazelcast requires at minimum JDK 17. Pull the latest source from the repository and use Maven install (or package) to build:

$ git pull origin master
$ ./mvnw clean package -DskipTests

It is recommended to use the included Maven wrapper script. It is also possible to use local Maven distribution with the same version that is used in the Maven wrapper script.

Additionally, there is a quick build activated by setting the -Dquick system property that skips validation tasks for faster local builds (e.g. tests, checkstyle validation, javadoc, source plugins etc) and does not build extensions and distribution modules.

Testing

Take into account that the default build executes thousands of tests which may take a considerable amount of time. Hazelcast has 3 testing profiles:

• Default:

  
  ./mvnw test
  

  to run quick/integration tests (those can be run in parallel without using network by using -P parallelTest profile).
• Slow Tests:

  
  ./mvnw test -P nightly-build
  

  to run tests that are either slow or cannot be run in parallel.
• All Tests:

  
  ./mvnw test -P all-tests
  

  to run all tests serially using network.

Some tests require Docker to run. Set -Dhazelcast.disable.docker.tests system property to ignore them.

When developing a PR it is sufficient to run your new tests and some related subset of tests locally. Our PR builder will take care of running the full test suite.

Creating PRs for Hazelcast SQL

When creating a PR with changes located in the hazelcast-sql module and nowhere else, you can label your PR with SQL-only. This will change the standard PR builder to one that will only run tests related to SQL (see run-sql-only above), which will significantly shorten the build time vs. the default PR builder. NOTE: this job will fail if you’ve made changes anywhere other than hazelcast-sql.

Creating PRs which contain only documentation

When creating a PR which changes only documentation (files with suffix .md or .adoc) it makes no sense to run tests. For that case the label docs-only can be used. The job will fail in case you’ve made other changes than in .md, .adoc or .txt files.

Copyright

Copyright © 2008-2024, Hazelcast, Inc. All Rights Reserved.

Visit www.hazelcast.com for more info.