VerneMQ can be monitored in several ways. We implemented native support for Graphite, MQTT $SYS tree, and Prometheus.

The metrics are also available via the command line tool:

vmq-admin metrics show

Or with:

vmq-admin metrics show -d

Which will output the metrics together with a short description describing what the metric is about. An example looks like:

# The number of AUTH packets received.
counter.mqtt_auth_received = 0

# The number of times a MQTT queue process has been initialized from offline storage.
counter.queue_initialized_from_storage = 0

# The number of PUBLISH packets sent.
counter.mqtt_publish_sent = 10

# The number of bytes used for storing retained messages.
gauge.retain_memory = 21184

Notice that the metrics:

mqtt_connack_not_authorized_sent
mqtt_connack_bad_credentials_sent
mqtt_connack_server_unavailable_sent
mqtt_connack_identifier_rejected_sent
mqtt_connack_unacceptable_protocol_sent
mqtt_connack_accepted_sent

Are no longer used (always 0) and will be removed in the future. They were replaced with mqtt_connack_sent using the return_code label. For MQTT 5.0 the reason_code label is used instead.

The output on the command line are aggregated by default, but details for a label can be shown as well, for example all metrics with the not_authorized label:

vmq-admin metrics show --return_code=not_authorized
counter.mqtt_connack_sent = 0

All available labels can be show using vmq-admin metrics show --help.

Start a VerneMQ cluster node

docker run --name vernemq1 -d vernemq/vernemq

Sometimes you need to configure a forwarding for ports (on a Mac for example):

docker run -p 1883:1883 --name vernemq1 -d vernemq/vernemq

This starts a new node that listens on 1883 for MQTT connections and on 8080 for MQTT over websocket connections. However, at this moment the broker won't be able to authenticate the connecting clients. To allow anonymous clients use the DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on environment variable.

docker run -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" --name vernemq1 -d vernemq/vernemq

Autojoining a VerneMQ cluster

This allows a newly started container to automatically join a VerneMQ cluster. Assuming you started your first node like the example above you could autojoin the cluster (which currently consists of a single container 'vernemq1') like the following:

docker run -e "DOCKER_VERNEMQ_DISCOVERY_NODE=<IP-OF-VERNEMQ1>" --name vernemq2 -d vernemq/vernemq

(Note, you can find the IP of a docker container using docker inspect <CONTAINER_NAME> | grep \"IPAddress\").

Checking cluster status

To check if the above containers have successfully clustered you can issue the vmq-admin command:

docker exec vernemq1 vmq-admin cluster show
+--------------------+-------+
|        Node        |Running|
+--------------------+-------+
|VerneMQ@172.17.0.151| true  |
|VerneMQ@172.17.0.152| true  |
+--------------------+-------+

Install VerneMQ

Once you have downloaded the binary package, execute the following command to install VerneMQ:

sudo yum install vernemq-<VERSION>.centos7.x86_64.rpm

or:

sudo rpm -Uvh vernemq-<VERSION>.centos7.x86_64.rpm

Activate VerneMQ node

Once you've installed VerneMQ, start it on your node:

service vernemq start

Verify your installation

You can verify that VerneMQ is successfully installed by running:

rpm -qa | grep vernemq

If VerneMQ has been installed successfully vernemq is returned.

Next Steps

Now that you've installed VerneMQ, check out How to configure VerneMQ.

Welcome to the VerneMQ documentation! This is a reference guide for most of the available features and options of VerneMQ. The Getting Started guide might be a good entry point.

The VerneMQ documentation is based on the VerneMQ Documentation project. Any changes on Github are automatically deployed to the VerneMQ online Documentation.

For a more general overview on VerneMQ and MQTT, you might want to start with the introduction.

For downloading the subscription-based binary VerneMQ packages and/or a quick description on how to compile VerneMQ from sources, see Downloads.

How to help improve this documentation

The VerneMQ Documentation project is an open-source effort, and your contributions are very welcome and appreciated. You can contribute on all levels:

• Language, style and typos

• Fixing obvious documentation errors and gaps

• Providing more details and/or examples for specific topics

• Extending the documentation where you find this useful to do

Note that the documentation is versioned according to the VerneMQ releases. You can click the "Edit on Github" button in the upper right corner of every page to check what branch and document you are on. You can then create a Pull Request (PR) against that branch from your fork of the VerneMQ documentation repository. (Direct edits on Github are possible for members of the documentation repository).

Installing VerneMQ

VerneMQ is a high-performance, distributed MQTT message broker. It scales horizontally and vertically on commodity hardware to support a high number of concurrent publishers and consumers while maintaining low latency and fault tolerance. To use it, all you need to do is install the VerneMQ package.

Choose your OS and follow the instructions:

• CentOS/RHEL

• Debian/Ubuntu

It is also possible to run VerneMQ using our Docker image:

• Docker

Starting VerneMQ

To start a VerneMQ broker, use the vernemq start command in your Shell:

vernemq start

A successful start will return no output. If there is a problem starting the broker, an error message is printed to STDERR.

To run VerneMQ with an attached interactive Erlang console:

vernemq console

A VerneMQ broker is typically started in console mode for debugging or troubleshooting purposes. Note that if you start VerneMQ in this manner, it is running as a foreground process that will exit when the console is closed.

You can close the console by issuing this command at the Erlang prompt:

q().

Once your broker has started, you can initially check that it is running with the vernemq ping command:

vernemq ping

The command will respond with pong if the broker is running or Node <NodeName> not responding to pings in case it’s not.

Starting using systemd/systemctl

If you use a systemd service file (as in the binary packages), you can start VerneMQ using the systemctl interface to systemd:

$ sudo systemctl start vernemq

Other systemctl commands work as well:

$ sudo systemctl stop vernemq
$ sudo systemctl status vernemq

Introduction and general setup

VerneMQ supports authentication and authorization using a number of popular databases and the below sections describe how to configure the different databases.

The database drivers are handled using the vmq_diversity plugin and it therefore needs to be enabled:

When using database based authentication/authorization the enabled-by-default file based authentication and authorization are most likely not needed and should be disabled:

In order to use a database for authentication and authorization the database must be properly configured and the auth-data (username, clientid, password, acls) to be present. The following sections show some sample requests that can be used to insert such data.

While the handling of authentication differs among the different databases, the handling of ACLs is roughly identical and make use of a JSON array containing one or many ACL objects per configured client.

A minimal publish & subscribe ACL JSON object takes the following form:

General ACL

The pattern is a MQTT topic string that can contain MQTT wildcards, but also the template variables %m (mountpoint), %u (username), and %c (client id) which are automatically substituted with the auth data provided.

Publish ACL

The publish ACL makes it possible to control the maximum QoS and payload size that is allowed, and if the message is allowed to be retained.

Moreover, the publish ACL makes it possible to modify the properties of a published message through specifying one or multiple modifiers. Please note that the modified message isn't re-validated by the ACL.

Subscribe ACL

The subscribe ACL makes it possible to control the maxium QoS a client is allowed to subscribe to.

Like the publish ACL, the subscribe ACL makes it possible to change the current subscription request by returning a custom set of topic/qos pairs. Please note that the modified subscription isn't re-validated by the ACL.

Password verification and hashing methods

When deciding on which database to use one has to consider which kind of password hashing and key derivation functions are available and required. Different databases provide different mechanisms, for example PostgreSQL provides the pgcrypto module which supports verifying hashed and salted passwords, while Redis has no such features. VerneMQ therefore also provides client-side password verification mechanisms such as bcrypt.

There is a trade-off between verifying passwords on the client-side versus on the server-side. Verifying passwords client-side of course means doing the computations on the VerneMQ broker and this takes away resources from other tasks such as routing messages. With hashing functions such as bcrypt which are designed specifically to be slow (proportional to the number of rounds) in order to make brute-force attacks infeasible, this can become a problem. For example, if verifying a password with bcrypt takes 0.5 seconds then on a single threaded core 2 verifications/second are possible and using 4 single threaded cores 8 verifications/second. So, the number of rounds/security paramenters have a direct impact on the max number of verifications/second and hence also the maximum arrival rate of new clients per second.

For each database it is specified which password verification mechanisms are available and if they are client-side or server-side.

PostgreSQL

To enable PostgreSQL authentication and authorization the following need to be configured in the vernemq.conf file:

In case your Postgresql database requires SSL, you'll have to tell the plugin:

Consult the vernemq.conf file for more info about additional options:

PostgreSQL hashing methods:

Creating the Postgres tables

The following SQL DDL must be applied, the pgcrypto extension is required if using the server-side crypt hashing method:

To enter new ACL entries use a query similar to the following:

CockroachDB

To enable CockroachDB authentication and authorization the following need to be configured in the vernemq.conf file:

Notice that if the CockroachDB installation is secure, then TLS is required. If using an insecure installation without TLS, then vmq_diversity.cockroachdb.ssl can be set to off.

CockroachDB hashing methods:

Creating the CockroachDB tables

The following SQL DDL must be applied:

To enter new ACL entries use a query similar to the following, the example is for the bcrypt hashing method:

MySQL

For MySQL authentication and authorization configure the following in vernemq.conf:

MySQL hashing methods:

It should be noted that all the above options stores unsalted passwords which are vulnerable to rainbow table attacks, so the threat-model should be considered carefully when using these. Also note the methods marked with * are no longer considered secure hashes.

Creating the MySQL tables

The following SQL DDL must be applied:

To enter new ACL entries use a query similar to the following, the example uses PASSWWORD to for password hashing:

MongoDB

For MongoDB authentication and authorization configure the following in vernemq.conf:

VerneMQ supports MongoDB's DNS SRV record lookup to fetch a seed list. Specify the hostname of hosted database as a srv option instead of host and port. VerneMQ will randomly choose a host/port combination from the seed list returned in the DNS SRV record. MongoDB SRV connections use TLS by default. You will need to configure TLS support for MongoDB for most SRV connections.

MongoDB supports a number of node types in replica sets. The built-in MongoDB support simply connects to the host and port specified. It does not differentiate between primary or secondary nodes in MongoDB replica sets.

MongoDB hashing methods:

Insert the ACL using the mongo shell or any software library. The passhash property contains the bcrypt hash of the clients password.

Redis

For Redis authentication and authorization configure the following in vernemq.conf:

Redis hashing methods:

Insert the ACL using the redis-cli shell or any software library. The passhash property contains the bcrypt hash of the clients password. The key is an encoded JSON array containing the mountpoint, username, and client id. Note that no spaces are allowed between the array items.