HNHacker News
TopNewBestAskShowJobs

dlandiak

12 karma · joined December 15, 2023

submissionscomments
dlandiak··on Show HN: TBMQ – Live demo of a high-performance MQTT broker built on Kafka
Hi HN! I'm Dima from the ThingsBoard team — the company behind the open-source IoT platform and TBMQ, our MQTT broker built on top of Apache Kafka.

Most MQTT brokers hit a ceiling when you need to scale horizontally or survive node failures without message loss. We built TBMQ on Kafka because it's battle-tested at scale across the industry — you get fault tolerance, high throughput, and horizontal scalability as a foundation rather than something you bolt on later.

TBMQ has a Community Edition (open-source) and a Professional Edition with advanced observability, enterprise features, and monitoring. The demo runs the PE version, so you can see the full feature set without installing anything.

Two ways to try it:

- No signup: Connect via MQTT directly to "demo.tbmq.io" — ports 1883 (TCP), 8883 (TLS), 443 (WSS), username - "demo", and no password. Test pub/sub, shared subscriptions, retained messages. Public environment, so no sensitive data.

- Quick SSO signup: Head to https://demo.tbmq.io for read-only access to the full professional UI — live session/subscription browsing, message flow dashboards, and more.

Happy to answer questions about the architecture or tradeoffs. Fair criticism welcome.

dlandiak··on TBMQ 2.1 levels up your MQTT stack with embedded integrations and Helm support
Yes, TBMQ currently uses Redis, but as a fully open-source broker that prioritizes open-source technologies, we’re following the community shift to Valkey and plan to support it in the next release.
dlandiak··on TBMQ 2.1 levels up your MQTT stack with embedded integrations and Helm support
TBMQ 2.1 introduces native HTTP, MQTT, and Kafka integrations—plus an official Helm chart to make deploying your MQTT infrastructure easier than before.
dlandiak··on TBMQ 2.1 levels up your MQTT stack with embedded integrations and Helm support
Our architecture introduces a new Integration Executor microservice, ensuring that outbound integrations have zero impact on the core MQTT broker’s performance.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
thank you, my friend :)
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
As for me, TBMQ proposes the same reasons to use it as ActiveMQ Artemis. ActiveMQ Artemis seems to be a great platform that states about its high performance, clustering support, and data persistence. We believe TBMQ offers the same advantages by relying on Kafka to provide high-throughput and low-latency delivery. Check out our performance results: https://thingsboard.io/docs/mqtt-broker/reference/100m-conne... We prioritize data durability by leveraging Kafka replication guarantees. The scalability of TBMQ is very easy to achieve without the need to configure anything. Just add nodes as you go to achieve better throughput/capacity/performance.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
I would recommend reviewing the following article: https://bloomberg.github.io/blazingmq/docs/introduction/comp...

Key points can be found there and everyone will choose their own priorities. For us these are: * https://bloomberg.github.io/blazingmq/docs/introduction/comp... * https://bloomberg.github.io/blazingmq/docs/introduction/comp... (Kafka is Java as well as TBMQ, Kafka already does not depend on Zookeeper) * https://bloomberg.github.io/blazingmq/docs/introduction/comp... * https://bloomberg.github.io/blazingmq/docs/introduction/comp... (for this we recommend reviewing our test - https://thingsboard.io/docs/mqtt-broker/reference/3m-through...)

dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
RabbitMQ with the MQTT plugin is actually a good choice for many scenarios, especially when you need a messaging system that supports the MQTT protocol alongside other messaging protocols like AMQP. However, as they themselves say, There are other good MQTT brokers out there, and some will be able to handle even more MQTT client connections than RabbitMQ because other brokers are specialised in MQTT only. TBMQ - is exactly such a MQTT broker that is designed to be scalable, fault-tolerant, and efficient. The recent performance tests showed the TBMQ quality. Additionally, TBMQ can be easily launched as a single server, two servers, three servers, and so on. Compared to RabbitMQ - https://www.rabbitmq.com/mqtt.html#requirements
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
As far as I know, it is a lightweight MQTT broker designed to take good advantage of multi-CPU environments. We provide cluster support out of the box and are designed to support and work great for various setups, from small-scale to enterprise-level implementations.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
We aim to be no worse than Emqx in this regard :) Planning to add auth modules support in the near future.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
Right, we are not stating anything without proving it in the first place.

You can find another test here: https://thingsboard.io/docs/mqtt-broker/reference/3m-through...

dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
We are not very familiar with NATS, does it support the whole functionality the MQTT broker should support? In addition, as it is not Java, but we are working with it, it could not be our solution.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
We do not agree, but everyone has an opinion we should respect :)
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
Exactly, you described it well. I have answered above for the reasons why HiveMQ was not the final choice.
dlandiak··on New scalable, fault-tolerant, and efficient open-source MQTT broker
You are correct. TBMQ is based on Spring Boot. We use Netty (https://netty.io/) as the source of the MQTT communication, and we build the MQTT features the MQTT broker should support ourselves on top of that. Kafka is used to implement scalable, fault-tolerant solution with reliable message persistence. Additionally, it helped to gain high-throughput processing and low-latency message delivery. We store published MQTT messages, client sessions, and subscriptions, etc., in Kafka. It proved its quality in practice and with performance tests. Redis is currently used for caching in cluster mode. In our roadmap, we also have a ticket to add the ability to store MQTT messages there. So, it will be a so-called Redis integration.

Please see the architecture doc for more details: https://thingsboard.io/docs/mqtt-broker/architecture/

We have implemented TBMQ to satisfy our enterprise customers' needs. Some of them used hivemq before but decided to ask us to build a new solution (TBMQ) as they were not fully satisfied with hivemq. One of the most important advantages we provided is cluster support in our open-source version compared to hivemq.