Technology Deep Dive

11:00 AM - 11:35 AM, PT , Thur Jun 17

Exclusive Producer: Using Pulsar to Build Distributed Applications

There are several patterns that are very useful when designing and building distributed applications, like leader election or “atomic broadcast”. In practice, implementing these concepts is a very difficult undertaking and it is often offloaded to external services. These services in turn need to be maintained and operated and might even expose very complex APIs or semantics.
“Exclusive Producer” is a new addition to the set of Pulsar’s features, and it allows an application to have the guarantee of exclusive access when writing to a topic and to discard any data from producers who lost the exclusive access.
During this session we are going to show how easy it is, using Exclusive Producer, to build very robust mechanisms of communications across different services or instances of a single service.
These mechanisms, like for example leader election, failover and “total order and atomic broadcast”, are the main building blocks to build consistent, correct and reliable distributed systems.
We will also present how the exclusive producer is implemented internally, and how it can guarantee its properties in the presence of all different sorts of failures.

Speaker

Matteo Merli

Pulsar PMC Chair and CTO, StreamNative

Matteo Merli is the PMC Chair of Apache Pulsar and CTO at StreamNative. He is co-creator of Pulsar while at Yahoo!, co-founder of Streamlio, Committer and PMC of Apache Pulsar and Apache BookKeeper. He works for The Apache Software Foundation for 4+ years, and has rich experience in distributed pub-sub messaging platform.