Ecosystem

11:20 AM - 11:50 AM, CEST , May 23

Build Low-code Stream Data Pipelines with Pulsar Transformations

Until now, when you wanted to transform the data from your Pulsar cluster, you could either write Pulsar Functions or use an advanced Stream Processing technology such as Apache Flink. The advantage of Pulsar Functions is that they are part of Pulsar and don’t require deploying additional technology. But compared to advanced low-code Stream Processing engines, they are not as friendly to use as you need to write code and understand Pulsar concepts such as Schemas, Records and so on. On the other hand Stream Processing engines can implement complex transformation of data with simple SQL but they imply deploying and running complex additional infrastructure which can be overkill for simple use-cases. In this talk we’ll see how the open-source Pulsar Transformations provides an alternative for low-code manipulation of data without the deployment of other technology than what already exists in Pulsar. We’ll also see how common use-cases such as modifying, filtering and routing data are implemented in minutes with it and how it can be chained with Pulsar IO connectors.

Speaker

Christophe Bornet

Senior Software Engineer, DataStax