Keynote

9:50 AM - 10:20 AM, CEST , May 14th

Flipkart’s Async Bus Migration from Kafka to Pulsar: Realtime switch from Kafka To Pulsar

Varadhi is Flipkart's managed RESTBus, featuring a message bus implementation with a REST interface and a message broker underneath. Initially powered by a Kafka cluster with approximately 350 brokers, 3,000 topics, and 9,000 subscriptions, Varadhi processes more than billion messages per day and facilitates hundreds of teams at Flipkart for their asynchronous communication between various micro services. In this session, we will introduce the rationale behind Varadhi's existence, glancing at the use cases it serves and the added value it brings atop traditional message brokers. We will focus on decisions to transition from Kafka to Pulsar, detailing the process of live migration without incurring downtime. We'll also address the challenges encountered during this migration journey and detail the solutions devised to overcome them.

Speaker

Anil Gowda

Software Development Engineer, Flipkart

I work as Software Engineer in Flipkart's Platform as a Service (PaaS) team. Within this innovative team, I am part of group which is responsible for crafting and maintaining the messaging platform for Flipkart, one of India's leading e-commerce giants. Focused on working building micro-services, distributed systems with experience on Kafka, Pulsar, Cassandra, Redshift.