Use Case

2:10pm-2:50pm UTC+8 , November 29

How BIGO builds Real-Time Message System with Apache Pulsar and Flink

Powered by Artificial Intelligence technology, BIGO's video-based products and services have gained immense popularity, with users in more than 150 countries. The products include Bigo Live (live streaming) and Likee (short-form video). Bigo Live is available in more than 150 countries and Likee has more than 100 million users and is popular among the Generation Z.
In the past few years, we have deployed many Kafka clusters to support real-time ETL and short-form video recommendations. The Apache Pulsar's layered architecture and new features, such as Low latency with durability, Horizontally scalable, Multi-tenancy etc, help us solve a lot of problems in production. We have adopted Apache Pulsar to build our Message Processing System, especially in Real-Time ETL, short-form video recommendation and AB-test Real-Time Data report.
In this talk, I will share our journey of adopting Apache Pulsar in our Real-Time Message Processing System, especially in Flink & Flink SQL along with Pulsar.

Speaker

Hang Chen

Hang Chen is the Leader of the Messaging Platform team from BIGO, and responsible for creating a centralized pub-sub messaging Platform which provides a vast number of service/application traffics. He introduced Apache Pulsar into their Messaging Platform and integrate with upstream and downstream systems, such as Flink, ClickHouse and other inner systems for Real-Time recommendation and analysis. He focus on pulsar performance tuning, new features development and pulsar ecosystem integration.