Ecosystem

11:50 AM - 12:20 PM PDT , June 17

The first step to Multi AZ architecture for Apache BookKeeper

As software is increasingly deployed in public cloud environments, foundational platforms such as Apache BookKeeper must also continuously evolve to effectively work in multi availability zone environments and be designed to work around problems unique to such environments.
We at Salesforce added to BookKeeper the ability to function effectively in a Multi-AZ public cloud environment. The first step to this was adding awareness in bookies about their location in the cluster. We propose an approach to this problem which when using a kubernetes based deployment in public cloud environments, makes use of information natively provided by Kubernetes API, to determine their ‘location in the cluster’ also known as their ‘networkLocation’. The second step was to make this information available to the client. This was accomplished by adding the aforementioned ‘networkLocation’ as a field in the ‘cookies’ in a multi AZ environment. The field would be used to store the position of the bookie in the cluster. To make this network location available to the client, we added a mapping allowing client code to read this directly from the cookie.
This talk goes over how we could then use this added information to tackle various challenges such as creation of ensembles, placement and replication of ledgers, tolerance to bookie/zone failures, upgrade scenarios and backward compatibility all the while satisfying the durability guarantees promised by BookKeeper in a public cloud environment.

Speaker

Anup Ghatage

Senior Member of Technical Staff, Salesforce

Anup works on Salesforce's Infrastructure platform. Previously, he has worked on database internals, query processing and storage at SAP, Cisco Systems and other companies for more than 7 years. Anup holds a BS from the University of Pune and an MS from Carnegie Mellon University. Ask him to perform some close-up magic / read your mind for you.

Ankit Jain

Member of Technical Staff, Salesforce

Ankit Jain is a software engineer in the database team at Salesforce, where he works on the storage layer. He holds a Master's in Computer Engineering from Carnegie Mellon University.