Business Cloud News
The Facebook-led WebScaleSQL project counts Twitter, LinkedIn and Google as members

The Facebook-led WebScaleSQL project counts Twitter, LinkedIn and Google as members

Web-scale companies are among the few that can truly claim they have a “big data problem”. In a bid to improve on and indeed leverage some of the features included in MySQL for true big data applications, Google, LinkedIn, Twitter and Facebook announced this week that the companies are collaborating on WebScaleSQL, a project focused on running and fine tuning MySQL at web-scale.

“To help the more than 1.23 billion people who use Facebook to share and connect with each other, we’ve had to build an expansive and incredibly advanced infrastructure — including one of the largest deployments of MySQL in the world. Along the way, we’ve learned and benefited from code changes made by the MySQL community,” explained Steaphan Greene, software engineer at Facebook.

“Together, we’re working to share a common base of code changes to the upstream MySQL branch that we can all use and that will be made available via open source.”

“Our goal in launching WebScaleSQL is to enable the scale-oriented members of the MySQL community to work more closely together in order to prioritize the aspects that are most important to us.”

The collaboration includes MySQL engineering staff from some of the world’s largest internet companies. The project is being led by Facebook and its founding members include Google, LinkedIn, and Twitter, but Greene said that the group is open and hopes to encourage others with the scale and resources to customise MySQL.

He said the group, whose members have at times struggled to optimise their databases to operate at increasingly massive scale, will keep a close eye on upstream MySQL development, and act as a branch of the broader MySQL community’s efforts.

The group of engineers have so far created a range of prototype database stress testing systems, coding tweaks that improve buffer pool flushing and query optimisation, and have developed a few other feature that allow MySQL to scale better. It’s working on other improvements that it hopes to feed back into the upstream MySQL community.

“We’re excited to expand our existing work on WebScaleSQL, and we think that this collaboration represents an opportunity for the scale-oriented members of the MySQL community to work together in a more efficient and transparent way that will benefit us all,” Greene said.

Facebook is coordinating a number of joint industry projects focused on creating a rising tide for cloud service providers globally. It runs the Open Compute Project, an open source project focused on developing open source datacentre architectures that improve reliability and drop the cost of hardware, and, a nonprofit organisation composes of telcos and technology companies and tasked with improving global access to affordable connectivity.