The world of distributed computing took on a new profile this year when Folding@home, a 20-year-old distributed computing project, found itself picking up thousands of new volunteers to help Covid-19 researchers generate more computing power to fold proteins and run other calculations needed for screening potential drug compounds to fight the novel coronavirus. Today, a startup that is also tapping the potential and opportunity in distributed computing is announcing a round of growth funding to continue its own work.
Anyscale, a startup founded by a team out of UC Berkeley who created the Ray open-source Python framework for running distributed computing projects, has raised $40 million.
It plans to use the capital to continue developing Anyscale, a platform built on Ray that will make Ray usable not just by high-level developers and computing specialists, but any technical people who are looking to run projects that require large amounts of computing power.
Ion Stoica, Anyscale’s executive chairman who co-founded the company with Robert Nishihara, Philipp Moritz and Berkeley professor Michael I. Jordan, said in an interview that the company is tapping into a moment spurred not just by the events of 2020 but by the bigger demand from companies — spurred by the growth of cloud computing, major digital transformation of their systems, and a need to go that extra mile to remain competitive. Organizations are becoming more ambitious in their technology strategies and goals, whether they are tech companies or not.
“At a high level, the trend that we see is that all applications are distributed and running on clusters, but building these applications is incredibly hard and requires teams with the right expertise,” said Stoica. “What we are trying to build will make it as easy to build a distributed computing project as it would be to run a program on your laptop. It will mean ordinary developers will be able to build scalable applications just like Google can build today.”
The company’s first build of Anyscale — which will let organizations build multi-cloud applications from a single machine and use serverless architecture that scales up and down to meet application demands – has yet to launch commercially: it is in a private beta and the plan is to launch it fully next year.
There has been interest from financial services, retail, and manufacturing companies, Stoica said, with companies working in design, informatics and medical research (and Covid-19 vaccines) also using the private beta.
The Series B is being led by previous investor NEA, with Andreessen Horowitz (a16z), Intel Capital, and Foundation Capital also participating. A16z led the company’s Series A less than a year ago (a $20 million round in December).
Intel, meanwhile, is a strategic investor. Along with other tech giants like Microsoft, Intel is using Ray’s distributing computing model to run projects.
Stoica — who also co-founded Databricks, Conviva and was one of the original developers of Apache Spark — and Nishihara declined to comment in an interview on Anyscale’s valuation, but Stoica confirmed that the round was oversubscribed. The company has now raised just over $60 million.
While the startup continues to build out Anyscale, in the last year it has also been making more headway with Ray, which they also maintain.
At the Ray Summit — Anyscale’s conference for developers run as a virtual event at the end of September — Anyscale released Ray 1.0, which provides, in addition to a universal serverless compute API, an expanded library to use on Ray 1.0. Nishihara described it as a “huge milestone,” not least because it is one step along the path for the bigger vision they have for Anyscale to be used by non-tech companies for tech work.
A typical example was a recent recommendation algorithm built by Intel for Burger King. “The thing that is hard to do is not making the recommendations but learning from the interactions that users are having, and the choices they are making, and having that experience reflected back very rapidly,” he said. It’s a process that can be done in other ways, but with a far less good user experience due to lags.
This past year Nishihara said that interest in Ray has seen “tremendous growth,” but that it’s hard to say whether that is because of people working from home or just wider computing trends.
“It’s clear if anything that the pandemic is accelerating the transition,” said Stoica. “Ray has good support for the cloud, including Azure, Google Cloud Platform and others, which makes it quite compelling.”
We’ve seen an interesting trend in enterprise IT, where startups are finding an opportunity in the market by making it possible for non-technical organizations to bridge the digital divide, by providing better access to the most technical advances in computing to organizations beyond those that can build and operate such tools themselves. Just as groups like Element AI are working on ways to democratize advances in AI, the same kind of tech built, acquired and used by the likes of Apple, Google and Amazon, so too is Anyscale looking to do the same in enterprise computing.
And the two areas of AI and computing, of course, are interconnected: these days you need vast amounts of computing power to run AI applications, something the average company typically lacks.
“The demand for distributed computing continues to increase with the widespread adoption of AI and machine learning in application development,” said Pete Sonsini, General Partner at NEA, in a statement. “Still, scaling applications on clusters remains extremely challenging. Serverless computing is emerging as the preferred platform for developing distributed applications. Unfortunately, today’s serverless offerings support only a limited set of applications, and most of them are cloud-specific—but not Ray and Anyscale. The company’s path thus far bears the hallmarks of a standout technology pioneer, and we’re thrilled to partner with the team through this next phase bridging their open source and commercial offerings.”
