Diffblue, a spin-out from Oxford University, uses machine learning to help developers automatically create unit tests for their Java code. Since few developers enjoy writing unit tests to ensure that their code works as expected, increased automation doesn’t just help developers focus on writing the code that actually makes a difference but also lead to code with fewer bugs. Current Diffblue customers include the likes of Goldman Sachs and AWS.
So far, Diffblue only offered its service through a paid — and pricey — subscription. Today, however, the company also launched its free community edition, Diffblue Cover: Community Edition, which doesn’t feature all of the enterprise features in its paid versions, but still offers an IntelliJ plugin and the same AI-generated unit tests as the paid editions.
The company also plans to launch a new lower cost ‘individual’ plan for Diffblue Cover soon, starting at $120 per month. This plan will offer access to support and other advanced features as well.
At its core, Diffblue uses unsupervised learning to build these unit tests. “What we’re doing is unique in the sense that there have been tools before that use what’s called static analysis,” Diffblue CEO Mathew Loge, who joined the company about a year ago, explained. “They look at the program and they basically understand the path through the program and try and work backwards from the path. So if the path gets to this point, what inputs do we need to put into the program in order to get here?” That approach has its limitations, though, which Diffblue’s reinforcement learning method aims to get around.
Once the process has run its course, Diffblue provides developers with readable tests. That’s important, Loge stressed, because if a test fails and a developer can’t figure out what happened, it’s virtually impossible for the developer to fix the issue. That’s something the team learning the hard way, as early version so Diffblue used a very aggressive algorithm that provided great test coverage (the key metric for unit tests), but made it very hard for developers to figure out what was happening.
With the community edition, which doesn’t offer the command-line interface (CLI) of Diffblue’s paid editions, developers can write their code in IntelliJ as before and then simply click a button to have Diffblue write the tests for that code.
“The Community Edition is designed to be very accessible. It is literally one click in the IDE and you get your tests. The CLI version is more sophisticated and it covers more cases and solves for teams and large deployments inside of an organization,” Loge explained.
The company plans to add support for other languages, including Python, JavaScript and C# over time, but as Loge noted, Java has long been a mainstay in the business world and the team felt like that would be the best language to start with. As Loge noted, though, the technology
Diffblue has actually been around for a bit. The company raised a $22 million Series A round led by Goldman Sachs and with participation from Oxford Sciences Innovation and the Oxford Technology and Innovations Fund in 2017. You obviously don’t raise that kind of money to focus only on unit tests for Java code. Besides support for more language, unit tests are just the first step in the company’s overall goal of automating more of the programming process with the help of AI.
“We started with testing because it’s an important and urgent problem, especially with the impact that it has on DevOps and the adoption of more rapid software cycles,” Loge said. The next obvious step is to then take a similar approach to automatically fixing bugs — and especially security bugs — in code as well.
“The idea is that there are these steppingstones to machines writing more and more code,” he said. “And also, frankly, it’s a way of getting developers used to that. Because developer acceptance is a crucial part of making this successful.”
