Imagine buying a dress online because a piece of code sold you on its ‘flattering, feminine flair’ — or convinced you ‘romantic floral details’ would outline your figure with ‘timeless style’. The very same day your friend buy the same dress from the same website but she’s sold on a description of ‘vibrant tones’, ‘fresh cotton feel’ and ‘statement sleeves’.
This is not a detail from a sci-fi short story but the reality and big picture vision of Hypotenuse AI, a YC-backed startup that’s using computer vision and machine learning to automate product descriptions for ecommerce.
One of the two product descriptions shown below is written by a human copywriter. The other flowed from the virtual pen of the startup’s AI, per an example on its website.
Can you guess which is which?* And if you think you can — well, does it matter?
Screengrab: Hypotenuse AI’s website
Discussing his startup on the phone from Singapore, Hypotenuse AI’s founder Joshua Wong tells us he came up with the idea to use AI to automate copywriting after helping a friend set up a website selling vegan soap.
“It took forever to write effective copy. We were extremely frustrated with the process when all we wanted to do was to sell products,” he explains. “But we knew how much description and copy affect conversions and SEO so we couldn’t abandon it.”
Wong had been working for Amazon, as an applied machine learning scientist for its Alexa AI assistant. So he had the technical smarts to tackle the problem himself. “I decided to use my background in machine learning to kind of automate this process. And I wanted to make sure I could help other ecommerce stores do the same as well,” he says, going on to leave his job at Amazon in June to go full time on Hypotenuse.
The core tech here — computer vision and natural language generation — is extremely cutting edge, per Wong.
“What the technology looks like in the backend is that a lot of it is proprietary,” he says. “We use computer vision to understand product images really well. And we use this together with any metadata that the product already has to generate a very ‘human fluent’ type of description. We can do this really quickly — we can generate thousands of them within seconds.”
“A lot of the work went into making sure we had machine learning models or neural network models that could speak very fluently in a very human-like manner. For that we have models that have kind of learnt how to understand and to write English really, really well. They’ve been trained on the Internet and all over the web so they understand language very well. “Then we combine that together with our vision models so that we can generate very fluent description,” he adds.
Image credit: Hypotenuse
Wong says the startup is building its own proprietary data-set to further help with training language models — with the aim of being able to generate something that’s “very specific to the image” but also “specific to the company’s brand and writing style” so the output can be hyper tailored to the customer’s needs.
“We also have defaults of style — if they want text to be more narrative, or poetic, or luxurious — but the more interesting one is when companies want it to be tailored to their own type of branding of writing and style,” he adds. “They usually provide us with some examples of descriptions that they already have… and we used that and get our models to learn that type of language so it can write in that manner.”
What Hypotenuse’s AI is able to do — generate thousands of specifically detailed, appropriately styled product descriptions within “seconds” — has only been possible in very recent years, per Wong. Though he won’t be drawn into laying out more architectural details, beyond saying the tech is “completely neural network-based, natural language generation model”.
“The product descriptions that we are doing now — the techniques, the data and the way that we’re doing it — these techniques were not around just like over a year ago,” he claims. “A lot of the companies that tried to do this over a year ago always used pre-written templates. Because, back then, when we tried to use neural network models or purely machine learning models they can go off course very quickly or they’re not very good at producing language which is almost indistinguishable from human.
“Whereas now… we see that people cannot even tell which was written by AI and which by human. And that wouldn’t have been the case a year ago.”
(See the above example again. Is A or B the robotic pen? The Answer is at the foot of this post)
Asked about competitors, Wong again draws a distinction between Hypotenuse’s ‘pure’ machine learning approach and others who relied on using templates “to tackle this problem of copywriting or product descriptions”.
“They’ve always used some form of templates or just joining together synonyms. And the problem is it’s still very tedious to write templates. It makes the descriptions sound very unnatural or repetitive. And instead of helping conversions that actually hurts conversions and SEO,” he argues. “Whereas for us we use a completely machine learning based model which has learnt how to understand language and produce text very fluently, to a human level.”
There are now some pretty high profile applications of AI that enable you to generate similar text to your input data — but Wong contends they’re just not specific enough for a copywriting business purpose to represent a competitive threat to what he’s building with Hypotenuse.
“A lot of these are still very generalized,” he argues. “They’re really great at doing a lot of things okay but for copywriting it’s actually quite a nuanced space in that people want very specific things — it has to be specific to the brand, it has to be specific to the style of writing. Otherwise it doesn’t make sense. It hurts conversions. It hurts SEO. So… we don’t worry much about competitors. We spent a lot of time and research into getting these nuances and details right so we’re able to produce things that are exactly what customers want.”
So what types of products doesn’t Hypotenuse’s AI work well for? Wong says it’s a bit less relevant for certain product categories — such as electronics. This is because the marketing focus there is on specs, rather than trying to evoke a mood or feeling to seal a sale. Beyond that he argues the tool has broad relevance for ecommerce. “What we’re targeting it more at is things like furniture, things like fashion, apparel, things where you want to create a feeling in a user so they are convinced of why this product can help them,” he adds.
The startup’s SaaS offering as it is now — targeted at automating product description for ecommerce sites and for copywriting shops — is actually a reconfiguration itself.
The initial idea was to build a “digital personal shopper” to personalize the ecommerce experence. But the team realized they were getting ahead of themselves. “We only started focusing on this two weeks ago — but we’ve already started working with a number of ecommerce companies as well as piloting with a few copywriting companies,” says Wong, discussing this initial pivot.
Building a digital personal shopper is still on the roadmap but he says they realized that a subset of creating all the necessary AI/CV components for the more complex ‘digital shopper’ proposition was solving the copywriting issue. Hence dialling back to focus in on that.
“We realized that this alone was really such a huge pain-point that we really just wanted to focus on it and make sure we solve it really well for our customers,” he adds.
For early adopter customers the process right now involves a little light onboarding — typically a call to chat through their workflow is like and writing style so Hypotenuse can prep its models. Wong says the training process then takes “a few days”. After which they plug in to it as software as a service.
Customers upload product images to Hypotenuse’s platform or send metadata of existing products — getting corresponding descriptions back for download. The plan is to offer a more polished pipeline process for this in the future — such as by integrating with ecommerce platforms like Shopify .
Given the chaotic sprawl of Amazon’s marketplace, where product descriptions can vary wildly from extensively detailed screeds to the hyper sparse and/or cryptic, there could be a sizeable opportunity to sell automated product descriptions back to Wong’s former employer. And maybe even bag some strategic investment before then… However Wong won’t be drawn on whether or not Hypotenuse is fundraising right now.
On the possibility of bagging Amazon as a future customer he’ll only say “potentially in the long run that’s possible”.
Joshua Wong (Photo credit: Hypotenuse AI)
The more immediate priorities for the startup are expanding the range of copywriting its AI can offer — to include additional formats such as advertising copy and even some ‘listicle’ style blog posts which can stand in as content marketing (unsophisticated stuff, along the lines of ’10 things you can do at the beach’, per Wong, or ’10 great dresses for summer’ etc).
“Even as we want to go into blog posts we’re still completely focused on the ecommerce space,” he adds. “We won’t go out to news articles or anything like that. We think that that is still something that cannot be fully automated yet.”
Looking further ahead he dangles the possibility of the AI enabling infinitely customizable marketing copy — meaning a website could parse a visitor’s data footprint and generate dynamic product descriptions intended to appeal to that particular individual.
Crunch enough user data and maybe it could spot that a site visitor has a preference for vivid colors and like to wear large hats — ergo, it could dial up relevant elements in product descriptions to better mesh with that person’s tastes.
“We want to make the whole process of starting an ecommerce website super simple. So it’s not just copywriting as well — but all the difference aspects of it,” Wong goes on. “The key thing is we want to go towards personalization. Right now ecommerce customers are all seeing the same standard written content. One of the challenges there it’s hard because humans are writing it right now and you can only produce one type of copy — and if you want to test it for other kinds of users you need to write another one.
“Whereas for us if we can do this process really well, and we are automating it, we can produce thousands of different kinds of description and copy for a website and every customer could see something different.”
It’s a disruptive vision for ecommerce (call it ‘A/B testing’ on steroids) that is likely to either delight or terrify — depending on your view of current levels of platform personalization around content. That process can wrap users in particular bubbles of perspective — and some argue such filtering has impacted culture and politics by having a corrosive impact on the communal experiences and consensus which underpins the social contract. But the stakes with ecommerce copy aren’t likely to be so high.
Still, once marketing text/copy no longer has a unit-specific production cost attached to it — and assuming ecommerce sites have access to enough user data in order to program tailored product descriptions — there’s no real limit to the ways in which robotically generated words could be reconfigured in the pursuit of a quick sale.
“Even within a brand there is actually a factor we can tweak which is how creative our model is,” says Wong, when asked if there’s any risk of the robot’s copy ending up feeling formulaic. “Some of our brands have like 50 polo shirts and all of them are almost exactly the same, other than maybe slight differences in the color. We are able to produce very unique and very different types of descriptions for each of them when we cue up the creativity of our model.”
“In a way it’s sometimes even better than a human because humans tends to fall into very, very similar ways of writing. Whereas this — because it’s learnt so much language over the web — it has a much wider range of tones and types of language that it can run through,” he adds.
What about copywriting and ad creative jobs? Isn’t Hypotenuse taking an axe to the very copywriting agencies his startup is hoping to woo as customers? Not so, argues Wong. “At the end of the day there are still editors. The AI helps them get to 95% of the way there. It helps them spark creativity when you produce the description but that last step of making sure it is something that exactly the customer wants — that’s usually still a final editor check,” he says, advocating for the human in the AI loop. “It only helps to make things much faster for them. But we still make sure there’s that last step of a human checking before they send it off.”
“Seeing the way NLP [natural language processing] research has changed over the past few years it feels like we’re really at an inception point,” Wong adds. “One year ago a lot of the things that we are doing now was not even possible. And some of the things that we see are becoming possible today — we didn’t expect it for one or two years’ time. So I think it could be, within the next few years, where we have models that are not just able to write language very well but you can almost speak to it and give it some information and it can generate these things on the go.”
*Per Wong, Hypotenuse’s robot is responsible for generating description ‘A’. Full marks if you could spot the AI’s tonal pitfalls