Can AI write recipes better than humans? We put it to the ultimate test and here's the result

Recipes can be passed down by word-of-mouth, scribbled on index cards, published in cookbooks. But they’ve always had one thing in common: They’re made by people.

Few things, in fact, are as packed with humanity as a recipe. Mixed and folded and baked into each one are their creators’ backgrounds, stories, tastes and feelings.

People have their limitations, though. They can’t read every mashed potato recipe on the internet before coming up with their own version. They can’t analyse thousands of techniques in search of the best way to make a pie crust.

Machines can. Computer systems driven by artificial intelligence can compose tweets and blog posts, create art, even generate computer code. And now they’re starting to write recipes.

These recipes have all the components of their handmade forebears: lists of ingredients, precise measurements, step-by-step instructions and introductory notes with (fabricated) personal touches. Their advantage, in theory, is that they draw on a vast trove of online information about food and cooking.

But are they any good? And can they improve on millenniums’ worth of lived culinary experience?

As home cooks, professional chefs and food-magazine editors know, the ultimate test for recipes is Thanksgiving dinner, a sprawling, varied spread that invites high expectations.

So we decided to enlist artificial intelligence — in this case, a technology called GPT-3 — to devise a holiday menu, which we then prepared and presented to a corps of taste testers: four New York Times cooking columnists.

The results say a lot about the technology’s potential, and the very purpose of a recipe.

Before we get to the verdict, let’s explain the science. Designed by OpenAI, one of the world’s most ambitious artificial intelligence labs, GPT-3 is a neural network, a mathematical system that can learn skills by analysing enormous amounts of data.

Some systems study images; in September, an AI-generated work took the top prize at a state-fair art contest. GPT-3 analyses digital text, including books, Wikipedia articles, tweets, chat logs, computer programmes and, yes, recipes. It can identify billions of distinct patterns in the way people connect words, numbers and symbols, and then use that knowledge to generate its own content — like a Thanksgiving menu of original recipes.

Artificial intelligence is poised to reshape several fields, from email marketing to computer programming. Recipe writing is not a common area of study, but a handful of researchers, including a team at the Massachusetts Institute of Technology, have begun to explore whether AI can master the skill.

In 2016, Janelle Shane, an optics research scientist who runs a machine-learning humour blog called AI Weirdness, began using systems like GPT-3 to create recipes, and then posted them. Early versions of the technology, she said, produced recipes that were a bit, well, weird. They called for nonsensical ingredients like “peeled rice” or “chopped flour.”

Today, she said, many AI recipes can seem indistinguishable from human-made ones.

“What it does really well is sound plausible,” Shane said. “So if you weren’t paying attention and someone was just reading this recipe out loud to you, you would be, ‘Oh yeah, that sounds like a perfectly ordinary recipe.’”

To create our AI Thanksgiving menu, we began by introducing ourselves to the GPT-3 system — in a surprisingly human way.

Mark Chen, an OpenAI research scientist, advised me, Priya Krishna, to get personal. Tell the system about yourself, he said: your family background, what flavours you like, which ingredients you tend to use often.

“The more details you provide in the prompt,” he said, “generally, the better the model performs.”

So after logging into GPT-3 on my laptop, I typed: “I am originally from Texas, and I grew up in an Indian American household. I love spicy flavours, Italian and Thai food, and desserts that are not too sweet. Some ingredients I frequently cook with are chaat masala, miso, soy sauce, herbs and tomato paste.”

Then I wrote, “Show me a Thanksgiving menu made for me.”

The first recipe GPT-3 produced was called “pumpkin spice chaat.” I was confused by the concept yet impressed by the inventiveness.

I asked follow-up questions to spur GPT-3’s creativity: Show me a few desserts tailored to my taste preferences. Show me a nontraditional Thanksgiving recipe. Show me a recipe for cranberry sauce that’s not too sweet and a little spiced.

Minutes later, I had a complete menu that seemed both plausible and intriguing: pumpkin spice chaat, green beans with miso and sesame seeds, naan stuffing, roasted turkey with a soy-ginger glaze, cranberry sauce that’s not too sweet and a little spiced (yes, that’s the full recipe name) and pumpkin spice cake with orange cream cheese frosting.

The dishes looked appetising enough. We used DALL-E, another OpenAI system that generates images, to create photos for each one. And we asked GPT-3 to provide introductions for each recipe, written from my point of view: “This roasted turkey recipe is inspired by the flavours of my childhood.” (It was not.)

Some of the ingredient lists seemed questionable. The naan stuffing called for 32 different components, including two cups of dried fruit. Most of the recipes were suspiciously light on salt and fat. Still, I was hopeful.

Cooking and tasting the recipes all but dashed that hope.

The cake was dense and more savoury than sweet. The naan stuffing tasted like a chana masala and a fruitcake that had gotten into a bar fight. The roast turkey recipe called for a single garlic clove to season a 12-pound bird, and no butter or oil; the result was dry and flavourless.

The chaat, laced with cilantro and baking spices, was a grassy-flavoured mush. The green beans and the cranberry sauce were edible but unremarkable.

Our taste-tasting columnists were even less kind.

“We’re not out of a job,” Melissa Clark said. “I don’t feel anything eating this food,” Yewande Komolafe added.

Genevieve Ko summed it up best: “There is no soul behind it.”

The recipes provided few clues on what cooks should look or smell for during the process, and no reasoning for why ingredients were added in a particular order.

Even before the tasting, Shane, the optics research scientist, suggested lowering our expectations. She called AI-generated dishes “the recipe equivalent of hotel room art.”

Yet the technology, even at this early stage, can have its uses. Chen, the OpenAI scientist, said it could help provide home cooks with some basic inspiration.

“You’ve got something in mind you want to make, you’re not quite sure how to make it, or you have a set of ingredients and I’m not quite sure how to combine it,” he said. “This could be a very quick way to give you a lot of different ideas.”

GPT-3 could also help to personalise an existing recipe by creating a version that’s less spicy, or that incorporates specific flavours, he said.

But cookbook author Nik Sharma, whose work intertwines food and science, worries about the inherent biases of these programmes. They draw on the English-language internet, where Western recipes predominate. And when these programmes do suggest non-Western dishes, he added, those may be the more generic ones familiar to Americans, like tandoori chicken.

“As food writers, our goal is to drive people into a new direction, to help challenge them, to help them understand the world,” he said. With artificial intelligence, “how do you do that?”

AI has already worked its way into the act of cooking. Home cooks use search engines to find recipes and ask virtual assistants like Amazon Alexa to convert teaspoons to tablespoons.

This technology is not a replacement for people, at least so far. It can nudge cooks in one direction or another. But it is still humanity — with its intuition, storytelling and warmth — that drives a good recipe.

The individual behind the dish is just as important as the recipe itself, Shane said. There is value in knowing that someone took the time to perfect a version of pie, stuffing or turkey, and share it.

“You want to know that these recipes mean something to somebody,” she said. That’s something artificial intelligence may never be able to provide.

Cook like a robot: A sample recipe

Here’s one of the six recipes that artificial intelligence generated — just as it was given to us, including a narrative introduction meant to sound like reporter Priya Krishna. The recipe hasn’t received the rigorous testing and editing we give the recipes in New York Times Cooking. So if you decide to make it, proceed with caution. (The real-life reporter called it “a grassy-flavoured mush.”)

Pumpkin Spice Chaat – Written by the AI system GPT-3

“I grew up in an Indian-American household, and I love spicy flavours. I often do not enjoy traditional Thanksgiving food because it is too bland. I like recipes that have different kinds of flavours and textures, that balance savoury with sweet and spicy and tangy. This recipe for pumpkin spice chaat is perfect for my tastes! It is made with pumpkin purée, chaat masala, ginger, cloves, nutmeg, black pepper, honey, lime juice, cilantro, mango, red onion, and chickpeas. It is a delicious and easy snack or side dish that is perfect for Thanksgiving.”

>> Ingredients

1 can pumpkin purée

1 tablespoon chaat masala

1 teaspoon ground ginger

1/4 teaspoon ground cloves

1/2 teaspoon ground cinnamon

1/2 teaspoon ground black pepper

1 tablespoon olive oil

1 tablespoon lime juice

2 tablespoons chopped cilantro

1/4 cup chopped roasted peanuts

1/2 cup yogurt

>> Instructions

1. In a bowl, mix together the pumpkin purée, chaat masala, ginger, cloves, cinnamon and black pepper.

2. Heat the oil in a pan over medium heat. Add the pumpkin mixture and cook for 5 minutes.

3. Add the lime juice and cilantro and cook for another minute.

4. Serve with the peanuts, yogurt and lime wedges.

– This article originally appeared in The New York Times.

• Business