Issue 88

The latest AI hype: GPT-3

Delivered on 28 July 2020 by Justin Pyvis. About a 4 min read.

I say this a lot but the term artificial intelligence (AI) is designed to mislead. What people call AI is really just human-calibrated algorithms scanning enormous datasets for the most appropriate response. It's great for certain things - such as loading containers on a ship - but not so good at, say, driving on roads with pesky, unpredictable humans.

For some, the ambiguity can be profitable. Take ScaleFactor, which promised "artificial intelligence-powered tools that could replace the accountant", and raised over $100m prior to COVID-19. Turns out it was snake oil:

Instead of software producing financial statements, dozens of accountants did most of it manually from ScaleFactor’s Austin headquarters or from an outsourcing office in the Philippines, according to former employees. Some customers say they received books filled with errors, and were forced to re-hire accountants, or clean up the mess themselves.

Then there's OpenAI, a company which last week launched "GPT-3" (Generative Pretrained Transformer 3), its latest AI model that had even the normally sensible economist Tyler Cowen salivating:

This year is likely to be remembered for the Covid-19 pandemic and for a significant presidential election, but there is a new contender for the most spectacularly newsworthy happening of 2020: the unveiling of GPT-3.
The core of GPT-3, which is a creation of OpenAI, an artificial intelligence company based in San Francisco, is a general language model designed to perform autofill. It is trained on uncategorized internet writings, and basically guesses what text ought to come next from any starting point. That may sound unglamorous, but a language model built for guessing with 175 billion parameters — 10 times more than previous competitors — is surprisingly powerful.

Let's put aside the fact that there is nothing open about OpenAI - GPT-3 is closed source and provided via a paid API - and instead focus on GPT-3 itself.

In terms of AI, GPT-3 is an enormous upgrade if for no other reason than the sheer number of parameters in its database (it has 175 billion versus say Google's T5, which had ~11 billion). But the fundamental technology is unchanged from anything in the last few years and as Tyler notes, GPT-3 does not try to pass the Turing test by being indistinguishable from a human in its responses. It's a big improvement on existing AI models, but it's not a step forward in terms of the technology nor in addressing the more fundamental flaws with AI that restrict its potential applications.

One of those applications is autonomous vehicles, which are being developed using similar AI technology to GPT-3, with the same limitations. But it seems the sector has finally acknowledged the reality of the situation, moving from the robotaxi dream to finding practical uses for it:

The sector is experiencing “autonomous disillusionment”, says Prescott Watson, principal at Maniv Mobility, an early-stage venture capital firm. Now, “the pitch is, ‘robotaxis are a pipe dream’, but let’s take this technology to do something more lucrative,” he adds.

Investors are still interested in autonomy but the focus has shifted towards practical services such as grocery delivery, automated warehouse robots, and autonomous functions restricted to highways.

Highway automation is plausible with the current AI technology. You don't have to dodge the dog in the driveway (or worse, the toddler) like a robotaxi might, and authorities could even designate a specific lane for automated vehicles, as they currently do for buses. According to Chuck Price, chief product officer at TuSimple, the biggest autonomy start-up devoted to trucking:

“Trucks go on a very predictable route, every day, all the time,” he says. “So if I want to build a profitable trucking business with autonomy, I map [the highway] from Los Angeles to Jacksonville, Florida, and I’ve got billions of dollars of revenue available on that route.”

I think it's wise to avoid the AI hype that surrounds products such as OpenAI's GPT-3 or Tesla robotaxis given the technology hasn't really progressed (note Elon Musk was a founder of OpenAI, which might explain its excellent marketing), but still be optimistic about the productivity improvements on offer when automation is properly-applied, such as to freight.

RIP digital tracing

It has been three months since Apple and Google released their contact tracing framework yet governments still can't get it right. Germany's app (built using the framework) somehow still cost ~€20 million. Ambiguous privacy policies have kept some users away. South Korea's database was exposed.

The only positive development lately is that Ireland donated the code for its COVID Tracker app (based on the Apple/Google framework) to the not-for-profit Linux Foundation. Hopefully some kind of completely decentralised, global tracing app will now be possible.

Slack tries to antitrust Microsoft

No surprise that it's happening in Europe. When you can't compete ... sue?

Slack surprised Microsoft with a competition complaint in Europe yesterday. After arguing for months that Microsoft Teams isn’t a true competitor to Slack and is more akin to Zoom, Slack finally admitted what was clear all along: Microsoft Teams is a competitor, and Slack is finding it hard to compete with Microsoft. It’s not a surprising admission, but if Slack is finding it hard to compete with Microsoft, then it’s going to face even greater headaches once Google finally gets its act together. After fumbling with communications apps for years, there are early signs that Google is now ready to take on Slack, Microsoft Teams, and Zoom.

I do worry about the return of antitrust around the world. Formerly dynamic, boundary-pushing businesses are now hiring lawyers, economists and content moderators instead of developers. The end result will be the utilitisation of the sector.

Genealogy breach

Please don't give your DNA away to a company or government. They will, eventually, get breached:

First GEDmatch, the DNA database that helped identify the Golden State Killer, was hacked. Then email addresses from its users were used in a phishing attack on another leading genealogy site, MyHeritage.

Issue 88: The latest AI hype: GPT-3 was compiled by Justin Pyvis and delivered on 28 July 2020.