What is happening at the center of GPT-4? That is what it is about: How much AI is advanced now and what is happening in the future?

Understanding the risks and potential of the new abilities of the machine is more important than knowing what they are. But while there’s broad agreement that ChatGPT and similar systems give computers significant new skills, researchers are only just beginning to study these behaviors and determine what’s going on behind the prompt.

An artificial Neural network is used to predict the words that will follow a given string of text in each version of OpenAI.

But the company’s CEO, Sam Altman, says further progress will not come from making models bigger. He told an audience at an event held at MIT late last week that giant models are going to be the end of the era. “We’ll make them better in other ways.”

Nick Frosst said that a feeling that going bigger will not work indefinitely rings true. He thinks that progress on transformer, the model of machine learning at the center of GPT-4 and its competitors, is beyond scaling. “There are lots of ways of making transformers way, way better and more useful, and lots of them don’t involve adding parameters to the model,” he says. Frosst says that new AI model designs, or architectures, and further tuning based on human feedback are promising directions that many researchers are already exploring.

That night, Bubeck got up, went to his computer, and asked GPT-4 to draw a unicorn using TikZ, a relatively obscure programming language for generating scientific diagrams. Bu Beck was using aversion of GPT-4 that only worked with text. But the code the model presented him with, when fed into a TikZ rendering software, produced a crude yet distinctly unicorny image cobbled together from ovals, rectangles, and a triangle. A grasp of the elements of the creature is required by Bubeck. “Something new is happening here,” he says. We might have something that is intelligence for the first time.

How intelligent AI is becoming—and how much to trust the increasingly common feeling that a piece of software is intelligent—has become a pressing, almost panic-inducing, question.