OpenAI: AI Language Generative Model for Web-Based Conversational Conversations and the Google/Singapore Digital Strategy Challenge (An interview with Sam Altman)
The artificial intelligence at the core of the game is old, not new. It is a version of an AI model called GPT-3 that generates text based on patterns it digested from huge quantities of text gathered from the web. The model available as a commercial APIs for programmers can answer questions and generate text, even if you don’t ask them. crafting the right prompt required a lot of work, it had to respond in a particular way.
GPT, an artificial intelligence model which takes a string of text and predicts what will come next, is built on top of chatGpt. OpenAI has gained prominence for publicly demonstrating how feeding huge amounts of data into transformer models and ramping up the computer power running them can produce systems adept at generating language or imagery. ChatGPT improves on GPT by having humans provide feedback to different answers to another AI model that fine-tunes the output.
OpenAI has not released full details on how it gave its text generation software a naturalistic new interface, but the company shared some information in a blog post. It says the team fed human-written answers to GPT-3.5 as training data, and then used a form of simulated reward and punishment known as reinforcement learning to push the model to provide better answers to example questions.
Jacob Andreas, an assistant professor who works on AI and language at MIT, says the system seems likely to widen the pool of people able to tap into AI language tools. He said that the mental model that he used to apply to other agents was being presented to him in a familiar interface.
The heady excitement inspired by ChatGPT has led to speculation that Google faces a serious challenge to the dominance of its web search for the first time in years. The company held a media event tomorrow about new features for Bing that are related to their work with the creator of the popular chat service. OpenAI’s CEO Sam Altman tweeted a photo of himself with Microsoft CEO Satya Nadella shortly after Google’s announcement.
For example, the query “Is it easier to learn the piano or the guitar?” would be met with “Some say the piano is easier to learn, as the finger and hand movements are more natural … Others say that it’s easier to learn chords on the guitar.” Pichai also said that Google plans to make the underlying technology available to developers through an API, as OpenAI is doing with ChatGPT, but did not offer a timeline.
Google chooses to proceed cautiously with the addition of the technology behind LaMDA to its products. Besides hallucinating incorrect information, AI models trained on text scraped from the Web are prone to exhibiting racial and gender biases and repeating hateful language.
Those limitations were highlighted by Google researchers in a 2020 draft research paper arguing for caution with text generation technology that irked some executives and led to the company firing two prominent ethical AI researchers, Timnit Gebru and Margaret Mitchell.
Other Google researchers who worked on the technology behind LaMDA became frustrated by Google’s hesitancy, and left the company to build startups harnessing the same technology. The company seems to have been inspired to speed its timelines for adding text generation capabilities to its products.