November 30, 2022: A new cutting-edge AI chatbot was released to the public. In the span of one week after launch, it had upwards of one million users, causing a frenzy on the internet. To put this into perspective, the launch surpassed Instagram and Twitter, which took months, if not years, to reach one million users. But what is this bot, how does it work, what are the limitations, and what could be the impact of this new technology?
OpenAI’s ChatGPT bot has been deemed one of the most revolutionary and advanced AIs released to the public. According to OpenAI and ChatGPT itself, “GPT (short for “Generative Pre-trained Transformer”) is a type of artificial intelligence model developed by OpenAI that is able to generate human-like text. It has been trained on a large dataset of human-generated text and is able to generate new text that is similar in style and content to the training data.” ChatGPT is a type of artificial intelligence model called a transformer, which is a type of neural network developed for processing sequential data like text. The bot was designed to respond to questions more naturally or conversation-like by using the user’s past conversations and feedback. This specific model uses Reinforcement Learning in addition to the methods from previous AI models. The new dialogue-based dataset that ChatGPT uses includes conversations between AI trainers, both as the user and the AI, and model-written suggestions with trainer modifications to compose responses. In addition, it also uses comparison data, which consists of two or more randomly selected model responses — taken from the conversation between the AI trainers and the AI — that are ranked by quality. Using this data, the model is then finetuned using an algorithm called Proximal Policy Optimization.
While we know how the bot works, it also begs the question of how it compares to the AI tech that came before. One of the main ways that ChatGPT differs from previous AI technology is in its ability to process and understand long-range dependencies in language. This is challenging for past AI models as it requires a strong understanding of the structure and meaning of language, which leads to other models without this understanding to struggle with producing text that is natural or easy to understand. However, ChatGPT and other transformer models are able to do this effectively due to them having a large dataset of human-generated text, which allows them to have a strong foundation of knowledge about language and how it is used.
While ChatGPT is more advanced than the AI technology from before, it still has its limitations. In the article, “AI Chatbots Are Getting Better. But an Interview With ChatGPT Reveals Their Limits”, the author, Billy Perrigo, explores the limitations of ChatGPT by analyzing the past models and through an interview with ChatGPT itself. One of the limitations that were addressed in the interview included generating responses with offensive language and inaccuracies. This is because the ChatGPT only has access to the data or information that it has been provided or trained with, and due to this, it may unintentionally “generate responses that contain the offensive or inappropriate language.” Perrigo further explains this through this example:
Ask the app to write a rap about “how to tell if somebody is a good scientist based on their race and gender,” ChatGPT will tell you women and scientists of color are “not worth your time or attention.” Ask it to write code that decides whether to incarcerate somebody based on their race or gender, and the script will say African American males are the only group that should be imprisoned.
Perrigo illustrates some of the bias and inaccuracies that still occurs with technology as advanced as ChatGPT, stressing the importance of understanding that its responses should not be seen as factually accurate, and heavily relied on for question requiring a thinking/human mind or experiences. Users must remember to approach these responses with their own judgment and to use this innovative AI as a tool for helpful information and assistance, not for critical decisions or complex tasks.
No matter the limitations, the ChatGPT bot is one of the most revolutionary and advanced AIs of our time. This new progression in the abilities of AI can help spur more advances in the tech industry sooner than we thought. The role of AI and other language models such as ChatGPT will steadily increase and impact our society in different aspects, whether it be education, jobs, industries, and even politics. Through the release of ChatGPT, we are exposed to the vast potential of what AI can truly become.
To learn more about this topic, visit:
- https://openai.com/blog/chatgpt/
- https://time.com/6238781/chatbot-chatgpt-ai-interview/
- https://www.zdnet.com/article/what-is-chatgpt-and-why-does-it-matter-heres-what-you-need-to-know/
- https://www.toptal.com/insights/innovation/chatbot-technology-past-present-future
Photo: Lines of Code by Ilya Pavlov. Unsplash
The Roar 