Are GPTs a Marketing Gimmick?
Are GPTs a Marketing Gimmick?
OpenAI opened up ChatGPT to a community of builders to build their own "GPTs" and the results are mixed. What is the usefulness of GPTs and why aren't they a revolutionary AI change.
OpenAI released a new feature where you can create "your own GPT" experience within ChatGPT. Builders of the new GPTs can adjust ChatGPT to act differently and read from custom documentation, all without needing any coding knowledge. Additionally, there's potential to make money off of these tools, which adds significantly to the marketability of these features. However, I struggle to see a revolutionary change with GPTs.
I see GPTs as something similar to a bookmark or a shortcut for an assistant. The same functionality exists easily in ChatGPT, but this is a faster means of achieving the same things. I use bookmarks frequently and after sitting with GPTs for a little while, I can see their usefulness. I just think that the value is limited.
To explore their usage, I built a GPT that can help with writing emails. It templates setting up a chat around what kind of email I'm sending, how that email should be written, and a little context helping to craft the letter. This allows me to write what I want to say using shorthand and it gives me back a structure that matches the intended recipient. You can try it out here.
It's a pretty handy GPT and has also helped me teach what's possible to those less familiar with the ChatGPT experience. The prompts and setup are built-in, and you can get right to chatting, which helps cut down on the confusion for those new to working with AI interfaces. Unfortunately, these aren't shareable with others who don't already have a ChatGPT Plus subscription, so the user base is limited.
But being accessible to new users isn't the only measure of usefulness. To be highly useful, GPTs need to deliver a great experience to the user around a specific task. To do that, it needs to be hard to distract it from that task. Say we want critical feedback on our writing, and it responds with something true and helpful that we don't want to hear. If we argue with it, it should hold its ground or navigate the conversation in a way that can help to convince us. But GPTs can be derailed by user requests and arguments, which means they'll most likely cave to your opinion rather than help you.
This makes using AI programming interfaces, like the OpenAI API, much more powerful for crafting excellent experiences. By interpreting user input in a program, each request can be modified so that the AI responds in a direct and intended way. While you need programming skills, the user experience can be significantly better.
One of the most memorable experiences of a stubborn AI has been in my experiences chatting with Pi. After some conversation, I tried to practice Korean with it. The AI unfortunately believed I was joking around and making up words. I tried to correct it and told it how I was learning Korean with my girlfriend. It laughed at me and couldn't believe I had a girlfriend. (Ouch...) Nothing I could say would derail it from its belief that I was joking with it about any topic.
This experience was unlike anything I'd experienced with ChatGPT. While the responses weren't following my commands, they did convince me that I was speaking with something that had its own agenda outside my own, which was compelling. Comparing that with the unconfident responses of ChatGPT responding to your criticism shows just how much more there is to explore outside a GPT-driven experience.
One other major component of GPTs is the new documentation integration. GPT builders can add documents to be referenced in conversations that improve the responses and provide information that the AIs have not been trained on.
However, there isn't a lot of control over how the documents are read by the GPT. Users may ask questions from the documents and get back responses that correctly reference the document but don't actually give you the knowledge that the document holds. This is because you don't have control over how the documents are read compared to hand-tuned retrieval systems. We made a YouTube video about this where you can find more information about how documents are tricky to reference with AI systems.
DALL·E 3 integration into GPTs seems unique and interesting. The integration of chat and image generation means that your control over the images is lessened, but the assistant can do a lot to facilitate image generation. If we could control a bit more about how documents are referenced, there could be some interesting avenues where GPTs could define a style or direction for image generation. Again, users generally have more control when dealing with the programming interfaces directly.
In all, I think GPTs provide a unique shortcut for your usual ChatGPT experience. While the reality of using them is limited, they may provide a helpful introduction to those who are less familiar with AI. Engineers, programmers, and scientists will likely see the edge cases quickly but may still benefit from some provided shortcuts. The experience isn't revolutionary, but it has some usefulness given the right thinking around what is provided.
I agree, custom GPTs are not revolutionary. However, they are an obvious evolution of ChatGPT. The addition of Assistants and how they can be integrated into custom LLM applications are more useful in my opinion. I think it's just an outcome of LLMs evolving from chat bots into agentic applications, from basic LLM chains into LLM agents. It makes sense for OpenAI to democratize chat agents, for both hype and research.