Where AI Apps Don’t Help
Where AI Apps Don’t Help
Doing something simple can still be an incredible difficult task for today’s AI Apps like ChatGPT
As I board a plane on a trip to Korea, I can’t help but think how little AI helped to prepare me for this trip. I researched destinations on travel sites and with Google Maps. Purchasing was straightforward on booking sites where I was able to use my points. Based on previous trips, I created a list on my iPad of everything I needed to pack. Thinking back on it I can’t help thinking… as someone with such a big focus on AI, why don’t I use these tools more?
To start, it’s important to think about why we use tools. We pick tools depending on the job we’re looking to complete. If you’re tired, you might look to make coffee or tea. Hungry? Maybe it’s time for a burger or some ramen. Common tools help us solve the issues we experience each day. AI tools like ChatGPT are just another tool to use.
The choice of tool is at times personal but is usually determined by what we know a tool can or can’t do. I am not usually interested in using a shoe as a hammer (unless I’m missing a hammer). So the fact that I’m not using AI tools for my trip means that I don’t find them very useful for the problem I have and I have better tools. Or, at least I perceive that I have better tools.
But what problem am I trying to solve? My biggest challenge is filling in the details of the trip. I know where I’m flying, I know what hotels I’ll be staying in, and I know a few places I want to go. I’m pretty decisive when I make the big decisions for this trip but the smaller details are harder for me. What train do I take to get between cities, what’s a good spot for dinner each day, or what would be a good gift to get my friends?
The small details are where I need help. I’d like this help to be based on my preferences too. General help on the small stuff gets you stuck in a tourist-infested restaurant with a waiting line around the block. I need an itinerary manager to fill in the gaps in my knowledge/research and give me recommendations.
I don’t find ChatGPT to be a very good itinerary manager. While the ability to converse and problem-solve is incredible, it still needs to get better to be able to fill the role. While there might be other options besides ChatGPT, the travel options on There’s An AI For That are a disappointing and too commercial. They also don’t fill out the role I have in mind either.
An itinerary manager needs to have good access to verified information, adjust their recommendations to my preferences, and keep track of key pieces of information.
Verified travel information online in countries outside North America is difficult to find. While ”every company has become an internet company” in the States, much of the rest of the world is still playing catch up. In Japan, software is considered a pretty boring career and much more emphasis is put on hardware development. This results in apps and websites that don’t offer the same level of services you might expect in Western countries. Google Maps still struggles to understand dense areas of many Asian cities.
Further complicating things, aggregation websites can be completely different across regions. While Google is popular with English speakers, KakaoTalk is the go-to in Korea. Unless you can read Korean, it’s likely that any of the information there is going to be inaccessible even with the amazing photo translation available in Google Translate. A good itinerary manager needs to be able to combine these different sources of information to come up with a good plan.
If you’re looking for better data sources, reviews and peer-to-peer forums like Reddit are often the most accurate. But this information is hard to get at without extra effort. Someone looking to learn more about any location will need to sort through relevant information that is quite old or recent information that isn’t relevant. Combing through the comments requires research, either by a person or by an AI.
ChatGPT’s Bing Search isn’t going to be able to really find most of this information. While ChatGPT Plugins can provide some travel integrations to make things a little easier, the ones that currently exist for travel are lightly-veiled service offerings for renting cars or booking flights.
You can paste in data that you find elsewhere on the internet but you may find that the results ChatGPT can glean from your research are too simplified to help in many cases. Hallucinations based on partial answers cause even more trouble.
The most popular AI tool, ChatGPT, remains untrustworthy for the task of finding accurate data.
But say we were okay with some research on our end, we’d still like to have ChatGPT give us some suggestions that are relevant to us. For that we need personalization. The ability for us to have our itinerary manager remember what kind of places we like, how we like to get places, what we should bring with us, and what we’re willing to spend money on.
ChatGPT is pretty good at getting feedback. If we ask for a suggestion of activities to do in a city and then ask for specifically indoor activities, it will take that feedback and iterate on it. But the number of add-on suggestions it can remember is limited. Keep asking it to tweak things and it may forget earlier tweaks you asked it to remember.
Researchers have discussed the limited memory capabilities of ChatGPT and other AI models. Even after extensions to the total size of memory and text it can work with, there are still many problems remembering details. It’s believed that training data is to blame and companies are already working to adjust training data that can work with larger text and keep more context in memory. For now, the current workaround for memory problems is to put the most important information at the beginning and end of any prompt.
ChatGPT manages how prompts incorporate the history of the conversation and may inject its content into these highly coveted positions. This limits the control power users have and can make the AI more forgetful than directly interfacing with GPT-4.
Because you’re working with a generalized chat application when you use ChatGPT rather than GPT-4 directly, the underlying prompts and algorithms do some work to remember things that might be important for future prompts. You don’t control what is remembered. While planning a trip, it may just completely forget important details that, if forgotten, could cause chaos to your trip plans.
Until memory problems are resolved in a generalizable way, ChatGPT remains an unreliable, inaccurate, and forgetful itinerary manager. While I’m trying to plan a trip, you can extrapolate this example into many other fields and roles. It highlights just how limiting AI applications are today.
The story of AI is a compelling one. We’re seeing incredible things happen all of the time. But it might take ten years or more before we get to grasp this technology to do more than generate art and writing. I’m excited and compelled to keep looking at how close we’re getting after a bit of relaxation on my trip. Catch you on the other side!