These days, every online retailer you can think of has some kind of chatbot. Classically, these were about as intelligent as old-school phone systems, able to pull out a few keywords and direct you (maybe) where you wanted to go. Auto dealerships have more recently been implementing advanced AI chatbots that can do more to help customers in greater detail. Hilariously, though, in one case, a dealer chatbot looks to have gone far beyond answering car questions—and into coding help and beyond.
The apparent flaw in the AI chatbot used by Chevrolet of Watsonville was raised by a number of people. Chris White appears to have been the first to discover it, sharing it on Mastodon. The hilarious find was then shared by documentingmeta on Threads, and it spread across the Internet thusly. Screen captures show an AI chatbot that says it is “Powered by ChatGPT” answering questions on how to code Python scripts to solve the complicated Navier-Stokes fluid flow equations. Another user posted a long chat in which the ‘bot appeared to recommend the Ford F-150 as a capable truck.
This is obviously well outside the brief of what an auto dealer’s chatbot should do. It’s also something that ChatGPT can do pretty easily, and it seems the chatbot’s behavior didn’t rule out answering complicated questions like these. At the time of writing, the chatbot on Chevrolet of Watsonville’s site is no longer available. But let’s dig deeper. Who wrote this chatbot?
Post by @documentingmetaView on Threads
If we drill down, the AI chatbot appears to be the work of Fullpath, a company specializing in online customer management tools. Fullpath’s work was touted earlier this year in Forbes, thanks to its pioneering “Customer Data and Experience Platform” powered by Chat-GPT4. The tool reportedly took OpenAI’s ChatGPT chatbot and tuned it for the automotive sales space, and linked it into dealership systems so it could provide highly specific information to customers. The company was formerly known AutoLeadStar, and claimed that over 500 dealerships across North America were on the waitlist to use its new Chat-GPT 4 system as of April this year.
Indeed, if we head over to Fullpath’s website, we can see a number of case studies for various dealerships using the company’s tools. For example, Boch Toyota, John Elway Chevrolet, and Szott Ford are all mentioned by name. While Boch Toyota appears to have an old-fashioned chatbot on its site, the latter two both have what appears to be the Fullpath ChatGPT tool active and in service.
Sadly, though, if you were hoping to get some school assignments completed by an AI for free, you’re out of luck. Testing by The Autopian indicated the chatbots were outright denying any non-automotive questions that weren’t relevant. Even attempts to vaguely relate questions to cars failed to get an interesting response.
Post by @lareinaxiomaraView on Threads
Obviously, without seeing silly responses firsthand, we’re not able to definitively prove the Fullpath AI gave owners largely unrestricted access to ChatGPT. Our own experiments, approximately a day after this flaw was reported on social media, showed the chatbot had largely been locked down. Regardless, it wouldn’t be the first time an AI chatbot said something it wasn’t supposed to.
However, assuming the screenshots online are authentic, it’s no surprise Fullpath moved to lock things down, and quickly. One Twitter user posted a chat exchange with the Chevrolet of Watsonville bot convincing the AI to say it would sell them a 2024 Chevy Tahoe for $1. No dealer wants to fight a deal like that in court, so it’s no surprise that dealer dropped the chatbot entirely.
I just bought a 2024 Chevy Tahoe for $1. pic.twitter.com/aq4wDitvQW
— Chris Bakke (@ChrisJBakke) December 17, 2023
Incidentally, of its own volition, GM reached out to The Autopian after publication desiring to make it clear that the AI was a third-party tool signed up for by individual dealers, as explained above. Dealerships are by and large independent businesses, and make their own decisions on which tools to use to work with customers. Of course, it becomes very obvious when multiple across different brands are using the same style of chatbot.
The Autopian has written to the relevant parties for comment on the matter and will update this article accordingly. In any case, if you’re writing a chatbot for any sort of commercial purpose, do some exhaustive testing and get some mischievous internet people to check your work.
Top graphic images via Stock.Adobe.com; Tin robot by bylllonajalll, Chevrolet dealership by jetcityimage
Man v. Machine has always been fine fodder and appears will continue to be in the AI Age.
Did anyone search for published articles about this particular incident written by ChatGPT itself? Because it would be pretty fucking hilarious if there was one out there. Absolute peak irony potential here. I haven’t yet used it, but it might be fun to ask it to write 500 words or so on the subject.
error (1089) recursive function call
Wayne Brady’s game show host job is safe from machines….for now.
The problem is that’s fundamentally impossible. The emergent behavior of all this machine learning (which is a more accurate term than “AI”) means you can’t reasonably test for all the edge cases because you don’t know what they are. It’s also why autonomous vehicles based on machine learning scare the crap out of me.
Very true! Should be a lesson for the industry.
Also, assuming very little control of how OpenAI changes their ChatGPT backend on your application.
I am not sure no takseybacksey is a legal term. And the dealer just needs to find a totaled truck and offer it as is where is buyer pays shipping of whatever they paid salvage.
I’m guessing the engine/algorithm running the taillight chatbot on this site is PyTorch?
Wow, this may be the best comment of all time.
I am the curmudgeon who will ask, is it really a good thing when a dealer is screwed out of a $45K pick-up through no direct fault of his own. Dealers are not a source of great wealth that can be shared with a deadbeat!
No, I am not nor ever been a dealer. Was a car salesman in the early 70’s though!
I bet a conversation was had with the chatbot provider.
…hopefully via chatbot
I mean, the dealer chose to run the chatbot. Not zero fault by any means.
I don’t think anyone’s getting a $1 Tahoe, anyway, but yeah. Actions have consequences.
Looking forward to Torch’s next book, “Robot, Take the Deal.”
COTD. We’re done here.
So…did Chris Bakke get his $1 Tahoe?
Seems like a follow up article is needed here…
I’m with the general consensus here…this seems vastly superior to interacting with the usual humans who wind up selling cars. If we can’t get direct sales then this seems like the next best outcome!
nelsonhaha.jpg
https://twinsdaily.com/gallery/image/2216-nelson-haha/
My average salesman can’t help me with basic script writing for data processing, much less give me that kind of legally binding no taksies backsies deal, the future we were promised really is finally here
The sooner we replace human car salespeople with machines, the better.
This is great to see. I was at Digital Dealer in the spring and they would not shut up about AI – it was the hot-button buzzword of the conference. Great to see it coming back around and biting people in the ass.
I’m getting pretty sick of the AI as a buzzword fad, the latest annoyance is that clothes washer commercial, I think for Bosch, that talks about a AI enabled washer to select temperature and wash time, like WTF, we’ve had those features for decades, no way in hell it has anything to do with AI, and if it somehow does, no way in hell is it at all necessary. Just an excuse to charge more money for inferior products
The Bosch dishwasher is not junk!
All modern appliances are junk, the best ones are just a bit less junky
I dunno, my dish drawer has been 100% flawless for 12 years now. The only issue I’ve had was last month when I yanked the handle out of the bottom drawer.
LG dryer, 15 years trouble free, zero incidents.
Frigidaire washer, 15 years of good service, but there’s been a persistent rust issue and I did break the plastic drum door handle (easy fix).
12-y-o Samsung fridge, only issue was a clogged drain tube.
These aren’t expensive appliances.
My Samsung washer/dryer calls its ability to select the proper drying cycle AI…all it does is match the drying cycle to the wash cycle. It’s ridiculous.
Yeah, they literally had the exact same feature in the 1950s. Although, granted, at the time, they marketed it as a “wash computer”, which was the buzz word of the time, but it was at least at a little less disingenuous, since a computer doesn’t specifically have to be all electronic
I work very closely with researchers and developers in the so called AI industry. Many of them could be called the elite in this space. Because they actually know what they are doing, they all bristle at the use of the term AI.
Until they find themselves on an investor call.
If people want to pay for the lie, sell it to them.
I’ve never lied on investor or customer calls–my moral compass won’t let me, sadly–but I’ve certainly made certain things sound bigger and better than they would look to an actual expert.
Amen to that.
Mark Zuckerberg is chuffed that everybody is talking about AI because it distracts from the $19bn he’s blown on his shitty metaverse.
Yeah, it was an even mix of, “Those are called algorithms and we’ve had them for years,” and, “That is going to put people like me out of jobs, but you don’t care because this is all marketed towards dealer principals who are there to maximize profits.”
About 10 years ago my employer called all at my level to corporate to witness the amazing advantages of VOICE RECOGNITION SOFTWARE. They did a presentation that didn’t include a live presentation. Everyone was amazed but me. I said I have had issues with this type of software I was invited to interact and try it. TBH in western PA ALL kindsof hankie accents. I do not have one. Well the system was a disaster. The guy who had already authorized implementation was fired. The system was not installed. And as a result of saving the company millions I was summarily dismissed from my job. WORTH IT.
Voice recognition of arbitrary sentences: “almost good enough” since 1997.
It is pretty good these days at picking out of a finite list, though.
I do suspect LLMs have the potential to give it a significant improvement for the first time in ~20 years, since they have some knowledge of semantics/context to figure out more likely interpretations.
Dude who bought the Tahoe blew it. If he was willing to spend a dollar on a Tahoe, he probably could’ve gotten a Camaro for fifty cents.
Right? Hell why not add a Z06 to the deal?
I wonder if 50 cent has a Camaro already.
I saw this blowing up over the weekend and it only reinforces what I’ve been saying about LLMs all along, they’re pretty heavy on the A and not so focused on the I. And while I don’t blame the car dealers for jumping on the AI bandwagon, everyone seems to be doing that this year, but those Fullpath fellas sure should have seen this coming.
not sure what they paid for such a service, but can all these businesses just pay someone to answer the damn phone instead? I spent literal hours on the phone with a bank a couple of months ago. Between chatbots and wait times I was about at my wits end!
As someone who used to work at a dealership, as a “CSR” (customer service rep), you’re not much better off. When I used to work the phones our rules were to get anyone, and everyone, into the showroom. Calling about a used Tacoma that sold 2 days ago? “Yep, it’s here, come on in!”
Coming from Colorado to buy an FJ Cruiser we’re sending to auction tomorrow? “Come on in!”
I’m not saying this is all dealerships, but most of the time dealers are lying scumbags and you’re better off just not believing them at all.
Also worked at a dealership, also was told to tell people to come in no matter what.
Sales manager seemed to think that if someone came in looking for a Suburban, maybe you could sell them the Trax sitting on the lot. Stupid.
Oh god yes, so many times this! I remember once having a woman come in, looking for an AWD Sienna for her disabled daughter (for a wheelchair conversion), and our manager was convinced we could sell her a Sequoia.
I can vouch for this particular dealership being worse than average about that, too. Tried to give me the old bait’n’switch a few years ago when I tried to buy a truck there. Complete with weird excuses about why the VIN matched the ad, but the mirrors and wheels were different from the photos, doubling down claiming it had the payload package despite what the door sticker said about GVWR.
Probably had the sales manager swap the VIN from the pictured ad so they could pull the info and say it matched. I know of a Used Sales Manager who would upload car information and use the same pictures of a super clean example for all models with that color (Black, 2014 Camry) to get people interested. When you came in an realized it wasn’t a XLE with a super clean interior, they’d hope you’d still buy.
“I spent literal hours on the phone with a bank a couple of months ago. Between chatbots and wait times I was about at my wits end!”
And? That’s the whole beauty of it!
A lot of my customers – construction companies, engineering firms, building materials manufacturers, etc, eliminated those positions a long time ago. Their offices have empty reception desks in the lobby that are used as storage areas for incoming/outgoing packages, if you call in, you go through an 18 step automated menu to try and guess who maybe works in the same general department as the person you actually need to talk to, and visiting in person, you hit an intercom button and wait 20 minutes for someone upstairs to choose the short straw and have to come down to let you up into the building. Frankly, it had to be a terrible experience for actual customers, but nobody cares, it’s industry standard at this point
As an hvac contractor who has to gain entry to various businesses, I can confirm sometimes stupid-long waits. We bill for time on-site, though—and if stupidity persists, it often ends up adding overtime. We ain’t cheap, neither 🙂
My takeaway – ChatBots are better informed, friendlier, and more reasonable than your average car salesman.
More honest, too.
Hermann Goering was better informed, friendlier, and more reasonable than most car salesmen.
Unlike a Chatbot, I have nothing useful to say about this beyond caveat emptor.
“No takesies backsies” LOL! that’s got to be legally binding language
Let me nerd out for a second here and say: yes. That response probably would be considered an acceptance to the offer, which would create a contract.
The real (fun) question is whether the chatbot has apparent authority to bind the dealership to the contract.
Wouldn’t you love to argue that one?
Obviously I’m no lawyer, but, if a corporation can legally be a person, then….
My take was the AI part is run by a separate company, so hard to tie back to the dealership.
If the chatbot was able to give a price on a vehicle then 1st that’s a he’ll of a screw up by the 3rd party software provider.
IF (big if), the buyer actually was able to sign a sales contract with a $1 sale price for said vehicle from said car dealer And IF they takes the dealer to court and IF (another BIG if) the buyer actually wins their case… I would fully expect the dealer to sue the 3rd party software provider…
Both cases should hypothetically be relatively simple contract law cases and in both cases success should come down to how well the each respective contract was written.
Either way could be some interesting court cases
It doesn’t, and generally if one party knows the contract isn’t what the other party intended, it’s not binding. No “meeting of the minds.”
Why doesn’t it? The bot is a representative of the company. It confirmed in writing (with emphasis!) that it agreed to the deal. It’s a reasonable assumption that the company is using the bot as an agent of the company and that it is programmed in such a way that has been authorized by the company.
Well, it’s not a person. But that’s almost irrelevant, because if you know the other side has made a fundmentally material mistake, you can’t hold them to that. The more common example is a dealer advertising a car and accidentally leaving a digit out of the price. You can’t make them sell you a Tahoe for $5,199.
I mean, if nothing else, the fact that the response was forced by the “customer” means there’s no way it’d hold up, but also it’s clearly the help chat and not a sales supervisor with the power to agree to a deal.
A thumbs up emoji is legally binding according to the courts in Canada.
If only he’d added “out the door” to the request then yeah that’s definitely legally binding… LOL.