When I was a geeky kid, reading all sorts of science fiction, the idea of advanced artificial intelligences came up often. Sometimes in the form of robots, sometimes in the form of a computer without a robotic body, but always something thought provoking as these synthetic brains breached the boundaries of sentience and started to grapple with the mysteries of identity and what it means to be something aware of one’s self. They asked what this thing called “love” is or wished to experience laughter or something equally profound.
What they almost never did was make buyback deals on used BMWs that would later be reneged upon.
And yet, here in our stupid reality, that’s what AI is actually doing, when it’s not too busy giving you bad advice or putting pictures of famous people you find online in skimpy clothes. This particular situation, CBC writes, happened at the BMW Toronto dealership, when 2021 BMW X3 XDrive 30i owner Zack Giacomelli, displeased from having to do expensive repairs on the car, decided to reach out to BMW Toronto to see if they were interested in buying the car back.
Happily, they were! At least, according to BMW Toronto’s online representative, named Quinn.
Quinn offered $27,162.79 for the BMW, covering the remaining balance on the car, with “no money owing, and no cash back,” according to texts with Quinn that Giacomelli shared with CBC. Giacomelli was very happy with this offer, and for the hell of it made a counteroffer of $28,000, which Quinn allegedly stated “sounds reasonable.” Quinn even allegedly suggested that Giacomelli come by the dealership to “lock in today at 3:30.”
Everyone should be happy, right? A deal is about to be struck!
Well, not quite. According to CBC,
“Moments later, Giacomelli said, a BMW Toronto sales consultant called to revoke the offer, explaining that Quinn wasn’t a real person, but rather an artificial intelligence chatbot that had made the offer in error.”
Why did the dealership allegedly rescind the offer? Because Quinn is not a human being: Quinn is an AI chatbot.
This of course brings up that perennial philosophical question: so what?
Yes, Quinn is an AI, but it’s also acting on behalf of the dealership, and customers who interact with Quinn have no way of knowing if Quinn is man or machine; just ask Alan Turing about that.
Giacomelli brings up an excellent point about all of this, telling CBC:
“If they’re going to be replacing their employees’ jobs with AI, then they need to be honouring what that AI says.”
Mr. Giacomelli, I couldn’t agree more.
You can’t have it both ways; a business can’t automate a job that was formerly a person with authority to make deals like this and then decide that no, what this machine says while operating as an agent of the dealership cannot be taken as actual, binding interactions with the dealership, like what the human agents they replaced did. How the hell is a customer supposed to know that? And if the AI will say things that aren’t actually what the dealership intends to do, then what’s the point of the damn AI?
If a human employee had made a deal like this, the dealership could likely have to honor it, or at the very least consider it. Just as a human can be an idiot and make bad decisions, an AI can be an idiot as well, and perhaps even could execute idiocy with efficiencies and speeds no human could hope to match.
An AI chatbot may be able to interact convincingly with a customer, but it fundamentally does not understand what it’s doing. It has no idea what cars actually are or how much money is actually worth or, well, anything about life in the world, really. It’s a bunch of branching conditionals and massive databases that delivers the illusion of sentience. The reality is it doesn’t “know” jack feces.
Eventually, the dealership did honor the $27,162.79 offer, so they did end up doing the right thing in the end. But even if they hadn’t, the law seems to be on Giacomelli’s side. Canadian law has already encountered this issue, as a case in 2024 ruled that companies (in this specific case, Air Canada) are indeed responsible for what their AI agents tell customers.
The legal decision makes its point and reasoning quite clear:
“Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives – including a chatbot. It does not explain why it believes that is the case. In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission. While a chatbot has an interactive component, it is still just a part of Air Canada’s website. It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.”
Where that 2024 case made clear that it “makes no difference whether the information comes from a static page or a chatbot,” it would also stand to reason, and perhaps even more strongly, that it makes no difference if an official communication from a car dealership comes from a human agent or a silicon one.
Hopefully, this case will be noticed by other businesses using a chatbot to interact with clients and customers. You can’t embrace the economics of getting rid of employees in favor of AI without accepting whatever some AI may say or do while representing the organization. AIs are, bluntly, idiots. They can’t help it; they’re not alive, and have no genuine understanding of the world. Is that really what a company wants representing it?
(top image: Jason Torchinsky, YouTube, CBC)
“Eventually, the dealership did honor the $27,162.79 offer,”
Well, they should actually have honoured the $28,000 counteroffer that the chatbot said “sounds reasonable”.
This is half the story to me. The other half of the story is:
“when 2021 BMW X3 XDrive 30i owner Zack Giacomelli, displeased from having to do expensive repairs on the car, decided to reach out to BMW Toronto to see if they were interested in buying the car back.”
Essentially as they age, BMWs become unreliable, expensive-to-fix pieces of junk much faster than the Japanese luxury brands. Sure they’re nice to drive… when they run.
But if you want a luxury car that is actually reliable and durable, you’re much better off with a Lexus, Acura or 6 cylinder Infiniti.
I’d add some caveats and a replacement to your list of alternatives:
“…a naturally-aspirated Lexus, a Mazda, or a naturally-aspirated 6-cylinder 2022+ Infiniti.”
I might even throw certain Lincolns and maybe a non-supercharged Volvo to the list just for a wider variety.
The most reliable Lincolns will be the non-ecoboost ones… which would be the ones with the hybrid powertrain since just about everything else will have the Ecoboost.
And the human employee would probably be fired immediately after. So are they going to fire the chatbot and hire a human? Because that’s the right thing to do.
Fire the one human that was tasked with implementing these AI Chat bots and hire another AI chatbot to oversee the other AI chatbots.
Soon it is going to be AI chatbots all the way down.
“Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions”
What? Seriously? I can’t believe this is real…am I in the Twilight Zone?
Do they even have a brain? Fucking morons
They absolutely have a brain. They’re angling for the best of all worlds (for them). The chatbot can negotiate a deal, and the customer’s held do it but the airline isn’t. All while not having to actually pay any humans to do the bulk of the work.
Lawyers are sometimes paid to say the darndest things. Whoever composed that sentence was probably well aware of how idiotic it sounds, but their job is to write it and hope for the best. They’re not necessarily morons, but they’re certainly hoping the judge is.
to be fair to lawyers (I know right?) their job is to vigorously defend their client, which as the other commentors say means they say the darndest things. (cue seashell picture). The courts response was basically the same as yours, just more diplomatic; This is a remarkable submission. That’s judge talk for “are you #$%^ing kidding me??!?!”
Just a couple of fuckups like this and AI will start to look a lot more like the liability the rest of us know it to be, and not the economical labor offload that managers fantasize about.
A couple?? This has been an issue in the legal world for years now. Lazy, stupid, and/or malicious lawyers keep getting in trouble, but you know many cases don’t get caught. It’s kind of insane how much AI is fucking up numerous industries to profit a handful of people who do not deserve it.
Here’s but one article listing numerous examples: https://arstechnica.com/tech-policy/2025/11/lawyers-keep-giving-weak-sauce-excuses-for-fake-ai-citations-in-court-docs/
Yup, screw-ups like this have been happening at an alarming rate, but they just don’t get reported on because so much is being invested into AI right now, media groups are likely very exposed to the financial ramifications of any kind of downturn in the AI market (never mind the bubble bursting). I was appaled to learn recently that the failure rate of AI models in something as simple as retrieving a phone number (something we were used to do with a very high success rate in a normal web search) may be as high as 36%. That’s a staggering percentage – over one third. The same technology is being trusted with all sorts of way more complex tasks, and probably failing at an even higher rate in those tasks. I see the ramifications of this concerningly high failure rate every day in my field of work and it’s seriously scary.
At the same time, this is something everyone can test for themselves: try asking an AI model some basic questions on a field that you’re specialised in, and see for yourself how much stuff it gets incredibly wrong. I’ve had this recent example that I don’t even have to explain in depth: an AI model told me the issue with an unanswered call that I was investigating was that it was missing the call.answered event that’s expected in all unanswered calls. I don’t think I need to explain that all unanswered calls are expecetd to not have a call.answered event in this context. AI models will generate this kind of idiotic results with an unparalleled level of confidence, so I’m very wary of using them to investigate more complex things that I don’t have a deep understanding of – despite being incentivised to do by my employer.
I did this exact thing, I have a videogame on the Steam PC platform, easily google-able and has a public page that’s commonly indexed with tons of information in plain-text. Asked Gemini about it and it just started spouting nonsense about halfway through.
I’m being hyperbolic. It’s only a matter of time, in my mind, before the costs of offloading real people jobs onto AI become incontrovertible even to managers, and AI gets shunted to the sidelines for the time being. A few deals like this one and the dealership will quickly realize that having the human handling calls was far more affordable.
I think as soon as someone successfully sues the companies deploying the software behind “Full Self Driving,” the first few trees of the landslide will fall over.
Before anyone says it, yes, yes: the waiver. The disclaimer you have to sign. Blah blah. It’s a mile long and users don’t read it. That kind of defense has been successfully defeated before, partly because it’s a mile long. It’s just a matter of time.
(curmudgeon hat on)
I’m glad that I wasn’t born later than I was, if only because it would’ve meant that I’d have to spend even more of my finite hours of life interacting with customer service phone trees, and more recently phone trees augmented with AI ‘helpers’ such as those now found in most phone trees for larger companies, and of course online text chat bot ‘assistants.’
Had I been born 20 years earlier, I’d likely be dead for a few years now, and thus, would have spent less time dealing with such things than I currently have (which is already too much). But if that were the case, I’d not have met my first serious girlfriend in 1982, and that would be too high a price to pay. 😉
PS: speaking of curmudgeons, I don’t use a cell/smartphone and have a landline instead, which I honestly prefer. However, most of the wiring in my house is over 60 years old, and it’s probably to blame for my phones having no dial tone for the past 10 days (the tech appointment is still a week away). So, for the past week and a half, I’ve received and made zero phone calls (except for borrowing a neighbor’s cell once to call AT&T to schedule a service call).
On the whole, living without ANY phone for ten days has been fantastic. Yes, a minor inconvenience now and then, but those are more than offset by the tangible increase in peace and quiet. If I were 20 or 30 and cut off from phone calls I’d probably be going nuts. But at twice that age, it’s a lovely vacation from a small but persistent source of daily stress and distraction.
Alright, that’s it. Back to cars and how they’re only going to get worse as AI continues to pervade all aspects of modern society.
I bought a ’22 Niro EV from a Honda dealer, they had bought the former rental vehicle at auction, as many do. The price online was very reasonable, and this was in the days of $4500 used EV tax credits. I communicated via their website and email, and was sent a dealership quote (letterhead and all) via pdf for the roughly $24000 price, minus the $4500 credit.
Arrive at the dealer and meet the sales guy I had been communicating with, who first thing said ‘the sales department really hopes you don’t buy this one’. There was fine print on the website that all prices were inclusive of whatever discounts might be applied, but whoever made up the quote for me had applied it erroneously. However, since they had officially sent me the quote, they honored the deal.
Most pleasant dealership visit I have ever had…
My brother and I were talking about AI coming up with terrible answers because it isn’t capable of understanding the things it inputs. Since I am a crass man, I used the analogy that AI is like a classroom of 1st graders being told George Washington died from Thomas Jefferson violently pegging him with wooden strap on. They are going to go home and repeat that to their parents without a second thought because they have no idea what that actually means, or that it is an obvious and absurd lie.
Yes…obviously a lie…
Somewhere in a hidden room deep in the national archives, an alarm is going off
Yes it is an obvious and absurd lie, but it contains a kernel of truth in that Jefferson trafficked a 14-year old enslaved girl into France and then got her pregnant when she was 16…
To be fair, a human BDC rep is not authorized to put a value on trades.
Then again, a human BDC rep would know that.
Exactly. And they would most likely check with their boss before making these offers. Why is the person with signing authority out of the loop until an appointment is made to finalize?
But the chat screenshot shows that the AI even said, “I will have the team review it for a value”. Which pretty much explicitly promises (lies) that the AI DID go talk to the appropriate authorizing authority.
That part made me laugh. They packed it with standard dealership blather to make it sound more real, but it had already made the offer on its own.
I’m surprised that there wasn’t actually an AI Sales Manager that the AI agent had to go back and check with. Had that happened, you know that the answer would have been, “Oh, man, I really want to make the deal but the manager says no way. How about we give you $15K?”
Hang on.
Good for basic math and bad information? Overpriced and costs good people their jobs?
Turns out Sales Management was the original artificial intelligence all along.