Last week, The Autopian received an email from a reader describing a situation involving his mother’s 2022 Volvo XC60. The car had received an over-the-air software update at some point during the night, and when the owner attempted to start the car the next day, it would not start. The car had somehow been put into anti-theft mode, which immobilizes the vehicle and prevents it from starting. This incident does not appear to be limited to our reader’s mother, and we have confirmed more cases and details, plus we’ve received confirmation from Volvo about the issue. But there’s still a lot we don’t know, and it brings up some interesting issues about software updates worth discussing.
The first dealership service department we reached out to is located in the same area as our reader’s mother – Englewood, New Jersey. Speaking with service staff there confirmed that there were many, many customers who had this exact same problem, with the representative reporting they were “inundated with phone calls,” all from Volvo owners who had effectively bricked cars.
One of the details that makes this problem more alarming is that this particular software update is one that happened automatically, without owner authorization; the service department we spoke with explained that as of April 4th Volvo intended to send over-the-air software updates in waves, and these updates seemed to be, at least in part, updates to fix some reported infotainment display-related glitches.
Volvo’s owner’s manual notes that while some OTA updates are placed in a Notifications area of the user interface, some — “depending on software version” — may be downloaded automatically. Our customer’s update appears to have fit in this latter category, which means there’s not a lot she could have done to prevent her car from bricking itself.
The update seems to have had a problem that puts the owner’s cars in the anti-theft mode, where it does not recognize the owner’s key and as a result can’t be started.
The technician we spoke with said performing a battery reset (disconnecting the battery for around 15 minutes and then reconnecting, effectively re-booting the car) seemed to fix the problem in at least a few cases, but they could not confirm this was a universal solution.
Understanding how many cars this affected or how widespread the issue is or was has been tricky; we called Volvo service departments all over the country, but only ones in New Jersey, Connecticut, and New York were aware of the problem, which suggests this may have been a regional issue. Service departments in Chicago, Houston, Reno, and Los Angeles, for example, did not seem to be aware of any problem.
Volvo representatives stated that they are aware of an “issue relating to software installation,” and that the company’s “quality team is working directly with our retail partners to ensure customer vehicles affected by the issue are remedied promptly,” but declined to give any further information.
What I think is important to consider here is the use of OTA software updates that happen automatically, without user input; is this actually a good idea?
Volvo makes good cars, and everything I’ve ever seen the company do suggests it takes quality very seriously, and yet this still happened, even if only in what appears to be a limited area. If it can happen to Volvo, it can happen to anyone, and I don’t need to remind you that even if they are stuffed full of computers and have software updates and all that, cars are very much not computers, because when cars have problems they can strand you somewhere unpleasant or worse, and not just leave you unable to see the last episode of Severance (which is pretty great).
Knowing that software updates on modern cars have enough interconnections with so many systems that the ability to actually start or drive the car could be impaired by an error of some kind, should there ever be an automatic update?
If you’re driving on a desert camping trip and you know you’ll be somewhere remote, why the hell would you want to risk a full software update, even if the chance of your car becoming immobile is even slight? If it ain’t broke, and all that.
Software is complicated, and so many systems are interconnected on modern cars that you really can’t be certain a software change to one system won’t affect another. For this reason, perhaps all software updates for cars should be voluntary, so people can be sure they’re in a position to comfortably deal with a possible failure, or, if not, wait until they are.
People may have very specific reasons for keeping software that’s not the latest version, and, as long as there’s no crucial safety issue, that should be their right.
Of course, this can make things more complicated, as it means there will always be some percentage of cars out there with older of possibly even buggy software, but I feel like this recent example is an excellent reminder why we still want owners to approve changes to the way their cars work.
Being in IT – patch management in every form is a bitch to deal with: push your updates quickly and automatically, but undoubtedly a bad patch will roll out every once in a while.
Having users approve manually also does not work, as many end-user can’t be bothered and you’ll end up with out of date software.
I’ll just stick with my old E36, E39 and soon E93, thank you!
Why should Volvo care if customers bother to install updates?
Being in IT, you should only be pushing patches to systems you are responsible for. Other people’s equipment isn’t your problem.
IMO there should be a law that OTA updates come with detailed _accurate_ descriptions of what changed so users can choose whether to install an update or not. Fix a security issue or a bug the customer is experiencing? Then users will likely choose to take the update. Update the address to the ad server? Take your update and shove it.
Our 2020 land rover discovery sport had a software update that couldn’t be rolled back to previous version, and it muted all navigation directions – factory navigation, android auto navigation, apple carplay nagivation. Music and calls were fine, just navigation. Now instead of getting vocal directions while focusing on the road ahead I had to focus on the screen to get directions, not ideal or safe.
The update was to fix the screen going blank at random times, and we were warned of the navigation sound issue by the dealership before the update even though we found out afterwards it WAS a known issue with multiple cars affected. Land Rover Corporate did not fix it for a year, I think someone tried to start a class action lawsuit last year that never got anywhere.
With all car powertrains having similar specs these days software has been the differentiator and these automakers are proving over and over that they’re not software experts.
* we were NOT warned before the software update..
This is why I drive old cars. When the wiring on my Midget catches fire, I’m just screwed and it’s my own fault.
There is a great deal of truly awful engineering and design in modern cars but this sort of wireless control system update is astonishingly bad. It makes me question to professional qualifications of Volvo’s chief engineer. I assume that he had final say in major engineering decisions.
I guess the first fundamental engineering flaw is that the critical engine and operational control systems are not isolated from auxiliary systems. Volvo’s chief engineer would have approved the potentially catastrophic systems integration of operational control systems and auxiliary systems and the OTA protocols.
This childish engineering screw-up could be much worse. A fly-by-wire throttle could have been commanded to set full-throttle at speeds over 30 MPH. Fly-by-wire steering could commanded to set full-lock-left at speeds over 60 MPH. If Volvo can brick your car over the air so can a Russian hacker.
What have they done? Turned middle school computer whiz kids loose on auto design and engineering?
What was said about voting software applies to cars also (the obligatory xkcd):
https://imgs.xkcd.com/comics/voting_software.png
Totally OT, but at the mention of Severance I feel the need to post this random piece of genius:
https://www.youtube.com/watch?v=eX4GOltUDdA
Another vote for Severance: It’s a surprisingly good and engaging show, and the season finale was outstanding.
Egads, that is a horrible issue. Not like this would affect the average 2022 Volvo owner, but people get fired over unexpectedly missing a day of work. This will definitely be an issue when this trickles down to entry-level cars. At that point, who is responsible? The owner for not having a working car, or the car company for remotely bricking an otherwise functional car? And can the owner make the car company compensate for lost wages and job placement assistance?
I’ve had at least two consumer electronics items that were intentionally bricked by the vendor. I received e-mails saying something like ‘As of XXX date, this item will no longer be supported’, and it stopped working on that date. It’s one thing to have to replace a perfectly functional home gizmo for a couple hundred dollars. I know I don’t want anyone to have that power over my vehicle.
Had the same thing happen to me with a Sony smart TV while it was still in warranty. I since broke the TV while moving house and don’t feel bad about it, but their response to it fucking sucked and I vowed never to buy another Sony product again.
Ever since Sony Music put a rootkit on audio CDs so they could prevent copying of the music on PCs (early 2000’s), I’ve refused to do business with Sony. They were sued, and either settled or lost. Your compensation was a digital copy of another album on Sony Music’s label.
Sony not only doesn’t care about their customers, but seems to want to actively piss them off.
Volvo have been disappointing in this area lately (and as my profile on here would attest, I don’t say that lightly). Between the switch to the Android-based operating system causing delay after delay in having Apple CarPlay implemented (to it being mooted that it may now never appear) to this… They’ve really dropped the ball.
I had hoped to own a current gen Volvo in fifteen or twenty years when they become cheap enough for me to afford one!
Testing. Testing. Testing. It seems this would show up in a pretty straightforward pre-release test. As incentive for robust testing protocols, release managers should be told that software updates will be tested first on the cars driven by the C-suite people.
A certain well-known EV company does that exactly.
This brings to mind two things:
Most of you 30-somethings will remember the late-90s, early-00’s in video gaming. You know, when you’d slam down $70 on a brandy new game, and the whole game was included. No DLC, lots of easter eggs.
Now we have games with “seasons”, and updates galore.
New cars are the same way. Buy the $40k car, get your DLC and updates.
“Volvo XC90 requires an update, please wait”
Love how my old school cars have minimal electronics…..less to diagnose
I have this car. I leased it through the Care By program in January, and this is the second time in two and a half months that I’ve had to take it to the service department for software issues. The first was when the modem died. Last week, along with probably thousands of others, the car was bricked and had to be towed. The service department was so swamped it took 3 days to get to it.
My previous cars have all been older manual transmission cars from the 80s and 90s. I got the Volvo as a stop gap before I buy an EV. But I’m really disillusioned with modern cars — OTA updates, infotainment, wi-fi — none of it makes the car any more useful or valuable to me. I don’t need 360° cameras, lane warnings, or auto-pilot features. I live in Boston, so I do like being able to start the car from inside the house. Other than that, these features add thousands to the price of the car and cause headaches for the user.
Totally agree (except 360 cameras is a nice and useful feature).
I’m fortunate enough that only one of our cars a new enough to even offer OTA update, lane warning, advanced cruise control, and remote start and door unlock via an app (yikes!). And, except for the OTA update, they were all optional features, and ours doesn’t have them.
As for the OTA, one of the first things I did when we got it was to disconnect the antenna and replace it with a dummy load (so the unit wouldn’t complain about the missing antenna). Problem solved.
The automatic bit isn’t that much of a concern for me. If you allow users to postpone updates they will, forever. In an era where your car is now linked to the cloud you have to be able to force updates for security. This is just bad code.
Change control. Cars *aren’t* computers. Don’t make a change that can screw your customers.
A great counterpoint to fully-optional software updates is safety and recalls. Tesla, for example, recently fixed a regulatory violation in their cars with a software update (disabling the ‘boombox’ feature while in motion). Ford has also pushed out a safety-related update to the Mach E over the air.
People deferring safety updates could be an issue, particularly if — as in the case of Tesla — they are a safety issue for those *outside* of the car.
This of course also ignores the fact that for Tesla, the update that includes the safety fix also changes the car’s user interface for the worse.
They need some sort of change control process. Changes need to be tiered (buggy displays entirely optional), safety updates maybe given a week where the user can choose the time when it’s done. Risks need to be assessed. Teams whose subsystems might be affected need to be notified for review. Getting a good change control process in place can be a PITA. But it’s worth it’s weight in gold for those who are on call, and users who don’t want to be screwed.
I can’t thank you enough for writing this up, Torch.
My wife and I purchased a 2022 XC60 last October, and it feels like a ticking time bomb.
We have only have a fraction of the problems I have been reading about on the Volvo forums. The infotainment software Volvo and Google delivered with the 2022s was absolutely not finished, and it feels like we are beta testers.
We were promised Apple CarPlay by the end of 2021, then by end of Q1 2022, now by “summer.” CarPlay is standard on some $20,000 cars. Wife is convinced its never coming.
It’s extremely frustrating, spending this much on a car from a reputable, long-time automaker, and having the software running the many of the functions just not work properly.
Every time we get in the car, it prompts us to do this software update, and I have to constantly remind my wife to ignore the prompt. The fact that you can accept the update and it will brick the car is shocking. We are an hour from the nearest dealer and can’t take that chance.
The fuming on the forums have turned into talk of a class action suit, and some dealers have even bought back cars from people who have had major issues from day 1. It’s odd that everyone seems to have a different level of issues.
But thanks again for exposing this issue, which needs wider exposure than the forums to maybe get Volvo to act and fix.
I should add that in our car there is a setting to not allow automatic updates, which is why it asks us to update to this new version every time we get in the car.
What you’re describing is how IT support gets put in an inescapable bind: lose user trust in updates, user refuses future updates, user never gets updates that resolve the issues they complain about. We end up with a user who will never be satisfied and a manufacturer who is unable to do anything to satisfy them.
And the cost of this to software developers is hidden but non-trivial. I’ve personally dealt with customers who were hitting a bug fixed in later versions but refused to upgrade until we proved beyond a reasonable doubt that it would fix the problem. It’s a lot of wasted effort because the software industry has taught people that updates are dangerous.
Yikes, kinda scary, even if you are at home, the whole
“The technician we spoke with said performing a battery reset (disconnecting the battery for around 15 minutes and then reconnecting, effectively re-booting the car)” bothers me,
Also, what happens if your car doesn’t recognize the remote, how do you gain access to the battery?
First World Problems, I know.
Because, like all cars, the XC60 has a physical bladed key that can open the car. By necessity, that’s a mechanical process, so—regardless of what the computer’s doing, or, more relevantly, regardless of if the battery is dead or not—if the key can turn in the lock, the door will open.
This shit should be sequestered. What I mean is, a defective update for the infotainment system should never be able to prevent the car from starting because *those systems should be separated*. It shouldn’t all be tangled up in a big mess where if something weird happens with the big iPad in the dashboard, it affects core functionality. If that’s possible, automakers need to take a step back, look hard at themselves in the mirror, and them smack themselves on their own noses with a rolled-up newspaper until the engineering department comes up with something better. Start from scratch if you have to. The current state of affairs is fucking unacceptable.
They could take some lessons from the aviation industry. If the TV in the seatback in front of you on the 737 doesn’t work, the wings don’t fall off.
This. 1000% this. And stop doing stupid things; like making doors that only open via electric solenoids. Basic functionality doesn’t need to be held hostage by an infotainment system. The core mission of the car is to *drive*
Yes but an errant cell phone call can cause the whole plane to crash. /s
I, too, fail to understand how the infotainment system is such a central part of a modern car’s functional electronics network. I remember when various cars, such as the 2006 Range Rover Supercharged I once had, used a MOST ring…such that one failed component would take out the whole system, like series Christmas lighting. But none of the car’s propulsion or security systems were part of that ring.
Not dissimilarly, the glitchy-ass infotainment system in our ‘22 Outback often crashes, and takes out
the entire EyeSight safety system every time it does. I find it odd that those would be so inextricably tied together.
Ironic how that late-’90s “if Microsoft made cars” email joke list that some of us forwarded around (or searched for on…Netscape? Sigh.) is now kinda coming true.
Automatic software updates are a terrible idea. I prefer my Volvo’s traditional method of immobilizing itself at awkward times via destruction of its Variomatic belts instead.
¿Por que no los dos?
Yes, sometimes both belts.
But not the suspenders, I hope.
Perhaps all such vehicles need an equivalent of the paper-clip CDROM trick to reset the system, or rollback to the previous version. Not that that’ll happen.
Now just imagine a system-breaking update is received by a self-driving car that’s in motion. One would THINK being parked in off/standby mode would be a requirement before updating, but anyone who’s had their Windows unexpectedly reboot for an update knows better.
(yes nowadays there’s almost always a warning/prompt beforehand, but it CAN still happen)
Or make it as simple as putting the key in the door lock and unlocking the door physically. If the key works then allow the ID chip to be re-validated
OTA Updates should always have a prompt for the owner to accept when the car is parked in a safe place. Volvo should have thought of this. Nobody likes changes being made to their stuff without giving permission. Remember the U2 album on iPhone fiasco. Nobody got hurt but folks were still mad as hell.
Volvo: “Bricky but good”
Sounds like the passive-aggressive tagline for a they-don’t-make-em-like-they-used-to enthusiast Volvo repair/restoration shop.
“Ours are bricky in the good way – no 404, try 240”
I think you’ll find that this is a deep cut cultural reference, which if we could post pictures on this site, we could demonstrate.
See Dudley Moore, 1990, Crazy People, Volvo ad.
Also, that’s 32 years ago. I’m getting old.
I’m 93% sure that my car is setup to get its updates over Wi-Fi only. The only Wi-Fi network I have it setup to connect to is the one at my house. So if an update bricks my car, at least it will be at home and not at some random place away from civilization. I also leave the windows rolled down when it is parked in the garage as it would make it a lot easier for a technician to unbrick it.
I realize this is not an option for people who do not have the luxury of parking their cars in a private garage.
If true, then we should be able to prevent updates by simply turning off Wi-Fi in the vehicle.
This assumes it is like your laptop/tablet/phone and only joins networks that you say are OK..
If the vehicle can join any open Wi-Fi network, well.. that is even worse, for the same reasons it is not always wise to do your taxes on Starbucks Wi-Fi..
How soon before nefarious hackers from western Asia / eastern Europe access your car’s update system and turn it into .. is potato.
>> So if an update bricks my car, at least it will be at home and not at some random place away from civilization.
Not necessarily. It could download the update and then install the update later, while you are away from home.
Louis Rossman is going to lose his shit over this.