Are Ford Executives Dumb?

[don.mullins]

My F-150 is supposed to be built at the end of this month. Since I have been waiting, I have read everything I could about the vehicle on this forum and elsewhere to learn about it. Found this video where a Ford executive discusses the fact that they ‘outsourced’ the 150+ modules to 150 companies including the software. Those companies own the IP (intellectually property) rights and Ford has to go back and ask them to write new software if there is a problem. He states that they will bring the process in-house but it will take many years for that to happen.

Can‘t seem to comprehend how any executive would agree to subcontract ‘core competencies’ to save very little money, (he says that it’s $350-500/vehicle). I see so many issues on this forum that seem to be software related -I.e. BMS, TSM, DSP, etc. Seems that Ford really dropped the ball on this one. When Apple had problems with their iPhone antenna (2 or 3), Steve Jobs called the executive in charge of antennas and fired him on the spot.

You are misunderstanding the context of the interview (if it is the one with Jim Farley).

Historically, the software in these modules has always been outsourced to the auto industry, The module came from the vendor with software from the beginning,

Jim was saying he understood the current state of the industry, and knows they have to change their deign paradigm to meet the future’s development needs.

Sponsored

 

[oncechance]

I'm a little surprised the Ford bot hasn't chimed in asking for VIN and they will look into it.

 

[thudnblunder]

Im a firm believer in getting it right the first time. Im fine with development work getting subbed out under Fords management, but Im not okay with an SDRL that eventually leads to compatibility issues and future updates. The amount of updates a truck should receive after leaving the factory better be damn close to 0. The transmission software is still a disaster.

Yup, the possibility of over the air updates opens the door to the very real risk of making the default approach “Ship it as is to meet the deadline, we will fix it later via an update”.

 

[BigJim]

You’re not entirely wrong.
But you underestimate the cost of developing that capacity in-house, which is ENORMOUS, and the time it takes to develop that expertise which is somewhere from years to decades.
So there really was no choice.
Don’t believe it? Just look at how long it took a major company whose sole focus is software, to develop a somewhat reliable product development cycle and to release quasi-reliable products. Microsoft has been at it for decades and arguably just recently got (most of the way) there. The battle field is littered with the dead bodies of companies that tried and failed.
I am not excusing the relatively poor quality of Ford’s products by any means, just pointing out the reality.
IMHO (41 year alumnus of the high tech computer world where complex system design is derigeur) Ford needs major improvements in:
- System design. You can’t just hang a bunch of subsystems off a shared bus or two and hope it’ll work.
- System design Validation. IMO this is the weakest spot right now.
- Debug Validation. Once you’ve debugged a problem, did you actually solve it? What other independent subsystems did you screw up?
- System Validation. This is different from System Design Validation.
- Most important of all, Configuration Management and Release Control. More than once, Ford has released two consecutive fixes with the SAME revision number!
- And of course the OTA system is a shambles. Amazing it works at all.
So yeah, they’ve got a LOT of work ahead of them and competent hw/sw engineers don’t grow on trees, nor does the necessary management skill.
All IMHO of course.

You make a lot of sense and all I could think is Ford should hire you. As far as the cost of developing, Ford is losing $ 4 billion on the lightning. There should be enough money to properly develop competent in-house software engineering. Although this obviously does not resolve the length of time to implement and the process of finding qualified talent.

 

[PaulGrun]

It would be ideal if Ford had the engineering resources to do it all in house but you could argue that Ford’s core competencies are engine design, transmission design, suspension, frame, body etc and the rest they turn to other specialists for who have a core competency in that domain.

We see it for things like *cough* airbags, *cough* wiper motors, or even things like bulbs, and tires.

Instead of cost cutting alone, this is a way to get access to skillsets that Ford does not have in house. A problem with insisting on everything done in house means you can’t use new technology unless you hire and train people for it. And you’re not as able to go try new technology because first you have to hire and build a team, and that needs time and approvals, and very soon you start falling behind your competitors.

As the vehicles get more complex, the software also gets progressively more complex to the point where an outsourced developer is very specialized in the type of software they develop. As AI becomes more embedded, you’ll see another wave of this since not everyone has the datasets, skill, or resources to train machine learning models.

Now how do you make it all work together? It’s not too different from how you’d do it in house. You build an API (Application Programming Interface) that is the protocol these components use to exchange data. This way everyone can work on their own piece of code, and make changes but as long as the API remains unchanged, it will all work together. This is how almost all software at scale operates today.

I will say though, it doesn’t make sense to me why Ford wouldn’t own rights to the code developed for them. They’re not a tiny shop running commodity off the shelf software. This stuff is developed for them. To me it sounds more like they don’t k ow how to change the code to make it do what they want without going through the vendor they used to develop it.

Even building the API is non-trivial.
Remember that Ford is using standardized networks (e.g. CAN bus), so depending on how far up the network stack that bus attaches dictates who defines the API for a given end node. I’m not familiar with the CAN bus protocols. My understanding is that Ford dies T control any of … not the wire and not the endpoints. So they’re stuck.
I agree it would be best if Ford at least had access to the source code, but apparently that isn’t the case - seemingly there are dozens of vendors so we can imagine the legal quagmire trying to get that done.
My understanding is that some vendors are easier to work with than others. Apparently, with some vendors, it’s “we’ll fix it when we get around to it”, and they may or may not tell Ford what they fixed.
It’s truly a mess.

 

[PaulGrun]

Yup, the possibility of over the air updates opens the door to the very real risk of making the default approach “Ship it as is to meet the deadline, we will fix it later via an update”.

This is ABSOLUTELY what is happening. Which is why you see fixes for fixes.
OTA methods have gone from, “ability to add new features”, or “ability to patch bugs”, to, “ship it. We’ll fix it later”. To engineering management it’s like a drug.

 

[Mike50]

My F-150 is supposed to be built at the end of this month. Since I have been waiting, I have read everything I could about the vehicle on this forum and elsewhere to learn about it. Found this video where a Ford executive discusses the fact that they ‘outsourced’ the 150+ modules to 150 companies including the software. Those companies own the IP (intellectually property) rights and Ford has to go back and ask them to write new software if there is a problem. He states that they will bring the process in-house but it will take many years for that to happen.

Can‘t seem to comprehend how any executive would agree to subcontract ‘core competencies’ to save very little money, (he says that it’s $350-500/vehicle). I see so many issues on this forum that seem to be software related -I.e. BMS, TSM, DSP, etc. Seems that Ford really dropped the ball on this one. When Apple had problems with their iPhone antenna (2 or 3), Steve Jobs called the executive in charge of antennas and fired him on the spot.

I don’t wish to insult you but I must reflect my own experience. Generally, when I accuse someone of ignorance or lack of intelligence, it turns into a display of my own ignorance. These people didn’t get to where they are by being retarded or stupid. They are the best and brightest, and they know things that we don’t. You best give them benefit of the doubt.

 

[Atlee]

Several mentions of iPhone and Tesla doing in house development in this thread.. iPhones are a premium product and people are willing to pay the premium.. So Apple can afford to do all the development in house..and charge over $1,000 for a tiny piece of hardware..
Tesla used to be able to command high premiums with a loyal customer base who were willing to pay the premium and live with many glitches and imperfections. With Tesla that is changing now as the committed buyers all own Teslas and now Tesla is having to reduce prices to attract non EV buyers..
This is not a luxury traditional automobile companies have.. Yes, they could do all the development in house but that would add significant time and cost to the development.

Instead, Ford and other companies go to component manufacturers who specialize in their products and sell their products to multiple companies and are able to spread the development costs.. If every auto company spent the money to develop every module, the prices would be prohibitive..

It is easy to criticize auto companies but they are typically choosing the most competitive path so they can sell their products at the current prices which we already think are too high! If every auto maker did all the development in house, the prices would be way higher!

I agree with this, but shouldn't Ford own the software?

 

[BigJim]

I don’t wish to insult you but I must reflect my own experience. Generally, when I accuse someone of ignorance or lack of intelligence, it turns into a display of my own ignorance. These people didn’t get to where they are by being retarded or stupid. They are the best and brightest, and they know things that we don’t. You best give them benefit of the doubt.

No insult taken. I am sure Ford has extremely bright engineers and the F-150 is generally a well designed vehicle But reading this forum, there are many issues that appear to be more software related and even Ford top management has reiterated their problems in software.

Besides improving software, I think Ford can up their game in customer service by improving communication. So many folks on this forum complain about not knowing the status of their vehicle because of unknown QC problems. if Ford communicated what they was happening and what they were doing to remedy the situation, I think that it would build brand loyalty and relieve a lot of frustrations.

 

[Murrk847]

Outsourcing is not always a bad thing. Who you outsource to is where the problems start. Being in the tech industry, a lot of stuff these days is being outsourced to Indian based company's who are not known for quality. It's not a race thing, it just so happens that India is the cheapest market that has technical competency. You are not going to see software development outsourced to Mexico based companies because technical competency is almost absent there.

Western Europe and American based software companies are greatly superior in ability but comes with a high labor cost. The average salary for an American software developer is north of 100k where as in India its 40-50k USD tops. They are also going through a "course correction" like America is where tech salaries are either stagnating or dropping from the covid-era tech salary boom.

Unfortunately, America is part of the global economy now and there is no end in sight.

 

[garagegymgoon]

My F-150 is supposed to be built at the end of this month. Since I have been waiting, I have read everything I could about the vehicle on this forum and elsewhere to learn about it. Found this video where a Ford executive discusses the fact that they ‘outsourced’ the 150+ modules to 150 companies including the software. Those companies own the IP (intellectually property) rights and Ford has to go back and ask them to write new software if there is a problem. He states that they will bring the process in-house but it will take many years for that to happen.

Can‘t seem to comprehend how any executive would agree to subcontract ‘core competencies’ to save very little money, (he says that it’s $350-500/vehicle). I see so many issues on this forum that seem to be software related -I.e. BMS, TSM, DSP, etc. Seems that Ford really dropped the ball on this one. When Apple had problems with their iPhone antenna (2 or 3), Steve Jobs called the executive in charge of antennas and fired him on the spot.

"Can‘t seem to comprehend how any executive would agree to subcontract ‘core competencies’ to save very little money, (he says that it’s $350-500/vehicle)."

Some basic math here...just counting the plants that produce F-Series trucks using the low end $350/vehicle with a low end units per day/shift number...this equals a total of $1,260,000/day and $7,560,000/week.

It's very easy to underestimate the sheer volume of vehicles that are built when talking about cost savings. The company will make some pretty extreme changes and invest a lot of money up front on projects that save PENNIES per vehicle. Let alone a whole dollar or as we are discussing here...hundreds of dollars per vehicle. The long term cost savings through the life cycle of a program are astronomical.

 

[EricR]

Does Mercedes develop their own ABS/stability control systems, or buy them from a vendor specializing in this technology?

The vendor is my bet.

Does BMW develop their own BLIS/collision-avoidace, or buy from a vendor who specializes in radar sensing systems for the automotive industry?

The vendor is my bet again.

Are the above items dependent on processors and software to function?

Absolutely!

So if you are Bendix, who develop ABS/stability control systems, are you going to let your huge investment in proprietary software (code) go out of house?

Absolutely not.

The automotive industry (and MANY other industries) have ALWAYS been dependent upon component vendors. For example, brake components, emission components, transmissions, suspension, etc.

The difference now is that many of these components rely on software to achieve their advanced level of functionality/features. The hardware and software cannot be separated. These vendors have teams of hardware and software engineers who are domain experts deeply immersed in the details of their specific component offerings.

So while I get some would want Ford to "own all the code" or "do it all in-house", it just isn't realistic.

Unless you want to drive Model T.

 

[Calson]

The smart software engineers tend to reside in places like the SF bay area or Seattle area where they are not 100% dependent on a single employer or nor need to wait for someone to die to get promoted. There is a big financial commitment for a person to move themselves and their family to places like Detroit or Austin, especially people of color or those who are female or have wives whose health, mental and physical, they care about.

I have managed software projects for Fortune 500 companies and there are mediocre, excellent, and exceptional software engineers. Few companies are willing to pay what it takes to attract the exceptional programmers and very few corporate managers (or HR people) understand the difference. Even with Microsoft they shut down their quality control and testing department with its 400 engineers as the managers decided to let customers be the testers of new code.

 

[AndreB]

It would be ideal if Ford had the engineering resources to do it all in house but you could argue that Ford’s core competencies are engine design, transmission design, suspension, frame, body etc and the rest they turn to other specialists for who have a core competency in that domain.

We see it for things like *cough* airbags, *cough* wiper motors, or even things like bulbs, and tires.

Instead of cost cutting alone, this is a way to get access to skillsets that Ford does not have in house. A problem with insisting on everything done in house means you can’t use new technology unless you hire and train people for it. And you’re not as able to go try new technology because first you have to hire and build a team, and that needs time and approvals, and very soon you start falling behind your competitors.

As the vehicles get more complex, the software also gets progressively more complex to the point where an outsourced developer is very specialized in the type of software they develop. As AI becomes more embedded, you’ll see another wave of this since not everyone has the datasets, skill, or resources to train machine learning models.

Now how do you make it all work together? It’s not too different from how you’d do it in house. You build an API (Application Programming Interface) that is the protocol these components use to exchange data. This way everyone can work on their own piece of code, and make changes but as long as the API remains unchanged, it will all work together. This is how almost all software at scale operates today.

I will say though, it doesn’t make sense to me why Ford wouldn’t own rights to the code developed for them. They’re not a tiny shop running commodity off the shelf software. This stuff is developed for them. To me it sounds more like they don’t k ow how to change the code to make it do what they want without going through the vendor they used to develop it.

Plus the holy grail reason of strategically owning the rights to the source code, is isolating Ford from a major problem on continuation if one of those 150+ subcontractors go bankrupt or get bought by an other company. Just my 2 cents after retiring from 33 years in IT.

 

[amschind]

The bigger question is: why do big automakers still exist? Powertrain stuff will all be electric motor/single gear transmissions purchased from BorgWarner, software is outsourced, electronics are outsourced, stamping is less and less attractive as structural composites become the norm, so what exactly does a giant automaker actually do? 50 years ago it was design and build engines and transmissions, but I think that we are currently driving the final generation of piston engines and transmissions.

I think that we will see a proliferation of small assemblers who buy off the shelf batteries/traction motors/fuel cells and focus on tooling to build vehicles that have more in common with a boat than a current generation vehicle. The huge metal stamping operations and indirect government support for the union pension plans are the last obstacles for that change, and we will see it in our lifetimes. Capitalism is the application of evolutionary theory to economics, and the only constant is change. Dinosaurs didn't die out, they just turned into birds.

Sponsored