Introducing Bluepilot! A Ford specific fork for Comma3X / OpenPilot

[HammaMan]

If they nail their new model as they're touting, I'll have to pick one up around black friday.

Sponsored

 

[kvandivo]

If they nail their new model as they're touting, I'll have to pick one up around black friday.

Assuming they don't channel their inner-Elon and that they actually hit their October target.... What's the odds that a new end-to-end model will require more compute horsepower in the car than the 3x can handle? Obviously the real compute intensive part is training the model, but how much additional compute will likely be needed in-vehicle? Anyone know or care to speculate?

 

[Ajzride]

Assuming they don't channel their inner-Elon and that they actually hit their October target.... What's the odds that a new end-to-end model will require more compute horsepower in the car than the 3x can handle? Obviously the real compute intensive part is training the model, but how much additional compute will likely be needed in-vehicle? Anyone know or care to speculate?

I don't think the compute power will drastically change for the ML models. The computer horsepower will change when they want to start solving the model on a larger field of vision (further away from the vehicle).

 

[HammaMan]

Models have been getting more efficient so they should be able to incorporate better/more logic without over-taxing the compute. Given their end goal they'll likely keep the details at 30k ft level.

 

[kvandivo]

Question that may or may not be public knowledge.. Does "the model" actually take advantage of all the inputs available to it for a given vehicle? My understanding is that the bluecruise capable vehicles like the properly-equipped F150s are using 4 cameras (front, back, side mirrors) and however many IR and radars.. Does "the model" use all of that, or does it fall back to a common denominator that works for all supported vehicles along with the comma 3x's included 2 cameras?

Just curious because it would obviously be best to know that all available input is being consumed for safety reasons if nothing else. We were amazed with what Mobileye etc could accomplish in our Model S just using the front-facing cameras, but safety obviously improved once the software started using data from the other cameras available.

 

[Ajzride]

Question that may or may not be public knowledge.. Does "the model" actually take advantage of all the inputs available to it for a given vehicle? My understanding is that the bluecruise capable vehicles like the properly-equipped F150s are using 4 cameras (front, back, side mirrors) and however many IR and radars.. Does "the model" use all of that, or does it fall back to a common denominator that works for all supported vehicles along with the comma 3x's included 2 cameras?

Just curious because it would obviously be best to know that all available input is being consumed for safety reasons if nothing else. We were amazed with what Mobileye etc could accomplish in our Model S just using the front-facing cameras, but safety obviously improved once the software started using data from the other cameras available.

the model only uses the 3 cameras, and common data such as acceleration, yaw, roll etc. other data like BSM and Radar can be incorporated in the vehicle specific code.

 

[HammaMan]

Question that may or may not be public knowledge.. Does "the model" actually take advantage of all the inputs available to it for a given vehicle? My understanding is that the bluecruise capable vehicles like the properly-equipped F150s are using 4 cameras (front, back, side mirrors) and however many IR and radars.. Does "the model" use all of that, or does it fall back to a common denominator that works for all supported vehicles along with the comma 3x's included 2 cameras?

Just curious because it would obviously be best to know that all available input is being consumed for safety reasons if nothing else. We were amazed with what Mobileye etc could accomplish in our Model S just using the front-facing cameras, but safety obviously improved once the software started using data from the other cameras available.

BC uses 5 radars and the windshield camera. These are 4 blis radars at the corners (including blis in the taillights) and the CCM cruise radar. Yes the comma device can see these signals if desired, but uses its own cameras for visual. The 360 cams are not used for BC. If they were it would actually be able to track straight in the lane by verifying position using the mirror cams and looking at the relation of the tires vs the lines (if you run a 360 cam calibration you can see that the system can utilize them, but limited compute capabilities causes it to either/or where it uses the compute - in calibration mode it's running its system on all 4 360 cams to fuse the image and post process it). It'd be substantially better if it used the cams, but its got limited compute. The data from the radars is rather limited basically reporting an angular direction and range to an object.

If ford used the cameras, it wouldn't need the corner radars. Telsa and open pilot use visual data. Ford is overly dependent on the CCM radar which discards all stationary objects which is why vehicles using BC have rear-ended vehicles stopped in the travel lanes (NTSB is investigating due to fatalities). Ford is very disappointing on the software front. When running the lane centering calibration, it should be watching the windshield camera and the side cameras and comparing the data to generate the necessary calibration data for the vehicle. In a curve, it'd be looking at the curve's details, what the windshield camera is reporting and what it's seeing in the side view cams. If they did this it'd be able to lane center like a boss without needing the the side views. Though it'd still be wise to occasionally 'check' the side view cameras to see its position in the lane and make sure the vehicle is still tracking according to its learned variables. The system doesn't even need to be engaged for this to occur. It'd simply be paying attention to you driving.

It could even go so far as to query / alert the driver "Has anything changed in the vehicle? My sensor calibration and your driving suggest you're not maintaining the lane or my sensors need calibrated". It'd then prompt for a calibration cycle to be run. This isn't currently how the system operates, but it could very well come in the future. The fact 1.4 hasn't released despite some of us having it suggests ford isn't happy with it and may have been working on something else / killed 1.4's future progress. I'm surprised they didn't just release it and call it 1.3 since 1.3 is in vehicles already. The NTSB actions likely have an impact on it as well. I hope they're just making a new model entirely but they haven't said anything at all, not even making hints on what's going on w/ their ADAS.

If ford does go more vision focused, BC could be offered to assist 2.0 vehicles that have 360 cams if they find someway to be mostly hands-free while by drastically extending the need to provide wheel torque feedback. For $1k tesla will upgrade older vehicle computers with newer hardware that has more compute capability which is required for additional vision processing. If ford priced their system appropriately, and it worked well, they could also provide a similar upgrade path. They just don't have good leadership and vision right now to realistically monetize vehicles they've already sold with a competent piece of software that adds a solid cost to benefit ratio to customers.

 

[Ajzride]

Just so no one gets confused (this was discussed at depth in the more hardware related thread), it isn't practical for Comma to use the onboard cameras. Every manufacturer uses a different connection style and communication protocol so the Comma hardware would have to be custom for every manufacturer, which is cost prohibitive for them. That doesn't even address having to code the camera reading differently for every manufacturer.

 

[HammaMan]

My projection for more advanced levels of ADAS is going to require a camera to the left of the driver's head but within the wiper coverage to add increased visibility for unprotected left turns. It still needs a centerish camera within the passenger wiper coverage as well. It's not as sleek as a AIO design, but it does make the system safer in such regards (it also provide stereoscopic vision aka distance). This is one reason I don't expect tesla to advance past L3 in their current hardware without the addition of being able to increase unprotected left turn visibility when vehicles are obstructing the view of traffic on streets that have head-on meeting of dedicated left turn lanes without the addition of spacing to offset traffic, particularly if a larger vehicle is completely obstructing any sort of distance viewing.

This image sorta explains it, but any driver with miles under their belt knows the feeling of trying to make an unprotected left and you're trying to peek around vehicles to observe oncoming traffic. A center mounted camera does little when even as a driver you've all but (and maybe do have) your head out the window to gain visibility. Especially if opposing traffic is curved behind the opposite lane's left turning vehicles. We have a few intersections like that around here and it's 'ready-set-hit-it' type of turn. Sure it'd be nice to wave a wand and force common-sense, visibility-enhancing road construction techniques, especially clearly marked lines across the countless municipalities responsible for road construction and markings. Often the poor design comes down to urban sprawl, lack of space for lanes, etc... as areas just outgrow their decadal old original planning and space road space allocation.

Vehicle 2 vehicle communication could solve this as new vehicles come online and they're each drawing a situational awareness picture that can be shared essentially feeding a vehicle trying to make such a turn, the status of other vehicles hidden. Dedicated frequency was allocated for this purpose but nobody ever took advantage of it. It should be an open protocol like wifi. Furthermore a OBD2 port /GPS/Transceiver may be a requirement for older vehicles as an anonymized beacon similar to ADSB transponders giving a small amount of telemetry data to fully autonomous vehicles enter the market. Not so much as a tracking device, but a beacon like trains and emergency vehicles emit to warn of their approach. Though I do fully expect this to morph into a pay-per-mile taxation on all vehicles instead of having the burden fall entirely onto any given municipalities registered vehicle owners / gas taxes. EVs for instance don't pay gas taxes which fund roads, though owners living in an area get a hefty tax on their registrations, but EVs passing through don't pay anything as not even charging has taxes on it. They're a mess here

A bit of a tangent but it's all connected and needs a serious look to make future technologies much easier to implement by having a standard telemetry beacon requirement for all vehicles thus allowing antique vehicles to continue to operate safely on the roadways with autonomous systems, especially semi trucks. Ai powered stop lights would also be able to more easily/quickly route traffic, especially if the antique vehicle systems have basic notifications like "slow down, the light is red" with lights jump instantly to red to allow a truck to pass through an intersection, only to go back to green if warranted. Without such systems, entire roadways will end up being gated to autonomous vehicles only while not requiring dedicated traffic control devices at all.

 

[GMPowerboost]

Hi all, I just got my replacement device as the first one I received had a cracked screen I’m guessing from shipment, and I was curious what settings are the most common changes were to the stock installation of it.

Does Bluepilot recognize stop signs and stop lights? Or is it mostly designed to follow someone else and slow down with them?

And what is enabled in experimental mode (just the reduced nuisance “steering limit exceeded”? (Just what was covered in post #2?)) And is there a reason the camera goes from zoom 1 (in calm mode) to ultra wide view on the screen too (with zoom 1 I cant see the hood, with ultra wide I can see almost to my dash)?

Also is the only message that the comma can push to the F150 dashboard is the “keep hands on steering wheel” even when something else is going on like pay attention or others?

Thanks All! Sorry if I am asking stuff that was already covered in the hardware thread or elsewhere.

 

[Ajzride]

I was curious what settings are the most common changes were to the stock installation of it.

See attachment:

Does Bluepilot recognize stop signs and stop lights? Or is it mostly designed to follow someone else and slow down with them?

if you enable "experimental longitudinal control alpha" and "experimental mode" it will attempt to recognize and stop for red lights and stop signs. It has about a 30% success rate on stop signs and about 60% on red lights.

And what is enabled in experimental mode (just the reduced nuisance “steering limit exceeded”? (Just what was covered in post #2?))

maybe I should have thought more carefully about my naming. the experimental branch of bluepilot is not related to the experimental mode inside the software. Shame on me for not choosing a different name. Right now the experimental branch is almost identical to stable. I plan to start working on the nuisance alarms next week.

And is there a reason the camera goes from zoom 1 (in calm mode) to ultra wide view on the screen too (with zoom 1 I cant see the hood, with ultra wide I can see almost to my dash)?

It's based on the situation. Anytime you are detected near or at an intersection it swaps to wide angle, then goes back to narrow when you leave the intersection.

Also is the only message that the comma can push to the F150 dashboard is the “keep hands on steering wheel” even when something else is going on like pay attention or others?

It also pushes the collision warning alert. Maybe a few others. We can make it push anything we can find in the canbus logs. Just not a high priority right now.

 

[Spiffy]

Some advise needed. I purchased a topdon smart scanner with scope. The smart their best android based scanner. It uses a VCI obd. It's rated for CAN FD.
2/3 of my modules are not available.
I'm starting to suspect the 3x.
I'm plan on unplug the connection at the ipma.
Has anyone else seen issues when trying to access the CAN? I'm at the beginning of my search for the issue.

 

[Ajzride]

Some advise needed. I purchased a topdon smart scanner with scope. The smart their best android based scanner. It uses a VCI obd. It's rated for CAN FD.
2/3 of my modules are not available.
I'm starting to suspect the 3x.
I'm plan on unplug the connection at the ipma.
Has anyone else seen issues when trying to access the CAN? I'm at the beginning of my search for the issue.

Check your battery status too.
but I’m not aware of anyone with a Comma trying to access other modules.

 

[Spiffy]

That's good to know. I have a major cut on my index finger and replacing the truck to stock involves all fingers on deck.
Do you know if non OEM scanners work on CAN FD? My scanner has a VCI that supports J2534, DoIP, CanFD, D-PDU, and RP1210 protocol.
I'm asking all around because I haven't used a non OEM diagnostic tool on this 2022 hybrid. The scanner is shop quality equal to alot of snap-on stuff.
Figures I need company support ant it's the weekend.

 

[HammaMan]

That's good to know. I have a major cut on my index finger and replacing the truck to stock involves all fingers on deck.
Do you know if non OEM scanners work on CAN FD? My scanner has a VCI that supports J2534, DoIP, CanFD, D-PDU, and RP1210 protocol.
I'm asking all around because I haven't used a non OEM diagnostic tool on this 2022 hybrid. The scanner is shop quality equal to alot of snap-on stuff.
Figures I need company support ant it's the weekend.

Are you saying modules on a particular bus are missing or are you expecting to see truck wide module reporting? Here's the modules/busses on my PB

Signals from other modules not queried by any other module on the bus you're connected to aren't going to show up as the GWM isn't relaying them. There's also a discrete bus not connected to the GWM for the hybrid operations. Be careful going hog wild on querying devices as you can request more info than can be delivered, especially at tight intervals. There's several busses for this reason.

Sponsored