​​Building a safety culture

How Kodiak’s operations program ensures safety.

I first started working on autonomous vehicle operations in 2013. Since then, I’ve operated self-driving vehicles on roadways across the country: in California, Nevada, Arizona, Texas, and Pennsylvania. In that time, I’ve come to realize that test vehicle operations are the crux of an AV company’s safety culture. The Kodiak Safety Report gives a great introduction to Kodiak’s operations program: given my long experience in the industry, I wanted to add to our story.

As Kodiak’s Truck Operations Manager, I’m responsible for ensuring that we get the most efficiency from our testing while maximizing the program’s overall safety. When I started at Kodiak, I wanted to leverage all of my experience to help build a fundamentally safe and effective operations program. In my experience, safety must be everyone’s responsibility, from leadership to engineers to operations. Even though operational planning and execution is often an afterthought in engineering-driven self-driving companies, running a safe testing program is the key to safe development.

A Kodiak driving team on the road

Our people

We believe safe operations start with a great team. Like most companies in the industry, Kodiak tests its vehicles with two-person teams:

• Safety Drivers are Commercial Drivers License-holding truck drivers with industry experience, who are responsible for monitoring the road and the behavior of the vehicle. Safety Drivers are also responsible for driving the vehicle anytime we are outside of our Operational Design Domain, the highway environment where the Kodiak Driver is designed to operate.
• System Operators are operations engineers monitor the operation of the Kodiak Driver during test runs, and communicate its intentions to the Safety Driver.

Because these roles are so critical, we adhere to strict hiring standards for both positions, and only hire a small percentage of the people that meet our requirements. All our Safety Drivers have at least three years of commercial driving experience, a sterling safety record, and must pass a pre-employment drug and background check. We also think it’s important that all Safety Drivers and System Operators are Kodiak employees — not contractors. This helps us be sure that our most safety-critical people are well-aligned with our company values.

Kodiak Safety Driver and System Operator training

Once hired, we put both Safety Drivers and System Operators through an extensive training program, designed to introduce them to the Kodiak Driver technology, general operation of the truck, and Kodiak’s safety culture. We take pride in how our training allows drivers and operators to seamlessly work together and communicate as a team.

Kodiak’s inclusive safety culture

At Kodiak, we’ve worked hard to build a strong and inclusive culture that prioritizes safety since even before our first test run. Communications between operations and engineering is critical: engineering and operations cannot be siloed from one another during any part of the development process. Operations must have clear expectations regarding safety requirements, and Engineering must carefully consider how operations fit into the development process. Meticulous planning goes into each step of the development cycle, from feature concept to on-board implementation, so that a lens of safety can be applied to every aspect of daily operation.

We also make use of our close Engineering/Operations relationship during our software release process. We require that engineering and operations teams scrutinize any code that goes on the vehicle before we begin real-world testing. Engineers flag pieces of code that they believe may have an operational impact, so the operations team can review the flagged code during our daily briefings. During a briefing, we make sure that we understand the specifics about what we are testing, and that each driving team understands the goals of the mission. Our Safety Drivers and System Operators are encouraged to talk with the engineer(s) who wrote the code one-on-one to gain better clarity.

The Operations team also participates in the regular Safety Meeting, an open forum where a cross-functional group of people from hardware, software, systems, software quality and driving operations discuss topics related to on-road safety. At the Safety Meeting, team members discuss what we are doing operationally, and brainstorm how we can do things better. Each sync ends with a risk reflection that allows us to address even minute issues that may bubble up.

Our processes

In addition to our great people and our safety culture, we have built specific policies and procedures to ensure we are following safety best practices. Our policies set clear expectations around what is required from vehicle operations, including standards for safe manual operation, best practices for when to disengage the system, and where drivers should keep their hands when the system is engaged.

Operators are held to high safety-critical standards. To enforce policies in the truck, we employ a sophisticated Driver Monitoring System that uses interior-facing cameras and AI-based algorithms to automatically detect and notify a driver when he or she may be drowsy or distracted. Drivers know that certain violations, such as touching a cell phone while driving, lead to immediate dismissal. These policies are communicated throughout Kodiak, to make sure that anyone who steps inside our truck understands our operations best practices.

Pre-trip inspection

Also critical to safety is keeping our trucks well-maintained. We have created clear processes for how often we check both our trucking platform and the modifications we make to install the Kodiak Driver — these processes go well-beyond what the Federal Motor Carrier Safety Administration mandates. Safety drivers conduct comprehensive pre-trip inspections every time they take a truck onto a test track or public road. This inspection includes both the components required by a traditional Pre-Trip inspection, such as brakes, fluid levels, and tire pressure, as well as a close inspection of the vehicle’s sensors, their connectors, and their mounts. Safety Drivers also inspect the truck and trailer every time the truck stops — at meal times, refueling stops, and rest breaks. In addition to pre-trip inspections, Kodiak technicians also conduct in-depth weekly, monthly, and quarterly inspections to both the truck and our AV equipment.

Once on the road, driving teams report any notable events they experience — nothing goes unaccounted for or swept under the rug. Safety Drivers use their extensive training to anticipate software behavior, and System Operators take careful notes on system operations to help give context to our engineering teams. Any issues related to system performance get included in an end of run report or, if safety critical, are escalated in real time to fleet managers immediately. Lastly, Engineering team members are encouraged to go on regular rides, to experience how the vehicle is evolving week-to-week.

After every test run, the Kodiak Driver’s performance is reviewed by the operations team and also analyzed event-for-event by our Software Triage team. Each week a cross-functional team of folks from software, systems, triage, product, and operations discuss the weeks on road operations and vehicle performance.

Kodiak’s human/machine interface

Kodiak’s focus on operational safety extends to how we build our self-driving trucks. Our engineering and operations teams have collaborated to design the Kodiak Driver so it first-and-foremost works with and for people. The technology must allow Safety Drivers to always remain in control of the technology, until the day we can demonstrate the tech is roadworthy on its own.

Dashboard indicator light

As such, we designed our Human Machine Interface (HMI) to be easy to understand, while minimizing or eliminating the distractions it offers to the Safety Driver. A simple, easy-to-understand dashboard light communicates the state of the system: Manually Driven/Not ready to engage, Manually Driven/Ready to Engage, Self-Driving, and Fallback. Each system state transition also has a unique audio cue, so the Safety Driver can manage system state transitions without taking his or her eyes off the road.

We also designed our hardware interface so that the Safety Driver can always disengage the self-driving system and retake control of the truck at any time. In addition to manual disengagements, the Kodiak Driver also automatically disengages when it encounters a scenario that is outside its ODD, and begins bringing the truck to a safe stop until the Safety Driver retakes control of the vehicle.

System Operator Visualizer

We’ve put just as much thought into the Visualizer, the interface System Operators use to monitor what the system’s sensors are detecting and how the system intends to react. The Visualizer allows System Operators to ensure that the perception system isn’t missing objects on the road, and that the planner intends to act safely and reasonably. For example, if the System Operator sees a vehicle up ahead that the system has not yet identified, he or she can give the Safety Driver advanced warning, so the Safety Driver can prepare for a potential intervention. We even use a screen protector, so that the Visualizer is not visible to Safety Drivers and can’t serve as a distraction.

What comes next

As Kodiak continues to grow, our operations program will continue to evolve. I’m eager to think about new problems we will have to solve as our tech begins to mature and our operations begin to shift. As the Kodiak Driver gets better and requires less human intervention, we will need to find ways to keep our Safety Drivers and System Operators engaged. As we begin to grow our fleet and take on more customers, we will begin to evaluate new Operational Design Domains and hire new Operations teams in new locales. Despite all this change, I know we will continue to keep safety first and always.