Uber ATG | Systems Directorate
Stand-Up Systems Engineering for Self Driving Vehicles
Uber's Advanced Technology Group (ATG) experienced a fatal accident in Tempe, Arizona in March of 2018. One of the outcomes of the investigation was the need for a strong Systems Engineering capability.
​
Nearly two years later, three attempts to stand up a proper Systems Engineering Directorate had failed.
My first assignment as Director of Systems Engineering & Tooling was to build out our Systems Engineering capacity and increase the flow of formally managed requirements to the Autonomy team.
​
By the time I moved to my next assignment five months later, we had ingested over 10,000 requirements and were continuing to refine and generate more.
Broad Mandate & Authority
Directorate Level (L7) Role
In this first part of a two-part assignment, I would stand up a formal Systems Engineering capability.
My responsibility was to help the Head of Systems Engineering & Test to build and integrate a new Systems Engineering Directorate into the existing Product, Safety, Autonomy Software, Program Management, and Quality teams, and prepare Systems Engineering for a full integration with the Test Directorate.
​
As the Director of Systems Engineering & Tooling, I would have full authority to reshape and build the team, tools, and processes required for success in preparation for the selection of and onboarding of a new Director.
Systems Engineering was struggling mightily. Team members were:
​
Spread across multiple floors of each building
Spread across multiple buildings on each campus, and
Spread across multiple campuses of the US
​
Not only were they physically separated, but they were philosophically separated. Nobody had established how Systems Engineering (SE) would be defined at Uber ATG. While many extremely capable engineers were available, they weren't all coming from the same industries or using the same technical language. And most of the team had one foot in SE, and the other in one of several old roles they used to support.
​
More importantly, key functions of System Engineering, such as centralizing and managing requirements, tracing requirements to each other, to implementations (code/hardware), and to tests, or performing Model Based Systems Engineering (MBSE), weren't yet a part of the broader culture at ATG.
​
This task was one of the bigger challenges of my career. Focus the team on the critical tasks, build the infrastructure required for them to be successful, hire on additional talent to carry the load, begin integrating SE into the existing culture at ATG that in large part didn't understand SE, and prepare for further integration with the Test Directorate later in 2020.
Physical Unity
SE personnel were scattered across multiple floors of multiple buildings. They were continually drawn into their "old" jobs by mere proximity, and seasoned SEs weren't sitting with the new recruits.
​
My first act was to negotiate a single space where all of SE could sit together and support each other, and work near the Product and Safety groups, our key partners.
Philosophical Unity
Some of our engineers had never worked formal SE, and the only requirements they had ever written were for Scenario based tests (not the product itself). Some had Aerospace, Automotive, or INCOSE knowledge, but would talk past each other without common and ATG specific definitions.
​
My second act was to pull the entire SE team (all sites) into one room for a few days and baseline them on SE and my expectations.
Identify Top Talent and Reorg
The team was missing command and control structures. Generic "pods" could be put on any task, but nobody formally owned any piece of the design. It was frustrating the experienced SEs and confusing the new grads.
My third act was to interview every team member, evaluate their skills, and then reorganize the team into to functional areas of responsibility with clear leads to manage different aspects of the design.
Standardize & Mandate a Requirements Database
Informal requirements were stored in all manner of formats across over 50 locations. Requirements weren't traced, or even formally reviewed. More importantly, Systems Engineering didn't own the requirements!
​
I quickly mandated JAMA as the only legitimate database for requirements, and that all requirements be migrated to JAMA and be deprecated elsewhere. Over 10,000 requirements were harvested!
Standardize and Staff MBSE
Soon after my arrival, several engineers approached me regarding Model Based Systems Engineering (MBSE). They had tried to get system modeling going, but they kept being pulled from the work for other "higher" priorities.
​
They had me at "MBSE". There is no way we can manually identify all the system's iterations and interactions. We needed scalable solutions. I assigned three engineers to begin immediately.
Formalize Requirements Reviews
With over 10,000 newly harvested requirements, the work was just beginning. Duplicates upon duplicates existed, with slight variations of wording that could be critical differences in the end design. Deduplicating and organizing would take months. But to what standards would we review?
​
I created a Requirements Quality Assurance standard and implemented in our formal processes. All requirements would be automatically reviewed in tooling where possible, and manually reviewed for everything else.
Formally Instantiate the V Model
While several engineers had tried to champion the V Model for Systems Engineering & Test, it had never been instantiated specifically for the ATG environment and product line.
​
I created a V Model for the Enterprise (Uber Rides + Self Driving Vehicle + AV System), a second one for the Self Driving Vehicle, and a third for the AV system, and then trained the entire organization how to select, discuss, and plan for each.
Systems Integration with Product and Safety
Product and Safety had each taken on roles that extended into the Systems Engineering area in the absence of a formal Systems Engineering team. But we couldn't just transfer functions; SE had to be built up first!
​
Over the 5 months I build out SE, our team took increasing load off of Product and Safety as we came up the curve on formal requirements management, system modeling, traceability, and system architecture. This required (sometimes painful) changes to everyone's RACI.
Systems Integration with Test
As ATG's Software group grew, so did the number of scenario based tests. But the tests were too brittle. What happened on the track under tight controls wasn't telling us enough about probable behaviors on public roads. What would happen when scenarios came together in complex, real-world environments?
​
I began working with the top Test managers of Development Test, Structured Test, and Operational Performance Testing, to create a flow of requirements that would
Systems Integration with Software
The Product team was accustomed to taking their visions directly to the Software Engineering group. But as the complexity of the systems, subsystems, and software interactions increased, a lack of Systems Architecture was killing morale. What does "good enough" look like?
​
Over the course of 2020, Systems began to define and pass thousands of meaningful requirements to Software, and help Test build meaningful Systems and Subsystems testing.
Systems Integration with Partners
Uber ATG started with a Suburban and a bunch of sensors on a steel rack! But as new partnerships and investors came on board, and vehicle platforms evolved, more sophisticated AV/Platform integration was required.
​
In 2020 I applied dedicated Systems Engineers to be the full-time liaison to both Volvo, our core platform partner, and Toyota, a second platform. The goal was to ensure all interfaces were well managed, and that next-generation platforms would meet our redundancy needs.
Training the Org on SE
Incorporating formal Systems Engineering is one of the most difficult growing pains and cultural shifts ATG had to go through. Some aspects of "design authority" and "architecture" were moving. Many of the SE terms and paradigms were unfamiliar to employees with a purely software company focus.
​
During the first 5 months of my assignment, and then continuing on as I stitched together Systems outputs as Test inputs from the Test side, I created and presented training, RFC's, and All Hands briefs to drive the transition and help all stakeholders understand what was changing, and why.
Results Matter
At 5-month mark, I was able to successfully do a warm handoff to the incoming Director with:
​
-
A logically and physically organized staff
-
10,000 formally managed requirements in the JAMA database
-
Detailed traceability, modeling, ticketing, and tooling all well underway
-
The broader organization part-way through the transition from many R&D activities to a formal Systems Architecture and Systems Requirements
​
My successor would still have a lot of work ahead of him for an organizational change of this magnitude. But he had been set up for success in every possible way. The real benefits of the new Systems Engineering & Tooling Directorate wouldn't be fully realized until the Test Directorate could be realigned to take advantage of these changes, some 5 months down the road.
​
And so began my next task as the Director of Test, well positioned and partnered with the new Director of Systems Engineering to start the process of aligning the Test group to the new Systems group and consolidate and solidify all the gains for Autonomy and the rest of the Software group.