Uber ATG | Test Directorate
Increase Throughput to Autonomy Software
Uber's Advanced Technology Group (ATG) had a large Test Directorate with three senior managers. The group was already pivoting to more simulation for Test.
But our Test Track was a limited resource, COVID-19 was impacting our ability to work in proximity, our organizations ​had evolved without a plan, and we weren't aligned to the new Systems Engineering Directorate.
​
The assignment was to step in as the Director of Test, identify every waste, bottleneck, and opportunity possible, and get 10X the capacity out of Test within months.
​
​
Align Test to Match Systems
Director of Test (L7)
At Uber’s Advanced Technology Group (ATG), I worked for the Head of Systems Engineering & Test.
In this second part of a two-part assignment, I would work to restructure and streamline Uber ATG's Test Directorate to align with the newly formed Systems Engineering & Tooling Directorate I just finished standing up. The Test Directorate consisted of:
-
Development Test - R&D, Prototypes, New Feature Testing, Interactive Work with Development
-
Structured Test - Offline Simulation, Triage, Test Plans,
-
Test Operations - Track and Public Roads, Operational Safety, Business Intelligence
​
As the Director of Test, I would reorganize the Testing Directorate so that it could align with the work I just completed as the Director of Systems Engineering & Tooling. My primary goal was to set up the three Test groups to operate at scale - and with sufficient velocity - to meet 2021 demands. Initial challenges included a limited track resource, unclear spans of control, lack of testing to formal Systems and Subsystems requirements, and operating in a Covid-19 constrained environment.
​
With three very capable senior managers at hand, this assignment would involve mentoring those managers and their subordinates, pathfinding to solve their common challenges, streamlining the testing flow, and improving our testing approaches to give better answers to Autonomy, Hardware, Systems, and other stakeholders. Most importantly, I worked with the managers to institute new paradigms - beyond simple scenario-based testing - that would improve the fidelity and robustness of the test results in edge cases and complex situations.
​
Steps I personally took:
UNDERSTAND THE AS-IS
-
Extensively interviewed personnel at every level to build up a profile of operational capabilities, measurements, and challenges
-
Engaged in both strategic and tactical meetings to watch how processes execute or struggle
-
Worked with teams to document process flows and Pareto excessive handoffs and wastes
-
Directed the Business Intelligence team to instrument processes with data from Jira ticketing, Jama requirements, and internal tooling data to identify bottlenecks, turnbacks, and resource limitations
-
Evaluated what was performed at track that could be done earlier/later to preserve resource
REORGANIZE THE TEAM
-
Reorganized the Test Directorate over several months working closely with the top managers of DTE, STE, and VO, laying out new structures for Subsystem Test, System Test, and Vehicle Test, aligned with the new Systems Engineering
​
BUILD INFRASTRUCTURE
-
Led negotiations and licensed University of Michigan's Mcity automated track source code
-
Identified core weaknesses between Product planning and Technology readiness
-
Launched and led Project Framework to define and manage AV technology tracks, tech roadmaps, and TRL measures
-
Worked with Platform Triage to restore Fault Management System capability
-
Expanded Business Intelligence measures to the entire Test Directorate
​
Results Matter
At the 12-month mark, the Test Directorate had undergone a tremendous metamorphosis.
-
The testing paradigm had shifted from testing narrowly defined and brittle scenarios - to testing based on systems and subsystems requirements that would persist across any scenario or combination of influences.
-
The three senior managers had agreed to a new testing flow, had agreed to ownership each piece of the Test Directorate (RACI matrix), and had agreed to where people needed to move to accomplish the next years objectives.
-
The Software and Hardware teams were seeing end to end linkages from Systems through Design to Testing for the first time.
-
My new Continuous Improvement program, "Velocity", had taken hold. It was generating both solutions to known barriers, and esprit de corps amongst the entire Systems Engineering & Test group.
​
​