Scaling Cognitive Systems Engineering with the Brittleness Audit Tool (BAT)

Problem:
Resilient Cognitive Solutions (RCS) has long excelled in applying Cognitive Systems Engineering (CSE) to improve human-machine team designs, with methods like functional modeling and decision-centered testing delivering significant real-world impacts. However, scaling these insights across large enterprises with numerous products and workflows posed a challenge. While CSE methods are effective, they can be complex and unfamiliar to many clients, particularly when applied to an entire portfolio rather than individual systems. RCS needed a way to extend their expertise and address latent design flaws across an organization without overwhelming clients.

Solution:
To address this challenge, RCS developed a process called the “Brittleness Audit” and an accompanying tool, the Brittleness Audit Tool (BAT). The concept of “brittleness” is borrowed from metallurgy, referring to hidden design flaws that may not be obvious under normal conditions but can lead to failure under stress or anomalous situations.

I led the design and development of the Brittleness Audit Tool and the creation of the tradecraft process that supports its use. Starting with initial design concepts and a loosely defined methodology, we iterated through multiple design cycles, testing the process on real workflows to refine the tool. Our goal was to empower users to map out workflows within their domains—including roles, tools, and tasks—and then answer targeted questions that help identify potential brittleness within those workflows.

Once brittleness is identified, the tool provides insights into the root causes of these flaws and offers recommendations for how to address them, ensuring that human-machine teams can perform effectively even under pressure.

Outcome:
The Brittleness Audit Tool (BAT) and its tradecraft now allow organizations to scale CSE principles across large portfolios, helping users identify and resolve latent flaws in their workflows. By empowering clients to proactively address potential issues, the BAT has enabled clients to ideate more resilient and robust designs for human-machine teams, preventing failures before they occur and enhancing overall performance under real-world conditions. This project marks a significant step in making CSE expertise more accessible and scalable for large enterprises.