Assurance-based Learning-enabled Cyber-Physical Systems

Hazard analysis and assurance cases are well-established approaches for assessing the safety of cyber-physical systems and it is already being extended to address the impact of the use of learning enabled controllers in the system. However, such static assurances cases are not suitable for dynamic situations where the context and the assumptions made while developing the assurance case may be invalid or only partially correct. This requires a dynamic assurance approach, which however is highly non-trivial and needs to consider the composition of various hypotheses about failures in different aspects of the system including the learning enabled component, safety requirements, potential hazard conditions in the operating environment, hazard prevention, and mitigation strategies. This paper introduces ReSonAte which uses the information gathered by hazard analysis and assurance cases to build Bow-Tie Diagrams to model hazard propagation paths and capture their relationships with the state of the system and environment. These Bow-tie diagrams are used to synthesize graphical models that are then used at runtime along with the information gathered from prior incidents about the possible environmental hazards and the hypothesis from failure diagnosers and system runtime monitors to estimate the hazard rates at runtime. These hazard rates are then used to determine the likelihood of unsafe system-level consequences captured in the bow-tie diagram.  We implement ReSonAte for an autonomous vehicle example in the CARLA simulator and through comprehensive simulations across 600 executions we show that there is a strong correlation between our risk estimates and eventual vehicular collisions. Our approach outperforms static risk estimates produced from the assurance cases. Also, the Bow-Tie based risk calculations on an average take 0.3 milliseconds at runtime in addition to the overhead introduced by the assurance monitors and runtime detectors used to identify various anomalies.