In this project, we design a novel architecture for sandboxing unverified controllers (e.g., deep neural network or black-box controllers from third parties) that uses a Safety Advisor and a Supervisor (Safe-visor). On one hand, Safety Advisor only focuses on the safety of the system, which should be treated as a fallback in case the unverified controllers are trying to perform some harmful actions. On the other hand, the unverified controller is designed for functionality, i.e. it is expected to realize some tasks which are much more complicated than purely keeping the system safe. To ensure a specific level of safety probability of the physical system, the supervisor specifies verifiable safety rules for the unverified controller to follow. The control inputs of the controller fed to the system are checked and can only be accepted when they are not disobeying the safety rules defined in the sandboxing mechanism. In general, the Safe-visor architecture can be used to sandbox any types of unverified controllers at run-time in order to guarantee the safety of the physical system. By sandboxing the unverified controller, we are able to exploit its advantages for realizing complex tasks while preventing the system from being threatened by its harmful behaviour, if any.

Active members for this topic

Related papers