My working life was mainly on Safety Critical Software Intensive Systems, however I am now active on a broader perspective of topics, as shown by the items now covered by this web-site.
As Low As Reasonably Practical (ALARP)
In the context of ALARP, the meaning of 'reasonable' is generally taken as being a financial balance, i.e. is the financial cost of any improvement is more than the gain from the reduction in risk. However it can be difficult to produce a quantitative argument, even for apparently straight forward cases. For example, the apparently simple justification for having two engines rather than three on an aircraft on an ALARP basis can become complex. The extra cost of the third engine, together with the cost of fitting it (aircraft structure, wiring, interfacing etc.) is relatively straight forward. The running cost (maintenance, extra weight/fuel) over the life of the aircraft is a bit more complex, bearing in mind that there is a cost due to reduced aircraft availability, assuming take-off is only allowed with all three engines fully operational. This extra cost could be absorbed by the aircraft manufacturer, which would have an effect on their profitability, or more likely passed on to the customer, possibly having an effect on their sales; either way the manufacturer may be less competitive than a rival company who were more 'creative' with their analysis.
On the 'benefit' side of the cost/benefit equation, the reduction in risk is more straight forward, the probability of having all three engines fail at the same time is less than with two; the consequence is the same, i.e. loss of aircraft. Typically it would be the aircraft operator (or their insurers) who would have to cover the cost of loss of an aircraft plus any compensation claims by victims' families. In theory the airline operators should be the driving force for safer aircraft, whereas in reality it rests more with the manufacturer and the regulatory authorities.
To try to apply the concept of ALARP to safety critical software is even more problematic. The cost of spending more man-hours on testing or reviews and the cost of analysis tools can be easily measured, but the benefit is far harder to determine. The software may be better in terms of fewer 'bugs', but the probability of reduced failure far harder to determine. Metrics can be gathered during the development process and with time, over numerous projects, some qualitative feel can be obtained, but it is more of an art than science.
Acceptable
Another approach to determining what is 'reasonable' is the various figures set by regulatory authorities within different industries as to what is considered to be an “acceptable” loss rate. So for civil aviation the figure for an aircraft loss resulting in multiple fatalities is a probability of 10**-9 per flight hour. The comparable figure for military aircraft is 10**-7, but as the fatalities as the result of the crash of a military aircraft is generally in single figures, whereas the fatalities for a civil aircraft are generally 100's, the acceptable probabilities are consistent. However, other areas where there is a risk of loss of live do not match. For example, the figures used by the Dutch nuclear safety authority is according to EUR20055, page 59, “the probability of an accident in which at least 10 people suffer acute death is restricted to a level of 10** -5 per year. If the number of fatalities increases by the factor of n, the probability should decrease by a factor of n**2” . Assuming the risk of an accident is 24 hours, 365 days per annum, the probability per hour becomes 10**-9 for single figure number of fatalities. For a fatality of up to 300, their figure for acceptable probability would be 10**-12. As an example of what is considered 'acceptable' risk at the other end of the spectrum a road crash fatality is about 10**-6, about an order of magnitude more dangerous than for a military aircraft!
This lack of consistency as to what we think is acceptable risk to human life is bothersome, to say the least. It may partially explain why safety standards in certain industries have not improved for years because the current (i.e. old) criteria is still considered to be 'acceptable'.
About Me
Terry has over 30 years experience in the software industry, operating as a consultant since 1982. His experience is mainly on real time systems such as autonomous control, navigation and radar systems. He has both developed and assessed software intensive safety critical systems, and held senior positions in both roles. Having an appreciation of various military and civil standards, both working to and assessing against them, he has been able to provide consultancy support to a number of projects on various issues regarding the implementation of critical software systems. Recently he joined the DO-178B update Working Group and took part in the public review of ECSS (ESA) standards.
Most recent major project was consultant support on the Eurofighter Typhoon project. Performed the roles of Air Vehicle Systems (AVS) Safety Assessment Team Leader and also Software Safety Co-ordinator. The AVS team consisted of 7 assessors covering general aircraft systems such as Fuel (including C of G monitoring), Landing Gear and Life Support. The Software Co-ordinator task was to establish a consistent approach to safety across the 10+ Risk Class 1 software systems including Flight Control and Display systems. The aim was to have an ‘integrated’ safety case for a system (aircraft) of diverse systems (avionic sub-systems). The sub-systems were incrementally developed over a fairly extensive (10+ year) period according to a comprehensive set of Eurofighter’s own standards, originally based around standards such as DO178A. Part of the assessors’ task was to ensure safety arguments were still valid according to what was now considered ‘best practice’, such as Def Stan 00-54. Also had the role of Lead Assessor on the Fuel Computer software (SPARK Ada 83 and Motorola 68000 assembler).
Links and Refs
EUR20055: 25 Years of Community Activities towards Harmonisation of Nuclear Safety Criteria and Requirements - Achievements and Prospects