Software Risk Analysis

[Pyrotex]

SOFTWARE PROBABILISTIC RISK ANALYSIS (SPRA)

I have finally determined what my new job is all about. They want me to come up with a way of assigning a probability to a module of software--the probability that it will execute in such a way that loss of assets will occur (LOA).

This isn't Quality Assurance. This isn't Software Reliability Analysis. This isn't Software Verification and Validation. So, don't go there!

You may ask, "what's the difference between SW PRA and SQA, SRA, SVV?" Excellect question. When I have a good answer, I'll get back to you. But for the nonce, just accept that it is a different animal altogether.

--What is the probability that LOA will occur because of SW?--

Hardware PRA is fairly straightforward, and I've been learning a lot about that lately. But software PRA is... is... currently undefined. The best I have done so far is build the following metaphor:

The "mission" takes place in "TEFO", a 4-dimensional space, with dimensions of time, environment, function and (human) operation. Knowing what you want to do, you carve out a cavern of "nominality" (nominal, expected). Knowing what you want the mission to accomplish, you settle on regions within TEFO which define the expected Environment the mission will deal with (high G's, high speeds, etc), the Functions the mission will have to perform (takeoff, navigation, communication, etc), the human Operations the people (including crew) involved will execute -- and all as a function of Time. Within TEFO, we carve out a cavern that makes room for the mission.

Within the TEFO cavern, we assemble our Requirements--like a scaffolding, that defines the outer limits of our mission. Hung from that scaffolding, the actual mission is constructed out of hardware, software and people. Within the volume of the actual mission itself, we define a smaller volume--let's call it the 99% envelope--and this defines the boundaries of all our testing and verification. So, SQA, SRA and SVV all apply to that portion of the mission within the 99% envelope.

A "mission" would be the actual flight. At any given instant in time, there is a point in the TEFO that defines the "mission event state". This point traces out the "mission event timeline" which starts at say, ignition, and goes all the way to say, landing.

--What is the probability that LOA will occur because of SW?--

We can assume the SQA, SRA and SVV can (mostly) eliminate flaws and bugs within the 99% envelope.

But SW PRA, as I currently understand it, addresses the probability that the mission event timeline will drift out of the 99% envelope, maybe even outside the requirements envelope--AND result in LOA--because of a failure of the software to adequately respond to these boundary edge conditions.

NOTE: The software doesn't have to FAIL, necessarily. It could be doing exactly what it was designed to do. Remember, there are other dimensions: the Environment and the Operations of human beings.

THE BIG Q: Does anyone out there know of resources, people, books, projects or have personal experience applicable to this problem???

In the meantime, I will use this thread as kind of a blog to record whatever I find out.

[freeztar]

Originally Posted by Pyrotex

What is the probability that LOM or LOC will occur because of SW?--

We can assume the SQA, SRA and SVV can (mostly) eliminate flaws and bugs within the 99% envelope.

But SPRA, as I currently understand it, addresses the probability that the mission event timeline will drift out of the 99% envelope, maybe even outside the requirements envelope--AND result in LOM or LOC--because of a failure of the software to adequately respond to these boundary edge condtions.

NOTE: The software doesn't have to FAIL, necessarily. It could be doing exactly what it was designed to do. Remember, there are other dimensions: the Environment and the Operations of human beings.

Excellent question Pyro, but I'm having trouble understanding the differentiation of SPRA from the collective others. I'm sure the answer is in the last paragraph I quoted above, but it's a bit elusive still. Can you give a practical example using, perhaps, the "Operations of human beings" dimension?

[Buffy]

I know this from nothing...

But if you want to gain some legitimacy, it may be worthwhile to look at the "accepted" approach for hardware, to wit: the basic idea on the hardware side is you have a countable number of components and a mean-time-before-failure and a system analysis that takes into account criticality and redundancy to determine time before system failure (ignoring whether its mission or crew for the moment: that's an implementation detail!).

It would appear that "software is completely different," but I'll argue that its not: "components" in software are "program modules" and while its hard to assess specific "MTBF" to a specific module since its one of a kind, what you can do is pull out one of those weird computer science concepts that not many people pay attention to any more: the good ol' "function point."

While its been a long time since I've gone near them, I'm pretty sure you can scare up statistics on "MTBF-per-function point" with appropriate analysis of the types of FPs and so on and end up using a methodology that's exactly like what's done on the hardware side.

Now if I were actually an astronaut, I'd be scared to death of this kind of "analysis," ("That's a bunch of crap," is what Gus Grissom would probably say) but its probably just as justifiable as the hardware stuff is...

You pulled it out of where?
Buffy

[Pyrotex]

Thanks dudes and dudettes.

The most famous incident where SW killed someone was the Therac 25.
A software error passed through exhaustive testing, and never caused any problems until a rare sequence of operations enabled it.

SW PRA is different from the other analyses because the output of SW PRA (just like HW PRA) is to assign a number -- the probability that that particular piece of SW will fail or enable a failure, and cause damage. It's not trying to fix the SW or even test it. It's not a methodology of creating good SW. It's not even about FINDING bugs or flaws.

What is the probability (per mission or per hour of operation) that software will cause hurt or damage? That's it. And maybe, what is the probable number of serious bugs still existing?

[freeztar]

Originally Posted by Pyrotex

The most famous incident where SW killed someone was the Therac 25.
A software error passed through exhaustive testing, and never caused any problems until a rare sequence of operations enabled it.

Great link!

SPRA is different from the other analyses because the output of SPRA (just like HW PRA) is to assign a number -- the probability that that particular piece of SW will fail or enable a failure, and cause damage. It's not trying to fix the SW or even test it. It's not a methodology of creating good SW. It's not even about FINDING bugs or flaws.

What is the probability (per mission or per hour of operation) that software will cause hurt or damage? That's it.

I understand now, at least partially.

[CraigD]

How about this approach?

1. Write a simulation of the spacecraft, human operators, and environment in which it operates
2. Install a particular collection of software in the simulation
3. Run the simulation many (N) times for a particular mission or collection of missions
4. Count the number (F) of runs that result LOM/LOC ;
5. The estimated probability of LOM/LOC for any reason is P=N/F
6. Vary the installed software by replacing a single component with new component C.
7. Repeat steps 3-5. Assuming P is low, the estimated probability of C having caused the failure is

It’s a completely empirical approach – nothing about the software must be known. You can test completely bad or incompatible components. If they are critical, will be nearly 1.

[Pyrotex]

Originally Posted by CraigD

How about this approach?

1. Write a simulation of the spacecraft, human operators, and environment in which it operates
2. Install a particular collection of software in the simulation
   ...

It’s a completely empirical approach – nothing about the software must be known. ....

Thanks for the suggestion. It seems well thought out. But maybe here is a clue. My "bosses" have tried and failed to come up with a way of assigning risk to software (SW).

Your approach and Buffy's approach to hardware (HW) risk analysis has been going on since at least the 1960's--nearly 50 years, maybe more. The empirical approach. Put 10,000 widgets in the fottasite, and run for 1,000 hours. 2,081 widgets failed. Probability of failure per hour is 2,081/10,000,000 or 0.00021 or 0.021%.

If the widget has to run for 50 hours during a mission, then Total Probability of failure is about 1 in 100.

But when you try to do SW that way, you can download 10,000 copies of it into a computer, and the computer HW may fail, but every copy of the SW will behave the same. They may all fail. They may all never fail. They may all fail once every 7 hours for unknown reasons. But there's no difference among identical copies of 1s and 0s, given that they are subjected to the same test. The empirical approach produces bad, nonsensical, untrustworthy or uninterpretable results.

My bosses have committed that they will, for the first time ever, come up with a way of assigning a probability of failure to SW. So far, it appears to me that they have tried and failed. The ball now appears to me to be in my court.

Now, my data may be incomplete as hell or just dead wrong. In case anybody is reading this post, I hereby disavow everything I have said so far. But I am left with the conclusion that SW PRA is not easy and not straightforward.

[Buffy]

Originally Posted by Pyrotex

But when you try to do SW that way, you can download 10,000 copies of it into a computer, and the computer HW may fail, but every copy of the SW will behave the same.

Sure, and that's why counting function points and comparing it to a *huge* sample of software that's all different is such a useful thing: Since software can be copied perfectly, there's no point in testing multiple copies, but its perfectly reasonable to assume that your average programmer will produce an average of x bugs per function point of each type. Each program will have a unique combination of number and type of function points, so you can use that to determine the number of expected bugs, and their severity.

I know this has been going on for a long time, and the question is why the response to the people who are doing it is still "I dunno"... The development managers I know have been keeping statistics like this (usually in lines of code rather than FPs) for ages just to rate programmers for when promotion/layoff time comes. Even if there were more complexities than I've mentioned here (and I know there are!), there's got to be enough data to start to give "fuzzy" numbers that are probably no less "justifiable" than the "test 2000 vacuum tubes" method...

Originally Posted by Pyrotex

But I am left with the conclusion that SW PRA is not easy and not straightforward.

No disagreement there!

Tell me you're really doing this just so they let you do Hypography at work...

Work expands to fill the available time,
Buffy

[Pyrotex]

Originally Posted by Buffy

...Tell me you're really doing this just so they let you do Hypography at work... ...

They're really doing this just so I can do Hypography at work.

[CraigD]

Originally Posted by Pyrotex

But I am left with the conclusion that SW PRA is not easy and not straightforward.

Confidence bolstered by the failure of the almighty google search engine to identify S[W][ ]PRA with anything more appropriate than “software project risk analysis”, “solid waste planning”, or (my favorite) the Scottish Plastic & Rubber Association, I’m left with the suspicion that SPRA is especially difficult because it’s a discipline you NASA folk are making up independently of the larger world – claims of its 20 year antiquity notwithstanding.

Not to disparage the invention of acronyms and disciplines to accompany them (some of the high points of my career have involved the invention of acyronyms ) but a lot of definition will be necessary before anybody can understand or contribute much in detail to any conversation about it.

The number and scope of questions that spring to mind in an effort to create this definition overwhelm my ability to render them very coherently, so I’ll just let fly with one:

Does the software failure have to be a “sin of commission or omission” – eg: turning on/off or failing to turn on/off some piece of the spacecraft at a critical moment such that all goes boom – or are “sins of ignorance” – eg; not detecting and alarming or reacting to an anomalous situation – also grounds to conclude that failure is due to software? If this is the case, arguable any failure can be cast as a failure of software: for example, the 1986 Challenger disaster could have been made survivable had sensors and software detected and reacted to the SRB abnormalities by separating the orbiter from them prior to the explosion.

If this is not the case, what is to prevent the probability of failure due to software from being made effective zero by not having any software, even though such a vehicle would almost certainly have much lower performance and much greater probability of failure due to non-software causes?