SOFTRELY - WEB v1.1
HELP

1) Introduction
The reliability of a software is its ability to perform a required function under given conditions for a given time interval, but itÆs also the probability that it will operate without failure under given conditions for a given time interval.
To evaluate the reliability of a software, we have to do some measurements, and solve a predictio problem : how is the software likely to behave from now, given the past record of failures.
A few metrics must be defined to analyse software reliability. The reliability is, as we said above, the probability that the software will execute without failure in a given environmment for a given period of time. The mean time to failure or MTTF is the time which is expected to elapse between the current time and the next failure. The median is the point of statistical distribution that a given quantity is equally likely to fall either side of. The ROCOF is the current rate at which failures are occuring. To analyse the trend of the failures, we use the LAPLACE test.
The uncertainty comes from two factors, the operational environment and the effect of fault removals. Models in software reliability are trying to model these two types of uncertainty.

2) Tools
Dramatic disagreements between these models on the same source can be observed sometimes. On one hand there is no universally accurate model, on the other hand some of them are sometimes allright. We have no way to select a model a priori, so that we need tools to measure the accuracy of the models on a given dataset. If we have the data t1, ..., ti-1, we want to predict the not yet observed Ti. The true distribution of Ti is Fi(t), but the distribution we compute in the model is . Knowing (ti) we want to know if the model was accurate on the last data subset considered. (All this is in terms of a one-step-ahead prediction.)

2.1 Medians
We can draw the medians, which are the points of statistical distribution that a given quantity is equally likely to fall either side of. That means that in these points, we have R(t)=1/2.

2.2 Prequential likelihood function
For the series of predictions of Tj+1, ..., Tj+n, the prequential likelihood is : We can now compare two models, by calculating the prequential likelihood ratio between model 1 and 2 :
If this ratio increases as n increases, we can say that model A discredits model B. This plot tells us which model is locally performing best.

2.3 U-Plot
If the predictions Fi were true, then would be realisations of independent identically distributed random variables on [0,1]. Thus their closeness to this is a measure of the accuracy of the prediction. In order to decide we draw a plot called u-plot, using the data subset and the distribution computed by the model. The closeness to the line of unit slope is measured through the Kolmogorov-Smirnov distance. If the plot is above the line then the prediction is too optimistic, if it is below the line, it is too pessimistic. Because this plot aggregates the datas we can only observe global trends on the dataset.

2.4 Model recalibration
Sometimes predictions seem to be merely biased, as we can evaluate it through the u-plot. We can use this past information to recalibrate the model. In order to compute the new model we use a spline-smooth version for the u-plot of the raw model.

2.5 Laplace test
This test analyses the trend of the failures. One can extract two types of information from such a graph : local and global changes.
When the values are positive (resp. negative), the fiability is globally increasing (resp. decreasing). On the other side, when the values are increasing (resp. decreasing), we have local variations of the fiability.

Part 2 : SOFTRELY - WEB v1.0

1) How to run the program

- Run Netscape
- Open location : http://www-inf.enst.fr/~popentiu/SOFTRELY-
WEB_v1.0/

2) Where to put the data set
- the dataset files must be put in the directory ~popentiu/SOFTRELY- WEB_v1.0/donnees/ - the model files obtained with J.-P. BlainÆs program R-Vision must be put in the directory ~popentiu/SOFTRELY-WEB_v1.0/modeles/ - then the program MakeList must be executed (by typing MakeList in the shell). It is located in the directory ~popentiu/SOFTRELY-WEB_v1.0/ . It must be executed in its directory.

3) How to use the program
You can load a dataset by clicking on the button "load a data set". This one will open a new window, containing the list of the disponible datafiles.To choose a dataset, click on it in the list and select OK.
Once a dataset is loaded it is presented in the two graphics of the first window. The graphic "Data set" presents inter-failure times, and the second graphic presents the cumulatives failures (in both graphics, versus total elapsed time).
Now you can select the tool you want to use : either you will look the u_plot, or the median, or the Prequential Likelihood (presented above). Just click on one of the three buttons of graphic selection. Each one raises a window containing a graphic and a list of models available.

3.1 The u-plot and the median windows
The u-plot window will include the graph itself and the value of the Kol\- mogorov-Smirnov distance for selected models in either raw and recalibrated forms. You can toggle views between raw and recalibrated parametrics by using the button on the right side of the graph. You may choose the models you want to see, by clicking on the related buttons, and you also may choose to see the recalibrated models. The graphs will be traced with the colors corresponding to the buttons.

3.2 The prequential likelihood window
On the left of the window, you have two lists of the models. The first one concerns the models you want to see, and the second one the model you want to be the reference, in order to compare each model with a reference model. By default the reference model is Littlewood-NHPP. You may change this by clicking on the name of a model in the right list. Like in the u-plot window you may switch between raw and recalibrated parametrics.

Softrely-WEBv1.0 lets you find out if a model is accurate for a particular dataset. It may easily be completed with other models, by writing the programs to compute medians, likelihood functions and uplot in both raw and recalibrated forms and simply adding the model to the list of models considered by the interface. The interface is simple to use and may give you alot of informations on a particular dataset. Our program allows everyone using Netscape or an equivalent software (supporting Java), to see the informations in a graphical form.

We will keep this notation until the end.