Figure 1: The CAME strategy

• Community: the necessity of a group or a team that is motivated and has the knowledge of software measurement to install software metrics. In general, the members of these groups are organized in metrics communities such as our German Interest Group on Software Metrics [3].

• Acceptance: the agreement of the (top) management to install a metrics program in the (IT) business area. This aspects is strong connected with the knowledge about required budgets and personnel resources.

• Motivation: the production of measurement and evaluation results in a first metrics application which demonstrates the convincing benefits of the metrics application. This very important aspect can be achieved by the application of essential results in the (world-wide) practice which are easy to understand and should motivate the management. One of the problem of this aspect is the fact that the management wants to obtain one single (quality) number as a summary of all measured characteristics.

• Engagement: the acceptance of spending effort to implement the software measurement as a permanent metrics system (with continued measurement, different statistical analysis, metrics set updates etc.). This aspect include also the requirement to dedicate personnel resources such as measurement teams etc.

The CAME framework itself consists of the following four phases:

• Choice: the selection of metrics based on a special or general measurement view on the kind of measurement and the related measurement goals -- as measurement objects,

• Adjustment: the measurement characteristics of the metrics for the specific application field - as attributes of the measurement objects,

• Migration: the semantic relations between the metrics along the whole life cycle and along the system architecture – as services of the measurement objects,

• Efficiency: the automation level as construction of a tool-based measurement - as implementation of the measurement objects.

The phases of this framework will be explained in the following sections including the role of the CAME tools.

The first framework step includes the choice of the software metrics and measures from the general software metrics tree which is based on the SQA literature and IEEE or ISO standards (see also [2], [5], [7], [11] and [12]) and is described in figure 2. In a first simplification, we consider the quality aspect as the only empirical criterion.

Figure 2: Measurement aspects

Measurement objects metrics tree (the first four levels):

• product metrics:
  • architecture metrics:
    • component metrics
    • configuration metrics
    • data-base metrics
  • run-time metrics:
    • task metrics
    • data handling metrics
    • human interface metrics
  • documentation metrics:
    • manual metrics
    • development metrics
    • marketing metrics
• process metrics:
  • management metrics:
    • project metrics
    • configuration metrics
    • SQA metrics
  • life cycle metrics:
    • phases metrics
    • milestone metrics
    • workflow metrics
  • CASE metrics:
    • method metrics
    • paradigm metrics
    • tool metrics
• resources metrics:
  • personnel metrics:
    • skill metrics
    • communication metrics
    • productivity metrics
  • software metrics:
    • paradigm metrics
    • performance metrics
    • replacement metrics
  • hardware metrics:
    • reliability metrics
    • performance metrics
    • availability metrics

Figure 3: The abstract metrics tree

• we consider only the nodes until the level five for metrics counting (total: 81 nodes),

• every node has the same significance.

• the metrics tree coverage level in a partial manner as the percentage of the used kinds of metrics in accordance with the fifth classification level,

• the metrics tree balance level as the tree coverage for the level four nodes only,

• the metrics standard level as percent of metrics which are based on or defined by (IEEE, ISO etc.) standards.

These level indicators leads to a value interval from zero to 100 (percent) with 81 (discrete) values for the tree coverage and 27 values for the tree balance. We will describe the coverage of 100 percent as formal completeness of the metrics choice. On the other hand, we can formulate plausible the "locality" of some GQM-based measurement approaches in practice

The adjustment is related to the experience (expressed in values) of the measured attributes for the evaluation. The adjustment includes the metrics validation and the determination of the metrics algorithm based on the measurement theory ([8] and [13]). The steps in the measurement adjustment are

• the determination of the scale type and (if possible) the unit,

• the determination of the favorable values (as thresholds) for the evaluation of the measurement component, e. g. by
  • discussion or brainstorming in the development or quality team,
  • analyzing and use the examples in the literature,
  • use of the thresholds of the metrics tools,
  • taking the results of appropriate case studies,

• the tuning of the thresholds as [10]
  • approximation during the software development from other project components,
  • application of a metrics tool for a chosen software product that was classified as a "good qualitative" example,

• the calibration of the scale (as transformation of the numerical scale part to the empirical) depends on the improvement of the knowledge in the problem domain.

In order to evaluate metrics applications from the adjustment point of view we define the following three reference numbers

• the ratio scale level as percentage of the used metrics with the ratio scale type,

• the thresholds validity level as percentage of the used thresholds which have an enterprise-based experience,

• the measurement level as the percentage of measures related to predicted or estimated metrics.

The values also range from zero to 100 percent and classify the "measurement level" of a metrics application or a metrics program.

The migration step should define the semantic connections of the (chosen) metrics. Some examples of these kinds of connections for software products are

• metrics tracing along the software life cycle, e. g. #notions (problem definition) -> #classes (specification) -> #new-defined-classes (design) -> #implemented-classes (implementation),

• metrics refinement along the software life cycle, e. g. informal description of a specified service (text metrics) -> PDL description of a service (design metrics) -> Java form of a service (code metrics) [9],

• metrics granularity related to the architecture, e. g. in an object-oriented development as the system, the class and the method [1].

These characteristics are also given for the process and resources area. Note, that the most existing software measurement approaches or frameworks do not include this step (see [2] and [8]). This step leads to a reduction of the number of the measurements for the metrics in the tree level six by introducing (new) relations between any two concrete software metrics. On the other hand, the metrics set will be expanded by adding the metrics necessary to control all product architecture components, process phases and resource versions.

In order to evaluate the metrics application from this "migration" point of view, we assume that

• for every node in the level six in the metrics tree exists a semantic connection to any other node in this level,
• the semantic connection can be expressed by an experience-based transformation, aggregation or specification of the metrics and will be described as migration.

We define the migration level as

• the metrics tracing level as percentage of the involved nodes in the level six of the metrics tree by considering the workflows of product stages, process phases and resources versions,

• the metrics refinement level as percentage of the involved nodes in the tree level six by splitting or detailing of the measured components,

• the metrics granularity level as percentage of the coverage of all architecture (structure) levels through the metrics in the tree level six.

The values of these metrics range that case of zero and 100 percent. We can establish in the 100 percent we have a complete semantic metrics network. In order to keep the possibility of correction or tuning the estimate models, we should consider all metrics including their redundancy effect by migration. During the application of our framework, the migration step can require a repetition of the adjustment step above.

This step includes the instrumentation or the automation of the measurement process by tools. It requires to analyze the algorithmic character of the software metrics and the possibility of the integration of tool-based "control cycles" in the software development or maintenance process. We will call the metrics tools as CAME (Computer Assisted software Measurement and Evaluation) tools [4]. In most cases, it is necessary to combine different metrics tools and techniques related to the measurement phases.

On the other hand, we are also required to integrate such tools related to the software development phases. A special tool descriptions are given in the SMLab URL at http://ivs.cs.uni-magdeburg.de/sw-eng/us/ (see also [4] and [6]). This CAME tool application also requires a metrics database with a lot of different forms of measurement values and used aspects [8]. The metrics database is the repository of a concrete enterprise-based software measurement and supports the statistical analysis, assessments and controlling information for the managed software development components such as projects, products and resources.

In order to evaluate the efficiency level we define

• the metrics tool-based level as percentage of metrics execution by CAME tools,

• the metrics controlling level as percentage of metrics which are involved in a controlling or improvement/SQA process,

• the metrics database level as percentage of stored or persistent values of all used metrics.

These metrics give an evaluation value between zero and 100 (percent) to characterize the level of the automation of the software measurement. The situation of 100 percent of these metrics values can be characterized as an automated controlling system which fulfills the CMM level four.

[1] Abreu, F. B.; Goulao, M; Esteves, R.: Toward the Design Quality Evaluation of Object-Oriented Software Systems. Proc. of the Fifth Int. Conf. on Software Quality, Austin, October 23-25, 1995, pp. 44-57

[2] De Panfilis, S., Kitchenham, B., Morfuni, N.: Experiences introducing a measurement program. Inf. and Software Technology, 39(1997), pp. 745-754

[3] Dumke, R.; Abran, A.: Software Measurement – Current Trends in Research and Practice. DUV Publ., Germany, 1999, (269 p.)

[4] Dumke, R.; Grigoleit, H.: Efficiency of CAME tools in software quality assurance. Software Quality Journal, 6(1997), pp. 157-169

[5] Dumke, R.; Winkler, A.: Management of the Component-Based Software Engineering with Metrics. Fifth Int. Symposium on Assessment of Software Tools, Pittsburgh, June 2-5, 1997, pp. 104-110

[6] Ebert, C.; Dumke, R.: Software-Metriken in der Praxis. Springer Publ., 1996 (314 p.)

[7] Eman, K.E.; Drouin, J.; Melo, W.: SPICE The Theory and Practice of Software Process Improvement and Capability Determination. IEEE Comp. Soc., 1998 (486 p.)

[8] Fenton, N. E.; Pfleeger, S. L.: Software Metrics – A Rigorous & Practical Approach. Thomson Publ., 1997 (638 p.)

[9] Hanebutte, N.: A Measurable Program Design Language. Diploma Thesis, University of Magdeburg, University of Idaho, January 1999 (177 p.)

[10] Lubahn, D.: Konzeption und Implementation eines C++-Meßtools Diploma Thesis, University of Magdeburg, March 1996 (126 p.)

[11] Moore, J W.: Software Engineering Standards. IEEE Computer Press, 1998 (296 p.)

[12] Zahran, S.: Software Process Improvement - Practical Guidelines for Business Success. Addison-Wesley Publ., 1998 (447 p.)

[13] Zuse, H.: A Framework of Software Measurement. DeGruyter Publ., 1997 (755 p.)