This is an old revision of the document!
Model Quality
Metrics improve development process, understanding of complexity, discovering/predicting faults and estimation of efforts. Often the choice of metrics exceed what is required and therefore it is advised to define what is the purpose of measurement.
Framework for Deciding Metrics
Goal-Question-Metric Framework
A useful framework to derive useful metrics is the Goal-Question-Metric (GQM) paradigm developed by Basili [2]. The GQM process is as follows:
Define goals of measurement (for “UML As Sketch” type models then goal is mainly focusing on communication.)
Define questions to which the answers allow the observer to determine whether the goal has been met
Each question is evaluated to determine what metrics are required to support the question's answer.
Krogstie's framework
Krogstie et al (referenced in [4]) is a framework which lays out that quality goals are defined as relations between blocks.
Syntatic quality - goal that all statements in the model are according to the syntax of the modeling language
Empirical quality - comprises comprehensibility matters such as layout and readability
Organizational quality - model fulfills the goal of modeling
Social quality - agreement between stakeholder's intepretations
No known empirical evaluation of this framework on models.
Quality Goals
Mohagheghi [1] show that there are 6 classes of quality goals for models:
Comprehensibility (emphasis for model sketches)
Confinement (emphasis for model sketches)
Correctness (emphasis for model design)
Completeness (emphasis for model design)
Consistency (emphasis for model design)
Changeability (emphasis for model design)
Comprehensibility - Model is understandable to its intended audience.
Confinement - Model is in agreement with its purpose
Correctness - Model includes correct elements & relationships. Does not violate rules and conventions
Completeness - All necessary information is included at the necessary degree of fidelity
Consistency - Model is without contradictions. Horizontal consistency is consistency between diagrams and views that belong same level of abstraction. Vertical consistency is consistency between models or diagrams at different levels of fidelity. Consistency also refers to semantics, i.e. same element does not have multiple interpretations
Changeability - Model supports continuous and rapid improvement and evolution
MIT xPRO propose the following “Qualities of Great Models”
Linked to Decision Making.
Model Credibility - it is believable
Clear Scope
Verification and Validation of Model - Model should show why it is the preferred option to do verification and validation of the system
Traceable and Analyzable
Understandable and Well Organized
Data extrapolation - where is the model valid
Complete relative to Scope
Internally consistent
Verifiable
Validatable
Elegant
Appropriate level of Fidelity
Allows Optimization - does it includes gradients or convexity
Avoid optimization on a black box
Reuse
Availability of Interfaces
Model Metrics
Model Size
Rationale includes:
Comparison of models (before and after, model A and model B on same system)
Measure of progress
Prediction of work effort
Examples:
Absolute size (e.g. number of elements)
Relative size (e.g. ratio between sequence diagrams and use cases)
Complexity (…)
Functionality (e.g. number of use cases)
Re-use (e.g. applicable if a profile is used, then the metric may be % of profile usage)
Design Metrics
Degree of polymorphism
Degree of information hiding
Degree of complexity
Dynamicity / Coupling - complexity of a class internal behavior based on message calls and state transitions
Depth of Inheritance Tree (DIT)
Cohesion - which part of class is needed to perform a single task
Number of Use Cases - number of use cases per class
Comprehensibility Metrics
Number of elements on diagram
Number of crossing lines on diagram
Number of entry and exit points on diagram
References
-
[2] V.R. Basili, G. Caldiera, and H.D. Rombach, “The Goal Question Metric Paradigm”, In Encyclopedia of Software Engineering, volume 2, John Wiley and Sons, 1994, pp. 528-532.
-
-