( This page is part of the NOVA system. )

nova/XOMO

Code to implement some of the USC COCOMO-family models. These models offer predictions for:

How long it will take to build a software; measured in elapsed months;
How many months of effort are required during those elapsed months (so your team size is effort/months);
How many defects will be introduced or removed from your software;
Number of projects threats; i.e how prone is your software to schedule overruns due to poor management decisions.

In standard practice, these models are tuned to local conditions using historical data. NOVA is an experiment in an alternate approach. When historical data is absent, NOVA uses AI search methods to find policies that are useful across the space of possible tunings.

Files

coc-lib.lisp

General support code for XOMO

COCOMO ranges
COCOMO structs
Accessing COCOMO Range Data
Guessing COCOMO ranges
Support functions

model-defaults.lisp

Ranges for a standard COCOMO model

effort-model.lisp

The COCOMO effort model

months-model.lisp

The COCOMO months model

defects-model.lisp

The COQUALMO defects model.

threats-model.lisp

The Madachy threat prediction model.

Variables in the COCOMO model

The following models are based on the following variables.

coc-lib.lisp: COCOMO ranges

(coco) converts COCOMO ranges numbers to a canonical form; e.g. "high" or "h" or "4" are all synonyms for "4".

(let ((names  
       '((xh . 6) (6 . 6) (extremelyhigh . 6)
	 (vh . 5) (5 . 5) (veryhigh . 5)
	 (h  . 4) (4 . 4) (high .  4)
	 (n  . 3) (3 . 3) (nominal . 3)
	 (l  . 2) (2 . 2) (verylow . 2)
	 (vl . 1) (1 . 1) (low . 1))))
  (defun coco (l) 
    (labels ((coc1 (x)
	       (or (cdr (assoc x names))
		   (warn "~a not a known coco-name" x)
		   x)))
      (mapcar #'coco1 l)))
  (defun coco-restrict (old new)
    (let* ((new1 (coco new))
	   (overlap (intersection new1 old)))
      (or overlap
	  (warn "~a does not intersect with ~a" old new)
          old)))
)

coc-lib.lisp: COCOMO structs

Many of the USC models take the form:

prediction = a * LOC^{(b+ sf1 + sf2+..)} * em1 * em2 * ...

where:

LOC = lines of code

"a" and "b" are quick-and-dirty tuning parameters that have the following relationship:

(defstruct one-a line) 
(defstruct one-b num) ; b falls underneath the a line

(defun ?one-a (r)
  "from a floating a-line, pull fixed values"
  (let ((line  (one-a-line r)))
    (?quantity (line-x1 line) 
	       (line-x2 line))))

(defun ?one-b (r &optional 
	       (a-val (! 'a)) 
	       (a-line (one-a-line 
			(geta 'a (db-settings *db*)))))
  "from a floating b, pull fixed values"
  ; note that "b" is defined in terms of "a"
  (let ((inc (?num (one-b-num r))))
    (max 0 (- (line-y a-val a-line ) inc))))

sf = scale factors that change the prediction by an exponential amount.

(defstruct (sf   (:include bag)) 
   effort  ; impact on effort
   rsf     ; impact on defect introduction during requirements
   dsf     ; impact on defect introduction during design
   csf     ; impact on defect introduction during coding
)

em = effort multipliers that change the prediction by a linear amount.

(defstruct (em   (:include bag)) 
   effort ; impact on effort
   rin    ; impact on defect introduction during requirements
   din    ; impact on defect introduction during design
   cin    ; impact on defect introduction during coding
)

There are also defect removal variables (called "dr") that remove defects.

(defstruct (dr   (:include bag)) 
   rout ; impact on defect removal during requirements
   dout ; impact on defect removal during design
   cout ; impact on defect removal during coding
)

These range of these values is some subset of

very low
low
nominal
high
very low
extremely low

Internally to these models, these "sf,em,dr" variables form straight lines. These lines are either

Hinged through the origin:

(defun  hinged-line (x m) (* m (- x 6)))

Hinged through the origin and offset a little:

(defun hinged-line-coqualmo (x m) (* m (- x 1)))

Pivoted through the point (x,y)=(3,1).

(defun pivoted-line (x m) (+ 1 (* m (- x 3))))

(The justification for the last one is simple- the intuition behind the effort multipliers is that when x=3 (i.e. "nominal"). a variable has no impact on the prediction. Given a product of variables, a value of "1" means that the net effect on the prediction is "unchanged". Hence, the pivoted functions travel through the line (x,y)=(3,1)

With those functions, it is possible to map a "x" variable into a "y" effect on the prediction:

(defun em2effort (x) (let ((x1 (! x))) (pivoted-line (em-range x1) (em-effort x1))))
(defun em2rin    (x) (let ((x1 (! x))) (pivoted-line (em-range x1) (em-rin    x1))))
(defun em2din    (x) (let ((x1 (! x))) (pivoted-line (em-range x1) (em-din    x1))))
(defun em2cin    (x) (let ((x1 (! x))) (pivoted-line (em-range x1) (em-cin    x1))))

(defun sf2effort (x) (let ((x1 (! x))) (hinged-line (sf-range x1) (sf-effort x1))))
(defun sf2rin    (x) (let ((x1 (! x))) (hinged-line (sf-range x1) (sf-rsf    x1))))
(defun sf2din    (x) (let ((x1 (! x))) (hinged-line (sf-range x1) (sf-dsf    x1))))
(defun sf2cin    (x) (let ((x1 (! x))) (hinged-line (sf-range x1) (sf-csf    x1))))

(defun dr2rout   (x) (let ((x1 (! x))) (hinged-line-coqualmo  (dr-range x1) (dr-rout  x1))))
(defun dr2dout   (x) (let ((x1 (! x))) (hinged-line-coqualmo  (dr-range x1) (dr-dout   x1))))
(defun dr2cout   (x) (let ((x1 (! x))) (hinged-line-coqualmo  (dr-range x1) (dr-cout   x1))))

Tuning a model changes the slopes of the "em,sf,dr" lines: The following structures store the minimum and maximum slopes seen in prior tunings.

Impact on efforts:

(defstruct (sfs   ; scale factors
             (:include num (max  -1.56  ) (min -1.015 ))))
(defstruct (em+  ; some effort multipliers increase effort
             (:include num (max   0.25  ) (min  0.073 ))))
(defstruct (em-  ; some effort multiplers decrease effort
             (:include num (max  -0.178 ) (min -0.078 ))))

Impact on defect introduction

(defstruct (rin+ ; some variables increase defects at requirements time
             (:include num (max   0.112 ) (min  0     ))))
(defstruct (rin- ; some variables decrease defects at requirements time
             (:include num (max  -0.146 ) (min -0.034 ))))

(defstruct (din+ ; some variables increase defects at design time
             (:include num (max   0.117 ) (min  0.08  ))))
(defstruct (din- ; some variables decrease defects at design time
             (:include num (max  -0.152 ) (min -0.038 ))))

(defstruct (cin+ ; some variables increase defects at coding time
             (:include num (max   0.117 ) (min  0.008 ))))
(defstruct (cin- ; some variables decrease defects at coding time
             (:include num (max  -0.152 ) (min -0.042 ))))

Impact on defect removal of scale variables:

(defstruct (rsf  (:include num (max  -0.146 ) (min  0.0   ))))
(defstruct (dsf  (:include num (max  -0.208 ) (min  0.0   ))))
(defstruct (csf  (:include num (max  -0.190 ) (min  0.0   ))))

Impact on defect removal of defect removal variables:

(defstruct (rdr  (:include num (max   0.117 ) (min  0.083 ))))
(defstruct (ddr  (:include num (max   0.130 ) (min  0.083 ))))
(defstruct (codr (:include num (max   0.147 ) (min  0.092 ))))

coc-lib.lisp: Accessing COCOMO Range Data

Functions to get/set variables:

(defmacro range? (x) ; declared as macro so we can "setf" it
  `(bag-range
    (bag-range 
     (cdr
      (assoc  ,x (db-settings *db*))))))

(defun range! (x y) ; simple as pie, once we've setf-ed range?
  (setf (range? x)  (as-list y)))

(defmacro kloc? ()
  `(cdr (assoc 'kloc (db-settings *db*))))

(defun kloc! (min max)
  (setf (kloc?) (make-num :min min :max max)))

coc-lib.lisp: Guessing COCOMO ranges

(defun ?em (x)
  "from a floating em, pull fixed values"
  (make-em 
   :range  (?bag (em-range  x))
   :effort (?num (em-effort x))
   :rin    (?num (em-rin    x))
   :din    (?num (em-din    x))
   :cin    (?num (em-cin    x))))

(defun ?sf (x)
  "from a floating sf, pull fixed values"
  (make-sf 
   :range  (?bag (sf-range  x))
   :effort (?num (sf-effort x))
   :rsf    (?num (sf-rsf    x))
   :dsf    (?num (sf-dsf    x))
   :csf    (?num (sf-csf    x))))

(defun ?dr (x)
  "from a floating defect remover, pull fixed values"
  (make-dr
   :range (?bag (dr-range x))
   :rout  (?num (dr-rout x))
   :dout  (?num (dr-dout x))
   :cout  (?num (dr-cout x))))

coc-lib.lisp: Support functions

(defun init-db (&optional (settings (cocomo-defaults)))
  "create defect settings"
  (setf *db* (init-db0 settings)))

(defun init-db0 (&optional (settings (cocomo-defaults)))
  (make-db :settings settings))

The (cocomo-defaults) function is defined below.

model-defaults.lisp

Using all the above we can create the default ranges for all the variables used in the USC models.

(defun cocomo-defaults ()
  (append 
    (kloc-defaults)
    (a-defaults)
    (b-defaults)
    (defect-removal-defaults)
    (em-defaults)
    (sf-defaults)))

All these "defaults" function must return a list of dotted pairs otherwise the "append" in cocomo-defaults won't work.

(defun kloc-defaults ()
  `((kloc . ,(make-num :min 2 :max 2000))))

(defun a-defaults ()
  `((a  . ,(make-one-a :line (point-to-line 2 1 10 0.6)))))

(defun b-defaults ()
  `((b . ,(make-one-b :num  (make-num :min 0 :max 1)))))

(defun defect-removal-defaults ()
 `((aa   . ,(make-dr :range (make-r16) :rout (make-rdr) :dout (make-ddr) :cout (make-codr)))
   (etat . ,(make-dr :range (make-r16) :rout (make-rdr) :dout (make-ddr) :cout (make-codr)))
   (pr   . ,(make-dr :range (make-r16) :rout (make-rdr) :dout (make-ddr) :cout (make-codr)))))

(defun sf-defaults ()
  `((flex . ,(make-sf :range (make-r25) :effort (make-sfs) :rsf (make-rsf) :dsf (make-dsf) :csf (make-csf)))
    (pmat . ,(make-sf :range (make-r15) :effort (make-sfs) :rsf (make-rsf) :dsf (make-dsf) :csf (make-csf)))
    (prec . ,(make-sf :range (make-r15) :effort (make-sfs) :rsf (make-rsf) :dsf (make-dsf) :csf (make-csf)))
    (resl . ,(make-sf :range (make-r15) :effort (make-sfs) :rsf (make-rsf) :dsf (make-dsf) :csf (make-csf)))
    (team . ,(make-sf :range (make-r15) :effort (make-sfs) :rsf (make-rsf) :dsf (make-dsf) :csf (make-csf)))))

(defun em-defaults() 
  `((acap . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (aexp . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (cplx . ,(make-em :range (make-r16) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (data . ,(make-em :range (make-r25) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (docu . ,(make-em :range (make-r15) :effort (make-em+) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (ltex . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (pcap . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin+) :din (make-din-) :cin (make-cin-)))
    (pcon . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (plex . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (pvol . ,(make-em :range (make-r25) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (rely . ,(make-em :range (make-r15) :effort (make-em+) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (ruse . ,(make-em :range (make-r26) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (sced . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (site . ,(make-em :range (make-r16) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))
    (stor . ,(make-em :range (make-r36) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (time . ,(make-em :range (make-r36) :effort (make-em+) :rin (make-rin+) :din (make-din+) :cin (make-cin+)))
    (tool . ,(make-em :range (make-r15) :effort (make-em-) :rin (make-rin-) :din (make-din-) :cin (make-cin-)))))

effort-model.lisp

The effort model returns total staff months (155 hours, includes all development and management activities) to complete the software.

(defun effort ()
  "return total development+management effort in months"
  (macrolet ((f (x)  `(em2effort ',x))
	     (s (x)  `(sf2effort ',x)))
    (let ((a    (! 'a))
	  (b    (! 'b))
	  (kloc (! 'kloc)))
      (* a 
	 (expt kloc 
	       (+ b 
		  (* 0.01 
		     (+ (s prec) (s flex) 
			(s resl) (s team) 
			(s pmat)))))
	 (f time) (f stor) (f data) (f pvol) 
	 (f ruse) (f rely) (f docu) (f acap) 
	 (f pcap) (f pcon) (f aexp) (f plex) 
	 (f ltex) (f tool)  (f cplx) (f site) 
	 (f sced)))))

months-model.lisp

The months model returns the calendar months required to build the system (so staff = effort/months).

(defun months (&optional (pm (effort)))
  "return elapsed development time (in months)"
  (macrolet ((f (x)  `(em2effort ',x))
	     (s (x)  `(sf2effort ',x)))
    (let* ((c             3.67)
	   (d             0.28)
	   (sced          (em-range (! 'sced)))
	   (scedPercents  '((1 .  75) (2 .  85) (3 . 100)
			    (4 . 130) (5 . 160)))
	   (scedPercent   (geta sced scedPercents))
	   (pmNs          (/ pm (f sced)))	                  
           (elessb        (* 0.01 
			     (+ (s prec) (s flex) 
				(s resl) (s team) 
				(s pmat))))
	   (f             (+ d (* 0.2 elessb))))
      (* c (expt pmNs f) (/ scedPercent 100)))))

Note that the months model is computed from the effort model. However, the relationship is not simple. The following plot shows effort/months computerd using this model. The relationship between these relations is

months = 9.0188*Ln(effort) - 20.815

and the correlation is not strong (R²=0.6557):

months
  180 ++--------+---------+---------+---------+---------+---------+--------++
      +         +         +         +         +         +  eff-mnths   A    +
  160 ++                           A                                       ++
      |               A                                                     |
  140 ++                                                                   ++
      |         A                                                           |
  120 ++                  A                                                ++
      |     A A  AA               A                                         |
  100 ++     AAA                                                           +A
      |  A A  AA                                                            |
      | A A AAA A   A AA A  A  A         A                       A          |
   80 ++AAA AAAA   A A  A                                                  ++
      |AAA   AAA   AA A                                                     |
   60 +AAAAAAAAA  AA    A A                                                ++
      AAAAAAAA AAA A                                                        |
   40 AAAAAAA                                                              ++
      AAAA                                                                  |
   20 AA                                                                   ++
      A         +         +         +         +         +         +         +
    0 A+--------+---------+---------+---------+---------+---------+--------++
      0       10000     20000     30000     40000     50000     60000     70000
                                      effort

defects-model.lisp

Not built yet.

threats-model.lisp

Not built yet.