\begin{figure*}[!t]
{\footnotesize
\begin{tabular}{|p{3in}|p{3in}|}\hline
\begin{description}
\item[{\bf Satellite}] {\em
Description:} Analysts at the NASA Jet Propulsion
Laboratory have built a semantic network describing the
interactions between 88 features of a proposed dead space
mission. Each edge was annotated with the numeric cost and
benefits of taking some action. Some of these nodes
represented base decisions within the project (e.g.
selection of  a particular type of power supply). The net
can be  executed using some random combination of the base
decisions and the resulting costs and benefits  collected.

{\em Assessment:}
The generated cost and benefit scores can be sorted
and divided into $X$ percentile bands.
An individual run can then be scored as a pair
$<Cost_i,Benefit_j>$ where $i$ and $j$ select one of the
percentile bands. In this application, the preferred band
would be lowest cost and highest benefit.

{\em References:} The semantic net editor for this
application was developed  by Feather et.al.~\cite{fea00}.
No prior report explores machine learning in this domain.
\item[{\bf CMM2:}]
{\em Description:} The expressiveness of the semantic net
processed  by  Feather et.al.~\cite{fea00} was extended by
Menzies \& Kiper to create a more general rule-based
language~\cite{me01e}. This language was used to encode
 a model of best practices in software engineering
(level 2 of the Software Engineering Institute's capability
maturity model, also known as CMM2~\cite{paulk93}).

{\em Assessment:} Menzies & Kiper doubted the accuracy of
the cost and benefit supplied by domain experts. Hence,
they wrote a simulator that randomly perturbed these
weights by $\pm50\%$. The goal of the l Same as with {\em
satellite}.
\end{description}
&
\begin{description}
\item[{\bf COCOMO:}] {\em
Description:} The goal of the COCOMO-II project is to build
an open-source software cost estimation
model~\cite{cocomoII}. Internally, the model contains a
matrix of  parameters that should be tuned to a particular
software organization. Using COCOMO-II, the Madachy risk
model can assess the risk of a software cost
over-run~\cite{madachy97}.

{\em Assessment:} For machine learning purposes, the goal
of using the Madachy model is to find a change to a
description of a software project that reduces the
likelihood of a poor risk software
project~\cite{me00e,me01f}.

\item[{\bf  Circuit:}] A qualitative description of a circuit
of 47 wires connecting 9 light bulbs and 16 other
components was coded in Prolog. The model was expressed as
a set of constraints; e.g. the {\tt sum} of the voltages of
components in series is the {\tt sum} of the voltage drop
across each component. The definition of {\tt sum} honored
the nondeterminism of qualitative arithmetic; e.g.  {\tt
sum(-,+,Any)} notes that the sum of a negative and a
positive value is unknown.

{\em Assessment:} The goal of the circuit was to illuminate
a space using the 9 light bulbs. The problem with the
circuit was out-of-control nondeterminism. On backtracking,
this seemingly simple circuit generated 35,228 solutions to
the constraints. Worse, over all those solutions, the
circuit was mostly dark: only two bulbs glowing (on
average). The goal of the machine learning was to find a
minimal set of changes to the circuit to increase the
illumination~\cite{me01g}.
\end{description}
\\\hline
\end{tabular}
} \caption{Some of the models used in  TAR2 validation
studies.}
\end{figure*}