From tim@menzies.us Sun Aug 31 18:00:59 2008
Date: Sun, 31 Aug 2008 09:53:58 -0400
From: Tim Menzies <tim@menzies.us>
To: Jamie Andrews <andrews@csd.uwo.ca>
Subject: Re: call me? 304 376 2859 or menzies.tim orn skype

    [ The following text is in the "UTF-8" character set. ]
    [ Your display is set for the "iso-8859-1" character set.  ]
    [ Some characters may be displayed incorrectly. ]

further to the below.

you reported to me that after fss, runtimes dropped to (i think) 43% of
the original.

this _sounds_ good but the key issue for the reveiwers will be "so what?"

and the so what is this. GAs have to search a space of 2^N options. a
reduction in the feature space of X% has exponential impacts on the
runtime and scalability

our results are preliminary,yes, but they indicated that that there could
be much productive future work  based around exponential decreases and
scale up from GAs+FSS

t

On Sun, Aug 31, 2008 at 9:24 AM, Tim Menzies <tim@menzies.us> wrote:
      dear jamie,

      if gmail is formatting things properly then all my
      additions(below) to the paper are NOT indented

      the paper reads very well. proud to be on it.

      in my view, the paper need a short list in intro, succinctly
      summarizing nighthawk's advantages over the methods from
      section V. your intro "contributions" list says what it is
      you did. it does not say what you found and why anyone should
      be over the moon excited by nighthawk.

      ditto with conclusion.

      lower-level details:

      1) have you considered \usepackage{cite} in the preamble? its
      orders the citations numerically.

      2) p11, graphics are cool. perhaps too big?

      3) table p17 too large. time for {\footnotesize
      \begin{tabular}... \end{tabular}} ?

4) p28: OLD "Another reason to prefer RELIEF is that we should be
able to integrate it with the rest  of Nighthawk."
p28: NEW: 

"Another reason to prefer RELIEF is that it can take advantage of
Nighthawk's  stochastic search."

5) editorial convention: i used UPPER CASE for program names. you
don't.
so for my words to be consistent with yours, RELIEF and WRAPPRER
should be Relief and Wrapper.
ditto with "SELECTED", below.

      6) p29. OLD <<<EOF
      For each run of Nighthawk on the units studied, this
      process resulted in a list of all the genes
      present in the run, with a merit score calculated for
      each.
      TBD by TIM: explain what this merit score signiļ¬^Įes
      EOF

      ok. here's the text i sent before but with a little
      more stuff in there on "merit"


Using the above discretization policy, we ran RELIEF 10 time sin a
10-way cross-validation study.
The data set was divided
into $10$ buckets. Each bucket was (temporarily) removed and RELIEF
was run on the remaining data. This produced a list of "merit"
figures for each feature (this "merit" value is an internal
heuristic measures generated from RELIEF and reflects the
difference ratio of neighboring instances that have different
regions). We took the maximum merit ($Max$) and generated a
$SELECTED$ list. A feature was $SELECTED$ if its merit $m$ was $m>
\alpha Max$ (e.g. at $\alpha=0.5$ then we selecting all features
that score at least half the maximum effort).

The following table shows the number of features that were SELECTED
in our 19 examples, using different values for $\alpha$. Note that
as $\alpha$ increases, we selected fewer and fewer features

\begin{tabular}{rl}
$\alpha$ & $| SELECTED | $\\\hline
0.5 & 439\\
0.6& 217 \\
0.7 & 112 \\
0.8 & 62 \\
0.9 & 39 \\
\end{tabular}

That is, this method allowed us to discover the genes that were
most important in selecting
for the {\em high plateau} and not the {\em slope} or {\em hole}.

7) missing references

      Could you please send me your BiBTeX references for the
      following?

      Warning--I didn't find a database entry for "hall03"


@article{hall03,
  journal =	 "IEEE Transactions On Knowledge And Data Engineering",
  year =	 2003,
  volume =       15,


  number=        6,
  pages =        "1437- 1447",
  title =	 "Benchmarking Attribute Selection Techniques for
                  Discrete Class Data Mining",
  author =	 "M.A. Hall and G. Holmes",


	note="Available from 
          \url{http://www.cs.waikato.ac.nz/~mhall/HallHolmesTKDE.pdf}"
}
 

      Warning--I didn't find a database entry for "miller02"


@book{miller02,
  author="A. Miller",
  title="Subset Selection in Regression (second edition)",


  isbn="1-58488-171-2",
  year="2002",
  publisher="Chapman \& Hall"
}
 

      Warning--I didn't find a database entry for "kohavi97"


@article{kohavi97,
  author =	 "Ron Kohavi and George H. John",
  title =	 "Wrappers for Feature Subset Selection",


  journal =	 "Artificial Intelligence",
  volume =	 "97",
  number =	 "1-2",
  pages =	 "273-324",
  year =	 "1997",
  url =		 "citeseer.nj.nec.com/kohavi96wrappers.html"


}
 

      Warning--I didn't find a database entry for "Kir92"


@INPROCEEDINGS{Kir92,
  TITLE =	 " A Practical Approach to Feature Selection",
  AUTHOR =	 {K. Kira and L. Rendell},


  BOOKTITLE =	 "The Ninth International Conference on Machine
                  Learning",
  PUBLISHER =	 "Morgan Kaufmann",
  PAGES =	 "pp. 249-256",
  YEAR =	 1992,
}
 

      Warning--I didn't find a database entry for "Kon94"


@INPROCEEDINGS{Kon94,
  TITLE =	 " Estimating attributes: Analysis and extensions of
                  relief",


  AUTHOR =	 "I. Kononenko",
  BOOKTITLE =	 "The Seventh European Conference on Machine Learning",
  PUBLISHER =	 "Springer-Verlag",
  PAGES =	 "pp. 171-182",
  YEAR =	 1994,


}
 



--
<tim.menzies
title=" dr (Ph.D.) and associate professor"
align=" csee, west virginia university"
cell=" esb 841A"
rss=" http://menzies.us/rss.php"
url=" http://menzies.us"
>

Reality is nothing but a collective hunch.
-- Lily Tomlin




--
<tim.menzies
title=" dr (Ph.D.) and associate professor"
align=" csee, west virginia university"
cell=" esb 841A"
rss=" http://menzies.us/rss.php"
url=" http://menzies.us"
>

Reality is nothing but a collective hunch.
-- Lily Tomlin