\begin{figure}
\begin{center}
\includegraphics[width=4in]{cycle.png}
\end{center}
\caption{HAMLET's search for associations.}\label{fig:cycle}
\end{figure}
HAMLET's search for association has to two essential phases, and a third optional phase
(see
\fig{cycle}). All these stages have the same goal: from a large set of possible associations,
extract the small subset that (a)~have occurred frequently in prior technical documents and which
(b)~the user will approve. 
 Before running any query:
\bi
\item
 Archival artifacts must be parsed into a network of nodes and edges. 
Each node is represented as a set of {\em terms} referenced in that node.
This set of terms can include every term in the archive so we call these the {\em wide vectors}.
\item
The wide vectors are unwieldy to process. Hence, we run some linear time {\em reduction}
methods to isolate the most informative columns. This produces  the {\em narrow vectors}.
Note that, when we run a query, we only use the parts of the query that appear in these
narrow vectors.
\item
HAMLET also clusters the narrow vectors into {\em groups}. These groups define cliches of repeated
structures.
\ei
When running a query:
\bi
\item We remove irrelevant detail from the query. Linear time methods remove white space 
and spurious word endings. 
Then the query is pruned back to include just the terms seen in the narrow vectors.
\item The resulting {\em tuned vector} is then matched to the the groups of vectors found
above. For computational reasons, we run this match in two-stages. First, we find the N nearest clusters (this generates the {\em relevant groups}). Second, searching just within those relevant groups, we perform exact matches to find related terms.
\item
The {\em candidate matches} found from the exact match are then ranked (by the size of the overlap of the query and the terms) then pruned using thresholding (find the neighboring items in th sort with the biggest difference between them; prune the items below that largest cliff).
\item
The user assesses the matched queries and declares that some are ``relevant'' and some aren't. 
This builds up a session log for this user working these kinds of queries. Once this log grows beyond a certain size, it is used to refine the tuned query such that the tuning favors nodes that
do not contain what the user has labeled ``irrelevancies'' and does contain what the user has called
``relevances''.
\ei
Optionally, we can visualize the results:
\bi
The N-dimensions of the vectors are mapped down to 3 dimensions, then visualized on the screen.
\ei
