(Warning: this article may contain heresies.
Also, it is quite dated since I wrote it in the mid- 1990s. So this is a passionate argument about stuff most folks don't care about anymore.)
Human "knowledge" is a context-sensitive, approximate and inaccurate hypotheses that requires continually testing. Compton argues that knowledge is a context-sensitive construct [1]. Models may be inappropriate when used out of the context in which they were elicited [2].
Knowledge representation theorists stress that our KBs are approximate surrogates of reality [3-5]; i.e. there accuracy is doubtful. Contrast this view with Ed Feigenbaum who described knowledge engineering as "mining the jewels in the expert's head" [6]; i.e. KBs were representations of what experts actually have in their heads. The changeover from the Feigenbaum "expertise transfer" view and the modern "knowledge-modelling" view seems to have occurred in the early nineties [7]. O'Hara notes that some KR theorists still make occasional claims that their KR theory has some psychological basis. However, when pressed, their public line is that representations are models/ surrogates only [8].
Popper argues that all knowledge is an hypothesis since nothing can ever be ultimately proved; our currently believed ideas are merely those that have survive active attempts to refute them [9]. Compton describes knowledge acquisition (KA) cycles where "test" is the dominate technique [10, 11]. Elsewhere we have argued that KE methodologies based on testing can out-perform alternative, more complicated, methodologies [12].
Our knowledge representation (KR) research is somewhat flawed if we uncritically enshrine our knowledge bases (KB). Clancey argues the knowledge structures found during knowledge acquisition (frames, rules, etc) are structures created on-the-fly in response to the specifics of the situation in which they were elicited (the example being studied, the experts used, etc); i.e. they have little/no isomorphism with structures present in an expert's information processing system. Clancey is silent on where these structures come from (but hints that the substrate may be neural). [13]
References