Lee Arnold summons the ghost of Kurt Gödel to make the careful argument that are minds might not be "mechanical": after all
Lee Arnold: It could remain problematic. There is also the possibility that if reason is an evolutionary development, we still may not be able to describe how it happened, nor run a Turing machine to evolve it again in silico. Consider Kurt Gödel's remarks on several consequences of his own theorem, quoted in Hao Wang, A Logical Journey: From Gödel to Philosophy (MIT Press, 1996):
The human mind is incapable of formulating (or mechanizing) all its mathematical intuitions. That is, if it has succeeded in formulating some of them, this very fact yields new intuitive knowledge, for example the consistency of this formalism. This fact may be called the 'incompletability' of mathematics. On the other hand, on the basis of what has been proved so far , it remains possible that there may exist (and even be empirically discoverable) a theorem-proving machine which in fact IS equivalent to mathematical intuition, but cannot be PROVED to be so, nor even be proved to yield only CORRECT theorems of finitary number theory. (pp.184-5)
Either the human mind surpasses all machines (to be more precise: it can decide more number-theoretical questions than any machine) or else there exist number-theoretical questions undecidable for the human mind. [It is not excluded that both alternatives may be true. -- Wang] (p.185)
My incompleteness theorem makes it likely that mind is not mechanical, or else mind cannot understand its own mechanism. If my result is taken together with the rationalistic attitude which Hilbert had and which was not refuted by my results, then [we can infer] the sharp result that mind is not mechanical. This is so, because, if the mind were a machine, there would, contrary to this rationalistic attitude, exist number-theoretic questions undecidable for the human mind. (p.186-7)
Gödel was also much more careful than many of today's scientists (it would appear) in his assumptions:
Turing, in his 1937, gives an argument which is supposed to show that mental procedures cannot carry farther than mechanical procedures. However, this argument is inconclusive, because it depends upon the supposition that a finite mind is capable of only a finite number of distinguishable states. (p.197)
A supposition which, he points out, is not established, though it strikes me that could be a clue to a new quantum explanation.
It should also be pointed out that, according to Wang, Gödel indeed held the strong convictions that the brain does NOT function basically like a computer, and that the brain does NOT suffice for the explanation of mental phenomena. Gödel led himself to a non-material idealism, and a Leibnitzian monadology, with a central monad [God].
This is a much, much stronger and deeper example than Thomas Nagel's claim that he knows that his reason has transcendental access to objective reality because he knows and knows that he knows, etc., that if the sun rises on his right that he is going north, and because he knows and knows that he knows, etc., that there ought to be an inheritance tax.
There do, after all, appear to be three possibilities: (i) pure this-universe materialism, (ii) this-universe as the surface with an ocean beneath, and (iii) we are meat-puppets. (i) and (ii) are indistinguishable--unless whatever is in the ocean, יהוה or Mind or Eternity or Cthulhu or ♞☀♉♈♁⚑, the bored third-grader doing xys history project in 2000000000 A.D., decides to break the glass and look in. Between (i) and (ii) on the one hand and (iii) on the other--well, "we are meat-puppets activated by soul-stuff dwelling elsewhere" looked like a very good hypothesis from a Bayesian perspective in the time of Descartes, and a less-good hypothesis now: nobody yet looking at the brain has yet found anything in there other than quarks, gluons, electrons, and photons doing what they normally, materialistically do. The materialist problem is very had, but the gaps in it are growing small…