What kind of a machine is writing this?

One of the most enjoyable things that ever happened to me in the God business was meeting John Lucas, the philosopher who came up with one of the early arguments against AI. It is much more subtle than it’s usually given credit for, but you can find it from his web site, and here is the key paragraph, which I reproduce for its elegance:

The argument is a dialectical one. It is not a direct proof that the mind is something more than a machine, but a schema of disproof for any particular version of mechanism that may be put forward. If the mechanist maintains any specific thesis, I show that [146] a contradiction ensues. But only if. It depends on the mechanist making the first move and putting forward his claim for inspection. I do not think Benacerraf has quite taken the point. He criticizes me both for “failing to notice” that my ability to show that the Gödel sentence of a formal system is true “depends very much on how he is given that system”2 and for putting the argument in the form of a challenge in which I challenge the mechanist to produce a definite specification of the Turing machine that he claims I am.3 Benacerraf thinks that the argument by challenge reduces the argument to a mere contest of wits between me and the mechanist. But we are not trying to see who can construct the smartest machine, we are attempting to decide the mechanist’s claim that I am a machine: and however clever the mechanist is, even if he were not a mere man but Satan himself, I, or at least an idealised and immortal I, could out-Gödel it, and see to be true something it could not. Benacerraf protests that “It is conceivable that another machine could do that as well.” Of course. But that other machine was not the machine that the mechanist was claiming that I was. It is the machine which I am alleged to be that is relevant: and since I can do something that it cannot, I cannot be it. Of course it is still open for the mechanist to alter his claim and say, now, that I am that other machine which, like me, could do what the first machine could not. Only, if he says that, then I shall ask him “Which other machine?” and as soon as he has specified it, proceed to find something else which that machine cannot do and I can. I can take on all comers, provided only they come one by one in the sense of each being individually specified as being the one that it is: and therefore I can claim to have tilted at and laid low all logically possible machines. An idealised person, or mind, may not be able to do more than all logically possible machines can, between them, do: but for each logically possible machine there is something which he can do and it cannot; and therefore he cannot be the same as any logically possible machine.

Of course this entirely fails to address the Dennett/Minsky point that we are shifting coalitions of machines — against that Lucas simply says that it is indeed a different question. No one can seriuosly doubt that we’re built from machines. But something assembled in that way can be more — or less — than a single logically coherent machine, which is the kind of thing that Turing was concerned with. Something assembled in that way may turn out to be a someone; that still doesn’t make them — or me — a single machine. This point seems to me so uncontroversial that I am still surprised by the rage which Lucas’s argument arouses.