**11. Discussion**

Although I enjoyed playing with the two 2012 and 2013 Loebner Turing test champions, I will now present my own serious reservations concerning the Turing test. My reservations must be read in view of the 2014 claim that a program "passed" the test be presented in detail below. In my view, some of the shortcomings of the Turing test as organized for the Loebner Prize contest that lessen the value of its results with respect to evaluating artificial intelligence and differentiating from human intelligence and consciousness are:


The questions appropriate for uncovering the nonhuman nature of a computer system should utilize knowledge of at least some of the following human capabilities that usually are manifested during the dialog of a human with a judge:

1.Memorization of previous stages of a dialog.


These human capabilities inspire artificial intelligence researchers in their efforts to advance the relevant technology. It is expected that programs written by programmers unfamiliar with the state of the art would fail to answer correctly questions that utilize these capabilities. In my two experiments reported above, I checked mainly for operational consciousness in line with our book's [6] main theme and the results show that this was sufficient for swiftly uncovering the machine nature of the two systems I interacted with. In addition, some lack of coherence can be observed apparently as a result of lack of dialog memorization by these systems.

I performed my experiments during the 2012–2013 period. I was subsequently surprised to learn that in the 2014 Loebner contest with the Turing test a program named "Eugene Goostman" succeeded to "pass" the test. The 2014 Turing test contest was run in London under the auspices of the University of Reading and the British Royal Society.

Cybernetics professor Kevin Warwick of the University of Reading who announced the result of the Test is reported in his Christian Monitor interview of 9th of June 2014 to state the following:

*"Some will claim that the test has already been passed. The words Turing test have been applied to similar competitions around the world. However, this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing test does not set the questions or topics prior to the conversations. We are, therefore, proud to declare that Alan Turing's test was passed for the first time on Saturday."*

The Christian Monitor's reporter commented:

"Despite Prof. Warwick's praise, a conversation with Mr. Goostman is decidedly underwhelming. He often appears not to be listening, fails to answer direct questions, and is inappropriately sarcastic and aggressive.

So, can machines think? We posed this question to Goostman and got an uncharacteristically direct answer. 'Machines can not think" Goostman told the Monitor."

The abovementioned difference of opinion between the media and scientists became most apparent in the wave of publicity and reactions that followed the announcement. The reactions ranged from naïve statements such as "supercomputer"

#### *Analysis Dialogs and Machine Consciousness DOI: http://dx.doi.org/10.5772/intechopen.112476*

first to pass Turing test, convinced judges it is alive to sober scientific analyses in line with my reservations presented above that tried to rationalize the event. The writer of the above mentioned "headline" falls victim of the usual confusion between the word "computer," which is used to refer to the general-purpose stored-program digital electronic device with the phrase "computer system" that refers to a computer equipped and running executing the instructions of a "program". This set of computer instructions, when executed by the computer, makes it behave as a special-purpose computer. When on September 15th, 2014, I googled the name of the 2014 prize winner program I got 125.000 snippets as the result of my search. This shows the wide publicity of the Turing test of the 2014 Loebner Prize contest. However, I tried in vain several times to interact with Eugene Goostman, but I always found it inactive.

I will now briefly review some of the recent scientific works that deal with the so called "Turing test" that are related to the definition of "artificial intelligence" since it is usually stated that if a program passes the test, it is then considered "intelligent" and hence satisfies the definition of "artificial intelligence." I must first emphasize that Turing in his seminal 1950 paper did not exactly propose the so called "Turing test", as organized for the Loebner Prize contest as a test of intelligence. In his own words in that paper, it is stated in page 442:

*"I believe that in about 50 years' time, it will be possible to program computers, … , to make them play the imitation game so well that an average interrogator will not have more than 70 percent chance of making the right identification after 5 minutes of questioning. The original question "Can machines think?" I believe it to be too meaningless to deserve discussion. … Conjectures are of great importance since they suggest useful lines of research."*

I think that the important fact is the formulation of his conjecture, which we have to judge whether it was verified in 2014, a mere 14 years later than the year specified by his prediction. It should be noted that there are some inexact phrases in Turing's conjecture that make this verification rather hard, for example, "average interrogator," "70 percent chance," and "five minutes of questioning". The following questions, among others, must be answered before the conjecture can become precise enough for scientific scrutiny:


Other scientists have also criticized the Turing text from both philosophical and technical points of view. Among other things, the fact is criticized that Turing did not provide a definition of the verb "think."

This is used in formulating the question "do machines think?". In view of such criticisms a new test to replace the Turing test called the "Winograd schema challenge" has been established.

The "Winograd schema challenge "has been suggested as a conceptually and practically appealing alternative to the Turing test.

The Winograd schema challenge is an alternative to the Turing test that is supposed to provide a more accurate measure of artificial intelligence. Rather than base the test on the sort of freeform conversation suggested by the Turing test, the Winograd schema challenge poses a set of multiple-choice questions.

An example of a Winograd schema question is the following:

*"The trophy would not fit in the brown suitcase because it was too big. What was too big? Answer 0: the trophy or Answer 1: the suitcase?"*

A human who answers these questions correctly typically uses his abilities in spatial reasoning, his knowledge about the typical sizes of objects, and other types of commonsense reasoning, to determine the correct answer. The 2015 Commonsense Reasoning Symposium, to be held at the AAAI Spring Symposium at Stanford from March 2325, 2015, will include a special session for presentations and discussions on progress and issues related to this Winograd schema challenge. Contest details can be found at: http://commonsensereasoning.org/winograd.html.

A program succeeding in the Winograd schema challenge needs reasoning abilities and background knowledge. It involves a coreference resolution task. The complexity of the task is increased by the fact that the Winograd sentences are not constrained by domain or sentence structure.

The winner program that meets the baseline for human performance will receive a grand prize of \$25,000. In the case of multiple winners, a panel of judges will base their choice on either further testing or examination of traces of program execution. If no program meets those thresholds, a first prize of \$3000 and a second prize of \$2000 will be awarded to the two highest-scoring entries. In the case of teams, the prize will be given to the team lead whose responsibility will be to divide the prize among their teammates as appropriate.

In [13] a pronoun resolver system is developed for the confined domain Winograd sentences. A classifier or filter was developed which takes input sentences and decides to accept or reject them based on some criteria.

Furthermore, he has developed four answering modules, which use world knowledge and inference mechanisms to try and resolve the pronoun.

In [14] they examine resolving complex cases of definite pronouns, specifically those for which traditional linguistic constraints on coreference, as well as commonly used resolution heuristics are not useful.

In [15] a test quite different from both the Turing test and the Winograd challenge is studied, namely the "Raven's Progressive Matrices" (RPM) Test. This test is based upon purely visual representations. A technique is introduced based on the calculation of confidence in an answer and the automatic adjustment of level of resolution if that confidence is insufficient.
