**4.1 Robot platform: DiGORO**

Figure 6 shows the robot "DiGORO" we previously developed (28). It is composed of the following hardware:


Service Robots 9

Learning Novel Objects for Domestic Service Robots 265

Bring me "Choco"

Dining table.

Where ?

(a) (b)

(c) (d)

start position and handed over the object to the user.

turns toward the object and says "I found X."

applied system through the task called supermarket task.

**5.1 Experiment 1: Evaluation of proposed method**

6. The robot goes to P (Fig.8(b)).

**5.1.1 Segmentation accuracy**

**5. Experiments**

phase.

segmentation.

Fig. 8. Scenery of the supermarket task. (a)The user ordered the robot to bring "choco", and the robot asked the user for needed information. (b)The robot navigated to the place where the user ordered. (c)The robot found the object and grasped it. (d)The robot returned to the

7. The robot looks around to find X using plane detection. If the robot can find X, the robot

In this section, experiments have been conducted to evaluate the proposed method and the

In this experiment, we evaluated segmentation accuracy. It is necessary to segment the object region from a complex background which generally exists in the domestic environment. We used motion attention discussed in Section 2 because the object is held by a user in the learning

The experiment was carried out in an ordinary living room, shown in Fig.9. We used 120 ordinary objects, as shown in Fig.10. A user taught the robot each object by showing and telling its name to the robot. The robot acquired 40 consecutive frames for each object and extracted the target object region from each image. Figure 11 shows examples of object

8. The robot grasps the object (Fig.8(c)) and returns to the start position (Fig.8(d)). 9. The task is completed when the robot brings three objects in total. Otherwise go to 1.

Fig. 7. Scenery of the learning phase. (a)The user gives a command for the robot to learn a new object. (b)The user shows the object from various directions.

### **4.2 Learning of novel object**

Before the task, we need to teach objects to the robot. The procedure of teaching objects is summarized below.


If the user moves the object incorrectly, the motion attention cannot segment out the object. For example, if the user moves the object from outside the observed frame into it, the unintentional region is segmented out. To avoid such a situation, we instructed the user to say "DiGORO, this is X", while showing the object to the robot and to keep on showing it until the robot says, "I've memorized X."

#### **4.3 Supermarket task**

Supermarket is a task that the robot brings three specified objects. The detailed procedure of this task is summarized below.


<sup>2</sup> In this paper, the utterances are translated into English.

8 Will-be-set-by-IN-TECH

(a) (b)

4. Visual features (SIFT descriptors and color histogram) are calculated.

1. The user says "DiGORO, bring me X (X is an OOV word)" (Fig.8(a)) .

to 1. If the user says "DiGORO, yes" go to next.

4. The user says "It is P (P is a name of the place.)".

If the user says "DiGORO, yes" go to next.

<sup>2</sup> In this paper, the utterances are translated into English.

new object. (b)The user shows the object from various directions.

Fig. 7. Scenery of the learning phase. (a)The user gives a command for the robot to learn a

Before the task, we need to teach objects to the robot. The procedure of teaching objects is

1. The user shows the object to the robot and say "DiGORO, this object is X (X is an OOV word)" in Japanese 2. The phoneme sequence and sound of the OOV word are extracted

3. The user moves the object. Then 40 images of the object are segmented out and captured

5. The phoneme sequence of the OOV word, sound of the OOV word and visual features of the object are registered in the object database. Then the robot says "I've memorized X." If the user moves the object incorrectly, the motion attention cannot segment out the object. For example, if the user moves the object from outside the observed frame into it, the unintentional region is segmented out. To avoid such a situation, we instructed the user to say "DiGORO, this is X", while showing the object to the robot and to keep on showing it until the robot says,

Supermarket is a task that the robot brings three specified objects. The detailed procedure of

2. The robot says "I'll bring you X. Is that correct?" Then if the user says "DiGORO, no", go

5. The robot says "I'll go to P. Is that correct?" Then if the user says "DiGORO, no", go to 4.

This object is "Kuma".

**4.2 Learning of novel object**

from the user's speech (Fig.7(a)). 2. The robot says "I start learning of X."

summarized below.

(Fig.7(b)).

"I've memorized X."

**4.3 Supermarket task**

this task is summarized below.

3. The robot says "Where is X?"

Start learning.

(c) (d)

Fig. 8. Scenery of the supermarket task. (a)The user ordered the robot to bring "choco", and the robot asked the user for needed information. (b)The robot navigated to the place where the user ordered. (c)The robot found the object and grasped it. (d)The robot returned to the start position and handed over the object to the user.

