Earticle

This study aims to develop a robotic mood transition for a robot that can express emotional states in a human-robot interaction system. The proposed method of this paper ensures that the user's emotions are reflected as the robot's emotions and that the robot can express continuous mood transitions. A robot's mood transition is generated based on the user’s current facial expression to achieve the interaction between the patient robot and caregivers. Thus, we present an advanced robotic system based on human-robot interaction in various fields by utilizing the robot’s facial expression method.

규칙 기반 Human–Robot Interaction(HRI)은 서비스 및 교육 로봇 환경에서 널리 활용되고 있으나, 실제 상호작용 과정에서는 반복 인사, 반응 지연, 의미 불일치와 같은 붕괴 현상이 빈번히 발생한다. 기존 연구는 이를 규칙 기반 접근의 일반적 한계로 진단해왔으나, 실제 서비스 맥락에서 붕괴가 어떠한 조건에서 발생하며 구조적으로 어떻게 설명될 수 있는지에 대한 경험적 분석은 충분하지 않았다. 본 연구는 Pepper–Choregraphe 기반 규칙형 HRI를 대상으로 20명의 참여자(총 144분 상호작용 데이터)를 관찰하여 29개의 붕괴 에피소드를 분석하였다. 그 결과, 붕괴는 네 가지 유형—규칙 반복 실행(31.0%), 질문–응답 흐름 단절(31.0%), 인식 노이즈로 인한 과잉 반응(17.2%), 사용자 맥락과의 의미 불일치(20.7%)—으로 나타났다. 붕괴는 사용자 행동의 변동성, 반응 지연, 인식 값의 불안정성, 맥락 누적 부족 등이 결합될 때 반복적으로 발생하는 경향을 보였다. 본 연구는 규칙 기반 HRI의 붕괴를 개별 오류가 아닌 구조적 특성에서 비롯된 반복 패턴으로 해석하고, 멀티 에이전트 관점에서 그 원인을 설명한다. 이는 규칙 기반 HRI의 적용 범위와 한계를 보다 정밀하게 규정하고, 향후 안정화 메커니즘 및 적응형 HRI 설계를 위한 기초를 제공한다.

Rule-based Human–Robot Interaction (HRI) is widely utilized in service and educational robot environments; however, breakdowns such as repetitive greetings, response delays, and semantic mismatches frequently occur during actual interactions. While prior studies have diagnosed these as general limitations of rule-based approaches, empirical analyses of the specific conditions under which breakdowns arise in real service contexts and their structural explanations remain insufficient. This study observed 20 participants (144 minutes of interaction data) in a Pepper–Choregraphe-based rule-type HRI environment, analyzing 29 breakdown episodes. The results identified four breakdown types—repetitive rule execution (31.0%), question-response flow disruption (31.0%), over-reactive responses due to perceptual noise (17.2%), and semantic mismatch with user context (20.7%). Breakdowns recurred when factors including user behavior variability, response delays, perceptual instability, and context accumulation deficits combined. This study interprets rule-based HRI breakdowns not as isolated errors but as recurring patterns stemming from structural characteristics, explained through a multi-agent perspective. It precisely delineates the applicability and limitations of rule-based HRI, providing a foundation for future stabilization mechanisms and adaptive HRI designs.

혼합현실(Mixed Reality, MR), 가상현실(Virtual Reality, VR)과 같은 인간-컴퓨터 상호작용(Human-Computer Interaction, HCI) 기술의 발전에 따라, 온디바이스 환경에서 인공지능 모델과 사용자 간의 실시간 상호작용의 중요성이 커지고 있다. 특히 별도의 컨트롤러 없이 1인칭 시점에서 손동작을 인식하는 기술은 사용자 친화적인 인터페이스를 구축한다는 역할을 하지만, 입력 영상의 시점 및 환경 변화 등으로 인해 자기가림(self-occlusion) 현상이 빈번하게 발생하며, 이는 손동작 인식 성능 저하의 주요 원인으로 작용한다. 본 논문에서는 이러한 문제를 완화하기 위해 영상 기반 공간적 특징과 손 관절 기반 시계열 특징을 결합한 멀티모달 모델을 제안한다. 자체 구축한 데이터셋을 활용한 실험을 통해, 멀티모달 모델이 단일 모달 기반 모델보다 향상된 인식 성능을 보임을 확인하였다. 나아가 제안된 손동작 인식 모델이 가상현실 환경에서의 인간-로봇 상호작용(Human-Robot Interaction, HRI)으로 확장될 수 있는 가능성을 논의한다. 본 연구는 1인칭 시점 멀티모달 손동작 인식 기술에 대한 실시간 환경 적용과 향후 VR-HRI 융합 환경에서의 응용 가능성을 제시한다는 점에서 의의를 가진다.

With the advancement of Human-Computer Interaction(HCI) technologies such as Mixed Reality(MR) and Virtual Reality(VR), the importance of real-time interaction between users and artificial intelligence models in on-device environments has been increasingly emphasized. In particular, hand gesture recognition from a first-person perspective without the use of physical controllers plays a crucial role in enabling user-friendly interfaces. However, first-person vision-based hand gesture recognition often suffers from frequent self-occlusion caused by viewpoint changes and environmental variations, which significantly degrades recognition performance. To alleviate these challenges, this paper proposes a multimodal hand gesture recognition model that combines video-based spatial features with joint-based temporal features. Experimental results obtained using a self-collected dataset demonstrate that the proposed multimodal model achieves improved recognition performance compared to single-modality-based approaches. Furthermore, this study discusses the potential of extending the proposed hand gesture recognition as a key intermediate layer that organically connects VR interfaces and HRI systems. This work highlights the applicability of first-person multimodal hand gesture recognition in real-time environments and presents its potential for future applications in integrated VR-HRI interaction frameworks.

대부분의 인간 로봇 상호작용 방법은 제스쳐, 음성, 터치스크린, 버튼등과 같이 로봇에 "직접적으로" 향하 거나 접촉을 통해 이루어지는 것이 보통이다. 그러나 이러한 전통적인 방법은 아직 기술의 완성도 때문에 많은 경우에 아직 사용자에게 불편함을 주고 있는 것이 사실이다 (예: 인식 불량). 이 논문에서는 간접적이 나 보다 안정적인 방법으로 로봇과 상호작용 하는 방법을 소개 한다. 세부적인 예로서, 프로젝터를 로봇에 장착하여 가깝고 넓은 면에 정보를 투사하고 인터액션 공간을 제공 하는 것이다. 이 논문에서는 이러한 간 접적인 HRI 방법의 요구사항과 기술적인 이슈들을 정리 하고, 두 실제 프로젝터 로봇의 구현을 소개 한다. 더 나아가서, 실험을 통하여 프로젝터 로봇을 이용한 간접적 HRI 방법의 효과와 장단점을 살펴본다. 실험 결과는 간접적 HRI 가 정보를 보다 명확하게 전달하는 데는 효과적이나 휴머노이드 로봇과 비교 하여 친밀 성이나 감정적인 접근성이 떨어지는 것으로 나타났으나 이러한 면은 프로젝터를 통한 컨텐츠나 로봇 자체 의 외부 모습을 바꾼다면 극복 할 수 있는 문제로 보여 진다.

Most current forms of Human Robot Interaction (HRI) use the “direct” interaction/contact with the robot, often causing unexpected inconvenience and usability problems. In this paper, we present a novel “indirect” HRI paradigm using projected display and remote multimodal interaction. The projector (on the moving robot) projects information on the nearby surface and provides a relatively large area through which indirect interaction can occur among the users and the robot. We first discuss and explain the indirect HRI paradigm and technical issues in realizing it in the form of a projector robot. We present two realizations: (1) a moving robot kiosk that projects information around itself on the floor and communicates with multiple users, and (2) a robot museum guide that projects information on the wall near the interested artifact. Experiments were carried out to assess the effectiveness of the proposed HRI paradigm, comparing various aspects of the indirect interaction to those of direct interaction with a humanoid robot and even a human. The experimental results indicate that while the proposed indirect HRI is effective in conveying information more clearly, it lacks the quality of friendliness and affect due to its current outer appearance. However, such aspects could be overcome by more friendly robot and content design.

To improve the robot's locomotion performance, it definitely needs the HRI (Human Robot Interaction) technology, the location awareness technology, and the real time location correction technology. In this paper, an HRI method that uses a circular coordinate system is proposed. The proposed method can implement robot locomotion with variable speed and directions instead of conventional method that controls speed and direction separately. Also, the nine characteristics of proposed circular coordinate system are to be described and the principles of user interfaces using the circular coordinate system are to be analyzed. For the performance evaluation, we have used a mobile robot that can be controlled through the wireless LAN and can perform differential drive. In addition, we have configured and tested a smart phone environment that can control the robot. The evaluation results show that the circular coordinate system reduces the locomotion time with accuracy.

Gesture based natural human-robot interface is an important function of robot teleoperation system. Such interface not only enables users to manipulate a remote robot by demonstration, but also ensures user-friendly interaction and software reusability in developing a networked robot system. In this paper, an application of gesture-based remote human-robot interaction is proposed using a Kinect sensor. The gesture recognition method combines depth information with traditional Camshift tracking algorithm by using Kinect and employs HMM in dynamic gesture classification. A Client/Server structured robot teleoperation application system is developed, which provides a favorable function of remotely controlling a dual-arm robot by gestural commands. Experiment results validate the practicability and effectiveness of the application system.

$\textbullet$ Developing "muscle shit" providing muscular support $\textbullet$ Based on a new concept: wearable robot $\textbullet$ Be applicable directly to human $\textbullet$ McKibben artificial muscles are sewn into a garment

For efficient interaction between humans and robots, robots should be able to understand the meaning and intention of human behaviors as well as recognize them. This paper proposes an interactive human intention reading method in which a robot develops its own knowledge about the human intention for an object. A robot needs to understand different human behavior structures for different objects. To this end, this paper proposes a hierarchical behavior knowledge network that consists of behavior nodes and directional edges between them. In addition, a human intention reading algorithm that incorporates reinforcement learning is proposed to interactively learn the hierarchical behavior knowledge networks based on context information and human feedback through human behaviors. The effectiveness of the proposed method is demonstrated through play-based experiments between a human and a virtual teddy bear robot with two virtual objects. Experiments with multiple participants are also conducted.

In this paper, we developed not only a reliable sound localization system including a VAD(Voice Activity Detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we proposed a way to integrate three systems in the human-robot interaction to compensate errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from undesired directions effectively. For the purpose of verifying our system's performances, we installed the proposed audio-visual system in a prototype robot, called IROBAA(Intelligent ROBot for Active Audition), and demonstrated how to integrate the audio-visual system.

This paper describes the configuration of a multi-agent system that can recognize human intentions. This system constructs ontologies of human intentions and enables knowledge acquisition and sharing between intelligent agents operating in different environments. This is achieved by using a bi-directional associative memory network. The process of intention recognition is based on fuzzy association inferences. This paper shows the process of information sharing by using ontologies. The purpose of this research is to create human-centered systems that can provide a natural interface in their interaction with people.

In network human-robot interaction, human can access services of a robot system through the network The communication is done by interacting with the distributed sensors via voice, gestures or by using user network access device such as computer, PDA. The service organization and exploration is very important for this distributed system. In this paper we propose a new agent-based framework to integrate partners of this distributed system together and help users to explore the service effectively without complicated configuration. Our system consists of several robots. users and distributed sensors. These partners are connected in a decentralized but centralized control system using agent-based technology. Several experiments are conducted successfully using our framework The experiments show that this framework is good in term of increasing the availability of the system, reducing the time users and robots needs to connect to the network at the same time. The framework also provides some coordination methods for the human robot interaction system.

There are different domains in which convergence might take place. Within the arts we can think of the convergence of different forms of expressive media such as music, theater, film, and dance. Concerning media, we also may think of convergence of technical representations of different kinds of modalities as in multimedia and multi-modal environments. Furthermore, in the sciences one can think of convergence of different approaches to research questions. Artistic human-robot interaction design with schema theory might provide a basis for the study of subjective and objective mind using new media art environments as laboratories. The subjective or individual mind is studied within the humanities. Cognitive science as situated cognition and the humanities might converge in investigating the mind. Cognitive musicology is given as an example of a possible converging way of studying the subjective and objective mind. The use of robots plays an essential part of such a converging approach. Artistic human-robot interaction design is proposed as a research strategy. Social schema theory is related to artistic human-robot interaction design. The humanities, I think, should actively take part in the formation of a society in which humans are socially interacting with robots or artificial agents in daily life, art, entertainment, edutainment, therapy, and teaching.

In this paper, we propose a robust fingertip extraction and extended Continuously Adaptive Mean Shift (CAMSHIFT) based robust hand gesture recognition for natural human-like HRI (Human-Robot Interaction). Firstly, for efficient and rapid hand detection, the hand candidate regions are segmented by the combination with robust $YC_bC_r$ skin color model and haar-like features based adaboost. Using the extracted hand candidate regions, we estimate the palm region and fingertip position from distance transformation based voting and geometrical feature of hands. From the hand orientation and palm center position, we find the optimal fingertip position and its orientation. Then using extended CAMSHIFT, we reliably track the 2D hand gesture trajectory with extracted fingertip. Finally, we applied the conditional density propagation (CONDENSATION) to recognize the pre-defined temporal motion trajectories. Experimental results show that the proposed algorithm not only rapidly extracts the hand region with accurately extracted fingertip and its angle but also robustly tracks the hand under different illumination, size and rotation conditions. Using these results, we successfully recognize the multiple hand gestures.

Nowadays, with progresses in robotic science, the design and implementation of a mechanism for human-robot interaction with a low workload is inevitable. One notable challenge in this field is the interaction between a single human and a group of robots. Therefore, we propose a new comprehensive framework for single-human multiple-robot remote interaction that can form an efficient intelligent adaptive interaction (IAI). Our interaction system can thoroughly adapt itself to changes in interaction context and user states. Some advantages of our devised IAI framework are lower workload, higher level of situation awareness, and efficient interaction. In this paper, we introduce a new IAI architecture as our comprehensive mechanism. In order to practically examine the architecture, we implemented our proposed IAI to control a group of unmanned aerial vehicles (UAVs) under different scenarios. The results show that our devised IAI framework can effectively reduce human workload and the level of situation awareness, and concurrently foster the mission completion percentage of the UAVs.

This paper presents a mobile robot carrier designed to carry a person using two modes: a mechanism with full support and another with partial support. The carrier is driven through guided control from an operator. Applied force is sensed by a force sensor mounted on the bottom of the handle. The measured force is filtered by the impedance function that generates the desired velocity to drive the motors. The inner loop PID controller is then required to follow the desired velocity, which is the reference input to the system. The impedance function is designed to make the driving condition comfortable for the driver by smoothing out abrupt starts and stops. Feasibility tests on the application of the impedance force control method to the carrier robot have been performed through experimental case studies aimed at evaluating the comfort level of prospective users: one is on a full support case when a user is riding on the carrier and another on a partial support case where the user is pushing the carrier.