Earticle

We generated a high resolution 3D object using a commercial DSLR camera and the low depth information of Kinect. High resolution depth map was obtained through camera calibration between the DSLR and Kinect sensor. 3D mesh model was created by the high resolution depth information and mapped with RGB color value. The experimental result showed that it is possible to create high resolution 3D object from the depth camera. This paper is expected to be applied to reality game graphic.

정보통신의 급격한 발전으로 인해 우리는 정보화 사회의 스마트시대에 살고 있다. ICT(정보통신기술)가융합된 스마트 팩토리는 대표적인 4차 산업혁명 시대의 기술로 주목받고 있다. 공장은 물류, 제조 및 유통 등 인간의 생산활동에 많은 부분을 차지한다. 이렇게 공장은 정보화 사회에 들어서면서 자동화에 있어서 많은 발전을 이루어왔다. 본 논문에서는 Kinect를 활용해 모션을 통한 공장의 편의시스템 구현에 대하여 연구하였다. Kinect는 사용자의 행동(Motion)을 촬영하고 영상의 정보를 통해 시스템을 제어하기 때문에 사용자에게 여러 가지 편의를 제공한다. 이 연구를 통해 영상처리로 스마트 팩토리를 구현하도록 했고 공장에서 일하는 사람들의 여러 가지 편의를 제공함으로써 생산의 능률에 영향을 줄 수 있을 것으로 기대 된다.

Due to the rapid development of information and communication, we live in the smart age of information society. Smart Factory, which integrates Information and communication technology, is being hailed as the technology of the 4th industrial revolution. As a result of entering the information society, the factory has made a lot of progress in automation. In this thesis, we used kinetization to research and implement the leisure system of the factory through motion. Kinect provides users with various convenience because they can take advantage of the user's actions and control the system through information. This study is expected to produce a smart factory with video processing, and it is expected to affect the efficiency of the production by providing various conveniences for people working in factories.

This research is the preliminary study for development of narrative contents for improving reading skills and behavior controlling capabilities of children with ADHD. Microsoft's Kinect, a non-contact motion sensing device, and NeuroSky's MindWave, a BCI device, are connected to Unity 3D engine to capture user's motion and brainwaves, respectively. The given narrative contents interactively respond to children with ADHD while they are reading the contents and performing the given tasks presented in the behavior training game system. This study examines whether the contents are suitable for the first and second grader children. For this, The interaction with the contents is first observed by five adults and two children. Then the system is tested by three ordinary first graders. During the experiments, the children easily figure out the given texts and do what the game contents drive for the player to react. All three kids concentrate on the reading contents without leaving their seats for average 37 minutes. This experiment shows that the contents are properly designed for the first and second graders in terms of reading speed, reading completion time, complexity level, and task performance. Based on this experimental study, the interactive narrative contents will be used to conduct full-scale experiments with children with ADHD.

CIn this paper, I discuss the minimization of the user interface of the interactive fairy tale content based on BCI and Motion Sensing Technology. For this reason, 3 version intervention interfaces of the interactive fairy tale content with Microsoft's Kinect as a non-contact motion sensing device and NeuroSky's MindWave EEG biosensor device as a BCI interface were produced and experimented. The experiments have been accomplished for four children of the first or the second graders by examining three different interactive intervention interfaces: 1) attention indicator by sparkling light; 2) the attention indicator plus behavior instructions; and 3) an indication bar for the attention level plus behavior instructions. As a result of this experimental observations, it shows that the visualized interface such as icons helps the participating children communicate with the fairy tale content system. In addition, they do communicate well without the visualized graphical interfaces when they are demanded by the realistic actions. Whereas, some communication problems have been found neither the realistic actions are requested nor the attention indicators are provided definitely.

This research introduces the interactive fairy tale system based on BCI and Motion Sensing Technology for Children with ADHD. ADHD is a specific disorder which shows deficits in sustained attention, behavioral inhibition, resistance to distraction, and the regulation of one’s activity level to the demands of a situation (hyperactivity or restlessness). The interactive fairy tale system implemented on the Unity3D engine is expected to keep track of a player's time-varying brainwave and motion logs through NeuroSky’s MindWave Brain-Computer Interface (BCI) and Microsoft’s Kinect motion sensing technology. The proposed system helps us take a closer look at the player's behavior than performing a video shoot or direct observation. In addition, as the system keeps the observed data, it is possible to constantly track the changes on the player's brainwave and behavior, and synthetically assess the effect of the proposed approach from many different aspects.

최근 산업의 발달로 인한 빌딩 등의 인텔리젼트화가 빠른 속도로 진행됨에 따라, 사무실환경, 주거환경 에 있어서 쾌적성, 효율성,편리성을 추구하게 되면서, 인간은 생활에서 다양한 디바이스를 사용 할 수 있 게 되었다. 특히, 스마트TV 과 스마트폰 등이 널리보급되면서 인간과 스마트 기기간의 인터랙션에 대한 관심이 증대되고 이러한 인터랙션을 위해 다양한 방법들이 연구되었지만 사람이 컨트롤러를 이용하여 인터랙션 해야 한다는 불편함과 한계점이 발생하였다. 본 논문에서는 사용자가 LED 제어를 컨트롤러 없이 제스처(손동작)를 통해 간편하게 인터랙션 및 제어 할 수 있도록 Kinect를 이용한 방법을 소개한다. Kinect 센서로부터 획득한 제스처의 관절 정보를 이용하 여 발광 소재를 제어하는 인터페이스를 설계 및 구현하였다. 제스처(손동작)을 구현한 인터페이스를 통 해 LED를 사용자가 원하는 대로 개별 제어 가능하다. 본 논문에서 설계한 인터페이스는 LED 제어 및 다 양한 분야에 응용하여 유용하게 사용될 수 있을 것으로 기대된다.

Recently, building is rapidly develop more intelligently because of the development of industries. And people seek such ascomfort, efficiency, and convenience in office environment and the living environment. Also, people were able to use avariety of devices. Smart TV and smart phones were distributed widely so interaction between devices and human has beenincrease the interest. A various method study for interaction but there are some discomfort and limitations using controllerfor interaction. In this paper, a user could be easily interaction and control LED through using Kinect and gesture (hand gestures) withoutcontroller. we designed interface which is control LED using the joint information of gesture obtained from Kinect. A usercould be individually controlled LED through gestures (hand movements) using the implementation of the interface. Weexpected developed interface would be useful in LED control and various fields.

Chroma key technique which composes images by separating an object from its background in specific color has restrictions on color and space. Especially, unlike general chroma key technique, image composition for stereo 3D display requires natural image composition method in 3D space. The thesis attempted to compose images in 3D space using depth keying method which uses depth information of Microsoft Kinect. Object was converted into point cloud type in 3D space after separating it from its background according to depth information. The image in which 3D virtual background and object are composed obtained and played stereo 3D images using a virtual camera. The research showed naturally composed stereo 3D images as well as the feasibility of applying stereoscopic 3D mixed reality image technology.

Objective To compare optical motion capture system (MoCap), attitude and heading reference system (AHRS) sensor, and Microsoft Kinect for the continuous measurement of cervical range of motion (ROM).Methods Fifteen healthy adult subjects were asked to sit in front of the Kinect camera with optical markers and AHRS sensors attached to the body in a room equipped with optical motion capture camera. Subjects were instructed to independently perform axial rotation followed by flexion/extension and lateral bending. Each movement was repeated 5 times while being measured simultaneously with 3 devices. Using the MoCap system as the gold standard, the validity of AHRS and Kinect for measurement of cervical ROM was assessed by calculating correlation coefficient and Bland?Altman plot with 95% limits of agreement (LoA). Results MoCap and ARHS showed fair agreement (95% LoA<10o), while MoCap and Kinect showed less favorable agreement (95% LoA>10o) for measuring ROM in all directions. Intraclass correlation coefficient (ICC) values between MoCap and AHRS in ?40o to 40o range were excellent for flexion/extension and lateral bending (ICC>0.9). ICC values were also fair for axial rotation (ICC>0.8). ICC values between MoCap and Kinect system in ?40o to 40o range were fair for all motions.Conclusion Our study showed feasibility of using AHRS to measure cervical ROM during continuous motion with an acceptable range of error. AHRS and Kinect system can also be used for continuous monitoring of flexion/extension and lateral bending in ordinary range.

최근 디지털 네이티브 학습자를 위한 교수방법인 게임 활용 학습에 대한 관심이 증가하고 있다. 이에 본 연구 에서는 초등학교 역사수업에 맞춰 개발된 Kinect 게임 활용 수업을 실시한 후 성공적으로 학습에 참여한 것으로 관찰된 학생 14명을 대상으로 인터뷰를 실시하여 참여적 학습 경험을 분석하였다. 연구 결과 학생들은 경쟁과 협력 놀이 요소에서는 ‘목표 달성’과 ‘상호작용’을, 디지털 놀이 요소에서는 ‘다감각적 자극’, ‘환상과 호기심’, ‘정 확한 피드백’, ‘우연’, ‘통제감’을, 신체활동 놀이 요소에서는 ‘체감된 인지’, ‘실재감’을 경험하면서 참여적 학습을 하는 것을 확인할 수 있었다. 이를 통해 Kinect 게임 활용 수업을 참여적 학습을 위한 교수학습방법으로 도입할 기초자료로서 도움을 주고자 한다.

Recently, there is an increasing interest in game‐based learning as a teaching method for digital native learners. This study set kinect game contributes to engaged learning as the competition and cooperation play (achievement goals, interaction), the digital game play (multisensory stimulation, fantasy and curiosity, chance, accurate feedback, control), and the body movement play (embodied cognition, presence). After performing classes using the motion recognition game developed for the elementary school history class, this study conducted semi‐structured interviews based on engaged learning elements of kinect game based learning for students who were successfully participating in learning. In the result, each element appeared to a successful learner. Based on these results, this study hopes to assist researchers as a basic evidence to introduce kinect game-based learning for engaged learning.

최근 카메라, 캠코더 및 CCTV 등의 사용이 활발해지면서 영상 처리 기술의 수요가 급증하고 있다. 특히 키넥트 센서와 같은 깊이(Depth) 카메라를 사용한 3D 영상 기술에 대한 연구개발이 더욱더 활성화되고 있다. 키넥트 센서는 RGB, 골격(Skeleton) 및 깊이(Depth) 영상을 통해 인체의 3D 골격 구조를 실시간 프레임 단위로 획득할 수 있는 고성능 카메라이다. 본 논문에서는 키넥트 센서를 사용하여 인체의 3D 골격 구조를 모션 캡처하고 범용으로 사용되고 있는 모션 파일 포맷(*.trc 및 *.bvh)으로 선택하여 저장할 수 있는 시스템을 개발한다. 또한 본 시스템은 광학식 모션 캡처 파일 포맷(*.trc)을 자기식 모션 캡처 파일 포맷(*.bvh)으로 변환할 수 있도록 하는 기능을 가진다. 마지막으로 본 논문에서는 키넥트 센서를 사용하여 캡처한 모션 데이터가 제대로 캡처되어졌는지 모션 캡처 데이터 뷰어를 통하여 확인한다.

Recently, there is increasing demand for image processing technology while activated the use of equipments such as camera, camcorder and CCTV. In particular, research and development related to 3D image technology using the depth camera such as Kinect sensor has been more activated. Kinect sensor is a high-performance camera that can acquire a 3D human skeleton structure via a RGB, skeleton and depth image in real-time frame-by-frame. In this paper, we develop a system. This system captures the motion of a 3D human skeleton structure using the Kinect sensor. And this system can be stored by selecting the motion file format as trc and bvh that is used for general purposes. The system also has a function that converts TRC motion captured format file into BVH format. Finally, this paper confirms visually through the motion capture data viewer that motion data captured using the Kinect sensor is captured correctly.

Recently, education game contents with advanced mobile and Web technology have been widely researched. However, considering of interaction involved for children, it is necessary user friendly interfaces and interesting interactions. In addition, it is important to design to encourage the user’s learning motivation as a way to the effect of educational game contents. Therefore, in this study, we designed and implemented the educational game contents system using inter-working between KINECT camera, PC, and android platform based humanoid robot. The designed system consisted of KINECT system for capturing hand gestures, game contents system for playing games, and android humanoid robot system for interacting with users. This system can lead to interests and willingness of users through handy interaction such as hand gestures and interesting quiz game contents. To improve effectiveness and userability, we plant to advance designed educational game contents system that can be operated on a smart TV system including a built-in camera as a further study.

Creating avatar animations are tedious and time-consuming task since the desired avatar poses should be specified for each of a large number of keyframes. This paper proposes a fast and handy method to create game character animation contents using the motion data captured from the Kinect sensor. A Kinect sensor captures and saves the human motion. The Kinect sensor provides the motion information in a simple form of coordinates of joint positions. Using the captured motion data we determine the set of bone transforms that makes up the human skeletal animation data. The animation data is utilized to determine the position of all the bones at the current time in the animation. For experimental purpose we create a simple avatar character. We express the character model by the MD5 format, in which the mesh data and animation data are separated. A set of twenty joint positions reflect a snapshot of the character pose. The sets are used to evaluate the bone transform matrices and construct our skeletal animation scheme. We verified our method by appling the captured Kinect motion data to character animation. Our approach provides an easy method for creating avatar animations.

The necessity of automation gradually increases by increasing industrialization at the present. Needed workforce is supplied on occasion of being short of manpower or threatening man’s health. It brings about a rapid improvement in the robotics world. All the used robot technologies in automatic systems performed by this time, fulfil the given commands as they are prearranged, or they can be momentarily controlled by the help of remote controls that are designed for the control of the system. In this thesis study, it is purposed that designed robot arms can be commanded simultaneously and one-to one with articulation motions of man’s arms by using Kinect technology. In this performed study, prearranged robot’s arms can be momentarily controlled by Kinect sensor. Motions of arm sensed by Kinect are transformed into mathematical data in software which is fixed for Kinect and then the process of leading servo motor, composed robot’s articulations, is achieved by sending the data to software fixed for Arduino. The ability to control Kinect as a software is supplied with WPF (Windows Presentation Foundation) and programming of Arduino is supplied with C++ program language.

본 연구에서는 마이크로소프트 키넥트 센서와 굽힘 감지 데이터 글러브를 이용하여 광업 분야 소프트웨어를제어할 수 있는 3차원 사용자 인터페이스를 개발하였다. 개발된 인터페이스는 키넥트 센서로 추적되는 사용자의 손위치에 따라 윈도우즈 환경에서 마우스 커서를 제어한다. 또한 굽힘 감지 데이터 글러브로 측정한 사용자 손가락의구부러짐 정도에 따라 윈도우즈 환경에서 마우스 클릭 이벤트를 처리한다. 인터페이스의 개발에는 오픈소스 하드웨어인 아두이노 Lilypad 보드와 National Instrument의 LabVIEW 소프트웨어가 사용되었다. 3차원 사용자 인터페이스를 광업 분야에서 사용되고 있는 몇 가지 3차원 소프트웨어에 적용해본 결과, 마우스나 키보드와 같은 전통적인 인터페이스보다 더 직관적으로 3차원 소프트웨어를 제어할 수 있음을 확인할 수 있었다.

In this study, we developed a 3D user interface (UI) using the Microsoft’s Kinect sensor and bend-sensing data glove to control software in the mining industry. The developed UI controls the mouse cursor in Windows according to the user’s hand position tracked by the Kinect sensor. In addition, it controls the mouse click events in Windows by measuring the user’s finger flexion using the bend-sensing data glove. An open-source hardware, Arduino Lilypad Board, and National Instrument’s LabVIEW software were used to develop the UI. Applications of the 3D UI to several 3D software utilized in the mining industry revealed that it can control the 3D software in more intuitive way than traditional UIs such as mouse and keyboard.

움직임 감지 입력 기기인 키넥트(Kinect)의 시공간 해상력을 알아보고자 사이먼 효과 검증과 뮐러-라이어 착시 판단과제에서 정량적 행동측정을 하였다. 실험 1에서는 사이먼 효과를 통하여 키넥트의 시간해상력을 알아보고자 하였다. 실험 2에서는 뮐러-라이어 착시 판단에서 실험 참가자의 착시선분 측정행동을 깊이 센서로 촬영하고 실시간으로 골격을 구성하여 정량적 행동측정 자료를 얻고, 측정된 자료를 통해 키넥트의 공간해상도를 알아보고자 하였고, 착시의 지각된 반응과 비교하였다. 실험 1의 결과는 사이먼 효과에서 키넥트를 이용하여 실험 참가자의 행동을 반응키로 하였을 때도 키보드 반응을 사용한 기존 사이먼 효과와 같은 결과를 얻을 수 있었다. 실험 2의 결과, 착시 선분의 길이에 따라 비교적 안정적으로 변화하는 운동 반응이 얻어져서 키넥트 공간해상도가 비교적 안정적이었다. 보다 중요하게, 지각반응에서 유의한 착시량이 얻어진 반면, 키넥트 반응에서는 유의한 착시량이 관찰되지 않았다. 이는 지각반응과 행동반응의 괴리를 드러내는 결과로서, 특정 실험상황에서 키넥트의 사용이 중요한 방법적 도구가 될 수 있음을 시사한다. 마지막으로 기존 실험 참가자의 행동측정 방법의 문제점을 키넥트와 비교해 보고 향후 발전 방향에 대해 논의하였다.

We examined the temporal and spatial resolution of Kinect which is a motion sensing input device manufactured by Microsoft. Specifically, two experiments were conducted to verify Simon effect and Muller-Lyer illusion by measuring quantitative behavior using Kinect. The purpose of experimental 1 was to evaluate the temporal resolution of Kinect in a Simon effect task. Simon effects were observed in both of the keyboard condition and the Kinect condition. In experimental 2, participants were asked to judge the length of the Mueller-Lyer test line compared to the comparison line either by perception or hand action. In the Kinect condition, hand-action judgement of the line increased with the line length linearly. More importantly, the hand-action condition did not show significant illusions while the perception condition showed significant illusions. This result suggests that Kinect could be a useful tool to explore a possible gap between perception and action. Finally, several methodological problems and future developments of the Kinect system were discussed.

We present a Microsoft Kinect-based hand recognition algorithm for an interactive image clipping system, which is widely used for environments such as public facilities and security environments where personal capturing devices including mobile phones are not allowed. User-friendly interface and accurate image capturing function are required for an image clipping system. We build the system by combining Microsoft Kinect, HD webcam and projector. The Kinect and webcam are used to capture the motions of users' hand and project is to display the user-selected area from the capturing material. Hand recognition is composed of three steps: (i) the region occupied by users' hand is extracted from an image, (ii) the fingertips of the extracted hand region are analyzed using k-curvature algorithm, and (iii) the height of the fingertip is estimated using the depth image from Kinect. The height of the fingertip informs whether users' finger touched the surface of the target. The region captured by the fingertip is clipped from the image and stored as the target image. The excellence of our hand recognition algorithm is proved through a user test.

디지털 카메라, 스마트폰, 타블렛과 같은 스마트 기기의 대중화와 소셜 네트워크 서비스를 통해서 사진과 같은 멀티미디어 데이터의 양이 빠르고, 급격하게 확산되고 있다. 사진 검색 방법은 키워드 기반의 검색 방법, 예제 기반의 검색 방법, 시각화 질의 기반의 검색 방법의 세 가지 분류될 수 있다. 이전에 연구된 사진 검색 기법은 일반 PC 환경에 최적화되었기 때문에 최근에 등장한 스마트 TV 환경에서 사진 검색하기 위한 방법으로 사용하는 것은 적합하지 않은 상황이다. 본 논문에서는 스마트 TV 환경에서 키넥트를 이용한 소셜 네트워크에 존재하는 사진 검색 시스템을 제안하였다. 이를 위해서 키넥트 센서를 사용하여 마우스의 컨트롤을 제어할 수 있도록 구현하였으며, 제안하는 시스템의 검색 결과는 임계값이 0.7일 때, 평균 재현율과 평균 정확도는 각각 81%, 80%의 성능을 보였다.

Advances of digital device technology such as digital cameras, smart phones and tablets, provide convenience way for people to take pictures during his/her life. Photo data is being spread rapidly throughout the social network, causing the excessive amount of data available on the internet. Photo retrieval is categorized into three types, which are: keyword-based search, example-based search, visualize query-based search. The commonly used multimedia search methods which are implemented on Smart TV are adapting the previous methods that were optimized for PC environment. That causes some features of the method becoming irrelevant to be implemented on Smart TV. This paper proposes a novel Visual Query-based Photo Retrieval Method in Smart TV Environment using a motion sensing input device known as Kinect Sensor. We detected hand gestures using kinect sensor and used the information to mimic the control function of a mouse. The average precision and recall of the proposed system are 81% and 80%, respectively, with threshold value was set to 0.7

The paper suggests Game Motion Library Tool that utilizes Kinect 2.0 to simplify pipeline of character motion production that fits into the game concept and to use saved motion data actively. In order to develop Game Motion Library Tool, motion library was constructed to be organize motion data with BVH file which was produced by using Max Script language that can enable FBX file and motion data collected through Kinect 2.0 to be used in 3ds Max, and then BVH file was brought to the 3ds Max to convert it to BIP file automatically and applied in characters.

Three-dimensional(3D) display technique is widely used in our daily life. Especially, to product augmented game contents which can interact with users, it is necessary to get high quality resolution image data to reconstruct 3D model more exquisitely. In this paper, we tried to expand depth image captured by Kinect using various interpolation methods(nearest neighbor, bilinear, bicubic) to adapt it to the size of original Kinect color image. To measure the quality of expanded depth image compared to original depth image, we used PSNR(Peak Signal-to-noise ratio) index. Besides, we implemented GPU parallel processing algorithm with OpenCL to interpolate a large amount of image data rapidly. As a result of the experiment, a bicubic interpolation method made an accurate depth image although it had a long time.