Earticle

무인기 기술의 발전에 따라 군사뿐만 아니라 다양한 민간 분야에서도 무인기 탐지 및 위치 추정 기술의 중요성이 부 각되고 있다. 딥러닝 기술의 발전으로 무인기 탐지 성능은 크게 향상되었으나, 정밀한 위치 추정에 대한 연구는 상 대적으로 미진하다. 본 연구에서는 RGB와 LiDAR로 구성된 다중 모달리티 센서팩을 설계하고, 이를 기반으로 한 데이터 세트를 수집하여 무인기 위치 추정을 위한 두 가지 시스템을 제안한다. 첫 번째 시스템은 LiDAR 포인트 클 라우드를 RGB 이미지에서의 객체 탐지 결과에 투영하여 위치를 추정하며, 두 번째는 RGB와 LiDAR 데이터를 결 합해 3D 객체를 직접 탐지한 후 위치를 산출한다. 연산 복잡도가 다소 증가하나, 두 번째 시스템은 UTM 좌표계 기준으로 동방(x) 0.77m, 북방(y) 2.07m, 평균 2.21m의 RMSE를 기록하며 더욱 우수한 정확도를 보였다. 본 연구는 다중 모달리티 기반의 무인기 위치 추정 기술이 실현 가능하고 효과적임을 보여주며, 향후 군사 및 민간 분 야에서의 실질적 응용 가능성을 제시한다.

As UAV technology rapidly evolves, the need for accurate detection and localization has become critical — not only for military and defense but also for a wide range of civilian applications. While recent advances in deep learning have significantly improved UAV detection, precise localization remains a less explored area. In this study, we design a multi-modal sensor pack combining RGB and LiDAR and build a custom dataset to develop and evaluate two UAV localization systems. The first system projects 3D LiDAR point clouds onto 2D object detections from RGB images. The second system fuses RGB and LiDAR inputs for 3D object detection, directly estimating UAV positions in 3D space. Although the second approach is more computationally intensive, it achieves better accuracy, with RMSE values of 0.77 m (x-axis), 2.07 m (y-axis), and 2.21 m overall in UTM coordinates. These findings confirm the promise of multi-modal UAV localization for both military and civilian applications.

최근 멀티모달 연구가 활발해지면서, 이미지-텍스트, 영상-텍스트, 오디오-텍스트 등 다양한 개념을 통합하여 인간처럼 사물을 받아들이는 기술이 발전하고 있다. 그 중에서도, contrastive languageimage pretraining 은 제로샷 학습과 대조 학습을 통해 라벨을 학습하지 않고도 이미지와 텍스트 간의 관계를 학습하여 이미지 분류를 하는 대표적인 대규모 언어 모델로, 다양한 분야에서 활용되고 있다. 그러나, 대규모 언어 모델의 성능은 프롬프트 엔지니어링에 크게 영향을 받는다. 이에 따라, 본 논문에서는 contrastive language-image pretraining 의 추론 과정에서 프롬프트 엔지니어링을 분석하고, 능숙한 활용을 위한 필수 고려 사항과 전략을 제시한다. 이를 통해, 사용자들이 contrastive language-image pretraining 을 더욱 효과적으로 활용할 수 있을 것으로 기대된다

Human-computer interaction (HCI) has great potential for applications in many fields. A HCI method has been developed based on audio-visual perceptive features and dynamic interactive pooling. A mode analysis based fusion strategy is proposed to fuse the interactive responses from audio-visual modalities especially for the case there are deviations from the modalities at the same time. The developed approach can be applied with a better performance even in single modal with multiple interactive users in the scenario. The users in the scenario are endowed with a fair chance to perform HCI by their interactive priorities no matter how many modalities used, and the interactive authority and priority are updated dynamically according to the variation of HCI content. The scenario and multiple users altogether are regarded as a dynamic HCI pooling. The diversity of interactive habits among multiple users is considered in an ordinary hardware from a crowded scene without any hypothesis for the scenario contents in advance. The pooling is updated dynamically to meet the user demands with the most intentions preferentially. Comparative study with state-of-the-arts has indicated the superior performance of the proposed method.

Delineating brain tumor boundaries from multi-modality magnetic resonance images (MRIs) is a crucial step in brain cancer surgical and treatment planning. In this paper, we propose a fully automatic technique for brain tumor segmentation from multi-modality human brain MRIs. We first use the intensities of different modalities in MRIs to represent the features of both normal and abnormal tissues. Then, the multiple classifier system (MCS) is applied to calculate the probabilities of brain tumor and normal brain tissue in the whole image. At last, the spatial-contextual information is proposed by constraining the classified neighbors to improve the classification accuracy. Our method was evaluated on 20 multi-modality patient datasets with competitive segmentation results.

급속한 유비쿼터스 환경의 발전과 디지털 컨버전스 기기의 대중화는 개인에게 동시에 많은 정보를 처리하는 효율성을 요구하게 되었고, 이러한 요구가 멀티태스킹의 보편화를 초래하게 되었다. 이는 다양한 기능의 활용이라는 편리함과 동시에, 사용성의 복잡함으로 인해 사용자의 인지적 혼란을 초래하였다. 따라서 멀티태스킹 수행 능력과 사용 경험을 향상시키기 위한 방안으로, 적절한 모달리티를 활용한 효과적인 멀티모달 인터랙션의 적용이 요구된다. 본 연구는 모바일 기기의 멀티태스킹 사용경험 향상을 위하여 멀티모달리티를 고려하여 적용할 수 있도록 멀티태스킹과 멀티모달리티의 관계를 모형화하는 것을 목적으로 한다. 본 연구를 통해 개발된 모형은 멀티태스킹과 멀티모달리티를 지원하는 다양한 디지털 기기를 디자인 설계하는 데 있어 적용이 가능하다. 더 나아가 다양한 인터랙션 디자인 연구의 기초 자료 및 디자인 개발의 기반 방법으로 활용이 가능할 것으로 기대된다.

The rapid development of the ubiquitous environment and the popularization of digital convergence equipment mean that individuals must now be efficient in processing vast amounts of information simultaneously. However, this requirement has resulted in multitasking becoming more common. Together with the convenience of utilizing various functions, this causes cognitive confusion in users due to complicated usability issues. Therefore, the application of effective multi-modal interaction which utilizes appropriate modalities is required to enhance multitasking performance and the user experience. The purpose of this research is to suggest a model that applies multi-modality to interactions to improve multitasking in a mobile environment. The model developed through this research can be applied to the design planning of various types of digital equipment in a mobile environment. Furthermore, it is expected that the framework will provide essential information for various interaction design studies as well as a basic method of design development.

오늘날 웹 디자인의 다양성은 급속하게 증가하여 사용자에게 최적의 만족감을 제공하기 위한 디자인 양식 효과에 대한 검증이 필요하게 되었다. 본 연구는 다중 양식의 이해를 위해 단일 부호화 이론과 이중 부호화 이론을 고찰하고, 프레임 이론에 대한 선행 연구들을 통해 프레이밍이 웹 사이트 사용자들의 지각과 정보처리결과에 매우 효과적임을 고찰하였다. 그리고 다중 양식을 달리 적용한 세 가지 타입의 파생 사이트 디자인을 제작하여 온라인 사용자들을 대상으로 다중 양식의 회상 효과를 측정하고, 그로 인한 프레임 촉진 효과를 조사하여 그 관계를 알아보았다. 그 결과, 다중 양식 디자인이 복합적일수록 사용자들은 더 높은 수준의 회상 효과를 나타냈으며, 프레임 촉진과 일정한 관계가 있음이 증명되었다.

The diversity of web design has been rapidly increased so the verification of the effect on design mode to offer the optimum satisfaction to users is needed today. This study examines the dual- and the single-coding theory in order to understand the multi-modality and proves a point that the frame is effective on web site users' perception and information-processing through the research on antecedent studies on frame theory. Three different types of micro site design according to the level of multi-modality were produced and displayed to participants to measure the recall effectiveness of the multi-modality and frame facilitation and then this study examines the relationship between those two. The more the multi-modality design is complex, the higher the users' recall is effective. The result shows that multi-modality on web site design and fame facilitation keeps close relationship.

Objective: Intracranial arteriovenous malformation (AVM) associated with aneurysm has been infrequently encountered and the treatment for this malady is challenging. We report here on our clinical experience with AVMs associated with arterial aneurysms that were managed by multimodality treatments, including clipping of the aneurysm, microsurgery, Gamma-knife radiosurgery (GKS) and Guglielmi detachable coil (GDC) embolization. Methods: We reviewed the treatment plans, radiological findings and clinical courses of 21 patients who were treated with GKS for AVM associated with aneurysm. Results: Twenty-seven aneurysms in 21 patients with AVMs were enrolled in this study. Hemorrhage was the most frequent presenting symptom (17 patients: 80.9%). Bleeding was caused by an AVM nidus in 11 cases, aneurysm rupture in 5 and an undetermined origin in 1. Five patients were treated for associated aneurysm with clipping followed by GKS for the AVM and 11 patients were treated with GDC embolization combined with GKS for an AVM. Although 11 associated aneurysms remained untreated after GKS, none of them ruptured and 4 aneurysms regressed during the follow up period. Two aneurysms increased in size despite the disappearance of the AVM nidus after GKS and then these aneurysms were treated with GDC embolization. Conclusion: If combined treatment using microsurgery, GKS and endovascular treatment can be adequately used for these patients, a better prognosis can be obtained. In particular, GKS and GDC embolization are considered to have significant roles to minimize neurologic injury.

.

Primary cardiac neoplasms are rare entities of which approximately 75% are benign and the remaining 25% malignant. Myxomas are the most common benign primary cardiac tumor (30%) and most commonly arise in the left atrium from the interatrial septum at the fossa ovalis. However, they also can originate in any cardiac chamber. Clinical presentation and patient symptomatology are determined by size, location, and mobility of the myxoma. This review will discuss the clinical presentation, natural history, pathology, and multimodality imaging features of cardiac myxomas.

Objective: The objective of this study was to examine multi-modal interaction effects of input-mode switching on the use of smart phones. Background: Multi-modal is considered as an efficient alternative for input and output of information in mobile environments. However, there are various limitations in current mobile UI (User Interface) system that overlooks the transition between different modes or the usability of a combination of multi modal uses. Method: A pre-survey determined five representative tasks from smart phone tasks by their functions. The first experiment involved the use of a uni-mode for five single tasks; the second experiment involved the use of a multi-mode for three dual tasks. The dependent variables were user preference and task completion time. The independent variable in the first experiment was the type of modes (i.e., Touch, Pen, or Voice) while the variable in the second experiment was the type of tasks (i.e., internet searching, subway map, memo, gallery, and application store). Results: In the first experiment, there was no difference between the uses of pen and touch devices. However, a specific mode type was preferred depending on the functional characteristics of the tasks. In the second experiment, analysis of results showed that user preference depended on the order and combination of modes. Even with the transition of modes, users preferred the use of multi-modes including voice. Conclusion: The order of combination of modes may affect the usability of multi-modes. Therefore, when designing a multi-modal system, the fact that there are frequent transitions between various mobile contents in different modes should be properly considered. Application: It may be utilized as a user-centered design guideline for mobile multi modal UI system.

In this paper, we propose pyramid appearance and global structure action descriptors on both RGB and depth motion history images and a model-free method for human action recognition. In proposed algorithm, we firstly construct motion history image for both RGB and depth channels, at the same time, depth information is employed to filter RGB information, after that, different action descriptors are extracted from depth and RGB MHIs to represent these actions, and then multimodality information collaborative representation and recognition model, in which multi-modality information are put into object function naturally, and information fusion and action recognition also be done together, is proposed to classify human actions. To demonstrate the superiority of the proposed method, we evaluate it on MSR Action3D and DHA datasets, the well-known dataset for human action recognition. Large scale experiment shows our descriptors are robust, stable and efficient, when comparing with the-state-of-the-art algorithms, the performances of our descriptors are better than that of them, further, the performance of combined descriptors is much better than just using sole descriptor. What is more, our proposed model outperforms the state-of-the-art methods on both MSR Action3D and DHA datasets.

.

Background: Ovarian cancer continues to pose a major challenge to physicians and radiologists. It is the third most common gynecologic malignancy and estimated to be fifth leading cancer cause of death in women, constituting 23% of all gynecological malignancies. Multi-detector computed tomography (MDCT) appears to offer an excellent modality in diagnosing ovarian cancer based on combination of its availability, meticulous technique, efficacy and familiarity of radiologists and physicians. The aim of this study was to compute sensitivity, specificity, positive and negative predictive values and diagnostic accuracy of 64-slice MDCT in classifying ovarian masses; 95% confidence intervals were reported. Materials and Methods: We prospectively designed a cross-sectional analytical study to collect data from July 2010 to August 2011 from a tertiary care hospital in Karachi, Pakistan. A sample of 105 women aged between 15-80 years referred for 64-MDCT of abdomen and pelvis with clinical suspicion of malignant ovarian cancer, irrespective of stage of disease, were enrolled by non-probability purposive sampling. All patients who were already known cases of histologically proven ovarian carcinoma and having some contraindication to radiation or iodinated contrast media were excluded. Results: Our prospective study reports sensitivity, specificity; positive and negative predictive values with 95%CI and accuracy were computed. Kappa was calculated to report agreement among the two radiologists. For reader A, MDCT was found to have 92% (0.83, 0.97) sensitivity and 86.7% (0.68, 0.96) specificity, while PPV and NPV were 94.5% (0.86, 0.98) and 86.7% (0.63, 0.92), respectively. Accuracy reported by reader A was 90.5%. For reader B, sensitivity, specificity, PPV and NPV were 94.6% (0.86, 0.98) 90% (0.72, 0.97) 96% (0.88, 0.99) and 87.1% (0.69, 0.95) respectively. Accuracy computed by reader B was 93.3%. Excellent agreement was found between the two radiologists with a significant kappa value of 0.887. Conclusion: Based on our study results, we conclude MDCT is a reliable imaging modality in diagnosis of ovarian masses accurately with insignificant interobserver variability.

다중 모달리티 영상정합은 서로 다른 성격의 두 영상의 중요정보를 결합하여 복합적 정보를 얻기 위해 널리 사용되는 영상처리 기법이다. 본 연구에서는 정합 대상 객체의 초기위치 및 방향에 종속적이지 않고, 낮은 정합오차 범위 내에서의 안정적인 정합을 지원하기 위하여 기존의 표면기반 정합 기법을 개선한 모멘트 정보 및 표면거리 기반의 정합 기법을 제시한다. 제안방법에서는 우선 정합대상객체의 표면 윤곽 점을 추출하고, 이를 기반으로 대상객체의 모멘트 정보를 추출하여, 표면거리 기반 상세 정합 이전에 모멘트 정보를 일치시키는 변환을 수행함으로써, 정합이전 대상객체의 위치 및 방향이 상이한 경우에 있어서도 정합이 안정적으로 수행되도록 한다. 또한 테스트 영상에 대한 표면 대표점 추출 시, 표면 코너추출법을 적용함으로써, 기존 표면 정보 기반 정합기법에서 일반적으로 사용하고 있는 무작위 샘플링 및 일정간격 샘플링에 의한 취약점을 보완한다. 본 논문에서 제안기법의 검증을 위하여 뇌 부위 자기공명단층영상(MRI)과 양자 방출 단층 촬영 영상(PET)을 적용하고, 정합오류율과 정합결과에 대한 2,3차원 가시화 영상의 육안평가를 통하여 정확성 및 안정성 측면을 검증한다.

Multi-modality image registration is a widely used image processing technique to obtain composite information from two different kinds of image sources. This study proposes an image registration method based on moment information and surface distance, which improves the previous surface-based registration method. The proposed method ensures stable registration results with low registration error without being subject to the initial position and direction of the object. In the preprocessing step, the surface points of the object are extracted, and then moment information is computed based on the surface points. Moment information is matched prior to fine registration based on the surface distance, in order to ensure stable registration results even when the initial positions and directions of the objects are very different. Moreover, surface comer sampling algorithm has been used in extracting representative surface points of the image to overcome the limits of the existed random sampling or systematic sampling methods. The proposed method has been applied to brain MRI(Magnetic Resonance Imaging) and PET(Positron Emission Tomography), and its accuracy and stability were verified through registration error ratio and visual inspection of the 2D/3D registration result images.

This paper presents an implementation of real-time speaker localization using audio-visual information. Four channels of microphone signals are processed to detect vertical as well as horizontal speaker positions. At first short-time average magnitude difference function(AMDF) signals are used to determine whether the microphone signals are human voices or not. And then the orientation and distance information of the sound sources can be obtained through interaural time difference and interaual level differences. Finally visual information by a camera helps get finer tuning of the speaker orientation. Experimental results of the real-time localization system show that the performance improves to 99.6% compared to the rate of 88.8% when only the audio information is used.

해마를 비롯하여, 뇌 기능과 밀접한 관련을 가지는 뇌 하위조직의 분석에 대한 최근 연구로 MR 영상 등의 해부학적 영상으로부터의 볼륨 추출, 형상 복원, 대칭성 비교 등을 들 수 있다. 이러한 연구들은 뇌의 해부학적 정보에만 의존함으로써 관심영역에 대한 신진대사 등의 분석에 한계를 가진다. 본 논문에서는 뇌 해마영역에 대하여 해부학적, 기능적 특성의 동시 분석이 가능한 프로시저를 제안한다. 먼저 해부학적 영상과 기능적 영상의 다중모달리티 영상정합을 수행하고 이를 기반으로 해마 SPECT 볼륨이 추출되며, 나아가 체적 측정 및 강도 분포 등의 정량분석을 수행함으로써 해부학적 영역의 기능정보에 대한 직관적이며 객관적인 분석이 가능하도록 하였다.

해마를 비롯하여, 뇌 기능과 밀접한 관련을 가지는 뇌 하위조직의 분석에 대한 최근 연구로 MR 영상 등의 해부학적 영상으로부터의 볼륨 추출, 형상 복원, 대칭성 비교 등을 들 수 있다. 이러한 연구들은 뇌의 해부학적 정보에만 의존함으로써 관심영역에 대한 신진대사 등의 분석에 한계를 가진다. 본 논문에서는 뇌 해마영역에 대하여 해부학적, 기능적 특성의 동시 분석이 가능한 프로시저를 제안한다. 먼저 해부학적 영상과 기능적 영상의 다중모달리티 영상정합을 수행하고 이를 기반으로 해마 SPECT 볼륨이 추출되며, 나아가 체적 측정 및 강도 분포 등의 정량분석을 수행함으로써 해부학적 영역의 기능정보에 대한 직관적이며 객관적인 분석이 가능하도록 하였다.