Earticle

본 논문은 객체 특징 탐색을 이용한 실시간 가상 모델 동기화 알고리즘을 제안한다. 제안된 알고리즘은 2차원 영상 에서 객체 특징 탐색을 사용하여, 실제 객체와 가상모델을 실시간으로 동기화 한다. 객체 특징 탐색 알고리즘은 색상을 이용 하여 객체를 개별적으로 분류하는 알고리즘과 각도를 이용하여 객체가 설치된 방향을 분석하는 알고리즘으로 구성된다. 제 안한 알고리즘을 사용하여 실제 물체의 움직임을 가상 모델에 동기화함으로써, 별도의 사용자 조작 도구 사용 없이 손을 사용하여 가상 물체를 움직이는 환경을 제공할 수 있다. 본 알고리즘은 향후 불특정한 모양과 색상 및 방향을 갖는 객체를 동기화하는 연구를 진행하며, VR/AR기술에 적용하여 사실적인 가상환경을 제공하는 것을 목표로 한다.

In this paper, we propose a real-time virtual model synchronization algorithm using object feature detection. The proposed algorithm may be useful to synchronize between real objects and their corresponding virtual models through object feature search in two-dimensional images. It consists of an algorithm to classify objects with colors individually, and an algorithm to analyze the orientation of objects with angles. We can synchronize the motion of the real object with the virtual model by providing the environment of moving the virtual object through the hand without specific controllers. The future research will include the algorithm to synchronize real object with unspecified shapes, colors, and directions to the corresponding virtual object.

본 논문에서는 효율적인 차량 객체를 추적하는 특징점 매칭을 이용한 다중 차량 객체 검 출 알고리즘을 제안한다. 제안하는 알고리즘은 효율적인 차량 객체 추적을 위해 FAST 알고 리즘을 이용해서 차량의 특징점을 추출한다. 그리고 5X5 영역으로 분할 된 영상에서 특징점 이 포함되면 True 포함되지 않으면 False로 해당 영역을 검은색으로 후처리하여 차량 객체을 제외한 불필요한 객체 정보를 제거한다. 그리고 후처리 된 영역을 차량의 최대 탐색창 크기 로 설정하고, 차량의 최외각 특징점을 이용한 최소 탐색창을 설정하여 Mean-Shift 알고리즘 의 탐색창 크기에 대한 단점을 보완하여 차량 객체 추적을 한다. 제안한 방법의 성능 평가하 기위해 SIFT, SURF 알고리즘을 비교하여 실험한다. 그 결과 SIFT 알고리즘에 비해서 약 4배 빠르고 SUFR 알고리즘의 처리 과정 보다는 효율적으로 검출하는 장점이 있다

In this paper, we propose a multi-vehicle object detection algorithm using feature point matching that tracks efficient vehicle objects. The proposed algorithm extracts the feature points of the vehicle using the FAST algorithm for efficient vehicle object tracking. And True if the feature points are included in the image segmented into the 5X5 region. If the feature point is not included, it is processed as False and the corresponding area is blacked to remove unnecessary object information excluding the vehicle object. Then, the post processed area is set as the maximum search window size of the vehicle. And A minimum search window using the outermost feature points of the vehicle is set. By using the set search window, we compensate the disadvantages of the search window size of mean-shift algorithm and track vehicle object. In order to evaluate the performance of the proposed method, SIFT and SURF algorithms are compared and tested. The result is about four times faster than the SIFT algorithm. And it has the advantage of detecting more efficiently than the process of SUFR algorithm.

최근 들어, 라이다 기술의 발전에 따라 정확한 거리 측정이 가능해지면서 라이다 기반의 3차원 객체 탐지 네트워크에 대한 관심이 증가하고 있다. 기존의 네트워크는 복셀화 및 다운샘플링 과정에서 공간적인 정보 손실이 발생해 부정확한 위치 추정 결과를 발생시킨다. 본 연구에서는 고수준 특징과 높은 위치 정확도를 동시에 획득하기 위해 어텐션 기반 융합 방식과 카메라-라이다 융합 시스템을 제안한다. 먼저, 그리드 기반의 3차원 객체 탐지 네트워크인 Voxel-RCNN 구조에 어텐션 방식을 도입함으로써, 다중 스케일의 희소 3차원 합성곱 특징을 효과적으로 융합하여 3차원 객체 탐지의 성능을 높인다. 다음으로, 거짓 양성을 제거하기 위해 3차원 객체 탐지 네트워크의 탐지 결과와 이미지상의 2차원 객체 탐지 결과를 결합하는 카메라-라이다 융합 시스템을 제안한다. 제안 알고리즘의 성능평가를 위해 자율주행 분야의 KITTI 데이터 세트를 이용하여 기존 알고리즘과의 비교 실험을 수행한다. 결과적으로, 차량 클래스에 대해 BEV 상의 2차원 객체 탐지와 3차원 객체 탐지 부분에서 성능 향상을 보였으며 특히 Voxel-RCNN보다 차량 Moderate 클래스에 대하여 정확도가 약 0.47% 향상되었다.

Recently, following the development of LIDAR technology which can detect distance from the object, the interest for LIDAR based 3D object detection network is getting higher. Previous networks generate inaccurate localization results due to spatial information loss during voxelization and downsampling. In this study, we propose an attention-based convergence method and a camera-LIDAR convergence system to acquire high-level features and high positional accuracy. First, by introducing the attention method into the Voxel-RCNN structure, which is a grid-based 3D object detection network, the multi-scale sparse 3D convolution feature is effectively fused to improve the performance of 3D object detection. Additionally, we propose the late-fusion mechanism for fusing outcomes in 3D object detection network and 2D object detection network to delete false positive. Comparative experiments with existing algorithms are performed using the KITTI data set, which is widely used in the field of autonomous driving. The proposed method showed performance improvement in both 2D object detection on BEV and 3D object detection. In particular, the precision was improved by about 0.54% for the car moderate class compared to Voxel-RCNN.

Tracking multiple objects in real-time videos represents a challenging area in the era of computer vision. This paper proposes a new method to track the multiple objects under different environment conditions such as rotation, illumination, blurred, occlusion, and many others. In addition, the kinect color depth image processing is used to estimate the distance of the objects. The tracking of multiple objects is formulated as classification task which competitively use the object features in the different video frames of the video sequences. To obtain the optimal configuration of feature classification, a neural network based framework is presented to make a global influence based on winner pixel estimation between the video frames. The objects are tracked efficiently in less time as compared with SIFT techniques and distance of objects is calculated with kinect based depth image processing. Experimental results are given for real-time scenes, and many experiments are conducted to examine the performance of the proposed approach. The proposed method resulted into efficient tracking of multiple objects in various conditions including rotation, scaling, occlusion, etc. The distance of multiple tracked objects is estimated using the kinect depth processing.

본 연구에서는 모델의 경량화를 위해 교사 모델의 출력 특징맵에서 3D 객체의 정보를 추출해 학생 모델의 다중 스케일 특징맵(Multi-scale feature map)에 맞게 증류하는 3D 객체 검출용 다중스케일 특징 지식 증류 기법인 M3KD (Multi-Scale Feature Knowledge Distillation for 3D Object Detection)를 제안한다. M3KD는 지식 증류 수행 시 학생 모델과 교사 모델의 다중 스케일 특징맵들 간 L2 손실(loss)을 사용해 특징맵 값의 차이를 줄이게 함으로써 학생 모델이 교사 모델의 백본을 모방하게 하여 학생 모델의 전체적인 정확도를 향상시키고, 기존의 이미지 분류 태스크(Task)에서 사용하는 클래스 로짓(Logits) 지식 증류를 적용해 교사 모델의 클래스 분류 로짓을 모방함으로써 학생 모델의 검출 정확도를 향상시킨다. 본 연구가 제안한 M3KD의 효과를 증명하기 위해 KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) 데이터 셋에서 실험을 진행하였으며, 이때 학습한 학생 모델이 교사 모델 대비 30%의 추론 속도 향상을 달성하였다. 또한, 정확도에서 기존의 학생 모델과 비교시 모든 클래스 및 모든 난이도에서 평균적으로 1.08%의 3D mAP (Mean Average Precision) 향상이 있음을 확인하였다. 또한 최신 지식 증류 기법인 PKD, SemCKD에 제안하는 기법을 추가로 적용하였을 시 기존 대비 0.42%, 0.52% 높은 정확도 (3D mAP)를 나타내 성능 향상을 달성하였다.

In this paper, we propose Multi-Scale Feature Knowledge Distillation for 3D Object Detection (M3KD), which extracting knowledge from the teacher model, and transfer to the student model consider with multi-scale feature map. To achieve this, we minimize L2 loss between feature maps at each pyramid level of the student model with the correspond teacher model so student model can mimic the teacher model backbone information which improves the overall accuracy of the student model. We apply the class logits knowledge distillation used in the image classification task, by allowing student model mimic the classification logits of the teacher model, to guide the student model to improve the detection accuracy. In KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset, our M3KD (Multi-Scale Feature Knowledge Distillation for 3D Object Detection) student model achieves 30% inference speed improvement compared to the teacher model. Additionally, our method achieved an average improvement of 1.08% in 3D mean Average Precision (mAP) across all classes and difficulty levels compared to the baseline student model. Furthermore, when integrated with the latest knowledge distillation methods such as PKD and SemCKD, our approach achieved an additional 0.42% and 0.52% improvement in 3D mAP, respectively, further enhancing performance.

With the recent development of deep learning, most computer vision-related tasks are being solved with deep learning-based network technologies such as CNN and RNN. Computer vision tasks such as object detection or object segmentation use intermediate features extracted from the same backbone such as Resnet or FPN for training and inference for object detection and segmentation. In this paper, an experiment was conducted to find out the compression efficiency and the effect of encoding on task inference performance when the features extracted in the intermediate stage of CNN are encoded. The feature map that combines the features of 256 channels into one image and the original image were encoded in HEVC to compare and analyze the inference performance for object detection and segmentation. Since the intermediate feature map encodes the five levels of feature maps (P2 to P6), the image size and resolution are increased compared to the original image. However, when the degree of compression is weakened, the use of feature maps yields similar or better inference results to the inference performance of the original image.

배회란 "어떤 곳을 중심으로 어슬렁거리며 이리저리 돌아다닌다." 라는 사전적인 의미를 가지고 있으며 대부분의 범죄가 발생하기 전에 이루어지는 행동이다. 따라서 배회행위를 검출함으로써 다양한 범죄를 사전에 예방할 수 있다. 본 논문에서는 Raspberry Pi를 이용한 배회행위 검출 알고리즘을 제안한다. 제안한 알고리즘은 적응형 차영상을 이용해 움직이는 객체를 검출하였고, 객체검출의 정확성을 높이기 위해 모폴로지 열림 연산을 사용한다. 객체가 검출되면 객체의 무게중심점을 이용하여 무게중심점의 각도 변화와 객체의 높이에 반비례하는 픽셀 이동 거리를 이용하여 배회행위를 검출한다. 배회행위로 검출되면 Raspberry Pi를 이용하여 경고음을 출력하여 사용자에게 알려준다.

The literal meaning of loitering is "to lingering aimlessly or as if aimless in or about a place". And most criminals show this kind of act before they actually commit crime. Therefore, detecting this kind of loitering can effectively prevent a variety of crime. In this paper, we propose a loitering-detection algorithm using the Raspberry Pi. Proposed algorithm uses an adaptive difference image to detect moving objects and morphology opening operation to enhance the accuracy of detection. The loitering- behavior is being detected by using the center of gravity of the object to see the changes of angle; and pixel movement distance to determine the height of the object. When the loitering-behavior is detected, it outputs the alarm to tell the users by using the Raspberry Pi.

전파를 이용한 통신이 활성화 되어 인터넷상의 네트워크에 연결하여 다양한 서비스가 제공되고 있는 현실에서 통신 서비스의 질을 높이기 위한 물체의 특징 점 탐지 및 추적의 중요성이 크게 대두되었다. 본 논문은 전파가 미치지 않는 음영 공간의 탐지와 추적을 위한 연구의 내용으로, Snakes 알고리즘을 이용하여 음영 공간을 탐지하고, 탐지된 음영 공간 내에서 어느 한 출발 지점에서 목표 지점까지의 경로를 추적하는 시스템의 기반을 제시하고자 하였다.

In the actuality that the various services are provided in connection with the network of internet by activating the communication using Propagation, the importance of the feature point and chase of an object is greatly raised to increase the quality of the detection and tracking of the communication service. This paper is to detect the shadow space by using Snakes Algorithms and Present a system's base which tracts the route from start to target points in the detected shadow space as a study for the detection and tracking the shadow space which does not reach the propagation.

특정한 제한을 두지 않는 복잡한 자연환경에서 사용자가 원하는 목표 물체만을 정확하게 검출하는 작업은 컴퓨터 비전 및 영상처리 분야에서 중요하지만 매우 어려운 문제 중의 하나이다. 본 논문에서는 반사가 존재하는 여러 환경에서 목표하는 물체를 강인하게 검출하는 새로운 방법을 제안한다. 제안된 방법에서는 먼저 스테레오 카메라를 이용하여 목표 물체를 촬영한 다음, 물체를 가장 잘 표현하는 라인과 코너 특징들을 추출한다. 그런 다음, 촬영된 좌우 영상으로부터 호모그래픽 변환을 이용하여 실제로 존재하지 않는 반사된 특징들을 효과적으로 제거한다. 마지막으로, 반사된 특징들을 제거한 실제 특징들만을 군집화하여 대상 물체만을 강건하게 검출한다. 본 논문의 실험결과에서는 제안된 알고리즘이 기존의 알고리즘에 비해서 반사가 존재하는 자연 환경에서 목표 물체를 보다 강인하게 검출한다는 것을 보여준다.

Detecting target objects robustly in natural environments is a difficult problem in the computer vision and image processing areas. This paper suggests a method of robustly detecting target objects in the environments where reflection exists. The suggested algorithm first captures scenes with a stereo camera and extracts the line and corner features representing the target objects. This method then eliminates the reflected features among the extracted ones using a homographic transform. Subsequently, the method robustly detects the target objects by clustering only real features. The experimental results showed that the suggested algorithm effectively detects the target objects in reflection environments rather than existing algorithms.

As digital twin technology for implementing smart shipyards advances, research on object detection using 3D point cloud has become increasingly necessary. Shipyards, in particular, are complex environments with coexisting indoor and outdoor settings, large structures, and diverse equipment, making the analysis of features from 3D point cloud objects essential for effective object detection. However, these environments, characterized by intricate geometries, varied equipment scales, and high-resolution data, generate massive datasets. Such large volumes of data lead to challenges in storage, transmission, processing, and learning resources and costs, notably intensifying computational and memory loads during the training of deep learning-based object detection models. Therefore, a lightweighting solution that ensures efficient processing while maintaining data quality is required. Accordingly, this study extracted geometric local features using Fast Point Feature Histograms (FPFH) from 3D point cloud acquired in a shipyard. An augmented dataset was then constructed based on these extracted feature points. Subsequently, an object detection model was trained using a 3D point cloud-based deep learning architecture to analyze the impact of these features on object detection accuracy. The results indicated that the FPFH-applied dataset maintained comparable accuracy to the original dataset. This research is anticipated to be valuably utilized for the development of automation technologies and digital twins aimed at constructing smart shipyards.

As digital twin technology for implementing smart shipyards advances, research on object detection using 3D point cloud has become increasingly necessary. Shipyards, in particular, are complex environments with coexisting indoor and outdoor settings, large structures, and diverse equipment, making the analysis of features from 3D point cloud objects essential for effective object detection. However, these environments, characterized by intricate geometries, varied equipment scales, and high-resolution data, generate massive datasets. Such large volumes of data lead to challenges in storage, transmission, processing, and learning resources and costs, notably intensifying computational and memory loads during the training of deep learning-based object detection models. Therefore, a lightweighting solution that ensures efficient processing while maintaining data quality is required. Accordingly, this study extracted geometric local features using Fast Point Feature Histograms (FPFH) from 3D point cloud acquired in a shipyard. An augmented dataset was then constructed based on these extracted feature points. Subsequently, an object detection model was trained using a 3D point cloud-based deep learning architecture to analyze the impact of these features on object detection accuracy. The results indicated that the FPFH-applied dataset maintained comparable accuracy to the original dataset. This research is anticipated to be valuably utilized for the development of automation technologies and digital twins aimed at constructing smart shipyards.