Earticle

본 연구는 라즈베리파이 기반 자율주행 자동차를 활용하여 학생들이 객체 이미지 데이터를 어노테이션(Annotation)하고 컴퓨터 비 전 기반의 인공지능을 개발하는 인공지능 교육을 위한 캠프 프로그램을 제안하여 인공지능에 대한 올바른 인식과 함께 인공지능에 대 한 흥미와 이해도를 향상시키고자 한다. 본 교육 프로그램은 학생들이 직접 이미지 데이터를 생성하고 객체인식 모델을 활용해 자율자 동차에 적용해볼 수 있는 프로젝트 형태로 구성하였으며, 학생들의 수준을 고려하여 기계 학습 및 이미지 추론이 가능한 블록 코딩을 활용한다. 자율주행 자동차의 센서와 작동 원리를 개괄적으로 이해하고, 자율주행 자동차 제어에 필요한 기초적인 문법과 이미지 추론 방법을 익혀 자율주행 자동차를 주행하고자 한다.

This study proposes a camp program for artificial intelligence education, utilizing a Raspberry Pi-based autonomous driving car, to enhance students' interest and understanding of artificial intelligence along with a correct perception of AI. The program aims to involve students in annotating object image data and developing artificial intelligence based on computer vision. The educational camp program is designed in a project format, allowing students to directly generate image data and create object recognition models to be applied to autonomous vehicles. Block coding, capable of machine learning and image inference, is employed, taking into consideration the students' proficiency levels. The program covers a broad understanding of the sensors and operating principles of autonomous vehicles, and it aims to familiarize students with basic syntax and image inference methods necessary for controlling autonomous driving cars.

Traditional image processing and pattern recognition research aimed at identifying the target image. As time goes by, on the basis of the recognition of the image, more and more research points to identify multiple targets in the image, and the corresponding block of the corresponding target is calibrated. Compared with the traditional image recognition, the problem of image annotation is a combination of multi classification and multi regression, which is more difficult and challenging. On the basis of deep convolutional neural network, this paper studies the image annotation algorithm based on region selection algorithm and support vector machine, and the algorithm is tested on the PASCAL VOC 2010 image data set. Experimental results show that compared with the current algorithm, this algorithm can be used to mark the image of multiple targets, the effect is obvious, and there is a great practical significance.

Automatic image annotation has been an active research topic in recent years due to its potential impact on both image understanding and semantic based image retrieval. However, the results of the state-of-the-art image annotation methods are still far from satisfaction due to the existence of semantic gap. Thus refining image annotation (RIA) has become one of the core research topics in computer vision and multimedia areas, whose purpose is to reserve the highly correlated annotations whereas remove the irrelevant or weakly relevant annotations by fully exploring the correlations of annotation keywords. RIA, to some extent, can effectively mitigate the semantic gap between low-level visual features and high-level semantic concepts. So in this paper, we focus on the latest development in image retrieval and provide a comprehensive survey on refining image annotation techniques. In particular, we analyze the key aspects of various RIA methods, including their original intentions and annotation models. Finally, we draw some important conclusions and highlight the potential research directions for the future.

Automatic image annotation (AIA) is an active topic of research in computer vision and pattern recognition. In the last two decades, large amount of researches on AIA have been proposed, mainly including classification-based methods and probabilistic modeling methods. As one of the most common methods for AIA, support vector machine (SVM) has been widely applied in the multimedia research community, especially for image classification, image annotation and retrieval. However, compared with various SVM methods and their corresponding applications in the literature, there is almost no review research and analysis about SVM related studies. So the current paper, to start with, elaborates the basic principles of SVM. Followed by it summarizes SVM with applications to image annotation from three aspects of SVM ensemble for AIA, SVM with mixture of kernels for AIA and hybrid SVM for AIA respectively. In addition, SVM exploited in several other applications are also briefly reviewed. Finally, we end this paper with a summary of some important conclusions and highlight the potential research directions of SVM in automatic image annotation for the future.

In this paper, we present a new method for automatic image annotation by applying semi- supervised learning based on the Bayesian framework. On the one hand, we employ the semi- supervised learning, i.e., transductive support vector machine (TSVM) to enhance the quality of training image data, which is a promising way to find out the underlying relevant data from the unlabeled ones. On the other hand, a simple yet very efficient Bayesian model is built to implement image annotation by the maximum a posteriori (MAP) criterion. The novelty of our method mainly lies in two aspects: exploiting TSVM to improve the quality of training image dataset and utilizing the Bayesian model to predict the candidate annotations for the unseen images. Experimental results on the standard Corel dataset demonstrate that the proposed method is superior or highly competitive to several state-of-the-art approaches.

In recent years, automatic image annotation (AIA) has been applied to cross-media retrieval usually due to its advantage of mining correlations of images and annotation texts efficiently. However, some AIA methods just annotate images as a unit and the accuracy of annotation may not be acceptable. In this paper, we propose a kind of probabilistic model which may assign keywords to an un-annotated image automatically based on a training dataset of images. Images in the training dataset are segmented into regions and a kind of vocabulary called blob is used to represent these image regions. Blobs are generated by using K-Means algorithm to cluster these image regions. Through this model, we can predict the probability of assigning a keyword into a blob. After the accomplishment of annotation, a keyword corresponds to one image region. Furthermore, the feature vectors of text documents are generated by TF.IDF method and images’ automatic annotation information is used to retrieve relevant text documents. Experiments on the IAPR TC-12 dataset and 500 Wikipedia webpages about landscape show the usefulness of applying probabilistic model of AIA to the cross-media retrieval.

Image annotation and retrieval has been a popular research topic for decades. Based on published journals from 2012 until 2015, a lot of research and studies has been focused on Content Based Image Retrieval (CBIR). In most cases, CBIR systems that use an image as the input query always face a problem called semantic gap due to the use of low-level features for similarity matching. The semantic gap will consequently reduce the performance of the CBIR systems. To overcome this problem, a new class of CBIR known as Annotation Based Image Retrieval (ABIR) has been developed. The ABIR systems, based on the annotation process of the images, employ bag of words for query. However it is found that employing pure bag of word only is not adequate to solve semantic problem. In this study, an attempt to use a Gaussian Mixture Model (GMM) based approach and spatial related information of the annotated objects has been performed in order to improve the performance of the ABIR systems. From the retrieval experiments, it is found that ABIR could achieve average performance up to 88% which is two times better than the CBIR systems in reducing the semantic problem.

Automatic image annotation (AIA) is expected to be a promising way to improve the performance of content-based image retrieval (CBIR). However, current image annotation results are always incomplete and noisy, and far from practical usage. In this paper, we incorporate semantic annotations into CBIR via query expansion scheme to improve retrieval accuracy. In the proposed method, semantic annotations of test images are obtained using a visual nearest-neighbor-based annotation model. And both visual features and annotation keywords are used to represent images. The similarity between two images is determined by their visual similarity and semantic similarity. The method is evaluated on the well-known Pascal VOC 2007 dataset using standard performance evaluation metric. The experimental results indicate that the performance of CBIR can be improved by incorporating semantic annotation via query expansion.

본 연구에서는 모바일 기기에서 증강현실 기술을 이용한 정보제공 시스템을 제안한다. ICT 기술의 발전으로 일상에서 사용하는 컴퓨터가 소형화되고 휴대가 간편하게 발전된 형태의 대표적인 예로 태블릿 PC와 스마트폰을 들 수 있다. 환경에 구애받지 않고 본인이 원하는 자료를 증강현실 기술을 이용하여 확인하고 학습할 수 있다. 카메라를 이용한 실영상에 원하는 이미지, 영상 등을 추가하여 다양한 도움을 받을 수 있는 것이다. 본 논문에서는 태블릿 PC나 스마트폰 환경에서 증강현실 기술을 이용한 정보제공용 시스템을 구현하였다. 이러한 시스템은 학습, 오락, 광고홍보 등 모든 분야에서 활용 가능하다.

In this paper, we propose an information service system using augmented reality technology on mobile devices. With the advancement in ICT technology the computer is smaller and easy to carry and developed into the tablet PC and the smartphone typically. The user can confirm and learn the desired data using the augmented reality technology, regardless of the environment. Padding the supplementary images or videos to the real image using the camera, we can have help from such additionally obtained images. In this paper, using an augmented reality technology on a tablet PC or smartphone environment, we implement a system for providing information to the user. This system can be utilized in all areas such as learning, entertainment, public relations and advertisement, etc.

본 논문에서는 이전의 의미 기반 이미지 어노테이션 및 검색 시스템 Moment(Mobile Semantic Image Annotation and Retrieval System)에 RDF(Resource Description Framework) 추론 기능을 사용한 어노테이션 방법을 제안한다. 이를 위하여 제안된 시스템은 Apache Jena Inference API를 통해 구현되였으며 각 이미지들이 가진 어노테이션의 개수가 증가되었다. 자동으로 추론된 결과 또한 SPARQL 질의를 통해 검색이 가능하며, 기존 어노테이션 결과에 대한 의미 검색을 더욱 효과적으로 할 수 있게 한다.

본 논문에서는 이전의 의미 기반 이미지 어노테이션 및 검색 시스템 Moment(Mobile Semantic Image Annotation and Retrieval System)에 RDF(Resource Description Framework) 추론 기능을 사용한 어노테이션 방법을 제안한다. 이를 위하여 제안된 시스템은 Apache Jena Inference API를 통해 구현되였으며 각 이미지들이 가진 어노테이션의 개수가 증가되었다. 자동으로 추론된 결과 또한 SPARQL 질의를 통해 검색이 가능하며, 기존 어노테이션 결과에 대한 의미 검색을 더욱 효과적으로 할 수 있게 한다.

The progress of mobile computing technology is bringing a large amount of multimedia contents such as image. Thus, we need an image retrieval system which searches semantically relevant image. In this paper, we propose a semantic image annotation and retrieval in mobile environments. Previous mobile-based annotation approaches cannot fully express the semantics of image due to the limitation of current form (i.e., keyword tagging). Our approach allows mobile devices to annotate the image automatically using the context-aware information such as temporal and spatial data. In addition, since we annotate the image using RDF(Resource Description Framework) model, we are able to query SPARQL for semantic image retrieval. Our system implemented in android environment shows that it can more fully represent the semantics of image and retrieve the images semantically comparing with other image annotation systems.

Since a large amount of multimedia image has dramatically increased, it is important to search semantically relevant image. Thus, several semantic image annotation methods using RDF(Resource Description Framework) model in mobile environment are introduced. Earlier studies on annotating image semantically focused on both the image tag and the context-aware information such as temporal and spatial data. However, in order to fully express their semantics of image, we need more annotations which are described in RDF model. In this paper, we propose an annotation method inferencing with RDFS entailment rules and user defined rules. Our approach implemented in Moment system shows that it can more fully represent the semantics of image with more annotation triples.

In this paper, I propose an automatic approach to annotating images dynamically based on MBRM(Multiple Bernoulli Relevance Models) using GLCM(Grey Level Co-occurrence Matrix). MBRM is more appropriate to annotate images compare with multinomial distribution. The model is used in limited test set, MSRC-v2 (Microsoft Research Cambridge Image Database). The results show that this model is significantly outperforms previously reported results on the task of image annotation and retrieval.

In this paper, I propose an automatic approach to annotating images dynamically based on MBRM(Multiple Bernoulli Relevance Models) using GLCM(Grey Level Co-occurrence Matrix). MBRM is more appropriate to annotate images compare with multinomial distribution. The model is used in limited test set, MSRC-v2 (Microsoft Research Cambridge Image Database). The results show that this model is significantly outperforms previously reported results on the task of image annotation and retrieval.

디지털 이미지의 양이 증가함에 따라 원하는 이미지를 정확하고 빠르게 찾을 수 있는 방법의 필요성이 증가하고 있다. 이미지 검색 방법으로는 이미지의 색상이나 명암과 같은 시각적 특성을 검색 조건으로 이용하는 내용 기반 검색과 이미지를 설명하는 키워드를 검색 조건으로 이용하는 키워드 기반 검색이 있다. 하지만 이러한 방법만으로는 사용자가 원하는 이미지를 정확하게 찾기 힘들다는 문제점이 제기되어 왔다. 따라서 최근에는 검색 도중 사용자의 응답을 받아 사용자의 요구를 파악함으로써 향상된 검색 결과를 제공하는 적합성 피드백에 대한 연구가 많이 진행되고 있다. 하지만 적합성 피드백을 이용하는 방법들도 원하는 결과를 얻기 위해서는 여러 번의 피드백을 필요로 하고 질의 수행이 완료된 후에는 얻어진 피드백 정보를 재사용하지 못한다는 단점이 있다. 따라서 본 논문에서는 이미지에 키워드를 연결한 후 사용자의 피드백 정보를 반영하여 키워드의 신뢰도를 조절함으로써 키워드 기반 이미지 검색의 정확도를 높일 수 있는 모델을 제안한다. 제안된 모델에서는 사용자로부터 피드백을 받은 이미지뿐만 아니라 긍정적 피드백을 받은 이미지들이 공통적으로 가지는 시각적 특성과 유사한 시각적 특성을 가지는 다른 이미지들까지도 키워드의 신뢰도를 조정함으로써 좀 더 빠른 시간 내에 검색 결과의 정확도를 높이도록 한다.

이미지검색을 위해서 객체의 시각적인 특징에 대한 저차원의 특징 정보를 추출하고 이미지에 의미를 부여하기 위하여 주석을 다는 것이 일반적이다. 하지만 주석 기반 검색에서는 주석으로 달아 놓은 단어와 정확한 매칭이 없다면 찾을 수가 없다. 이러한 문제를 해결하기 위해 재질의 질의어 확장과 같은 기법을 써서 문제를 해결해 왔으나 여전히 개념적 매칭이 아닌 스트링 매칭의 문제를 안고 있다고 볼 수 있다. 이에 본 논문에서는 이미지 관련 Text에서 단어를 추출한 후 추출된 단어들간의 개념 관계를 WordNet을 이용하여 표현한 주석 시스템을 제안한다. 이 시스템은 단순 스트링 매칭이 아닌 개념적 매칭에 의한 개념 기반 검색을 지원할 수 있다.