Earticle

The measurement of informatization program effectiveness continues to be a central concern of both academics and practitioners. However, most studies of it have been inclined too heavily toward testing validity and reliability of evaluation measure to strengthen the theoretical rigor. Drawbacks of this approach have been pointed out as being unsuitable for evaluating the effectiveness of informatization program due to lack of practical relevance. The paper comprises four sections to solve the problem. 1) Through literature review of theoretical conceptual models in IS effectiveness and the domestic and foreign cases on informatization program, we establish the evaluation framework which consists of evaluation phases, 6 evaluation areas, and 22 evaluation items, and developed evaluation measures of each items. 2) Different researchers have addressed different aspects of attribute of measure. With comparing them to each other, 3 mutually exclusive criteria on coincidence analysis(usability of measurement, reliability of measurement, and validity of content) are identified. 3) Data are gathered from respondents in informatizaiton program-related organizations such as policy agency, management agency, support agency, and recipient agency. 4) The degree of coincidence of each evaluation measure are analyzed by course of an coincidence analysis procedure, which includes the reliability test, arithmetical test, and criteria validity test. The academic implication of this study is that researchers can gain an insight into the coincidence analysis to approach the problem of deciding on an indicator of performance for informatization program. There is also practical implication for applying the potential means of improving the practical relevance of conducting evaluating the informatization program.

본 연구는 정보화지원사업의 성과평가모형 개발에 관한 유용한 정보를 제공하기 위한 목적을 가지고 다음과 같은 절차에 의해 수행되었다. ① 기술적/기능적, 경제적/재무적, 해석적/대안적 성과평가 패러다임에 대한 고찰을 통해 이들의 철학적․방법론적 관점을 비교․분석하였다. ② 정보시스템 효과평가와 관련된 이론적 모형을 비롯하여 실무분야에서 활용되고 있는 국내․외 평가사례에 대한 비판적 분석을 통해 기존의 정보화성과평가체계 구축․운영상의 문제점 및 한계점을 살펴보았다. ③ 이상적 성과평가모형이 갖추어야할 접근방법, 방법, 기법, 절차 등을 체계적으로 통합시킨 5W1H기반의 성과평가체계를 정립하였다. ④ 성과평가체계의 제구성요소들간의 상호관계에 관한 논리적 추론을 통해 3개 평가시점, 6개 평가영역, 22개 평가항목으로 구성된 개념적 성과평가모형을 개발하였다. 본 연구의 결과는 정보화지원사업의 성과관리 및 평가를 위한 이론적 프레임워크를 구성하기 위한 유용한 기준으로 활용될 수 있다는 이론적 의의를 가지며, 실무적인 측면에서도 다양한 유형의 정보화지원사업의 바람직한 성과관리를 위한 포괄적이고 분석적인 시각을 제공해 줄 수 있다는 기여점을 가진다.

This study amis to provide the IS community with useful information on the development of performance evaluation model in the area of IS(information systems) research. To achieve the purpose of research, the follows are conducted. We begin with a general overview of theoretical paradigms, which are technical/functional, economic/financial, and interpretive/alternative paradigms, and present philosophical and methodological perspectives of them. Subsequently, a number of theoretical evaluation models with respect to IS effectiveness as well as the domestic and foreign cases with relation to informatization project and program are critically reviewed. Then, diverse approaches, methods and techniques related to performance evaluation systems are systematically integrated within the framework of 5W1H. Lastly, the conceptual performance evaluation model which consisted of 3 evaluation phases, 6 evaluation areas, and 22 evaluation items is developed through logically reasoning the interrelationship between components of the performance evaluation system. For researchers, the implications of this study can be utilized significant basis for constructing the theoretical framework of evaluation method and management of the informatization program. For practitioners, the integrated performance evaluation model developed in this study might provide the comprehensive and analytical view for the desirable performance management of the various types of informatization program.

농어촌지역의 정보격차 해소, 주민의 소득증대, 지역경제 활성화를 위해 2001년부터 정보화마을을 조성하기 시작하여 2009년 11월 현재 현재 전국 15개 광역시도 시군구 기초자치단체에서 358개의 정보화마을을 운영하고 있다. 정보화마을 운영이 안고 있는 문제점으로는 지역정보화 자체가 안고 있는 문제점, 정보화마을운영에 따른 문제점, 정보화마을 사업 평가가 안고 있는 문제점 등을 들 수 있다. 이러한 문제점을 제대로 파악하기 위한 평가는 지난 10여년 동안 두세 차례에 불과하며, 사업에 대한 연구도 미미하다는 점 또한 알게 되었다. 이 연구에서는 2000년부터 시작된 정보화사업평가와 정보화수준평가를 위해 행정자치부와 정보통신부가 주가 된 정보화사업 평가지표를 바탕으로 2005년, 2006년 만들어진 정보화마을 평가지표를 참조하여 정보화마을에 대한 평가를 정보화사업 평가 차원에서 진행하고자 하였다. 연구를 통하여 알게 된 시사점으로는 먼저 방법론적 차원에서 정보화마을 사업 평가도 정보화사업 평가 지표 보완과 개발을 통해 주기적으로 평가가 가능하다는 점을 알게 되었으며, 다음으로 정보화마을사업 내용면에서 볼 때 사업 본래 목표에 부합되는 긍정적인 효과를 거둘 수 측면도 있음을 알게 되었다.

The key words of this study is the evaluation on the informatization program, information village project evaluation and information village. The purpose of this study is to evaluate the informatization program focusing on the Information Village. For the analysis some indicators were pointed out and the model was developed that was partially used in my 2002 research. The indicators include two main factors. The first factor, independent variable, contains three variables like as businessman, affairs effectiveness, business, law and institution, correlation, organizational context). The second factor, dependent variable, involves three variables such as user's satisfaction, possibility of business success, competitiveness. Two hypotheses were tested. The first is described as 'the relations among variables composing the independent variables that would be shown 'statistically significant'. And the second is written as 'the level of influences of the independent variables to dependent variables would be shown 'statistically significant' to the variable of e_Government. The statistical methodology such as factor analysis, ANOVA, F-test etc. were used for this study. Two hypotheses were totally accepted at the level of 5%. Finally some implications are indicated. The one is the new methodology for evaluating the informatization program, and the other is that the information village project would be meaningful, in the some parts, for solving the regional digital divide and promoting the rural people's income.

본 연구에서는 정보화지원사업에 대한 성과평가를 위해 도출된 각 평가항목의 측정지표들을 실제로 평가업무에 적용함에 있어서 그 사용의 용의성 및 신뢰성, 그리고 내용적인 적합성이 어느 정도 확보되어 있는지를 검증하는 일련의 부합성 분석절차를 개발${\cdot}$확립하였다.

본 연구는 『중소기업기술정보진흥원』에서 실시하고 있는 ‘중소기업 정보화지원사업’의 신청기업을 선정하기 위한 평가지표의 상대적인 중요도를 알아보기 위하여 2개의 평가기준, 9개의 평가영역, 31개의 평가지표로 계층화 및 모델링하고, 중소기업 정보화지원사업 평가위원 풀에 속해 있는 학계, 업계, 연구소 등의 전문가들을 대상으로 AHP(Analytic Hierarchy Process)분석기법과 Expert Choice 2000(Second Edition)을 활용해 성과지표의 가중치를 차별적으로 도출한다. 또한 다양한 측면의 민감도 분석을 통해 가중치의 변화에 따른 대안의 우선순위가 어떻게 변화하는지를 검토해 볼 수 있도록 하여 이를 통해 ‘중소기업 정보화지원사업’의 바람직한 신청기업 선정을 위한 전략적인 시사점을 제공하고자 한다. AHP분석 절차는 (1단계)계층도 모델링 → (2단계)쌍대수치비교 → (3단계)일관성 비율판정 → (4단계)평가지표의 우선순위 선정 → (5단계)민감도 분석의 절차로 진행되었으며, 본 연구를 통해 도출되는 성과민감도(Performance Sensitivity), 동적민감도(Dynamic Sensitivity), 경사민감도(Gradient Sensitivity)의 분석을 통해 의사결정과 관련된 정보의 변화에 따른 민감도를 분석할 수 있게 하였다. 즉 평가기준의 가중치를 변화시켜 나감으로써 평가지표의 우선순위가 어떻게 변화하는지를 검토하여, 중소기업 정보화지원사업의 신청기업을 선정하는 중요한 변수로 활용될 수 있다.

The purpose of this research is going to obtain differently the relative importance by targeting to the assessment category and its factors of assessment criteria(site assessment and evaluation assessment) to select the applied companies for 'enforcement business of management innovation which based on IT" and "digitalized business for manufacturing site" among the “SMEs Informatization Supporting Program” which TIPA carries out and to use it as a reasonable and useful data for selecting the applied companies through this. The procedure of AHP analysis of this research is as the following:(Step1)Modelling of Hierarchical layer (Step2)Pairwise Numerical Comparisons(Step3)Decision of Consistency Ratio(Step4)choice of Priority for valuation factors(Step5)Analysis of Sensitivity -(Step1) Hierarchical Modelling : We had been created the evaluation criteria and its alternative for "the weighted values for evaluation area and its factors in order to choose the "SMEs Informatization Supporting Program" using "Hierarchical modelling among Valuation factors for Informatization support business for SMEs" and we have been made the modelling for its hierarchical layer. -(Step2) Pairwise Numerical Comparisons : To investigate the relative importance regarding to the evaluation area and its criteria, we had been input the pairwise numerical value using the pairwise Numerical Comparisons among kinds of evaluation nodes. -(Step3) Decision of Consistency Ratio : We had been generated the Consistency Ratio using Priorities derived from Pairwise Comparisons and confirmed the consistency of answers by setting the consistency ratio to less than 0.2, considered the hierarchical models of this research that were made up 2 pairwise comparison objects for layer 1, 9 pairwise comparison objects for layer 2 and 31 pairwise comparison objects for layer 3.(SanhHoon Kim, JumKi Choi, 2007). It had been selected 25 persons to be satisfied the consistency less than 0.2 among 35 available respondents and processed the decision of consistency ratio after selecting 25 participants which were number of limitation provided in Expert Choice 2000(2nd edition)version by ascending order. -(Step4) choice of Priority for valuation factors : The priority choice for valuation factors and weighted value of valuation index which selects the company of an "SMEs Informatization Supporting Program" had been chosen according to the final results of the local index value and global index value with a combination of the answers by 25 available respondents. -(Step5) Analysis of Sensitivity : The Analysis of Sensitivity is shown the changing rate of valuation factors depending on a relative importance for the company selection among applied companies for "SMEs Informatization Supporting Program" and Decision analysis by assessors could be considered the changing of a relative importance for evaluation criteria depending on diverse situations and the relative decision change by decision maker, etc. As the results of this research, the relative importance of evaluation criteria had been shown 30.2% in "site assessment" and 69.8% in "evaluation assessment" was shown more than 2 times of "site assessment". When the criteria area of "site assessment" and the relative importance about evaluation criteria had been compared, it had been recorded the highest weighted value because the item of " a will for carrying out the business and environments" recorded 64.2% as well as we could found that "the carrying out the Informatization environments" recorded at 19.3% and "validity of assignment" recorded 16.5% as a similar level. In case of "the evaluation assessment", the "reality" of 28.8% had been analysed to have a more important than "Support ability" of 21.8%, "support strategy"of 19.9% and "reasonability of assignment" of 12.5% and "Operational management" and "business management" were shown the lowest weighted value as 0.8%. It could be understood that the various conditions for ...