Earticle

Cerebral palsy (CP) is a chronic disorder characterized by physical dis-ability and disruption of brain function. We evaluated the effects of treadmill exercise on motor and memory functions in relation with phosphatidylinositol 3-kinase (PI3K)-Akt pathway using CP rat model. Rota-rod test, step-down avoidance task, 5-bromo-2’-deoxyuridine (BrdU) immunohistochemistry, and western blot for synapsin I, postsyn-aptic density-95 (PSD-95), PI3K, Akt, and glycogen synthase kinase-3β (GSK-3β) were performed. CP was induced by maternal lipopolysac-charide (LPS)-injection with sensorimotor restriction. Five weeks after birth, the rats in the exercise groups were made to run on the treadmill for 30 min per one day, 5 times a week, during 4 weeks. Motor and memory functions were impaired in the LPS-induced CP rats and tread-mill exercise increased motor and memory functions in the CP rats. Cell proliferation in the hippocampus was suppressed in the LPS-induced CP rats and treadmill exercise increased hippocampal cell proliferation in the CP rats. Expressions of synapsin I, PSD-95, phosphorylated (p)-PI3K, and p-Akt were decreased in the LPS-induced CP rats and tread-mill exercise enhanced the expressions of synapsin I, PSD-95, p-PI3K, and p-Akt in the CP rats. GSK-3β expression was increased in the LPS-induced CP rats and treadmill exercise suppressed GSK-3β ex-pression in the CP rats. The present results suggest that treadmill exer-cise might improve motor and memory functions through activation of PI3K-Akt pathway.

딥러닝은 비선형 데이터를 다룰 수 있으면서, 내부적인 특성들과 연관 관계 등을 학습할 수 있다는 점에서 많은 분야에서 쓰이고 있다. 딥러닝이 비선형 데이터를 다루는 것이 가능한 이유는 활성화 함수가 사용되기 때문이다. 활성화 함수로는 비선형 함수를 이용하여 입력과 결과가 비선형성을 띌 수 있게 된다. 그런데 활성화 함수의 종류는 다양하며 각각 특성이 다르다. 따라서 어떤 활성화 함수가 사용이 되는지에 따라 결과의 차이가 발생할 수 있다. 본 연구에서는 활성화 함수의 차이가 딥러닝 학습에서 영향을 미치는 지에 대해 알아보고자 한다. 먼저 딥러닝 모델 중 하나인 GAN을 세포의 유전자 데이터를 이용하여 여러 모델 학습을 시킨다. 그 과정에서 각 GAN 모델의 생성자와 식별자는 다른 모델의 활성화 함수와 다른 조합으로 학습된다. 그 후 서로 다른 활성화 함수 조합으로 학습된 GAN의 생성자를 통해 가짜 데이터를 생성한다. 최종적으로 각 모델이 만들어 낸 가짜 데이터와 실제 데이터와의 상관 분석을 통해서 생성된 모델 간의 성능을 비교하여, 활성화 함수의 차이가 딥러닝 학습에 영향을 미치는 지 확인한다.

원자력발전소, 의료용 가속기 등 고선량 지역 및 능동형 열중성자 검출기를 사용하기 힘든 환경에서는 방사화 포일 기반 보너구 스펙트로메터를 사용하여 중성자 에너지 스펙트럼 측정을 수행한다. 중성자 방사화 포일을 보너구의 열중성자 검출기로 사용하는 경우, 중성자 방사화 포일의 위치 변동에 따른 보너구 반응도의 변화 및 중성자 방사화 포일의 질량과 반응도 사이의 상관관계 등 특성연구가 선행되어야만 한다. 본 연구에서는 MCNPX 모사계산을 통하여 중성자 방사화 포일 면에 수직입사하는 경우 중성자 방사화 포일의 위치 의존성이 크다는 사실과 중성자 방사화 포일의 질량과 반응도 사이에 선형관계가 없음을 알 수 있었다. 본 연구를 통하여 중성자 방사화 포일 기반 보너구의 반응함수 계산 방법 및 중성자 방사화 포일 위치 및 질량 차이에 따른 반응도 변화를 연구함으로써 중성자 방사화 포일 기반 보너구의 반응함수 결정방법을 확립하였다. 본 연구결과를 바탕으로 보너구 반응함수를 계산하여 추후 중성자 방사화 포일 기반 보너구 스펙트로메트리에 적용할 예정이다.

Activation foil-based Bonner sphere spheres are used to obtain neutron energy spectra of nuclear power plants or accelerator-produced neutrons. The position and the foil mass dependence of response functions should be studied carefully before measurement of Bonner spheres. This study showed that the normal incidence to the foil surface made a large shift of responses while parallel and isotropic incidence made no position dependence. The correlation between foil mass and response was not linear. Therefore, the response functions of activation-foil based Bonner spheres should be calculated for every different foil mass and the direction of Bonner spheres for parallel incidence will be preferred for radioactive neutron source or accelerator target produced neutrons.

Neurocomputing in complex domain has yielded second generation neural networks. The neural network, which is based on complex value, contains different layers. The attributes of these layers are biases, weights, inputs and outputs. These attributes are also complex numbers. The signal processing, speech processing, learning and prediction of motion on plane are few areas in which complex domain neurocomputing is applied., since in the above said areas, the inputs and outputs are represented by complex values. It has been observed that the neural network with complex value can easily perform the transformation of geometric figures. The examples of transformations are rotation, parallel displacement of straight lines and circles. The neural network can extend to complex domain by the application of transformation. A number in complex domain is composed of different entities i.e. two real numbers and phase information. The two real numbers and phase information of any point on plane is naturally embedded in this number.

Deep reinforcement learning (RL) significantly influences autonomous vehicle development by optimizing decision-making and adaptation to complex driving environments through simulation-based training. In deep RL, an activation function is used, and various activation functions have been proposed, but their performance varies greatly depending on the application environment. Therefore, finding the optimal activation function according to the environment is important for effective learning. In this paper, we analyzed nine commonly used activation functions for RL to compare and evaluate which activation function is most effective when using deep RL for autonomous vehicles to learn highway merging. To do this, we built a performance evaluation environment and compared the average reward of each activation function. The results showed that the highest reward was achieved using Mish, and the lowest using SELU. The difference in reward between the two activation functions was 10.3%.

A very complex task in deep learning such as image classification must be solved with the help of neural networks and activation functions. The backpropagation algorithm advances backward from the output layer towards the input layer, the gradients often get smaller and smaller and approach zero which eventually leaves the weights of the initial or lower layers nearly unchanged, as a result, the gradient descent never converges to the optimum. We propose a two-factor non-saturating activation functions known as Bea-Mish for machine learning applications in deep neural networks. Our method uses two factors, beta (β) and alpha (α), to normalize the area below the boundary in the Mish activation function and we regard these elements as Bea. Bea-Mish provide a clear understanding of the behaviors and conditions governing this regularization term can lead to a more principled approach for constructing better performing activation functions. We evaluate Bea-Mish results against Mish and Swish activation functions in various models and data sets. Empirical results show that our approach (Bea-Mish) outperforms native Mish using SqueezeNet backbone with an average precision (AP50val) of 2.51% in CIFAR-10 and top-1accuracy in ResNet-50 on ImageNet-1k. shows an improvement of 1.20%.

The contrast of an image is a feature which determines how image looks better visually. In this paper, we are analysing the capability of activation functions for contrast enhancement. Activation functions are classically used in neural network. In this paper, Activation function creates a mask which is operated on the image on pixel by pixel basis. On the basis of activation function the pixel value of image is changed which improves the contrast of image. We have used various activation functions such as sigmoid function, bipolar sigmoid function, RAMP function, hyperbolic tangent function. Contrast enhancement using these activation functions has been successfully applied on several dark and bright images. For performance assessment we have used Peak Signal to Noise Ratio (PSNR), absolute mean brightness error (AMBE), and Structure Similarity Index (SSIM). From experimental result, it is observed that RAMP function and hyperbolic tangent function have better image enhancement capability.

자율주행은 자동차가 사람 없이 운전할 수 있도록 해 주며 최근 인공지능 기술의 발전에 힘입어 매우 활발히 연구되고 있다. 인공지능 기술 중에서도 특히 심층 강화 학습이 가장 효과적으로 사용되는데 이를 위해서는 적절한 활성 화 함수를 이용한 신경망 구축이 필수적이다. 여태껏 많은 활성화 함수가 제시됐으나 적용 분야에 따라 서로 다른 성능 을 보여주었다. 본 논문은 교차로에서 자율주행을 학습하기 위해 심층 강화 학습을 사용할 때 어떤 활성화 함수를 사용 하는 것이 효과적인지 성능을 비교 평가한다. 이를 위해 평가에서 사용할 성능 메트릭을 정의하고 각 활성화 함수에 따른 메트릭의 값을 그래프로 비교하였다. 그 결과 Mish를 사용할 경우 보상이 다른 활성화 함수보다 평균적으로 높은 것을 알 수 있었고 보상이 가장 낮은 활성화 함수와의 차이는 9.8%였다.

Autonomous driving allows cars to drive without people and is being studied very actively thanks to the recent development of artificial intelligence technology. Among artificial intelligence technologies, deep reinforcement learning is used most effectively. Deep reinforcement learning requires us to build a neural network using an appropriate activation function. So far, many activation functions have been suggested, but different performances have been shown depending on the field of application. This paper compares and evaluates the performance of which activation function is effective when using deep reinforcement learning to learn autonomous driving on highways. To this end, the performance metrics to be used in the evaluation were defined and the values of the metrics according to each activation function were compared in graphs. As a result, when Mish was used, the reward was higher on average than other activation functions, and the difference from the activation function with the lowest reward was 9.8%.

자율주행 기술은 최근 심층 강화학습의 도입으로 큰 발전을 이루고 있다. 심층 강화 학습을 효과적으로 사용하기 위해서는 적절한 활성화 함수를 선택하는 것이 중요하다. 그 동안 많은 활성화 함수가 제시되었으나 적용할 환경에 따라 다른 성능을 보여주었다. 본 논문은 고속도로에서 자율주행을 학습하기 위해 강화 학습을 사용할 때 어떤 활성화 함수를 사용하는 것이 효과적인지 12개의 활성화 함수 성능을 비교 평가한다. 이를 위한 성능 평가 방법을 제시하였고 각 활성 화 함수의 평균 보상 값을 비교하였다. 그 결과 GELU를 사용할 경우 가장 높은 평균 보상을 얻을 수 있었으며 SiLU는 가장 낮은 성능을 보여주었다. 두 활성화 함수의 평균 보상 차이는 20%였다.

Autonomous driving technology has recently made great progress with the introduction of deep reinforcement learning. In order to effectively use deep reinforcement learning, it is important to select the appropriate activation function. In the meantime, many activation functions have been presented, but they show different performance depending on the environment to be applied. This paper compares and evaluates the performance of 12 activation functions to see which activation functions are effective when using reinforcement learning to learn autonomous driving on highways. To this end, a performance evaluation method was presented and the average reward value of each activation function was compared. As a result, when using GELU, the highest average reward could be obtained, and SiLU showed the lowest performance. The average reward difference between the two activation functions was 20%.

최근 강화 학습을 통해 게임을 학습하는 인공지능 에이전트를 만드는 연구가 활발히 진행되고 있다. 게임을 에이전트 에게 학습 시킬 때 어떠한 딥러닝 활성화 함수를 사용하는지에 따라 그 학습 성능이 달라진다. 본 논문은 2D 슈팅 게임 환경 에서 에이전트가 강화 학습을 통해 게임을 학습할 경우 어떤 활성화 함수가 최적의 결과를 얻는지를 비교 평가 한다. 이를 위해 비교 평가에서 사용할 메트릭을 정의하고 각 활성화 함수에 따른 메트릭 값을 학습 시간에 따라 그래프로 나타내었다. 그 결과 ELU (Exponential Linear Unit) 활성화 함수에 1.0으로 파라미터 값을 설정할 경우 게임의 보상 값이 다른 활성화 함수보다 평균적으로 높은 것을 알 수 있었고, 가장 낮은 보상 값을 가졌던 활성화 함수와의 차이는 23.6%였다.

Recently, there has been active researches about building an artificial intelligence agent that can learn how to play a game by using re-enforcement learning. The performance of the learning can be diverse according to what kinds of deep learning activation functions they used when they train the agent. This paper compares the activation functions when we train our agent for learning how to play a 2D shooting game by using re-enforcement learning. We defined performance metrics to analyze the results and plotted them along a training time. As a result, we found ELU (Exponential Linear Unit) with a parameter 1.0 achieved best rewards than other activation functions. There was 23.6% gap between the best activation function and the worst activation function.

주차 공간의 부족함을 획기적으로 해결할 수 있는 자율주행 자동차는 심층 강화 학습을 통해 큰 발전을 이루고 있다. 심층 강화 학습에는 활성화 함수가 사용되는데, 그동안 다양한 활성화 함수가 제안되어 왔으나 적용 환경에 따라 그 성능 편차가 심했다. 따라서 환경에 따라 최적의 활성화 함수를 찾는 것이 효과적인 학습을 위해 중요하다. 본 논문은 자율주행 자동차가 주차를 학습하기 위해 심층 강화 학습을 사용할 때 어떤 활성화 함수를 사용하는 것이 가장 효과적인 지 비교 평가하기 위해 강화 학습에 주로 사용되는 12개의 함수를 분석하였다. 이를 위해 성능 평가 환경을 구축하고 각 활성화 함수의 평균 보상을 성공률, 에피소드 길이, 자동차 속도와 비교하였다. 그 결과 가장 높은 보상은 GELU를 사용한 경우였고, ELU는 가장 낮았다. 두 활성화 함수의 보상 차이는 35.2%였다.

Autonomous vehicles, which can dramatically solve the lack of parking spaces, are making great progress through deep reinforcement learning. Activation functions are used for deep reinforcement learning, and various activation functions have been proposed, but their performance deviations were large depending on the application environment. Therefore, finding the optimal activation function depending on the environment is important for effective learning. This paper analyzes 12 functions mainly used in reinforcement learning to compare and evaluate which activation function is most effective when autonomous vehicles use deep reinforcement learning to learn parking. To this end, a performance evaluation environment was established, and the average reward of each activation function was compared with the success rate, episode length, and vehicle speed. As a result, the highest reward was the case of using GELU, and the ELU was the lowest. The reward difference between the two activation functions was 35.2%.

Objectives: Fuzzy cognitive maps (FCMs) representing causal knowledge of relationships between medical concepts have been used as prediction tools for clinical decision making. Activation functions used for inferences of FCMs are very important factors in helping physicians make correct decision. Therefore, in order to increase the visibility of inference results, we propose a method for designing certain types of activation functions by considering the characteristics of FCMs. Methods: The activation functions, such as the sinusoidal-type function and linear function, are designed by calculating the domain range of the functions to be reached during the inference process of FCMs. Moreover, the designed activation functions were applied to the decision making process with the inference of an FCM model representing the causal knowledge of pulmonary infections. Results: Even though sinusoidal-type functions oscillate and linear functions monotonously increase within the entire range of the domain, the designed activation functions make the inference stable because the proposed method notices where the function is used in the inference. And, the designed functions provide more visible numeric results than do other functions. Conclusions: Comparing inference results derived using activation functions designed with the proposed method and results derived using activation functions designed with the existing method, we confirmed that the proposed method could be more appropriately used for designing activation functions for the inference process of an FCM for clinical decision making.

본 연구는 화성 기상 데이터를 대상으로 활성화 함수와 어텐션 메커니즘이 시계열 모델의 성능에 미치는 영향을 평가하기 위해 비교 및 분석한다. 화성의 기상 데이터는 대기 밀도가 낮고, 급격한 온도 변동 및 복잡한 지형 등으로 인해 비선형적이고 불규칙적이다. 본 연구에서는 LSTM, BiLSTM, GRU, BiGRU 아키텍처를 사용하여 다양한 활성화 함수와 어텐션 메커니즘의 효과를 평가한다. 실험에 사용된 활성화 함수는 ReLU, Leaky ReLU, ELU, GELU, Swish, SELU이며, 모델 성능은 MAE와 RMSE 지표로 측정된다. 실험 결과, 어텐션 메커니즘을 통합함으로써 MAE와 RMSE가 모두 향상되었으며, Swish와 ReLU는 최저 온도 예측에서 가장 우수한 성능을 보였다. 반면, GELU와 ELU는 기압 예측에서 성능이 저하되었다. 이러한 결과는 복잡한 시계열 예측의 모델 정확도를 향상하기 위해 적절한 활성 함수와 어텐션 메커니즘을 선택하는 것이 중요함을 보여준다.

In this paper, we propose a comparative analysis to evaluate the impact of activation functions and attention mechanisms on the performance of time-series models for Mars meteorological data. Mars meteorological data are nonlinear and irregular due to low atmospheric density, rapid temperature variations, and complex terrain. We use long short-term memory (LSTM), bidirectional LSTM (BiLSTM), gated recurrent unit (GRU), and bidirectional GRU (BiGRU) architectures to evaluate the effectiveness of different activation functions and attention mechanisms. The activation functions tested include rectified linear unit (ReLU), leaky ReLU, exponential linear unit (ELU), Gaussian error linear unit (GELU), Swish, and scaled ELU (SELU), and model performance was measured using mean absolute error (MAE) and root mean square error (RMSE) metrics. Our results show that the integration of attentional mechanisms improves both MAE and RMSE, with Swish and ReLU achieving the best performance for minimum temperature prediction. Conversely, GELU and ELU were less effective for pressure prediction. These results highlight the critical role of selecting appropriate activation functions and attention mechanisms in improving model accuracy for complex time-series forecasting.

This paper concerns the problem of global robust stability of a time-delay discontinuous system with a positive-defined connection matrix under polytopic-type uncertainty. In order to give the stability condition, we firstly address the existence of solution and equilibrium point based on the properties of M-matrix, Lyapunov-like approach and the theories of differential equations with discontinuous right-hand side as introduced by Filippov. Second, we give the delay-independent and delay-dependent stability condition in terms of linear matrix inequalities (LMIs), and based on Lyapunov function and the properties of the convex sets. One numerical example demonstrate the validity of the proposed criteria.

In conventional RBF network, the activation functions of hidden layers generally are symmetric functions like gaussian function. This has been considered to be one of the limiting factors for the network to speed up learning of actuately describing a given function. To avoid this criticism, we propose a pseudo gaussian function (PGF) whose deviation is changed according to the direction of incoming pattern. This property helps to estimate the given function more effectively with a minimal number of centers because of its flexibility of functional representation. A level set method is used to describe the asymmetric shape of deviation of the pseudo gaussian function. To demonstrate the performance of the proposed network ...

The present investigation was under taken to study whether the tumor-associated macrophages (TAM) of Daltons lymphoma (DL), a spontaneous transplantable murine T cell lymphoma can be activated to tumoricidal state by crude thymus extract. Intraperitoneal administration of thymus extract to DL-bearing mice resulted in activation of TAM with an enhanced IL-1, TNF and antigen presenting ability. It was found that treatment with thymus extract could also enhance the phagocytic and cytotoxic activity of TAM. However, only a marginal increase in arginase activity was observed. Till date to the best of our knowledge the effect of crude thymus extract on the activation of tumor associated macrophages has not been investigated, this study provides a new piece of information in the area of thymus therapy.

본 논문에서는 OpenAI Gym 환경에서 제공하는 CartPole-v1에 대해 강화학습을 통해 에이전트를 학습시키고, 학습에 적용되는 활성화함수의 성능을 비교분석하였다. 본 논문에서 적용한 활성화함수는 Sigmoid, ReLU, ReakyReLU 그리고 softplus 함수이며, 각 활성화함수를 DQN(Deep Q-Networks) 강화학습에 적용했을 때 보상 값을 비교하였다. 실험결과 ReLU 활성화함수를 적용하였을 때의 보상이 가장 높은 것을 알 수 있었다.

제주도 동부 중산간 지역은 화산암으로 구성된 지하지질로 인해 지하수위의 변동폭이 크고 변동양상이 복잡하여 인공신경망(Artificial Neural Network, ANN) 모델 등을 활용한 지하수위의 예측이 어렵다. ANN에 적용되는 활성화함수에 따라 지하수의 예측성능은 달라질 수 있으므로 활성화함수의 비교분석 후 적절한 활성화함수의 사용이 반드시 필요하다. 본 연구에서는 5개 활성화함수(sigmoid, hyperbolic tangent(tanh), Rectified Linear Unit(ReLU), Leaky Rectified Linear Unit(Leaky ReLU), Exponential Linear Unit(ELU))를 제주도 동부 중산간지역에 위치한 2개 지하수 관정에 대해 비교분석하여 최적 활성화함수 도출을 목표로 한다. 또한 최적 활성화함수를 활용한 ANN의 적용성을 평가하기 위해 최근 널리 사용되고 있는 순환신경망 모델인 Long Short-Term Memory(LSTM) 모델과 비교분석 하였다. 그 결과, 2개 관정 중 지하수위 변동폭이 상대적으로 큰 관정은 ELU 함수, 상대적으로 작은 관정은 Leaky ReLU 함수가 지하수위 예측에 적절하였다. 예측성능이 가장 낮은 활성화함수는 sigmoid 함수로 나타나 첨두 및 최저 지하수위 예측 시 사용을 지양해야 할 것으로 판단된다. 도출된 최적 활성화함수를 사용한 ANN-ELU 모델 및 ANN-Leaky ReLU 모델을 LSTM 모델과 비교분석한 결과 대등한 지하수위 예측성능을 나타내었다. 이것은 feed-forward 방식인 ANN 모델을 사용하더라도 적절한 활성화함수를 사용하면 최신 순환신경망과 대등한 결과를 도출하여 활용 가능성이 충분히 있다는 것을 의미한다. 마지막으로 LSTM 모델은 가장 적절한 예측성능을 나타내어 다양한 인공지능 모델의 예측성능 비교를 위한 기준이 되는 참고모델로 활용 가능하다. 본 연구에서 제시한 방법은 지하수위 예측과 더불어 하천수위 예측 등 다양한 시계열예측 및 분석연구에 유용하게 사용될 수 있다.

In this study, compared and analyzed various Activation Functions to present a methodology for developing a natural intelligence-based prediction system. As a result of the analysis, ELU was the best with RMSE: 62.87, R<SUP>2</SUP>: 0.96, and the error rate was 4%. However, it is considered desirable to construct a prediction system by combining each algorithm model for optimization.

When we apply the neural networks to applications, we need to select proper architecture of the network and the activation function of the network is one of most important characteristics. In this research, we propose a method to make a network using multiple activation functions. The performance of the proposed method is investigated through the computer simulations on various regression problems.