Earticle

Human-computer interaction (HCI) has great potential for applications in many fields. A HCI method has been developed based on audio-visual perceptive features and dynamic interactive pooling. A mode analysis based fusion strategy is proposed to fuse the interactive responses from audio-visual modalities especially for the case there are deviations from the modalities at the same time. The developed approach can be applied with a better performance even in single modal with multiple interactive users in the scenario. The users in the scenario are endowed with a fair chance to perform HCI by their interactive priorities no matter how many modalities used, and the interactive authority and priority are updated dynamically according to the variation of HCI content. The scenario and multiple users altogether are regarded as a dynamic HCI pooling. The diversity of interactive habits among multiple users is considered in an ordinary hardware from a crowded scene without any hypothesis for the scenario contents in advance. The pooling is updated dynamically to meet the user demands with the most intentions preferentially. Comparative study with state-of-the-arts has indicated the superior performance of the proposed method.