Attention-enhanced Gradual Machine Learning

Introduction

The existing GML solution for ER supposes that features play independent roles in gradual inference. Unfortunately, this assumption may be untenable in real scenarios because features are usually correlated with each other. Attention-enhanced GML (AGML) proposes an automatically optimize feature weighting by feature correlation analysis. The AGML has the following characteristics:

1. New method of feature representation

The existing attention neural networks usually leverage pre-trained models such as BERT to represent text as fixed-length vectors. These models map semantically similar text to close points in the same vector space. However, the scenario of GML brings about new challenges because GML requires to map features with similar distributions to close points in the same vector space. AGML proposes a new method of spectral feature representation to map features with similar distributions into the same vector space.

2. Attention-enhanced gradual inference

Based on spectral feature representation, the proposed attention neural netwrok can effectively learn the decisive features given arbitrary combinations of features.

Publications

AGML Framework