Details of Research Outputs

TitleKnowledge Refinery: Learning from Decoupled Label
Author (Name in English or Pinyin)
Qianggang Din; Sifan Wu; Tao Dai; Jiadong Guo; 付樟华; Shutao Xia
Date Issued2020-12-02
Conference NameTheThirty-FifthAAAIConferenceonArtificial Intelligence(AAAI-21)
Source PublicationAAAI
Conference DateFebruary 2, 2021 - February 9, 2021
Conference PlaceVirtual Conference
Publication PlacePALO ALTO
PublisherAssociation for the Advancement of Artificial Intelligence
AbstractRecently, a variety of regularization techniques have been widely applied in deep neural networks, which mainly focus on the regularization of weight parameters to encourage generalization effectively. Label regularization techniques are also proposed with the motivation of softening the labels while neglecting the relation of classes. Among them, the technique of knowledge distillation proposes to distill the soft label, which contains the knowledge of class relations. However, this technique needs to pre-train an extra cumbersome teacher model. In this paper, we propose a method called Knowledge Refinery (KR), which enables the neural network to learn the relation of classes on-the-fly without the teacher-student training strategy. We propose the definition of decoupled labels, which consist of the original hard label and the residual label. To exhibit the generalization of KR, we evaluate our method in both fields of computer vision and natural language processing. Our empirical results show consistent performance gains under all experimental settings. Copyright © 2021, Association for the Advancement of Artificial Intelligence ( All rights reserved
KeywordNatural language processing systems Deep neural networks Personnel training Refining Generalisation Learn+ Neural-networks Regularisation Regularization technique Soft labels Student training Teacher models Teachers' Weight parameters
Indexed ByEI ; CPCI-S
WOS Research AreaComputer Science ; Education & Educational Research
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications ; Education, Scientific Disciplines
WOS IDWOS:000680423507039
EI Accession Number20222012122170
EI Classification Number461.4 Ergonomics and Human Factors Engineering - 723.2 Data Processing and Image Processing - 802.3 Chemical Operations - 912.4 Personnel
Original Document TypeConference article (CA)
Education discipline科技类
Published range国内外公开发行
Data SourceEI
Citation statistics
Document TypeConference paper
CollectionInstitute of Robotics and Intelligent Manufacturing
School of Science and Engineering
Corresponding AuthorTao Dai
Recommended Citation
GB/T 7714
Qianggang Din,Sifan Wu,Tao Daiet al. Knowledge Refinery: Learning from Decoupled Label[C]. PALO ALTO:Association for the Advancement of Artificial Intelligence,2020:7228-7235.
Files in This Item:
There are no files associated with this item.
Related Services
Usage statistics
Google Scholar
Similar articles in Google Scholar
[Qianggang Din]'s Articles
[Sifan Wu]'s Articles
[Tao Dai]'s Articles
Baidu academic
Similar articles in Baidu academic
[Qianggang Din]'s Articles
[Sifan Wu]'s Articles
[Tao Dai]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Qianggang Din]'s Articles
[Sifan Wu]'s Articles
[Tao Dai]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.