AutoBERT-Zero: Evolving BERT backbone from scratch

Keyphrases

• Attention Structure 50%
• Branch Weight 50%
• Candidate Evaluation 50%
• Convolution Block 50%
• Copyright 50%
• Downstream Task 50%
• Evaluation Model 50%
• Evolution Strategy 50%
• Fast Model 50%
• Flexible Search 50%
• Global Self-attention 50%
• High-score 50%
• Inductive Bias 50%
• Intralayer 50%
• Large Animal Model 50%
• Local Dependence 50%
• Model Capacity 50%
• Model Size 50%
• Natural Language Processing 50%
• Neural Architecture Search 50%
• Operations Priorities 100%
• Pre-trained Language Model 100%
• Processing Task 50%
• RoBERTa 50%
• Search Algorithm 100%
• Search Evaluation 50%
• Search Model 50%
• Search Space 100%
• Self-attention Layer 50%
• Training Strategy 50%
• Transformer-based 50%
• Weight Sharing 50%

Computer Science

• Bidirectional Encoder Representations From Transformers 100%
• Candidate Model 25%
• Capacity Model 25%
• Evaluation Models 25%
• Evolution Strategy 25%
• Math Operation 25%
• Natural Language Processing 25%
• Neural Architecture Search 25%
• Pre-Trained Language Models 50%
• Processing Task 25%
• Search Algorithm 50%
• Search Space 50%