GITHUB SNAPSHOT

automl/awesome-transformer-search

TopGit repo profile for automl/awesome-transformer-search, with GitHub repository stats and README context.

Stars

★ 270

Forks

⑂ 30

Language

—

Topic

—

License

—

Updated

3.2y ago

Contributors

—

Homepage

—

View on GitHub ↗

QUICK VERDICT

On GitHub, automl/awesome-transformer-search has picked up 270 stars. A curated list of awesome resources combining Transformers with Neural Architecture Search

Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.

WHY NO REVIEW YET

TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.

Snapshot

Stars★ 270

Forks⑂ 30

Language—

Topic—

License—

Homepage—

Read our review methodology ↗

Top contributors

Show top contributors

Top 21 contributors to automl/awesome-transformer-search

Title	Venue	Group
Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models	NeurIPS'22	MSR
Training Free Transformer Architecture Search	CVPR'22	Tencent & Xiamen University
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models	AutoML Conference 2022 Workshop Track	MSR
Searching the Search Space of Vision Transformer	NeurIPS'21	MSRA, Stony Brook University
UniNet: Unified Architecture Search with Convolutions, Transformer and MLP	ECCV'22	SenseTime
Analyzing and Mitigating Interference in Neural Architecture Search	ICML'22	Tsinghua, MSR
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search	ICCV'21	Sun Yat-sen University
Memory-Efficient Differentiable Transformer Architecture Search	ACL-IJCNLP'21	MSR, Peking University
Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition	arxiv [Aug'20]	Google Research
AutoTrans: Automating Transformer Design via Reinforced Architecture Search	NLPCC'21	Fudan University
NASABN: A Neural Architecture Search Framework for Attention-Based Networks	IJCNN'20	Chinese Academy of Sciences
NAT: Neural Architecture Transformer for Accurate and Compact Architectures	NeurIPS'19	Tencent AI
The Evolved Transformer	ICML'19	Google Brain

Title	Venue	Group
𝛼NAS: Neural Architecture Search using Property Guided Synthesis	ACM Programming Languages'22	MIT, Google
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training	ICLR'22	Meta Reality Labs
AutoFormer: Searching Transformers for Visual Recognition	ICCV'21	MSR
GLiT: Neural Architecture Search for Global and Local Image Transformer	ICCV'21	University of Sydney
Searching for Efficient Multi-Stage Vision Transformers	ICCV'21 workshop	MIT
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers	CVPR'21	Bytedance Inc.

Title	Venue	Group
AutoBERT-Zero: Evolving the BERT backbone from scratch	AAAI'22	Huawei Noah’s Ark Lab
Primer: Searching for Efficient Transformers for Language Modeling	NeurIPS'21	Google
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models	ACL'21	Tsinghua, Huawei Naoh's Ark
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search	KDD'21	MSR, Tsinghua University
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing	ACL'20	MIT

Title	Venue	Group
SFA: Searching faster architectures for end-to-end automatic speech recognition models	Computer Speech and Language'23	Chinese Academy of Sciences
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search	ICASSP'21	MSR
Efficient Gradient-Based Neural Architecture Search For End-to-End ASR	ICMI-MLMI'21	NPU, Xi'an
Evolved Speech-Transformer: Applying Neural Architecture Search to End-to-End Automatic Speech Recognition	INTERSPEECH'20	VUNO Inc.

Title	Venue	Group
RWKV: Reinventing RNNs for the Transformer Era	arxiv [May'23]	EleutherAI
Patches are All You Need ?	TMLR'23	CMU
Seperable Self Attention for Mobile Vision Transformers	TMLR'23	Apple
Parameter-efficient Fine-tuning for Vision Transformers	AAAI'23	MSR & UCSC
EfficientFormer: Vision Transformers at MobileNet Speed	NeurIPS'22	Snap Inc
Neighborhood Attention Transformer	CVPR'23	Meta AI
Training Compute Optimal Large Language Models	NeurIPS'22	DeepMind
CMT: Convolutional Neural Networks meet Vision Transformers	CVPR'22	Huawei Noah’s Ark Lab
Patch Slimming for Efficient Vision Transformers	CVPR'22	Huawei Noah’s Ark Lab
Lite Vision Transformer with Enhanced Self-Attention	CVPR'22	Johns Hopkins University, Adobe
TubeDETR: Spatio-Temporal Video Grounding with Transformers	CVPR'22 (Oral)	CNRS & Inria
Beyond Fixation: Dynamic Window Visual Transformer	CVPR'22	UT Sydney & RMIT University
BEiT: BERT Pre-Training of Image Transformers	ICLR'22 (Oral)	MSR
How Do Vision Transformers Work?	ICLR'22 (Spotlight)	NAVER AI
Scale Efficiently: Insights from Pretraining and FineTuning Transformers	ICLR'22	Google Research
Tuformer: Data-Driven Design of Expressive Transformer by Tucker Tensor Representation	ICLR'22	UoMaryland
DictFormer: Tiny Transformer with Shared Dictionary	ICLR'22	Samsung Research
QuadTree Attention for Vision Transformers	ICLR'22	Alibaba AI Lab
Expediting Vision Transformers via Token Reorganization	ICLR'22 (Spotlight)	UC San Diego & Tencent AI Lab
UniFormer: Unified Transformer for Efficient Spatial-Temporal Representation Learning	ICLR'22	SIAT-SenseTime
Hierarchical Transformers Are More Efficient Language Models	NAACL'22	Google Research, UoWarsaw
Transformer in Transformer	NeurIPS'21	Huawei Noah's Ark
Long-Short Transformer: Efficient Transformers for Language and Vision	NeurIPS'21	NVIDIA
Memory-efficient Transformers via Top-k Attention	EMNLP Workshop '21	Allen AI
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows	ICCV'21 best paper	MSR
Rethinking Spatial Dimensions of Vision Transformers	ICCV'21	NAVER AI
What makes for hierarchical vision transformers	arxiv [Sept'21]	HUST
AutoAttend: Automated Attention Representation Search	ICML'21	Tsinghua University
Rethinking Attention with Performers	ICLR'21 Oral	Google
LambdaNetworks: Modeling long-range Interactions without Attention	ICLR'21	Google Research
HyperGrid Transformers	ICLR'21	Google Research
LocalViT: Bringing Locality to Vision Transformers	arxiv [April'21]	ETH Zurich
Compressive Transformers for Long Range Sequence Modelling	ICLR'20	DeepMind
Improving Transformer Models by Reordering their Sublayers	ACL'20	FAIR, Allen AI
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned	ACL'19	Yandex

Title	Venue	Group
Transformers in Vision: A Survey	ACM Computing Surveys'22	MBZ University of AI
A Survey of Vision Transformers	TPAMI'22	CAS
Efficient Transformers: A Survey	ACM Computing Surveys'22	Google Research
Neural Architecture Search for Transformers: A Survey	IEEE xplore [Sep'22]	Iowa State Uni

Title	Venue	Group
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models	arxiv'23	Amazon Alexa AI

Related repositories

FREQUENTLY ASKED

Quick answers

Does automl/awesome-transformer-search have a project website?

No homepage URL was recorded for automl/awesome-transformer-search in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.

How active is development on automl/awesome-transformer-search?

The most recent commit recorded on automl/awesome-transformer-search was 3.2 years ago, based on the GitHub push timestamp. The repository has 30 forks — one of the better signals of community interest.

How many stars does automl/awesome-transformer-search have?

automl/awesome-transformer-search has 270 GitHub stars — refresh the page for the live number, or check github.com/automl/awesome-transformer-search. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.

Is automl/awesome-transformer-search open source?

TopGit's metadata for automl/awesome-transformer-search does not record a license. Most public repositories on GitHub ARE open source, but the exact terms vary — verify by opening the LICENSE file directly.

What topics is automl/awesome-transformer-search associated with?

GitHub's repository topics for automl/awesome-transformer-search: "neural-architecture-search", "transformer". TopGit's editorial category is open-source.

Where do I read more about automl/awesome-transformer-search?

This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/automl/awesome-transformer-search is the definitive source.

Read full README in the tab above.

Curious whether awesome-transformer-search is right for you?

Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about awesome-transformer-search.

Ask ChatGPT Ask Claude Ask Perplexity Ask Google AI Mode Ask Kimi Ask Grok

GitHub

automl/awesome-transformer-search

Snapshot

Top contributors

Awesome Transformer Architecture Search:

General Transformer Search

Domain Specific Transformer Search

Vision

Natural Language Processing

Automatic Speech Recognition

Transformers Knowledge: Insights, Searchable parameters, Attention

Transformer Surveys

Foundation Models

Misc resources

Related repositories

Quick answers

Curious whether awesome-transformer-search is right for you?