As an open-source project, huawei-noah/Pretrained-Language-Model has picked up 3.2k stars on GitHub (Python). Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
This repository provides the latest pretrained language models and its related optimization techniques developed by Huawei Noah's Ark Lab.
Directory structure
PanGu-α is a Large-scale autoregressive pretrained Chinese language model with up to 200B parameter. The models are developed under the MindSpore and trained on a cluster of Ascend 910 AI processors.
NEZHA-TensorFlow is a pretrained Chinese language model which achieves the state-of-the-art performances on several Chinese NLP tasks developed under TensorFlow.
NEZHA-PyTorch is the PyTorch version of NEZHA.
NEZHA-Gen-TensorFlow provides two GPT models. One is Yuefu (乐府), a Chinese Classical Poetry generation model, the other is a common Chinese GPT model.
TinyBERT is a compressed BERT model which achieves 7.5x smaller and 9.4x faster on inference.
TinyBERT-MindSpore is a MindSpore version of TinyBERT.
DynaBERT is a dynamic BERT model with adaptive width and depth.
BBPE provides a byte-level vocabulary building tool and its correspoinding tokenizer.
PMLM is a probabilistically masked language model. Trained without the complex two-stream self-attention, PMLM can be treated as a simple approximation of XLNet.
TernaryBERT is a weights ternarization method for BERT model developed under PyTorch.
TernaryBERT-MindSpore is the MindSpore version of TernaryBERT.
HyperText is an efficient text classification model based on hyperbolic geometry theories.
BinaryBERT is a weights binarization method using ternary weight splitting for BERT model, developed under PyTorch.
AutoTinyBERT provides a model zoo that can meet different latency requirements.
PanGu-Bot is a Chinese pre-trained open-domain dialog model build based on the GPU implementation of PanGu-α.
CeMAT is a universal sequence-to-sequence multi-lingual pre-training language model for both autoregressive and non-autoregressive neural machine translation tasks.
Noah_WuKong is a large-scale Chinese vision-language dataset and a group of benchmarking models trained on it.
Noah_WuKong-MindSpore is a MindSpore version of Noah_WuKong.
CAME is a Confidence-guided Adaptive Memory Efficient Optimizer.
How active is development on huawei-noah/Pretrained-Language-Model?
The most recent commit recorded on huawei-noah/Pretrained-Language-Model was 2.6 years ago, based on the GitHub push timestamp. The repository has 638 forks — one of the better signals of community interest.
How many stars does huawei-noah/Pretrained-Language-Model have?
huawei-noah/Pretrained-Language-Model has 3.2k GitHub stars — refresh the page for the live number, or check github.com/huawei-noah/Pretrained-Language-Model. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is huawei-noah/Pretrained-Language-Model open source?
TopGit's metadata for huawei-noah/Pretrained-Language-Model does not record a license. Most public repositories on GitHub ARE open source, but the exact terms vary — verify by opening the LICENSE file directly.
What is huawei-noah/Pretrained-Language-Model?
huawei-noah/Pretrained-Language-Model (huawei-noah/Pretrained-Language-Model) is a Python project on GitHub. From the project's own README: Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Where do I read more about huawei-noah/Pretrained-Language-Model?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/huawei-noah/Pretrained-Language-Model is the definitive source.
Read full README in the tab above.
Want a second opinion on Pretrained-Language-Model?
Ask an AI that can read this page — one click and you get its take on Pretrained-Language-Model.
/Pretrained-Language-Model