TopGit theo dõi xiph/LPCNet trên GitHub, đã đạt 1.2k sao. Efficient neural speech synthesis
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
Important: LPCNet is no longer being actively developed. It will remain
available but for most applications, users are encouraged to switch to the
Framewise Autoregressive GAN (FARGAN). FARGAN achieves better quality than
LPCNet with just 600 MFLOPS complexity. That's 1/5 of the complexity of the
most optimized LPCNet and 1/20 of the original LPCNet.
J.-M. Valin, A. Mustafa, Jan Büthe, Very Low Complexity Speech Synthesis Using Framewise Autoregressive GAN (FARGAN) with Pitch Prediction, IEEE Signal Processing Letters, 2024, arXiv:2405.21069.
See our demo page for comparisons
with LPCNet, HiFi-GAN, CARGAN and FWGAN.
The PyTorch source code
along with an optimized C implementation are available as part of the larger
Opus codec implementation (FARGAN is used for PLC
and deep redundancy within Opus).
LPCNet
Low complexity implementation of the WaveRNN-based LPCNet algorithm, as described in:
J.-M. Valin, J. Skoglund, LPCNet: Improving Neural Speech Synthesis Through Linear Prediction, Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP), arXiv:1810.11846, 2019.
J.-M. Valin, U. Isik, P. Smaragdis, A. Krishnaswamy, Neural Speech Synthesis on a Shoestring: Improving the Efficiency of LPCNet, Proc. ICASSP, arxiv:2106.04129, 2022.
K. Subramani, J.-M. Valin, U. Isik, P. Smaragdis, A. Krishnaswamy, End-to-end LPCNet: A Neural Vocoder With Fully-Differentiable LPC Estimation, Proc. INTERSPEECH, arxiv:2106.04129, 2022.
For coding/PLC applications of LPCNet, see:
J.-M. Valin, J. Skoglund, A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet, Proc. INTERSPEECH, arxiv:1903.12087, 2019.
J. Skoglund, J.-M. Valin, Improving Opus Low Bit Rate Quality with Neural Speech Synthesis, Proc. INTERSPEECH, arxiv:1905.04628, 2020.
J.-M. Valin, A. Mustafa, C. Montgomery, T.B. Terriberry, M. Klingbeil, P. Smaragdis, A. Krishnaswamy, Real-Time Packet Loss Concealment With Mixed Generative and Predictive Model, Proc. INTERSPEECH, arxiv:2205.05785, 2022.
J.-M. Valin, J. Büthe, A. Mustafa, Low-Bitrate Redundancy Coding of Speech Using a Rate-Distortion-Optimized Variational Autoencoder, Proc. ICASSP, arXiv:2212.04453, 2023. (blog post)
Introduction
Work in progress software for researching low CPU complexity algorithms for speech synthesis and compression by applying Linear Prediction techniques to WaveRNN. High quality speech can be synthesised on regular CPUs (around 3 GFLOP) with SIMD support (SSE2, SSSE3, AVX, AVX2/FMA, NEON currently supported). The code also supports very low bitrate compression at 1.6 kb/s.
The BSD licensed software is written in C and Python/Keras. For training, a GTX 1080 Ti or better is recommended.
This software is an open source starting point for LPCNet/WaveRNN-based speech synthesis and coding.
Using the existing software
You can build the code using:
./autogen.sh
./configure
make
Note that the autogen.sh script is used when building from Git and will automatically download the latest model
(models are too large to put in Git). By default, LPCNet will attempt to use 8-bit dot product instructions on AVX*/Neon to
speed up inference. To disable that (e.g. to avoid quantization effects when retraining), add --disable-dot-product to the
configure script. LPCNet does not yet have a complete implementation for some of the integer operations on the ARMv7
architecture so for now you will also need --disable-dot-product to successfully compile on 32-bit ARM.
It is highly recommended to set the CFLAGS environment variable to enable AVX or NEON prior to running configure, otherwise
no vectorization will take place and the code will be very slow. On a recent x86 CPU, something like
export CFLAGS='-Ofast -g -march=native'
should work. On ARM, you can enable Neon with:
export CFLAGS='-Ofast -g -mfpu=neon'
While not strictly required, the -Ofast flag will help with auto-vectorization, especially for dot products that
cannot be optimized without -ffast-math (which -Ofast enables). Additionally, -falign-loops=32 has been shown to
help on x86.
You can test the capabilities of LPCNet using the lpcnet_demo application. To encode a file:
./lpcnet_demo -encode input.pcm compressed.bin
where input.pcm is a 16-bit (machine endian) PCM file sampled at 16 kHz. The raw compressed data (no header)
is written to compressed.bin and consists of 8 bytes per 40-ms packet.
To decode:
./lpcnet_demo -decode compressed.bin output.pcm
where output.pcm is also 16-bit, 16 kHz PCM.
Alternatively, you can run the uncompressed analysis/synthesis using -features
instead of -encode and -synthesis instead of -decode.
The same functionality is available in the form of a library. See include/lpcnet.h for the API.
To try packet loss concealment (PLC), you first need a PLC model, which you can get with:
where error_pattern.txt is a text file with one entry per 20-ms packet, with 1 meaning "packet lost" and 0 meaning "packet not lost".
noncausal_dc is the non-causal (5-ms look-ahead) with special handling for DC offsets. It's also possible to use "noncausal", "causal",
or "causal_dc".
Training a new model
This codebase is also meant for research and it is possible to train new models. These are the steps to do that:
where the first file contains 16 kHz 16-bit raw PCM audio (no header) and the other files are output files. This program makes several passes over the data with different filters to generate a large amount of training data.
and it will generate an h5 file for each iteration, with model_name as prefix. If it stops with a
"Failed to allocate RNN reserve space" message try specifying a smaller --batch-size for train_lpcnet.py.
You can synthesise speech with Python and your GPU card (very slow):
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/xiph/LPCNet là nguồn chính thức.
xiph/LPCNet có phải mã nguồn mở không?
Có — xiph/LPCNet phát hành theo license BSD-3-Clause, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/xiph/LPCNet.
xiph/LPCNet là gì?
xiph/LPCNet (xiph/LPCNet) là dự án C trên GitHub. Theo mô tả gốc: Efficient neural speech synthesis
Đọc đầy đủ README ở tab phía trên.
Muốn nghe thêm một ý kiến về LPCNet?
Hỏi một AI đọc được trang này — một cú bấm là có ngay nhận định về LPCNet.