aliyun/SimAI

中文  ｜  English  ｜  日本語

SimAI

Latest News

Recent Updates

• [2026/04] SimAI 1.6 Released! Key updates:

  • GPU memory modeling for inference simulation (parameter counting & KV cache).
  • Linear interpolation for decode time estimation (replacing nearest-neighbor).
  • Prefill-Decode Disaggregation memory planning (independent budgets for Prefill/Decode).

• [2025/12] SimAI 1.5 Released! This release brings end-to-end simulation for multi-request inference workloads. Key features include:

  • Advanced Inference Simulation: Model complex scenarios with Prefill/Decode separation.
  • Modern Model Support: Now includes DeepSeek, Qwen3Moe and Qwen3Next. See AICB's README for more detailed information.
  • Request Scheduling: Request scheduling is now handled by a component adapted from Microsoft's Vidur. See Vidur-Alibabacloud's README for more detailed information.

• [2025/11] AICB now supports generating prefill/decode inference workloads for DeepSeek, Qwen3-MoE and Qwen3-Next.

• [2025/09] AICB now supports generating training workloads for DeepSeek. Thanks to @parthpower for this contribution.

• [2025/06] The code of SimCCL is first released in the branch SimCCL and will be released in SimCCL repository soon.

We warmly welcome contributions from the community! If you are interested in helping shape the future of SimAI, please feel free to open an issue to discuss your ideas or submit a pull request.

Documentation

See Tutorial for full documentation.

Table of Contents

• SimAI Overview
  • Introduction
  • Components
  • Scenario
  • Citation
• Quick Start
  • Setup
  • Use SimAI-Analytical
  • Use SimAI-Simulation
  • Use Multi-requests Inference Simulation

SimAI Overview

Introduction

SimAI is the industry's first full-stack, high-precision Simulator for AI large-scale inference and training. It provides detailed modeling and simulation of the entire LLM training process, encompassing framework, collective communication, network layers, and more. This comprehensive approach offers end-to-end performance data, enabling researchers to:

• Analyze inference/training process details
• Evaluate the time consumption of AI tasks under specific conditions
• Evaluate E2E performance gains from various algorithmic optimizations including:
  • Framework parameters settings
  • Collective communication algorithms
  • NCCL environment variables
  • Network transmission protocols
  • Congestion control algorithms
  • Adaptive routing algorithms
  • Scale-up/out network topology modifications
  • ...

Components

        |--- AICB
SimAI --|--- SimCCL
        |--- astra-sim-alibabacloud
        |--- ns-3-alibabacloud
        |--- vidur-alibabacloud

Building on pure simulation capabilities, SimAI has evolved into a versatile full-stack toolkit comprising four components (aicb, SimCCL, astra-sim-alibabacloud, ns-3-alibabacloud). These components can be combined in various ways to achieve different functionalities. Below, we present the main usage scenarios for SimAI. We encourage users to explore even more possibilities with this powerful tool.

Below is the architecture diagram of the SimAI Simulator:

astra-sim-alibabacloud is extended from astra-sim. We are grateful to the astra-sim team for their excellent work and open-source contribution. We have integrated NCCL algorithms and added some new features.

Scenario

SimAI supports three major operation modes to meet different simulation requirements:

SimAI-Analytical offers fast simulation by abstracting network communication details using bus bandwidth (busbw) to estimate collective communication time. While it currently supports user-defined busbw, automatic busbw calculation feature is coming soon.

SimAI-Simulation provides full-stack simulation with fine-grained network communication modeling. It leverages NS3 or other network simulators (NS3 currently open-sourced) to achieve detailed simulation of all communication behaviors, aiming for high-fidelity reproduction of actual training environments.

SimAI-Physical (Beta) enables physical traffic generation for CPU RDMA cluster environments. This mode generates NCCL-like traffic patterns, allowing in-depth study of NIC behaviors during LLM training. It is currently in internal testing phase.

Citation

SimAI work has been accepted by NSDI'25 Spring, for more details, please refer to our paper below:

SimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large Language Model Training with Scalability and Precision.

[pdf] / [slides] / [video]

We encourage innovative research and extensions based on SimAI. Welcome to join our community group or reach out via email for discussion. We may provide technical support.

Quick Start

Here are some simple examples. SimAI full tutorials can be found here: SimAI@Tutorial, aicb@Tutorial, [SimCCL@Tutorial], [ns-3-alibabacloud@Tutorial]

Setup

You can follow the instructions below to quickly set up the environments and run SimAI.

From Source Code

The following code has been successfully tested on GCC/G++ 9.4.0, python 3.8.10 in Ubuntu 20.04.

You can use the official Ubuntu 20.04 image, and do not install ninja.

(For generation workloads, it's recommended to leverage NGC container images directly.)

# Clone the repository
$ git clone https://github.com/aliyun/SimAI.git
$ cd ./SimAI/

# Clone submodules
$ git submodule update --init --recursive
# Make sure use the newest commit
$ git submodule update --remote

# Compile SimAI-Analytical
$ ./scripts/build.sh -c analytical

# Compile SimAI-Simulation (ns3)
$ ./scripts/build.sh -c ns3

Use SimAI-Analytical

$ ./bin/SimAI_analytical -w example/workload_analytical.txt -g 9216 -g_p_s 8 -r test- -busbw example/busbw.yaml

For calculating bus bandwidth automatically, please try the following command:

$ ./bin/SimAI_analytical -w ./example/workload_analytical.txt -g 9216 -nv 360 -nic 48.5 -n_p_s 8 -g_p_s 8 -r example-

Use SimAI-Simulation

# Create network topo
$ python3 ./astra-sim-alibabacloud/inputs/topo/gen_Topo_Template.py -topo Spectrum-X -g 128 -gt A100 -bw 100Gbps -nvbw 2400Gbps

# Running
$ AS_SEND_LAT=3 AS_NVLS_ENABLE=1 ./bin/SimAI_simulator -t 16 -w ./example/microAllReduce.txt -n ./Spectrum-X_128g_8gps_100Gbps_A100 -c astra-sim-alibabacloud/inputs/config/SimAI.conf

Use Multi-requests Inference Simulation

For detailed information, please refer to the README file in the vidur-alibabacloud directory. This module leverages AICB to profile the computation time of inference workloads. Due to its reliance on specific hardware-accelerated libraries like DeepGEMM and FlashMLA, it is exclusively compatible with NVIDIA GPUs based on the Hopper (SM90) and Blackwell (SM100) architectures.

# Build from Dockerfile
docker build -t image:latest .
docker run --gpus all -it --rm image:latest

Note: Please add ENV FLASH_MLA_DISABLE_SM100=1 to Dockerfile if using Hopper GPUs.

To quickly validate all supported inference scenarios (Qwen3-Next-80B, DeepSeek-671B, Qwen3-MoE-235B), use the bundled 4-scenario test suite:

# Prerequisites: conda activate vidur
bash vidur-alibabacloud/examples/vidur-ali-scenarios/run_scenarios.sh --all
# Or run a single scenario:
bash vidur-alibabacloud/examples/vidur-ali-scenarios/run_scenarios.sh --scenario 1

Prerequisites: Requires conda activate vidur environment. See Environment Setup for details.

For detailed scenario configuration table and output file descriptions, see Vidur-AlibabaCloud README.

Acknowledgments

A huge thanks to the following people and organizations who have contributed to this project:

• TianHao Fu (Peking University) and TELOS-syslab
• Parth Parikh (KEYSIGHT)
• Sarah-Michelle Hammer & Ziyi Wang (TU-Berlin)
• Xinyue Li (BUPT)
• Tong Chen (Zhejiang University)
• Ming Wang (BUPT)
• Tao Jiang (Institute of Computing Technology, Chinese Academy of Sciences)

...and many other individual contributors from the community (See the Contributors to aliyun/SimAI).

We also thank Chenning Li (MIT CSAIL) who initiated the cooperation on integrating SimAI into M4, a new, innovative simulator.

This project still welcomes more contributions and suggestions.

Contributing

We welcome all contributions! Please read the following guides before getting started:

Contact us

Please email Gang Lu ([email protected]), Feiyang Xue ([email protected]) or Qingxu Li ([email protected]) if you have any questions.

Welcome to join the SimAI community chat groups, with the DingTalk group on the left and the WeChat group on the right.