基于vLLM-Ascend的Qwen3.5-122B模型Atlas 800I A3单机混部部署实践-北京尧图网络

作者昇腾实战派知识地图https://blog.csdn.net/Lumos_Lovegood/article/details/161601003背景概述本文档将介绍基于vLLM-Ascend的Qwen3.5-122B模型在Atlas 800I A3上的单机混部部署实践包括支持的特性、特性配置、环境信息以及性能测试典型case。基本信息软件版本设备信息组网形态总卡数数据格式0.18.0NPU: Atlas 800I A3-560T, HBM 128GCPU: Kunpeng 920 (80核-2900MHz)内存: 32根64G5200MHzOS: OpenEuler 22.03 LTS-SP4-2W4A8C16服务化配置低时延exportASCEND_RT_VISIBLE_DEVICES0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportHCCL_IF_IPXXXXexportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE1024exportOMP_NUM_THREADS1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000exportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE1exportVLLM_ASCEND_ENABLE_FUSED_MC21vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-nameqwen3.5\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--max-model-len5000\--max-num-batched-tokens8192\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config{cudagraph_capture_sizes:[1,6,12,18,24,30,36,42,48,54,72,78,84,90,96,102,108,144,192], cudagraph_mode:FULL_DECODE_ONLY}\--speculative_config{method: qwen3_5_mtp, num_speculative_tokens: 5}\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config{enable_cpu_binding: true, enable_shared_expert_dp: true}典型测试用例平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)20482048MLADP1TP4500009624035001500MLADP1TP48000064160163841024MLADP1TP4184320328032768512MLADP1TP43430401640655361024MLADP1TP467584016401310721024MLADP1TP41331200410高吞吐exportASCEND_RT_VISIBLE_DEVICES0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportHCCL_IF_IPXXXexportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE1024exportOMP_NUM_THREADS1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000exportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE1exportVLLM_ASCEND_ENABLE_FUSED_MC21exportVLLM_ASCEND_ENABLE_FLASHCOMM11vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-nameqwen3.5\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--enable-expert-parallel\--max-model-len8000\--max-num-batched-tokens4096\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config{cudagraph_capture_sizes:[1,4,8,12,16,24,32,48,56,64,72,84,96,108,112,128,160,172,196,200,212,232,256,160,172,196,200,212,232,256,272,288,312,328,344,360,384,400,416,432,448,480,512], cudagraph_mode:FULL_DECODE_ONLY}\--speculative_config{method: qwen3_5_mtp, num_speculative_tokens: 3, enforce_eager: true}\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config{enable_cpu_binding: true, enable_shared_expert_dp: true}典型测试用例平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)20482048MLADP1TP250000600150035001500MLADP1TP280000320800163841024MLADP1TP21843209624032768512MLADP1TP23430403690655361024MLADP1TP267584032801310721024MLADP1TP213312001640测试命令参考aisbench官方测试指南。aisbench测试命令vllm-ascend社区官网特别声明以上配置均未开启Prefix Cache若实际生产环境需要使用该特性参考vLLM-Ascend社区参数指南开启–enable-prefix-caching

基于vLLM-Ascend的Qwen3.5-122B模型Atlas 800I A3单机混部部署实践

相关推荐

AI助手系统开发：角色定义、架构设计与Python实现

AI期刊论文工具平台哪家好？2026年期刊写作工具测评

竞争对手外链分析与复制策略：借同行的死链每天白嫖3个链接

更多资讯

电工证有几种在哪考上岗必备全攻略

武汉电工证年审吗保姆级教程：避开中介陷阱，自己搞定年审流程

工地太忙根本没时间复习考试？重复考低压电工证网上查询全攻略

肇庆电工证在哪里低学历也能拿证多久拿证

盐城低压电工证报名全流程指南：别被中介坑了，电子证书怎么拿

武汉电工证中天教育保姆级教程：学历不高也能顺利报考

今日更新

高压电工证哪里可以挂多久拿证全攻略：实操考试心里没底怕挂科别慌

电工证的理论题目在哪考，外省证怎么转过来

工地太忙根本没时间复习考试，职业学校的电工证值不值得考？

本周精选

安监局电工证分数查询怎么查才能知道含金量高不高

新疆电工作业干活累吗良心建议助你避开考不过的坑

中牟县电工证怎么报名外省证能转吗

本月热门

面向复杂光照与角度的宠物面部识别鲁棒性增强策略

基于DRV8818与STM32的双极步进电机驱动方案实战解析

DRV8818PWPR+PIC18F45K80双极步进电机控制方案实战

读完文章，想聊聊您的网站？