Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记-北京尧图网络

这里写自定义目录标题Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记第一步 mindie镜像下载安装第二步第三步第四步第五步第六步第七步启动服务其他容器简单推理测化命令参考连接Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记第一步 mindie镜像下载安装镜像地址https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48fdocker pull --platformarm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0b2-300I-Duo-py311-openeuler24.03-lts第二步dockerrun-it-d\--shm-size64g\--namemindie_310\--device/dev/davinci0\--device/dev/davinci1\--device/dev/davinci2\--device/dev/davinci3\--device/dev/davinci4\--device/dev/davinci5\--device/dev/davinci_manager\--device/dev/hisi_hdc\-v/usr/local/Ascend/driver:/usr/local/Ascend/driver:ro\-v/usr/local/dcmi:/usr/local/dcmi:ro\-v/usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro\-v/usr/local/Ascend/firmware:/usr/local/Ascend/firmware:ro\-v/usr/local/sbin:/usr/local/sbin:ro\-v/data/alg:/root/alg\-p29511:29511\-p29512:29512\-p29513:29513\-p1025:1025\-p1026:1026\-p1027:1027\-p1121:1121\swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0-300I-Duo-py311-openeuler24.03-lts\bash第三步dockerexec-itmindie_310 /bin/bashcd/usr/local/Ascend/atb-models/requirements/models pipinstall-rrequirements_qwen3vl.txt第四步cd/usr/local/Ascend/mindie/latestchmod750mindie-servicechmod-R550mindie-service/binchmod550mindie-service/libchmod440mindie-service/lib/*chmod550mindie-service/lib/grpcchmod440mindie-service/lib/grpc/*chmod-R550mindie-service/includechmod-R550mindie-service/scriptschmod750mindie-service/logschmod750mindie-service/confchmod640mindie-service/conf/config.jsonchmod700mindie-service/securitychmod-R700mindie-service/security/*第五步# 配置CANN环境默认安装在/usr/local目录下source/usr/local/Ascend/ascend-toolkit/set_env.sh# 配置加速库环境source/usr/local/Ascend/nnal/atb/set_env.sh# 配置模型仓环境变量source/usr/local/Ascend/atb-models/set_env.sh# MindIEsource/usr/local/Ascend/mindie/latest/mindie-llm/set_env.shsource/usr/local/Ascend/mindie/latest/mindie-service/set_env.sh# 开启MindIE日志打印exportMINDIE_LOG_TO_STDOUTtrue第六步cd/usr/local/Ascend/mindie/latest/mindie-servicecdmindie-service/confvimconfig.json 修改httpsEnabled:false修改modelName:qwen2-7b,modelWeightPath:/home/weight,其余参数视情况修改注意maxSeqLen:1280000,#2560倍数maxInputTokenLen:1024000,1024倍数maxPrefillTokens:1024000,需大于maxInputTokenLen第七步启动服务nohup./bin/mindieservice_daemonoutput.log21或 ./bin/mindieservice_daemon 回显 Daemon start success!表明启动成功测试importtimeimportrequestsimportjsondeftest_generation_speed(prompt输出工地安全准则,max_tokens1204000,runs3):urlhttp://127.0.0.1:1025/v1/chat/completionstotal_tokens0total_time0foriinrange(runs):print(f\n--- Run{i1}---)start_timetime.time()generated_tokens0responserequests.post(url,headers{Content-Type:application/json},json{model:qwen3-vl-qwen3-vl-a3b,messages:[{role:user,content:prompt}],max_tokens:max_tokens,# 增加生成长度temperature:0.7,stream:True},timeout3600,# 增加超时时间streamTrue)ifresponse.status_code200:print(生成内容: ,end,flushTrue)forlineinresponse.iter_lines():ifline:lineline.decode(utf-8)ifline.startswith(data: ):ifline.strip()data: [DONE]:breaktry:chunkjson.loads(line[6:])contentchunk[choices][0][delta].get(content,)ifcontent:print(content,end,flushTrue)generated_tokenslen(content.split())except:passprint()# 换行else:print(fRun{i1}: Error{response.status_code}-{response.text})continuedurationtime.time()-start_time speedgenerated_tokens/durationifduration0else0print(fRun{i1}: ~{generated_tokens}tokens in{duration:.2f}s →{speed:.2f}tokens/s)total_tokensgenerated_tokens total_timedurationiftotal_time0:print(f\nAverage:{total_tokens/total_time:.2f}tokens/s over{runs}runs)if__name____main__:test_generation_speed()输出其他容器简单推理测化命令torchrun--standalone--nnodes1--nproc_per_node1\examples/run_pa.py\--model_path/root/alg/qwen3-vl-8b\--input_texts什么是深度学习深度学习和机器学习有什么区别请举例说明深度学习的应用场景。\--max_output_length512\--max_batch_size3\--trust_remote_code参考连接https://www.hiascend.com/document/detail/zh/mindie/300/envdeployment/instg/docs/zh/user_guide/install/source/image_usage_guide.mdhttps://gitcode.com/Ascend/MindIE-LLM/blob/v3.0.0/docs/zh/user_guide/quick_start/quick_start.md

Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记

相关推荐

IIM-20670与STM32L162ZE的高精度运动跟踪方案

MouseTester终极指南：5分钟免费检测你的鼠标真实性能

工业设备产值与能耗监测管理系统方案

更多资讯

巴彦淖尔市高压电工证内部消息：学历不高也能拿证，全流程手把手教

2020的电工证全国通用怎么考？电工证备考全攻略

富阳区电工证培训机构通过率揭秘：外省证书怎么转过来

钳工证和电工证哪个好网上查询实操考试怕挂科怎么办

高压电工证年审报名的良心建议，怕考不过白交培训费？

苏州电工证号查询系统怎么考？网上查询教你一步步搞定

今日更新

电工证考试及格线你知道多少？值不值得考

南通电工证发证单位考前押题怎么搞？工地太忙根本没时间复习考试

黄陵县高低压电工证怎么考？上岗必备全流程避坑指南

本周精选

安监局电工证分数查询怎么查才能知道含金量高不高

新疆电工作业干活累吗良心建议助你避开考不过的坑

中牟县电工证怎么报名外省证能转吗

本月热门

面向复杂光照与角度的宠物面部识别鲁棒性增强策略

基于DRV8818与STM32的双极步进电机驱动方案实战解析

DRV8818PWPR+PIC18F45K80双极步进电机控制方案实战

读完文章，想聊聊您的网站？