成都网站开发公司南京网站建设哪家专业

张

张建站

2026/4/6 19:21:32

10分钟阅读

成都网站开发公司,南京网站建设哪家专业,百度识图搜索网页版,竭诚网络网站建设开发引言模型量化是将浮点模型转换为低精度整数模型的技术#xff0c;可以显著降低模型大小、提升推理速度并减少功耗#xff0c;是模型部署的重要优化手段。华为CANN平台提供了完善的量化工具链#xff0c;支持训练后量化和量化感知训练#xff0c;能够在保持模型精度的同时…引言模型量化是将浮点模型转换为低精度整数模型的技术可以显著降低模型大小、提升推理速度并减少功耗是模型部署的重要优化手段。华为CANN平台提供了完善的量化工具链支持训练后量化和量化感知训练能够在保持模型精度的同时实现高效推理。本文将详细介绍如何使用CANN进行模型量化帮助开发者在边缘设备和云端实现高效部署。相关链接CANN组织ops-nn仓库一、CANN量化技术原理1.1 量化基础量化是将连续的浮点数映射到离散的整数的过程对称量化零点为0范围对称非对称量化零点可调范围不对称逐层量化每层使用不同的量化参数逐通道量化每个通道使用不同的量化参数1.2 CANN量化方案CANN支持多种量化方案训练后量化PTQ直接量化已训练模型无需重新训练量化感知训练QAT在训练过程中模拟量化精度更高混合精度量化关键层保持FP16其他层使用INT8二、CANN训练后量化PTQ2.1 基础PTQ示例使用CANN对ResNet模型进行训练后量化importtorchimporttorch_npuimporttorchvision.modelsasmodelsfromtorch_npu.quantizationimportquantize_dynamic,prepare,convertdefcann_ptq_basic():CANN基础训练后量化# 加载预训练模型modelmodels.resnet50(pretrainedTrue)model.eval()# 准备校准数据calibration_datatorch.randn(100,3,224,224)# 使用CANN动态量化quantized_modelquantize_dynamic(model,{torch.nn.Linear,torch.nn.Conv2d},dtypetorch.qint8)# 保存量化模型torch.save(quantized_model.state_dict(),resnet50_quantized.pth)# 性能对比importtime# FP32推理model_fp32model.to(npu:0)input_fp32torch.randn(1,3,224,224).to(npu:0)torch_npu.synchronize()starttime.time()for_inrange(100):_model_fp32(input_fp32)torch_npu.synchronize()time_fp32(time.time()-start)/100# INT8推理model_int8quantized_model.to(npu:0)input_int8torch.randn(1,3,224,224).to(npu:0)torch_npu.synchronize()starttime.time()for_inrange(100):_model_int8(input_int8)torch_npu.synchronize()time_int8(time.time()-start)/100print(fCANN量化性能对比:)print(fFP32推理:{time_fp32*1000:.2f}ms)print(fINT8推理:{time_int8*1000:.2f}ms)print(f加速比:{time_fp32/time_int8:.2f}x)# 模型大小对比importos torch.save(model.state_dict(),model_fp32.pth)torch.save(quantized_model.state_dict(),model_int8.pth)size_fp32os.path.getsize(model_fp32.pth)/(1024**2)size_int8os.path.getsize(model_int8.pth)/(1024**2)print(f\nCANN模型大小对比:)print(fFP32模型:{size_fp32:.2f}MB)print(fINT8模型:{size_int8:.2f}MB)print(f压缩比:{size_fp32/size_int8:.2f}x)if__name____main__:cann_ptq_basic()2.2 使用校准数据集的PTQ使用真实数据进行CANN量化校准importtorchimporttorch_npufromtorch_npu.quantizationimportQuantStub,DeQuantStub,prepare,convertimporttorchvision.transformsastransformsfromtorch.utils.dataimportDataLoaderimporttorchvision.datasetsasdatasetsclassCANNQuantizedModel(nn.Module):CANN量化模型包装器def__init__(self,model):super().__init__()self.quantQuantStub()self.modelmodel self.dequantDeQuantStub()defforward(self,x):xself.quant(x)xself.model(x)xself.dequant(x)returnxdefcalibrate_cann_model(model,calibration_loader):使用CANN校准量化模型model.eval()# 准备量化model.qconfigtorch.quantization.get_default_qconfig(fbgemm)model_preparedprepare(model,inplaceFalse)# 使用校准数据print(CANN量化校准中...)withtorch.no_grad():fori,(images,_)inenumerate(calibration_loader):ifi100:# 使用100个batch进行校准breakimagesimages.to(npu:0)_model_prepared(images)if(i1)%100:print(f校准进度:{i1}/100)# 转换为量化模型model_quantizedconvert(model_prepared,inplaceFalse)print(CANN量化校准完成!)returnmodel_quantizeddefcann_ptq_with_calibration():带校准的CANN训练后量化# 加载模型modelmodels.resnet50(pretrainedTrue)modelCANNQuantizedModel(model)modelmodel.to(npu:0)# 准备校准数据transformtransforms.Compose([transforms.Resize(256),transforms.CenterCrop(224),transforms.ToTensor(),transforms.Normalize(mean[0.485,0.456,0.406],std[0.229,0.224,0.225])])calibration_datasetdatasets.ImageFolder(root./imagenet/val,transformtransform)calibration_loaderDataLoader(calibration_dataset,batch_size32,shuffleFalse,num_workers4)# 执行CANN量化校准quantized_modelcalibrate_cann_model(model,calibration_loader)# 保存量化模型torch.save(quantized_model.state_dict(),resnet50_calibrated_int8.pth)# 评估精度evaluate_accuracy(quantized_model,calibration_loader)defevaluate_accuracy(model,data_loader):评估CANN量化模型精度model.eval()correct0total0withtorch.no_grad():forimages,labelsindata_loader:imagesimages.to(npu:0)labelslabels.to(npu:0)outputsmodel(images)_,predictedtorch.max(outputs.data,1)totallabels.size(0)correct(predictedlabels).sum().item()accuracy100*correct/totalprint(fCANN量化模型精度:{accuracy:.2f}%)returnaccuracyif__name____main__:cann_ptq_with_calibration()三、CANN量化感知训练QAT3.1 QAT基础实现在训练过程中使用CANN模拟量化importtorchimporttorch.nnasnnimporttorch_npufromtorch_npu.quantizationimportQuantStub,DeQuantStub,prepare_qat,convertclassCANNQATModel(nn.Module):CANN量化感知训练模型def__init__(self,num_classes1000):super().__init__()self.quantQuantStub()self.conv1nn.Conv2d(3,64,7,2,3)self.bn1nn.BatchNorm2d(64)self.relunn.ReLU()self.maxpoolnn.MaxPool2d(3,2,1)self.layer1self._make_layer(64,64,3)self.layer2self._make_layer(64,128,4,stride2)self.layer3self._make_layer(128,256,6,stride2)self.layer4self._make_layer(256,512,3,stride2)self.avgpoolnn.AdaptiveAvgPool2d((1,1))self.fcnn.Linear(512,num_classes)self.dequantDeQuantStub()def_make_layer(self,in_channels,out_channels,num_blocks,stride1):layers[]layers.append(nn.Conv2d(in_channels,out_channels,3,stride,1))layers.append(nn.BatchNorm2d(out_channels))layers.append(nn.ReLU())for_inrange(num_blocks-1):layers.append(nn.Conv2d(out_channels,out_channels,3,1,1))layers.append(nn.BatchNorm2d(out_channels))layers.append(nn.ReLU())returnnn.Sequential(*layers)defforward(self,x):xself.quant(x)xself.conv1(x)xself.bn1(x)xself.relu(x)xself.maxpool(x)xself.layer1(x)xself.layer2(x)xself.layer3(x)xself.layer4(x)xself.avgpool(x)xtorch.flatten(x,1)xself.fc(x)xself.dequant(x)returnxdeftrain_cann_qat():CANN量化感知训练# 创建模型modelCANNQATModel(num_classes1000)modelmodel.to(npu:0)# 配置CANN量化model.qconfigtorch.quantization.get_default_qat_qconfig(fbgemm)model_preparedprepare_qat(model,inplaceFalse)# 准备数据train_loaderprepare_dataloader()# 定义优化器criterionnn.CrossEntropyLoss()optimizertorch.optim.SGD(model_prepared.parameters(),lr0.01,momentum0.9,weight_decay1e-4)# QAT训练循环model_prepared.train()forepochinrange(10):running_loss0.0fori,(inputs,labels)inenumerate(train_loader):inputsinputs.to(npu:0)labelslabels.to(npu:0)optimizer.zero_grad()# 前向传播CANN模拟量化outputsmodel_prepared(inputs)losscriterion(outputs,labels)# 反向传播loss.backward()optimizer.step()running_lossloss.item()ifi%10099:print(fEpoch [{epoch1}/10], Step [{i1}], fLoss:{running_loss/100:.4f})running_loss0.0# 转换为真正的量化模型model_prepared.eval()model_quantizedconvert(model_prepared,inplaceFalse)# 保存量化模型torch.save(model_quantized.state_dict(),model_qat_int8.pth)print(CANN QAT训练完成!)returnmodel_quantizedif__name____main__:modeltrain_cann_qat()3.2 逐层量化策略对不同层使用不同的CANN量化策略defconfigure_layer_wise_quantization(model):配置CANN逐层量化策略# 对不同层设置不同的量化配置forname,moduleinmodel.named_modules():ifisinstance(module,nn.Conv2d):iflayer1innameorlayer2inname:# 前面的层使用8位量化module.qconfigtorch.quantization.get_default_qconfig(fbgemm)else:# 后面的层使用更高精度module.qconfigtorch.quantization.QConfig(activationtorch.quantization.MinMaxObserver.with_args(dtypetorch.quint8,qschemetorch.per_tensor_affine),weighttorch.quantization.MinMaxObserver.with_args(dtypetorch.qint8,qschemetorch.per_tensor_symmetric))elifisinstance(module,nn.Linear):# 全连接层保持高精度module.qconfigNone# 不量化returnmodel四、CANN混合精度量化4.1 关键层保持FP16对关键层保持FP16精度defcann_mixed_precision_quantization(model):CANN混合精度量化# 标记需要保持FP16的层sensitive_layers[fc,layer4]forname,moduleinmodel.named_modules():# 检查是否是敏感层is_sensitiveany(layerinnameforlayerinsensitive_layers)ifisinstance(module,(nn.Conv2d,nn.Linear)):ifis_sensitive:# 敏感层不量化保持FP16module.qconfigNoneprint(f保持FP16:{name})else:# 其他层量化为INT8module.qconfigtorch.quantization.get_default_qconfig(fbgemm)print(f量化为INT8:{name})# 准备和转换model_preparedprepare_qat(model)# ... 训练 ...model_quantizedconvert(model_prepared)returnmodel_quantized4.2 动态量化范围调整根据激活值分布动态调整CANN量化范围classCANNDynamicQuantization:CANN动态量化def__init__(self,model):self.modelmodel self.activation_stats{}defcollect_stats(self,data_loader):收集激活值统计信息self.model.eval()# 注册钩子收集激活值hooks[]forname,moduleinself.model.named_modules():ifisinstance(module,(nn.Conv2d,nn.Linear)):hookmodule.register_forward_hook(self._make_hook(name))hooks.append(hook)# 运行数据收集withtorch.no_grad():forinputs,_indata_loader:inputsinputs.to(npu:0)_self.model(inputs)# 移除钩子forhookinhooks:hook.remove()def_make_hook(self,name):创建钩子函数defhook(module,input,output):ifnamenotinself.activation_stats:self.activation_stats[name]{min:float(inf),max:float(-inf)}self.activation_stats[name][min]min(self.activation_stats[name][min],output.min().item())self.activation_stats[name][max]max(self.activation_stats[name][max],output.max().item())returnhookdefapply_quantization(self):应用CANN动态量化forname,moduleinself.model.named_modules():ifnameinself.activation_stats:statsself.activation_stats[name]# 根据统计信息设置量化参数qminstats[min]qmaxstats[max]# 设置自定义量化配置module.qconfigtorch.quantization.QConfig(activationtorch.quantization.MinMaxObserver.with_args(quant_minint(qmin),quant_maxint(qmax),dtypetorch.quint8),weighttorch.quantization.default_weight_observer)print(fCANN量化{name}: range[{qmin:.2f},{qmax:.2f}])returnself.model相关链接CANN组织ops-nn仓库通过本文的实战案例我们详细介绍了如何使用CANN进行模型量化。CANN提供的训练后量化和量化感知训练技术可以在保持模型精度的同时显著提升推理速度、降低模型大小和功耗。掌握这些技术开发者可以高效地将模型部署到资源受限的边缘设备上。