中国做视频网站有哪些内容河南省百城建设提质工程官网网站

张

张建站

2026/4/27 7:26:32

10分钟阅读

中国做视频网站有哪些内容,河南省百城建设提质工程官网网站,四川建筑信息平台,建站平台与自己做网站ChatGLM-6B模型API开发指南#xff1a;FastAPI实现 1. 引言想给自己的应用加上智能对话能力#xff1f;ChatGLM-6B这个开源模型是个不错的选择#xff0c;它支持中英文对话#xff0c;效果不错#xff0c;而且对硬件要求相对友好。但直接调用模型代码不够灵活#xff…ChatGLM-6B模型API开发指南FastAPI实现1. 引言想给自己的应用加上智能对话能力ChatGLM-6B这个开源模型是个不错的选择它支持中英文对话效果不错而且对硬件要求相对友好。但直接调用模型代码不够灵活怎么让其他服务也能方便地使用呢这就是今天要解决的问题用FastAPI给ChatGLM-6B包装一层RESTful API。学完这篇教程你就能搭建一个属于自己的对话API服务让任何能发HTTP请求的应用都能享受AI对话能力。我们会从环境准备开始一步步带你完成API开发、并发处理、性能优化最后还会分享一些实际使用中的小技巧。不用担心复杂我用最直白的方式讲解保证你能跟上。2. 环境准备与模型加载2.1 安装依赖包首先确保你的环境有Python 3.8然后安装必要的依赖pip install fastapi uvicorn transformers torch这些包各自有不同作用FastAPI用来构建APIuvicorn是服务器transformers和torch用来加载和运行模型。2.2 准备ChatGLM-6B模型你可以从Hugging Face或ModelScope下载模型。如果网络不太好建议用ModelScopegit clone https://www.modelscope.cn/ZhipuAI/ChatGLM-6B.git chatglm-6b cd chatglm-6b git checkout v1.0.16下载完成后检查一下模型文件大概有12GB左右确保磁盘空间足够。2.3 基础模型加载代码先写一个简单的模型加载脚本测试一下是否能正常运行from transformers import AutoTokenizer, AutoModel model_path ./chatglm-6b # 你下载模型的路径 # 加载tokenizer和模型 tokenizer AutoTokenizer.from_pretrained(model_path, trust_remote_codeTrue) model AutoModel.from_pretrained(model_path, trust_remote_codeTrue).half().cuda() model model.eval() # 测试一下 response, history model.chat(tokenizer, 你好, history[]) print(response)如果能看到你好我是人工智能助手 ChatGLM-6B...这样的回复说明模型加载成功了。3. 基础FastAPI服务搭建3.1 创建最简单的API现在开始用FastAPI搭建服务from fastapi import FastAPI from pydantic import BaseModel app FastAPI(titleChatGLM-6B API, version1.0) class ChatRequest(BaseModel): prompt: str history: list [] max_length: int 2048 top_p: float 0.7 temperature: float 0.95 app.post(/chat) async def chat_endpoint(request: ChatRequest): # 这里会处理对话请求 return {response: 待实现, history: request.history}先定义了一个简单的API结构ChatRequest类描述了请求应该包含哪些参数。3.2 集成模型推理把模型加载和推理集成到API中from fastapi import FastAPI from pydantic import BaseModel from transformers import AutoTokenizer, AutoModel import torch app FastAPI(titleChatGLM-6B API, version1.0) # 全局变量保存加载的模型 tokenizer None model None app.on_event(startup) async def load_model(): global tokenizer, model model_path ./chatglm-6b print(正在加载模型...) tokenizer AutoTokenizer.from_pretrained(model_path, trust_remote_codeTrue) model AutoModel.from_pretrained(model_path, trust_remote_codeTrue).half().cuda() model model.eval() print(模型加载完成) class ChatRequest(BaseModel): prompt: str history: list [] max_length: int 2048 top_p: float 0.7 temperature: float 0.95 app.post(/chat) async def chat_endpoint(request: ChatRequest): response, updated_history model.chat( tokenizer, request.prompt, historyrequest.history, max_lengthrequest.max_length, top_prequest.top_p, temperaturerequest.temperature ) return { response: response, history: updated_history, status: 200 }这里用了app.on_event(startup)装饰器确保服务启动时自动加载模型而不是每次请求都加载。3.3 启动服务创建一个启动脚本# run.py import uvicorn if __name__ __main__: uvicorn.run(main:app, host0.0.0.0, port8000, reloadFalse)运行这个脚本就能启动服务了python run.py4. 处理并发请求4.1 理解FastAPI的并发模型FastAPI基于ASGI天生支持异步但模型推理是计算密集型任务会阻塞事件循环。我们需要用一些技巧来处理。4.2 使用线程池处理阻塞操作from concurrent.futures import ThreadPoolExecutor import asyncio # 创建线程池 thread_pool ThreadPoolExecutor(max_workers4) def run_model_chat(prompt, history, max_length, top_p, temperature): 在线程中运行模型推理 response, updated_history model.chat( tokenizer, prompt, historyhistory, max_lengthmax_length, top_ptop_p, temperaturetemperature ) return response, updated_history app.post(/chat) async def chat_endpoint(request: ChatRequest): # 将模型推理放到线程池中执行避免阻塞主事件循环 loop asyncio.get_event_loop() response, updated_history await loop.run_in_executor( thread_pool, run_model_chat, request.prompt, request.history, request.max_length, request.top_p, request.temperature ) return { response: response, history: updated_history, status: 200 }这样处理之后API就能同时处理多个请求了不会因为一个请求在推理就让其他请求干等着。4.3 限制并发数为了避免内存溢出可以限制同时处理的请求数from fastapi import HTTPException from concurrent.futures import ThreadPoolExecutor, TimeoutError import asyncio # 使用有界队列的线程池 thread_pool ThreadPoolExecutor( max_workers2, # 根据你的GPU内存调整 thread_name_prefixmodel_worker ) app.post(/chat) async def chat_endpoint(request: ChatRequest): try: loop asyncio.get_event_loop() # 设置超时时间避免请求卡住 response, updated_history await asyncio.wait_for( loop.run_in_executor( thread_pool, run_model_chat, request.prompt, request.history, request.max_length, request.top_p, request.temperature ), timeout30.0 # 30秒超时 ) return { response: response, history: updated_history, status: 200 } except asyncio.TimeoutError: raise HTTPException(status_code408, detail请求超时) except Exception as e: raise HTTPException(status_code500, detailf内部错误: {str(e)})5. 性能优化技巧5.1 模型量化减少内存占用如果显存紧张可以使用模型量化app.on_event(startup) async def load_model(): global tokenizer, model model_path ./chatglm-6b print(正在加载量化模型...) tokenizer AutoTokenizer.from_pretrained(model_path, trust_remote_codeTrue) model AutoModel.from_pretrained(model_path, trust_remote_codeTrue).quantize(8).half().cuda() model model.eval() print(8bit量化模型加载完成)8bit量化后显存占用从13GB降到约10GB4bit量化可以降到6GB但效果会稍微差一点。5.2 使用更高效的推理配置def run_model_chat(prompt, history, max_length, top_p, temperature): # 设置torch推理模式提升效率 with torch.inference_mode(): response, updated_history model.chat( tokenizer, prompt, historyhistory, max_lengthmin(max_length, 2048), # 限制最大长度 top_ptop_p, temperaturetemperature ) return response, updated_historytorch.inference_mode()比torch.no_grad()更高效专门为推理场景优化。5.3 批处理请求如果需要处理大量相似请求可以考虑实现批处理class BatchChatRequest(BaseModel): prompts: list max_length: int 2048 top_p: float 0.7 temperature: float 0.95 app.post(/batch_chat) async def batch_chat_endpoint(request: BatchChatRequest): results [] for prompt in request.prompts: response, history await run_single_chat( prompt, [], request.max_length, request.top_p, request.temperature ) results.append({prompt: prompt, response: response}) return {results: results}6. 完整API示例6.1 完整的main.pyfrom fastapi import FastAPI, HTTPException from pydantic import BaseModel from transformers import AutoTokenizer, AutoModel from concurrent.futures import ThreadPoolExecutor import asyncio import torch import time app FastAPI( titleChatGLM-6B API, description基于FastAPI的ChatGLM-6B对话API, version1.0.0 ) # 全局变量 tokenizer None model None thread_pool ThreadPoolExecutor(max_workers2) class ChatRequest(BaseModel): prompt: str history: list [] max_length: int 2048 top_p: float 0.7 temperature: float 0.95 class HealthResponse(BaseModel): status: str model_loaded: bool timestamp: str def run_model_chat(prompt, history, max_length, top_p, temperature): 在推理模式下运行模型 with torch.inference_mode(): response, updated_history model.chat( tokenizer, prompt, historyhistory, max_lengthmin(max_length, 2048), top_ptop_p, temperaturetemperature ) return response, updated_history app.on_event(startup) async def load_model(): global tokenizer, model try: model_path ./chatglm-6b print(f{time.strftime(%Y-%m-%d %H:%M:%S)} 开始加载模型...) tokenizer AutoTokenizer.from_pretrained(model_path, trust_remote_codeTrue) model AutoModel.from_pretrained(model_path, trust_remote_codeTrue).half().cuda() model model.eval() print(f{time.strftime(%Y-%m-%d %H:%M:%S)} 模型加载完成) except Exception as e: print(f模型加载失败: {str(e)}) raise app.on_event(shutdown) async def shutdown_event(): thread_pool.shutdown() print(线程池已关闭) app.get(/health, response_modelHealthResponse) async def health_check(): return { status: healthy, model_loaded: model is not None, timestamp: time.strftime(%Y-%m-%d %H:%M:%S) } app.post(/chat) async def chat_endpoint(request: ChatRequest): if model is None or tokenizer is None: raise HTTPException(status_code503, detail模型未加载完成) try: loop asyncio.get_event_loop() response, updated_history await asyncio.wait_for( loop.run_in_executor( thread_pool, run_model_chat, request.prompt, request.history, request.max_length, request.top_p, request.temperature ), timeout30.0 ) return { response: response, history: updated_history, status: 200, time: time.strftime(%Y-%m-%d %H:%M:%S) } except asyncio.TimeoutError: raise HTTPException(status_code408, detail请求超时) except Exception as e: raise HTTPException(status_code500, detailf内部错误: {str(e)}) if __name__ __main__: import uvicorn uvicorn.run(app, host0.0.0.0, port8000)6.2 测试API启动服务后可以用curl测试curl -X POST http://localhost:8000/chat \ -H Content-Type: application/json \ -d {prompt: 你好请介绍一下你自己, history: []}或者用Python代码测试import requests import json api_url http://localhost:8000/chat data { prompt: 写一首关于春天的诗, history: [], max_length: 500, top_p: 0.8, temperature: 0.9 } response requests.post(api_url, jsondata) if response.status_code 200: result response.json() print(回复:, result[response]) else: print(请求失败:, response.text)7. 实际使用建议7.1 部署注意事项GPU内存管理根据你的GPU显存调整并发数8GB显存建议max_workers116GB可以设2-3使用反向代理生产环境建议用nginx做反向代理处理SSL和负载均衡监控和日志添加日志记录监控API使用情况和模型性能7.2 常见问题处理模型响应慢可以尝试减小max_length或者使用量化模型显存不足减少并发数使用4bit量化或者尝试CPU部署响应质量不高调整temperature和top_p参数temperature越高越有创意top_p越高越专注7.3 进一步优化方向模型缓存对常见问题缓存答案减少模型调用流式响应实现逐词输出改善用户体验异步日志使用异步方式记录日志避免阻塞健康检查添加更详细的健康检查接口监控GPU内存使用情况8. 总结整套方案搭下来其实核心思路很简单用FastAPI提供HTTP接口用线程池处理模型推理的阻塞操作再加上一些性能优化和错误处理。实际用起来效果还不错响应速度基本能满足要求而且部署起来也挺方便的。最重要的是现在任何能发HTTP请求的应用都能用上ChatGLM-6B的能力了无论是Web应用、移动应用还是其他服务。如果你显存比较紧张记得用量化模型如果请求量比较大可以考虑加个缓存层。还有什么问题的话欢迎交流讨论。获取更多AI镜像想探索更多AI镜像和应用场景访问 CSDN星图镜像广场提供丰富的预置镜像覆盖大模型推理、图像生成、视频生成、模型微调等多个领域支持一键部署。