snaily大佬原帖：接力v佬，aipro流式支持 - #17，来自 snaily
感谢大佬们的分享，我来水水
因为CF不好申请，我就试了下直接走代理，发现竟然也ok,所以就抄了这个过来。
前提条件：
1.需要一个代理工具(梯子)
2.需要安装python以及python库
pip install fastapi
pip install requests
pip install uvicorn
食用方法：
step1：创建一个aipro.py文件，并把下面代码复制到文件中。
ps:注意 PROXIES_URL需要替换成你本地的梯子
from fastapi import FastAPI,Request
from fastapi import FastAPI, Request, Response, HTTPException, status
import requests
TG_API_URL = 'https://multillm.ai-pro.org'
PROXIES_URL = {"http":"http://127.0.0.1:7890","https":"http://127.0.0.1:7890"}
app = FastAPI()
@app.get("/")
async def read_root():
return {"Hello": "World"}
@app.post("/v1/chat/completions")
async def chat_completions(request: Request):
global TG_API_URL,PROXIES_URL
burp0_url = TG_API_URL +"/api/openai-completion"
burp0_headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36", "Content-Type": "application/json"}
burp0_json=await request.json()
burp0_json["stream"]=True
ret=requests.post(burp0_url, headers=burp0_headers, json=burp0_json,proxies=PROXIES_URL)
return ret.json()
@app.api_route("/{_:path}", methods=["POST", "GET", "PUT", "DELETE", "OPTIONS", "HEAD", "PATCH", "TRACE"])
async def catch_all(_):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not Found")
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=3699)
step2:直接命令行运行该文件(运行的命令可能如下)
python aipro.py
如果你得到如下，则说明是启动成功，如果是报错，就具体再看吧。
PS D:\project\AIfree36\MY36> python .\aipro.py
INFO: Started server process [43692]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:3699 (Press CTRL+C to quit)
step3:测试代码
把下面代码添加到一个新的文件如test.py,然后保存，并运行。
import requests
import json
url = "http://127.0.0.1:3699/v1/chat/completions"
payload = json.dumps({
"model": "gpt-4-turbo",
"messages": [
{
"role": "user",
"content": "讲个笑话"
}
],
"stream": True
})
headers = {
'Authorization': 'Bearer sk-666666',
'User-Agent': 'Apifox/1.0.0 (https://apifox.com)'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
在命令行后运行应该如下;
PS W:\fastapi_code\Taipro> python test.py
{"id":"chatcmpl-9HLOVhtGlaNcuvTgUOvTXX7rm7EsM","object":"chat.completion","created":1713920843,"model":"gpt-4-turbo-2024-04-09","choices":[{"index":0,"message":{"role":"assistant","content":"有个人辞职了，领导非常惊讶，问他为什么。他说：“我在公司网站上看到‘你的工资可能比你想象的要多’，算了算之后，觉得自己赚太多了，所以决定不干了！”"},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":82,"total_tokens":94},"system_fingerprint":"fp_76f018034d"}
这样就说明，本地的就部署好了。
base url:http://127.0.0.1:3699
key:随便填，或者不填，反正都没有校验的。
，我之前还跟老黄牛一样复制手动修改规整header信息
我代码都是东拼西凑的
请问一下为什么我test跑通了但是在nextchat里面这样调用就出现Failed to fetch 
