我没看太懂，是不是意思是说，请求AI的时候，直接走cloudflare就可以了？
During Developer Week in April 2024, we announced General Availability of Workers AI, and today, we are excited to announce that AI Gateway is Generally Available as well. Since its launch to beta in September 2023 during Birthday Week, we’ve proxied over 500 million requests and are now prepared for you to use it in production.
在 2024 年 4 月的 “开发者周”（Developer Week）期间，我们宣布 Workers AI 全面可用，今天，我们很高兴地宣布 AI Gateway 也已全面可用。自 2023 年 9 月 "生日周 "期间推出测试版以来，我们已经代理了超过 5 亿个请求，现在您可以在生产中使用它了。
AI Gateway is an AI ops platform that offers a unified interface for managing and scaling your generative AI workloads. At its core, it acts as a proxy between your service and your inference provider(s), regardless of where your model runs. With a single line of code, you can unlock a set of powerful features focused on performance, security, reliability, and observability – think of it as your control plane for your AI ops. And this is just the beginning – we have a roadmap full of exciting features planned for the near future, making AI Gateway the tool for any organization looking to get more out of their AI workloads.
AI Gateway 是一个人工智能运营平台，为管理和扩展生成式人工智能工作负载提供了一个统一的界面。其核心是充当您的服务与推理提供商之间的代理，无论您的模型在哪里运行。只需一行代码，您就可以解锁一系列以性能、安全性、可靠性和可观察性为重点的强大功能，将其视为人工智能操作的控制平面。而这仅仅是个开始–在不久的将来，我们的路线图上将会有更多令人兴奋的功能，这将使 AI Gateway 成为任何希望从其人工智能工作负载中获得更多收益的组织所需的工具。
Why add a proxy and why Cloudflare?
为什么要添加代理，为什么是 Cloudflare？
The AI space moves fast, and it seems like every day there is a new model, provider, or framework. Given this high rate of change, it’s hard to keep track, especially if you’re using more than one model or provider. And that’s one of the driving factors behind launching AI Gateway – we want to provide you with a single consistent control plane for all your models and tools, even if they change tomorrow, and then again the day after that.
人工智能领域发展迅速，似乎每天都有新的模型、提供商或框架出现。鉴于变化如此之快，很难跟踪，尤其是当您使用不止一种模型或提供商时。这也是我们推出 AI Gateway 背后的驱动因素之一–我们希望为您的所有模型和工具提供一个统一的控制平面，即使它们明天发生变化，后天又发生变化。
We’ve talked to a lot of developers and organizations building AI applications, and one thing is clear: they want more observability, control, and tooling around their AI ops. This is something many of the AI providers are lacking as they are deeply focused on model development and less so on platform features.
我们与许多开发人员和构建人工智能应用的组织进行过交谈，有一点很清楚：他们希望在人工智能运营方面获得更多的可观察性、控制性和工具性。这是许多人工智能提供商所缺乏的，因为他们都专注于模型开发，而较少关注平台功能。
Why choose Cloudflare for your AI Gateway? Well, in some ways, it feels like a natural fit. We’ve spent the last 10+ years helping build a better Internet by running one of the largest global networks, helping customers around the world with performance, reliability, and security – Cloudflare is used as a reverse proxy by nearly 20% of all websites. With our expertise, it felt like a natural progression – change one line of code, and we can help with observability, reliability, and control for your AI applications – all in one control plane – so that you can get back to building.
为什么选择 Cloudflare 作为您的人工智能网关？从某些方面来说，这感觉就像是天作之合。在过去的十多年里，我们一直在运营全球最大的网络之一，帮助世界各地的客户提高性能、可靠性和安全性，从而构建了一个更好的互联网。凭借我们的专业知识，这感觉就像一个自然而然的过程–只需修改一行代码，我们就能帮助您的人工智能应用实现可观察性、可靠性和控制性–所有这些都在一个控制平面内–这样您就可以继续构建。
Here is that one line code change using the OpenAI JS SDK. And check out our docs to reference other providers, SDKs, and languages.
下面是使用 OpenAI JS SDK 进行的一行代码更改。请查看我们的文档，参考其他提供商、SDK 和语言。
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'my api key', // defaults to process.env["OPENAI_API_KEY"]
baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/openai"
});
What’s included today? 今天包括哪些内容？
After talking to customers, it was clear that we needed to focus on some foundational features before moving onto some of the more advanced ones. While we’re really excited about what’s to come, here are the key features available in GA today:
在与客户交流之后，我们清楚地认识到，在开发一些更先进的功能之前，我们需要重点关注一些基础功能。我们对即将推出的功能感到非常兴奋，以下是 GA 目前可用的主要功能：
Analytics: Aggregate metrics from across multiple providers. See traffic patterns and usage including the number of requests, tokens, and costs over time.
分析：汇总来自多个提供商的指标。查看流量模式和使用情况，包括请求数、令牌数和随时间变化的成本。
Real-time logs: Gain insight into requests and errors as you build.
实时日志：在构建过程中深入了解请求和错误。
Caching: Enable custom caching rules and use Cloudflare’s cache for repeat requests instead of hitting the original model provider API, helping you save on cost and latency.
缓存：启用自定义缓存规则，使用 Cloudflare 缓存来处理重复请求，而不是访问原始模型提供商 API，从而帮助您节省成本和延迟。
Rate limiting: Control how your application scales by limiting the number of requests your application receives to control costs or prevent abuse.
速率限制：通过限制应用程序接收的请求数量来控制应用程序的扩展方式，以控制成本或防止滥用。
Support for your favorite providers: AI Gateway now natively supports Workers AI plus 10 of the most popular providers, including Groq and Cohere as of mid-May 2024.
支持您最喜爱的提供商：AI Gateway 现在原生支持 Workers AI 以及 10 个最受欢迎的提供商，包括从 2024 年 5 月中旬开始支持 Groq 和 Cohere。
Universal endpoint: In case of errors, improve resilience by defining request fallbacks to another model or inference provider.
通用端点：如果出现错误，可通过定义请求回退到另一个模型或推理提供程序来提高恢复能力。
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug} -X POST \
--header 'Content-Type: application/json' \
--data '[
{
"provider": "workers-ai",
"endpoint": "@cf/meta/llama-2-7b-chat-int8",
"headers": {
"Authorization": "Bearer {cloudflare_token}",
"Content-Type": "application/json"
},
"query": {
"messages": [
{
"role": "system",
"content": "You are a friendly assistant"
},
{
"role": "user",
"content": "What is Cloudflare?"
}
]
}
},
{
"provider": "openai",
"endpoint": "chat/completions",
"headers": {
"Authorization": "Bearer {open_ai_token}",
"Content-Type": "application/json"
},
"query": {
"model": "gpt-3.5-turbo",
"stream": true,
"messages": [
{
"role": "user",
"content": "What is Cloudflare?"
}
]
}
}
]'
What’s coming up? 接下来有什么活动？
We’ve gotten a lot of feedback from developers, and there are some obvious things on the horizon such as persistent logs and custom metadata – foundational features that will help unlock the real magic down the road.
我们从开发人员那里得到了很多反馈，一些显而易见的功能即将推出，如持久日志和自定义元数据，这些基础功能将帮助我们在未来的道路上释放真正的魔力。
But let’s take a step back for a moment and share our vision. At Cloudflare, we believe our platform is much more powerful as a unified whole than as a collection of individual parts. This mindset applied to our AI products means that they should be easy to use, combine, and run in harmony.
但是，让我们退后一步，分享一下我们的愿景。在 Cloudflare，我们相信我们的平台作为一个统一的整体要比作为一个单独部分的集合强大得多。将这一思想应用于我们的人工智能产品，意味着它们应该易于使用、组合并和谐运行。
Let’s imagine the following journey. You initially onboard onto Workers AI to run inference with the latest open source models. Next, you enable AI Gateway to gain better visibility and control, and start storing persistent logs. Then you want to start tuning your inference results, so you leverage your persistent logs, our prompt management tools, and our built in eval functionality. Now you’re making analytical decisions to improve your inference results. With each data driven improvement, you want more. So you implement our feedback API which helps annotate inputs/outputs, in essence building a structured data set. At this point, you are one step away from a one-click fine tune that can be deployed instantly to our global network, and it doesn’t stop there. As you continue to collect logs and feedback, you can continuously rebuild your fine tune adapters in order to deliver the best results to your end users.
让我们想象一下下面的旅程。您首先接入 Workers AI，使用最新的开源模型运行推理。接下来，您启用 AI Gateway 以获得更好的可见性和控制，并开始存储持久日志。然后，您想开始调整推理结果，因此您需要利用持久日志、我们的提示管理工具和内置的评估功能。现在，您正在做出分析决策，以改进推理结果。每一次数据驱动的改进，您都希望得到更多。因此，您实施了我们的反馈应用程序接口（API），该程序接口可帮助注释输入/输出，本质上就是建立一个结构化数据集。在这一点上，您离一键微调仅一步之遥，而微调可以立即部署到我们的全球网络中。随着不断收集日志和反馈，您可以持续重建微调适配器，以便为最终用户提供最佳结果。
This is all just an aspirational story at this point, but this is how we envision the future of AI Gateway and our AI suite as a whole. You should be able to start with the most basic setup and gradually progress into more advanced workflows, all without leaving Cloudflare’s AI platform. In the end, it might not look exactly as described above, but you can be sure that we are committed to providing the best AI ops tools to help make Cloudflare the best place for AI.
How do I get started?
AI Gateway is available to use today on all plans. If you haven’t yet used AI Gateway, check out our developer documentation and get started now. AI Gateway’s core features available today are offered for free, and all it takes is a Cloudflare account and one line of code to get started. In the future, more premium features, such as persistent logging and secrets management will be available subject to fees. If you have any questions, reach out on our [Discord channel]
We protect entire corporate networks, help customers build Internet-scale applications efficiently, accelerate any website or Internet application, ward off DDoS attacks, keep hackers at bay, and can help you on your journey to Zero Trust.
我们保护整个企业网络，帮助客户高效构建互联网规模的应用程序，加速任何网站或互联网应用程序，抵御 DDoS 攻击，防止黑客入侵，并帮助您实现零信任。
Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.
从任何设备访问 1.1.1.1，开始使用我们的免费应用程序，让您的互联网更快、更安全。
To learn more about our mission to help build a better Internet, start here. If you’re looking for a new career direction, check out our open positions.
欲了解更多有关我们帮助建设更美好互联网的使命，请从这里开始。如果您正在寻找新的职业方向，请查看我们的空缺职位。