OpenCode
68a9071cf1
Local branch had diverged from origin/master (sibling commits on the same base). Rewrote local history linearly on top of origin/master, folding in all local content: exa/hot-platform discovery hunters, leaks ledger, nightly sweep orchestrator, updated .gitignore and skill docs, plus the latest hunt output and vault state. Remote-only files (vault keys, channel scripts) were restored rather than dropped, so the resulting tree is a full union of both sides.
1 line
189 KiB
JSON
1 line
189 KiB
JSON
{"repo": "apache/apisix", "files": {"docs/en/latest/plugins/ai-request-rewrite.md": {"5f33466e26c8224ca5fc7fb092b6d4f6aa0690c3": {"t": 1785747233.3032188, "n": 28729, "h": "8aefaaaa90649b9d", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin forwards client requests to LLM services for processing before sending them upstream, enabling AI-driven redaction, enrichment, and reformatting.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n<head>\n <link rel=\"canonical\" href=\"https://docs.api7.ai/hub/ai-request-rewrite\" />\n</head>\n\nimport Tabs from '@theme/Tabs';\nimport TabItem from '@theme/TabItem';\n\n## Description\n\nThe `ai-request-rewrite` Plugin processes client requests by forwarding them to LLM services for transformation before relaying them to Upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The Plugin supports integration with OpenAI, DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible APIs.\n\n## Plugin Attributes\n\n| Name | Type | Required | Default | Valid values | Description |\n| --- | --- | --- | --- | --- | --- |\n| `max_req_body_size` | integer | False | 67108864 | >= 1 | Maximum request body size in bytes buffered into memory. Requests with a larger body are rejected. |\n| `prompt` | string | True | | | The prompt to send to the LLM service for rewriting the client request. |\n| `provider` | string | True | | [openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM service provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When set to `openai-compatible`, the Plugin proxies requests to the custom endpoint configured in `override`. When set to `azure-openai`, the Plugin also proxies requests to the custom endpoint configured in `override` and additionally omits the `model` parameter from the request body sent to Azure OpenAI. |\n| `auth` | object | True | | | Authentication configurations. |\n| `auth.header` | object | False | | | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `auth.query` | object | False | | | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `options` | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |\n| `options.model` | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |\n| `override` | object | False | | | Override setting. |\n| `override.endpoint` | string | False | | | LLM provider endpoint. Required when `provider` is `openai-compatible`. |\n| `timeout` | integer | False | 30000 | 1 - 60000 | Request timeout in milliseconds when requesting the LLM service. |\n| `keepalive` | boolean | False | true | | If true, keep the connection alive when requesting the LLM service. |\n| `keepalive_timeout` | integer | False | 60000 | ≥ 1000 | Keepalive timeout in milliseconds for requests to the LLM service. |\n| `keepalive_pool` | integer | False | 30 | ≥ 1 | Keepalive pool size for connections to the LLM service. |\n| `ssl_verify` | boolean | False | true | | If true, verify the LLM service's SSL certificate. |\n\n## How It Works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\nThe examples use OpenAI as the LLM service. To follow along, obtain an OpenAI [API key](https://openai.com/blog/openai-api) and save it to an environment variable:\n\n```shell\nexport OPENAI_API_KEY=<your-api-key>\n```\n\n:::note\n\nYou can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:\n\n```shell\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n```\n\n:::\n\n### Redact Sensitive Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to redact sensitive information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `provider` is set to `openai`, the OpenAI API key is passed in the `Authorization` header, and the `prompt` instructs the LLM to identify and mask sensitive information:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with some personally identifiable information:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\\"content\\\": \\\"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\\\"}\",\n ...,\n \"json\": {\n \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\"\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.170\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Reformat Data\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to reformat data before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to convert natural language queries into structured JSON:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Book a flight from NYC to LA on April 10, 2022.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\n \\\"intent\\\": \\\"BookFlight\\\",\\n \\\"parameters\\\": {\\n \\\"origin\\\": \\\"NYC\\\",\\n \\\"destination\\\": \\\"LA\\\",\\n \\\"date\\\": \\\"2022-04-10\\\"\\n }\\n}\",\n ...,\n \"json\": {\n \"intent\": \"BookFlight\",\n \"parameters\": {\n \"date\": \"2022-04-10\",\n \"destination\": \"LA\",\n \"origin\": \"NYC\"\n }\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.167\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Summarize Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to summarize information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to summarize lengthy input while preserving key details:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with lengthy content:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Hey! So, I'\\''m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\\''m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I'\\''m also a big foodie, especially into ramen and sushi, so I'\\''d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don'\\''t mind traveling into busy areas for sightseeing. Oh, and I'\\''d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I'\\''d love to catch them if possible. Also, what'\\''s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.\",\n ...,\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.171\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Send Request to an OpenAI-Compatible LLM\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin with an OpenAI-compatible LLM provider by setting `provider` to `openai-compatible` and configuring the custom endpoint in `override.endpoint`.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <your-api-key>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n plugins:\n ai-request-rewrite:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer <your-api-key>\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n"}, "070853d6a1c98b4c2e69c502a0ed577c3d5570bc": {"t": 1785771576.952029, "n": 28729, "h": "8aefaaaa90649b9d", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin forwards client requests to LLM services for processing before sending them upstream, enabling AI-driven redaction, enrichment, and reformatting.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n<head>\n <link rel=\"canonical\" href=\"https://docs.api7.ai/hub/ai-request-rewrite\" />\n</head>\n\nimport Tabs from '@theme/Tabs';\nimport TabItem from '@theme/TabItem';\n\n## Description\n\nThe `ai-request-rewrite` Plugin processes client requests by forwarding them to LLM services for transformation before relaying them to Upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The Plugin supports integration with OpenAI, DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible APIs.\n\n## Plugin Attributes\n\n| Name | Type | Required | Default | Valid values | Description |\n| --- | --- | --- | --- | --- | --- |\n| `max_req_body_size` | integer | False | 67108864 | >= 1 | Maximum request body size in bytes buffered into memory. Requests with a larger body are rejected. |\n| `prompt` | string | True | | | The prompt to send to the LLM service for rewriting the client request. |\n| `provider` | string | True | | [openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM service provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When set to `openai-compatible`, the Plugin proxies requests to the custom endpoint configured in `override`. When set to `azure-openai`, the Plugin also proxies requests to the custom endpoint configured in `override` and additionally omits the `model` parameter from the request body sent to Azure OpenAI. |\n| `auth` | object | True | | | Authentication configurations. |\n| `auth.header` | object | False | | | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `auth.query` | object | False | | | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `options` | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |\n| `options.model` | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |\n| `override` | object | False | | | Override setting. |\n| `override.endpoint` | string | False | | | LLM provider endpoint. Required when `provider` is `openai-compatible`. |\n| `timeout` | integer | False | 30000 | 1 - 60000 | Request timeout in milliseconds when requesting the LLM service. |\n| `keepalive` | boolean | False | true | | If true, keep the connection alive when requesting the LLM service. |\n| `keepalive_timeout` | integer | False | 60000 | ≥ 1000 | Keepalive timeout in milliseconds for requests to the LLM service. |\n| `keepalive_pool` | integer | False | 30 | ≥ 1 | Keepalive pool size for connections to the LLM service. |\n| `ssl_verify` | boolean | False | true | | If true, verify the LLM service's SSL certificate. |\n\n## How It Works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\nThe examples use OpenAI as the LLM service. To follow along, obtain an OpenAI [API key](https://openai.com/blog/openai-api) and save it to an environment variable:\n\n```shell\nexport OPENAI_API_KEY=<your-api-key>\n```\n\n:::note\n\nYou can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:\n\n```shell\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n```\n\n:::\n\n### Redact Sensitive Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to redact sensitive information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `provider` is set to `openai`, the OpenAI API key is passed in the `Authorization` header, and the `prompt` instructs the LLM to identify and mask sensitive information:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with some personally identifiable information:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\\"content\\\": \\\"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\\\"}\",\n ...,\n \"json\": {\n \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\"\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.170\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Reformat Data\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to reformat data before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to convert natural language queries into structured JSON:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Book a flight from NYC to LA on April 10, 2022.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\n \\\"intent\\\": \\\"BookFlight\\\",\\n \\\"parameters\\\": {\\n \\\"origin\\\": \\\"NYC\\\",\\n \\\"destination\\\": \\\"LA\\\",\\n \\\"date\\\": \\\"2022-04-10\\\"\\n }\\n}\",\n ...,\n \"json\": {\n \"intent\": \"BookFlight\",\n \"parameters\": {\n \"date\": \"2022-04-10\",\n \"destination\": \"LA\",\n \"origin\": \"NYC\"\n }\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.167\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Summarize Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to summarize information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to summarize lengthy input while preserving key details:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with lengthy content:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Hey! So, I'\\''m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\\''m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I'\\''m also a big foodie, especially into ramen and sushi, so I'\\''d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don'\\''t mind traveling into busy areas for sightseeing. Oh, and I'\\''d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I'\\''d love to catch them if possible. Also, what'\\''s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.\",\n ...,\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.171\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Send Request to an OpenAI-Compatible LLM\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin with an OpenAI-compatible LLM provider by setting `provider` to `openai-compatible` and configuring the custom endpoint in `override.endpoint`.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <your-api-key>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n plugins:\n ai-request-rewrite:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer <your-api-key>\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n"}, "44946016806b12273b39aa122f6c3ab649406169": {"t": 1785771584.260139, "n": 28568, "h": "1f880168c01bd32c", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin forwards client requests to LLM services for processing before sending them upstream, enabling AI-driven redaction, enrichment, and reformatting.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n<head>\n <link rel=\"canonical\" href=\"https://docs.api7.ai/hub/ai-request-rewrite\" />\n</head>\n\nimport Tabs from '@theme/Tabs';\nimport TabItem from '@theme/TabItem';\n\n## Description\n\nThe `ai-request-rewrite` Plugin processes client requests by forwarding them to LLM services for transformation before relaying them to Upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The Plugin supports integration with OpenAI, DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible APIs.\n\n## Plugin Attributes\n\n| Name | Type | Required | Default | Valid values | Description |\n| --- | --- | --- | --- | --- | --- |\n| `prompt` | string | True | | | The prompt to send to the LLM service for rewriting the client request. |\n| `provider` | string | True | | [openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM service provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When set to `openai-compatible`, the Plugin proxies requests to the custom endpoint configured in `override`. When set to `azure-openai`, the Plugin also proxies requests to the custom endpoint configured in `override` and additionally omits the `model` parameter from the request body sent to Azure OpenAI. |\n| `auth` | object | True | | | Authentication configurations. |\n| `auth.header` | object | False | | | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `auth.query` | object | False | | | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `options` | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |\n| `options.model` | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |\n| `override` | object | False | | | Override setting. |\n| `override.endpoint` | string | False | | | LLM provider endpoint. Required when `provider` is `openai-compatible`. |\n| `timeout` | integer | False | 30000 | 1 - 60000 | Request timeout in milliseconds when requesting the LLM service. |\n| `keepalive` | boolean | False | true | | If true, keep the connection alive when requesting the LLM service. |\n| `keepalive_timeout` | integer | False | 60000 | ≥ 1000 | Keepalive timeout in milliseconds for requests to the LLM service. |\n| `keepalive_pool` | integer | False | 30 | ≥ 1 | Keepalive pool size for connections to the LLM service. |\n| `ssl_verify` | boolean | False | true | | If true, verify the LLM service's SSL certificate. |\n\n## How It Works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\nThe examples use OpenAI as the LLM service. To follow along, obtain an OpenAI [API key](https://openai.com/blog/openai-api) and save it to an environment variable:\n\n```shell\nexport OPENAI_API_KEY=<your-api-key>\n```\n\n:::note\n\nYou can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:\n\n```shell\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n```\n\n:::\n\n### Redact Sensitive Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to redact sensitive information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `provider` is set to `openai`, the OpenAI API key is passed in the `Authorization` header, and the `prompt` instructs the LLM to identify and mask sensitive information:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with some personally identifiable information:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\\"content\\\": \\\"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\\\"}\",\n ...,\n \"json\": {\n \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\"\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.170\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Reformat Data\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to reformat data before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to convert natural language queries into structured JSON:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Book a flight from NYC to LA on April 10, 2022.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\n \\\"intent\\\": \\\"BookFlight\\\",\\n \\\"parameters\\\": {\\n \\\"origin\\\": \\\"NYC\\\",\\n \\\"destination\\\": \\\"LA\\\",\\n \\\"date\\\": \\\"2022-04-10\\\"\\n }\\n}\",\n ...,\n \"json\": {\n \"intent\": \"BookFlight\",\n \"parameters\": {\n \"date\": \"2022-04-10\",\n \"destination\": \"LA\",\n \"origin\": \"NYC\"\n }\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.167\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Summarize Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to summarize information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to summarize lengthy input while preserving key details:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with lengthy content:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Hey! So, I'\\''m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\\''m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I'\\''m also a big foodie, especially into ramen and sushi, so I'\\''d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don'\\''t mind traveling into busy areas for sightseeing. Oh, and I'\\''d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I'\\''d love to catch them if possible. Also, what'\\''s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.\",\n ...,\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.171\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Send Request to an OpenAI-Compatible LLM\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin with an OpenAI-compatible LLM provider by setting `provider` to `openai-compatible` and configuring the custom endpoint in `override.endpoint`.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <your-api-key>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n plugins:\n ai-request-rewrite:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer <your-api-key>\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n"}, "a551344684017e09a9aea1608033b8c919fa9dba": {"t": 1785771585.5806224, "n": 9342, "h": "13d17897e92fd468", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| provider_conf | No | Object | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |\n| provider_conf.project_id | Yes | String | Google Cloud Project ID. |\n| provider_conf.region | Yes | String | Google Cloud Region. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.gcp | No | Object | Configuration for Google Cloud Platform (GCP) authentication. |\n| auth.gcp.service_account_json | No | String | Content of the GCP service account JSON file. This can also be configured by setting the `GCP_SERVICE_ACCOUNT` environment variable. |\n| auth.gcp.max_ttl | No | Integer | Maximum TTL (in seconds) for caching the GCP access token. Minimum: 1. |\n| auth.gcp.expire_early_secs| No | Integer | Seconds to expire the access token before its actual expiration time to avoid edge cases. Minimum: 0. Default: 60. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "5b7170def6deccffb593bdab13fb705f4202f75d": {"t": 1785771587.3878741, "n": 8387, "h": "c7debc03bfe10675", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "03278a308fae8bb0194b788beafcd076847ed1cf": {"t": 1785771588.6895874, "n": 8376, "h": "6a832b632ce36ad2", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic, openrouter, gemini, and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "80c587e42f60a9eacd22c100cb9b34040412084f": {"t": 1785771590.795142, "n": 8368, "h": "8582c2380fdc289e", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic, openrouter, and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "9f3716ddf5002cd4a0fdffc35a4d4e693b75f6f6": {"t": 1785771592.1666431, "n": 8356, "h": "a39cafd6abe39f87", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, openrouter and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "d29878bfda09aab853f030596af59fc5b27c675d": {"t": 1785771593.8600914, "n": 8344, "h": "e0a1a1781eca0b66", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "ff8c350a37ef7f1bdfbb330de3fc2fe771d1eb5a": {"t": 1785771595.1978998, "n": 8330, "h": "54f247b140104e69", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, aimlapi and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible or aimlapi services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "8179784fe68f67fbf79e0110ba8f6f3a5ad56c63": {"t": 1785771596.6013799, "n": 8162, "h": "5a75715eff2fa3db", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n## Description\n\nThe `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.\n\n## Plugin Attributes\n\n| **Field** | **Required** | **Type** | **Description** |\n| ------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------ |\n| prompt | Yes | String | The prompt send to LLM service. |\n| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek and openai-compatible |\n| auth | Yes | Object | Authentication configuration |\n| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |\n| options | No | Object | Key/value settings for the model |\n| options.model | No | String | Model to execute. Examples: \"gpt-3.5-turbo\" for openai, \"deepseek-chat\" for deekseek, or \"qwen-turbo\" for openai-compatible services |\n| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |\n| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000|\n| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |\n| keepalive_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |\n| keepalive_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |\n| ssl_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |\n\n## How it works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\n:::note\n\nYou can fetch the admin_key from config.yaml and save to an environment variable with the following command:\n\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n\n:::\n\n### Redact sensitive information\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"gpt-4\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\nNow send a request:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"4111 1111 1111 1111\",\n \"ssn\": \"123-45-6789\",\n \"address\": \"123 Main St\"\n }'\n```\n\nThe request body send to the LLM Service is as follows:\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{\\n\\\"name\\\":\\\"John Doe\\\",\\n\\\"email\\\":\\\"john.doe@example.com\\\",\\n\\\"credit_card\\\":\\\"4111 1111 1111 1111\\\",\\n\\\"ssn\\\":\\\"123-45-6789\\\",\\n\\\"address\\\":\\\"123 Main St\\\"\\n}\"\n }\n ]\n}\n\n```\n\nThe LLM processes the input and returns a modified request body, which replace detected sensitive values with a masked format then used for the upstream request:\n\n```json\n{\n \"name\": \"John Doe\",\n \"email\": \"john.doe@example.com\",\n \"credit_card\": \"**** **** **** 1111\",\n \"ssn\": \"***-**-6789\",\n \"address\": \"123 Main St\"\n}\n```\n\n### Send request to an OpenAI compatible LLM\n\nCreate a route with the `ai-request-rewrite` plugin with `provider` set to `openai-compatible` and the endpoint of the model set to `override.endpoint` like so:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <some-token>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n"}, "906d883e6ef865f238d1911efca71ad5e8931d75": {"t": 1786068112.4280593, "n": 28729, "h": "8aefaaaa90649b9d", "c": "---\ntitle: ai-request-rewrite\nkeywords:\n - Apache APISIX\n - AI Gateway\n - Plugin\n - ai-request-rewrite\ndescription: The ai-request-rewrite plugin forwards client requests to LLM services for processing before sending them upstream, enabling AI-driven redaction, enrichment, and reformatting.\n---\n\n<!--\n#\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements. See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n-->\n\n<head>\n <link rel=\"canonical\" href=\"https://docs.api7.ai/hub/ai-request-rewrite\" />\n</head>\n\nimport Tabs from '@theme/Tabs';\nimport TabItem from '@theme/TabItem';\n\n## Description\n\nThe `ai-request-rewrite` Plugin processes client requests by forwarding them to LLM services for transformation before relaying them to Upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The Plugin supports integration with OpenAI, DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible APIs.\n\n## Plugin Attributes\n\n| Name | Type | Required | Default | Valid values | Description |\n| --- | --- | --- | --- | --- | --- |\n| `max_req_body_size` | integer | False | 67108864 | >= 1 | Maximum request body size in bytes buffered into memory. Requests with a larger body are rejected. |\n| `prompt` | string | True | | | The prompt to send to the LLM service for rewriting the client request. |\n| `provider` | string | True | | [openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM service provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When set to `openai-compatible`, the Plugin proxies requests to the custom endpoint configured in `override`. When set to `azure-openai`, the Plugin also proxies requests to the custom endpoint configured in `override` and additionally omits the `model` parameter from the request body sent to Azure OpenAI. |\n| `auth` | object | True | | | Authentication configurations. |\n| `auth.header` | object | False | | | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `auth.query` | object | False | | | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured. |\n| `options` | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |\n| `options.model` | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |\n| `override` | object | False | | | Override setting. |\n| `override.endpoint` | string | False | | | LLM provider endpoint. Required when `provider` is `openai-compatible`. |\n| `timeout` | integer | False | 30000 | 1 - 60000 | Request timeout in milliseconds when requesting the LLM service. |\n| `keepalive` | boolean | False | true | | If true, keep the connection alive when requesting the LLM service. |\n| `keepalive_timeout` | integer | False | 60000 | ≥ 1000 | Keepalive timeout in milliseconds for requests to the LLM service. |\n| `keepalive_pool` | integer | False | 30 | ≥ 1 | Keepalive pool size for connections to the LLM service. |\n| `ssl_verify` | boolean | False | true | | If true, verify the LLM service's SSL certificate. |\n\n## How It Works\n\n\n\n## Examples\n\nThe examples below demonstrate how you can configure `ai-request-rewrite` for different scenarios.\n\nThe examples use OpenAI as the LLM service. To follow along, obtain an OpenAI [API key](https://openai.com/blog/openai-api) and save it to an environment variable:\n\n```shell\nexport OPENAI_API_KEY=<your-api-key>\n```\n\n:::note\n\nYou can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:\n\n```shell\nadmin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/\"//g')\n```\n\n:::\n\n### Redact Sensitive Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to redact sensitive information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `provider` is set to `openai`, the OpenAI API key is passed in the `Authorization` header, and the `prompt` instructs the LLM to identify and mask sensitive information:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with some personally identifiable information:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\\"content\\\": \\\"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\\\"}\",\n ...,\n \"json\": {\n \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\"\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.170\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Reformat Data\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to reformat data before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to convert natural language queries into structured JSON:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Convert natural language queries into structured JSON format with intent and extracted parameters.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Book a flight from NYC to LA on April 10, 2022.\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"{\\n \\\"intent\\\": \\\"BookFlight\\\",\\n \\\"parameters\\\": {\\n \\\"origin\\\": \\\"NYC\\\",\\n \\\"destination\\\": \\\"LA\\\",\\n \\\"date\\\": \\\"2022-04-10\\\"\\n }\\n}\",\n ...,\n \"json\": {\n \"intent\": \"BookFlight\",\n \"parameters\": {\n \"date\": \"2022-04-10\",\n \"destination\": \"LA\",\n \"origin\": \"NYC\"\n }\n },\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.167\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Summarize Information\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin to summarize information before the request reaches the Upstream service.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin. The `prompt` instructs the LLM to summarize lengthy input while preserving key details:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"methods\": [\"POST\"],\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"provider\": \"openai\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer '\"$OPENAI_API_KEY\"'\"\n }\n },\n \"options\":{\n \"model\": \"gpt-4\"\n },\n \"prompt\": \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n methods:\n - POST\n plugins:\n ai-request-rewrite:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer ${OPENAI_API_KEY}\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n method: POST\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n methods:\n - POST\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n provider: openai\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: gpt-4\n prompt: \"Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative.\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n\nSend a POST request to the Route with lengthy content:\n\n```shell\ncurl \"http://127.0.0.1:9080/anything\" -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"content\": \"Hey! So, I'\\''m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\\''m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I'\\''m also a big foodie, especially into ramen and sushi, so I'\\''d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don'\\''t mind traveling into busy areas for sightseeing. Oh, and I'\\''d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I'\\''d love to catch them if possible. Also, what'\\''s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!\"\n }'\n```\n\nYou should receive a response similar to the following:\n\n```json\n{\n \"args\": {},\n \"data\": \"The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.\",\n ...,\n \"method\": \"POST\",\n \"origin\": \"192.168.97.1, 103.97.2.171\",\n \"url\": \"http://127.0.0.1/anything\"\n}\n```\n\n### Send Request to an OpenAI-Compatible LLM\n\nThe following example demonstrates how to use the `ai-request-rewrite` Plugin with an OpenAI-compatible LLM provider by setting `provider` to `openai-compatible` and configuring the custom endpoint in `override.endpoint`.\n\n<Tabs groupId=\"api\">\n<TabItem value=\"admin-api\" label=\"Admin API\">\n\nCreate a Route and configure the `ai-request-rewrite` Plugin:\n\n```shell\ncurl \"http://127.0.0.1:9180/apisix/admin/routes/1\" -X PUT \\\n -H \"X-API-KEY: ${admin_key}\" \\\n -d '{\n \"uri\": \"/anything\",\n \"plugins\": {\n \"ai-request-rewrite\": {\n \"prompt\": \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\\''s license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\",\n \"provider\": \"openai-compatible\",\n \"auth\": {\n \"header\": {\n \"Authorization\": \"Bearer <your-api-key>\"\n }\n },\n \"options\": {\n \"model\": \"qwen-plus\",\n \"max_tokens\": 1024,\n \"temperature\": 1\n },\n \"override\": {\n \"endpoint\": \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n }\n }\n },\n \"upstream\": {\n \"type\": \"roundrobin\",\n \"nodes\": {\n \"httpbin.org:80\": 1\n }\n }\n }'\n```\n\n</TabItem>\n<TabItem value=\"adc\" label=\"ADC\">\n\nCreate a Route with the `ai-request-rewrite` Plugin:\n\n```yaml title=\"adc.yaml\"\nservices:\n - name: ai-request-rewrite-service\n routes:\n - name: ai-request-rewrite-route\n uris:\n - /anything\n plugins:\n ai-request-rewrite:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer <your-api-key>\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n upstream:\n type: roundrobin\n nodes:\n - host: httpbin.org\n port: 80\n weight: 1\n```\n\nSynchronize the configuration to the gateway:\n\n```shell\nadc sync -f adc.yaml\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"Ingress Controller\">\n\n<Tabs groupId=\"k8s-api\">\n<TabItem value=\"gateway-api\" label=\"Gateway API\">\n\n```yaml title=\"ai-request-rewrite-gw.yaml\"\napiVersion: v1\nkind: Service\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n type: ExternalName\n externalName: httpbin.org\n---\napiVersion: apisix.apache.org/v1alpha1\nkind: PluginConfig\nmetadata:\n namespace: aic\n name: ai-request-rewrite-plugin-config\nspec:\n plugins:\n - name: ai-request-rewrite\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n---\napiVersion: gateway.networking.k8s.io/v1\nkind: HTTPRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n parentRefs:\n - name: apisix\n rules:\n - matches:\n - path:\n type: Exact\n value: /anything\n filters:\n - type: ExtensionRef\n extensionRef:\n group: apisix.apache.org\n kind: PluginConfig\n name: ai-request-rewrite-plugin-config\n backendRefs:\n - name: httpbin-external-domain\n port: 80\n```\n\n</TabItem>\n<TabItem value=\"ingress\" label=\"APISIX Ingress Controller\">\n\n```yaml title=\"ai-request-rewrite-ic.yaml\"\napiVersion: apisix.apache.org/v2\nkind: ApisixUpstream\nmetadata:\n namespace: aic\n name: httpbin-external-domain\nspec:\n ingressClassName: apisix\n externalNodes:\n - type: Domain\n name: httpbin.org\n---\napiVersion: apisix.apache.org/v2\nkind: ApisixRoute\nmetadata:\n namespace: aic\n name: ai-request-rewrite-route\nspec:\n ingressClassName: apisix\n http:\n - name: ai-request-rewrite-route\n match:\n paths:\n - /anything\n upstreams:\n - name: httpbin-external-domain\n plugins:\n - name: ai-request-rewrite\n enable: true\n config:\n prompt: \"Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \\\"*** **** **** 1234\\\") for credit card numbers. Ensure the JSON structure remains unchanged.\"\n provider: openai-compatible\n auth:\n header:\n Authorization: \"Bearer your-api-key\"\n options:\n model: qwen-plus\n max_tokens: 1024\n temperature: 1\n override:\n endpoint: \"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions\"\n```\n\n</TabItem>\n</Tabs>\n\nApply the configuration to your cluster:\n\n```shell\nkubectl apply -f ai-request-rewrite-ic.yaml\n```\n\n</TabItem>\n</Tabs>\n"}}}} |