AWSManagedRulesCommonRuleSet blocking legitimate chat completion calls from LibreChat Agents with Google Search #133
Description
We call this LiteLLM deployment from LibreChat. We use LibreChat Agents with the Tool option of Google Search. Sometimes, the data returned from the Google Search and submitted to LiteLLM for chat completion will match a condition in the WAF rule set AWSManagedRulesCommonRuleSet; this appears to be a function of the string patterns in the returned Google Search results.
Some possible candidate strings include...
Internally-truncated links that include patterns that may match directory traversal:
"formattedUrl": "https://www.ucop.edu/academic-personnel.../salary-scale-revisions.html",
Possibly the template parameter value, which may resemble injection patters or perhaps file access attempts:
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json"
(However, it may be some other rule within AWSManagedRulesCommonRuleSet that is blocking; indeed, it may very well be, since I suspect that the samples quoted above likely appear in all returned Google search results, even those that make it through the WAF!)
But I tested by manually adding an early Allow rule in my WAF that allows from an IP Set that include the IP address that my application stack presents when it calls the LiteLLM deployment. Once this was set the block would not happen, even after many attempts to reproduce it.
For completeness of reporting, here is the complete data structure of one blocked request:
{
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json"
},
"queries": {
"request": [
{
"title": "Google Custom Search - salary scales for workers site:ucop.edu",
"totalResults": "5360",
"searchTerms": "salary scales for workers site:ucop.edu",
"count": 5,
"startIndex": 1,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "67b34b92ceda04563"
}
],
"nextPage": [
{
"title": "Google Custom Search - salary scales for workers site:ucop.edu",
"totalResults": "5360",
"searchTerms": "salary scales for workers site:ucop.edu",
"count": 5,
"startIndex": 6,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "67b34b92ceda04563"
}
]
},
"context": {
"title": "River general web search (safe)"
},
"searchInformation": {
"searchTime": 0.179046,
"formattedSearchTime": "0.18",
"totalResults": "5360",
"formattedTotalResults": "5,360"
},
"items": [
{
"kind": "customsearch#result",
"title": "Job Code Lookup",
"htmlTitle": "Job Code Lookup",
"link": "https://tcs.ucop.edu/",
"displayLink": "tcs.ucop.edu",
"snippet": "Search for campus-specific non-academic grade table salary ranges. © Copyright 2023, UC Regents. University ...",
"htmlSnippet": "Search for campus-specific non-academic grade table <b>salary ranges</b>. © Copyright 2023, UC Regents. University ...",
"formattedUrl": "https://tcs.ucop.edu/",
"htmlFormattedUrl": "https://tcs.ucop.edu/",
"pagemap": {
"metatags": [
{
"theme-color": "#1976d2",
"viewport": "width=device-width, initial-scale=1"
}
]
}
},
{
"kind": "customsearch#result",
"title": "Academic salaries | UCOP",
"htmlTitle": "Academic <b>salaries</b> | UCOP",
"link": "https://www.ucop.edu/academic-personnel-programs/compensation/index.html",
"displayLink": "www.ucop.edu",
"snippet": "Read more about UC's unique faculty compensation system (pdf). The effective date of salary rates on the Academic Salary Scales is indicated on each scale.",
"htmlSnippet": "Read more about UC's unique faculty compensation system (pdf). The effective date of <b>salary rates</b> on the Academic <b>Salary Scales</b> is indicated on each scale.",
"formattedUrl": "https://www.ucop.edu/academic-personnel-programs/.../index.html",
"htmlFormattedUrl": "https://www.ucop.edu/academic-personnel-programs/.../index.html",
"pagemap": {
"metatags": [
{
"viewport": "width=device-width, initial-scale=1.0",
"id": "a4ab3a0c8030655355d69fa31c9d99ae"
}
]
}
},
{
"kind": "customsearch#result",
"title": "Salary scale revisions | UCOP",
"htmlTitle": "<b>Salary scale</b> revisions | UCOP",
"link": "https://www.ucop.edu/academic-personnel-programs/compensation/salary-scale-revisions.html",
"displayLink": "www.ucop.edu",
"snippet": "Revised Academic Student Employee, Graduate Student Researcher, and Postdoctoral Scholar Salary Scales",
"htmlSnippet": "Revised Academic Student <b>Employee</b>, Graduate Student Researcher, and Postdoctoral Scholar <b>Salary Scales</b>",
"formattedUrl": "https://www.ucop.edu/academic-personnel.../salary-scale-revisions.html",
"htmlFormattedUrl": "https://www.ucop.edu/academic-personnel.../<b>salary</b>-scale-revisions.html",
"pagemap": {
"metatags": [
{
"viewport": "width=device-width, initial-scale=1.0",
"id": "a562aedd8030655355d69fa37c062ed2"
}
]
}
},
{
"kind": "customsearch#result",
"title": "Salary and pay | UCOP",
"htmlTitle": "<b>Salary</b> and <b>pay</b> | UCOP",
"link": "https://www.ucop.edu/local-human-resources/your-career/compensation/salary-and-pay.html",
"displayLink": "www.ucop.edu",
"snippet": "... job grades and salary ranges established within an organization. The salary structure at OP will be expressed in terms of job grades. See Career Tracks Salary ...",
"htmlSnippet": "... job grades and <b>salary ranges</b> established within an organization. The <b>salary structure</b> at OP will be expressed in terms of job grades. See Career Tracks Salary ...",
"formattedUrl": "https://www.ucop.edu/local-human-resources/your.../salary-and-pay.html",
"htmlFormattedUrl": "https://www.ucop.edu/local-human-resources/your.../<b>salary</b>-and-pay.html",
"pagemap": {
"metatags": [
{
"viewport": "width=device-width, initial-scale=1.0",
"id": "a54bdd87803065533fd3d0eb1df0bd58"
}
]
}
},
{
"kind": "customsearch#result",
"title": "Search non-academic title and pay information by any of the ...",
"htmlTitle": "Search non-academic title and <b>pay</b> information by any of the ...",
"link": "https://tcs.ucop.edu/non-academic-titles",
"displayLink": "tcs.ucop.edu",
"snippet": "This system can be used to view information and rates for University of California job codes, both academic and non-academic.",
"htmlSnippet": "This system can be used to view information and <b>rates</b> for University of California job codes, both academic and non-academic.",
"formattedUrl": "https://tcs.ucop.edu/non-academic-titles",
"htmlFormattedUrl": "https://tcs.ucop.edu/non-academic-titles",
"pagemap": {
"metatags": [
{
"theme-color": "#1976d2",
"viewport": "width=device-width, initial-scale=1"
}
]
}
}
]
}