feat: ChatGPT Plugins/OpenAPI specs for Plugins Endpoint (#620)

* wip: proof of concept for openapi chain

* chore(api): update langchain dependency to version 0.0.105

* feat(Plugins): use ChatGPT Plugins/OpenAPI specs (first pass)

* chore(manifest.json): update pluginKey for "Browser" tool to "web-browser"
chore(handleTools.js): update customConstructor key for "web-browser" tool

* fix(handleSubmit.js): set unfinished property to false for all endpoints

* fix(handlers.js): remove unnecessary capitalizeWords function and use action.tool directly
refactor(endpoints.js): rename availableTools to tools and transform it into a map

* feat(endpoints): add plugins selector to endpoints file
refactor(CodeBlock.tsx): refactor to typescript
refactor(Plugin.tsx): use recoil Map for plugin name and refactor to typescript
chore(Message.jsx): linting
chore(PluginsOptions/index.jsx): remove comment/linting
chore(svg): export Clipboard and CheckMark components from SVG index and refactor to typescript

* fix(OpenAPIPlugin.js): rename readYamlFile function to readSpecFile
fix(OpenAPIPlugin.js): handle JSON files in readSpecFile function
fix(OpenAPIPlugin.js): handle JSON URLs in getSpec function
fix(OpenAPIPlugin.js): handle JSON variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add description for variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(loadSpecs.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(Plugin.tsx): remove unnecessary callback parameter in getPluginName function
fix(getDefaultConversation.js): fix browser console error: handle null value for lastConversationSetup in getDefaultConversation function

* feat(api): add new tools

Add Ai PDF tool for super-fast, interactive chats with PDFs of any size, complete with page references for fact checking.
Add VoxScript tool for searching through YouTube transcripts, financial data sources, Google Search results, and more.
Add WebPilot tool for browsing and QA of webpages, PDFs, and data. Generate articles from one or more URLs.

feat(api): update OpenAPIPlugin.js

- Add support for bearer token authorization in the OpenAPIPlugin.
- Add support for custom headers in the OpenAPIPlugin.

fix(api): fix loadTools.js

- Pass the user parameter to the loadSpecs function.

* feat(PluginsClient.js): import findMessageContent function from utils
feat(PluginsClient.js): add message parameter to options object in initializeCustomAgent function
feat(PluginsClient.js): add content to errorMessage if message content is found
feat(PluginsClient.js): break out of loop if message content is found
feat(PluginsClient.js): add delay option with value of 8 to generateTextStream function
feat(PluginsClient.js): add support for process.env.PORT environment variable in app.listen function
feat(askyourpdf.json): add askyourpdf plugin configuration
feat(metar.json): add metar plugin configuration
feat(askyourpdf.yaml): add askyourpdf plugin OpenAPI specification
feat(OpenAPIPlugin.js): add message parameter to createOpenAPIPlugin function
feat(OpenAPIPlugin.js): add description_for_model to chain run message
feat(addOpenAPISpecs.js): remove verbose option from loadSpecs function call

fix(loadSpecs.js): add 'message' parameter to the loadSpecs function
feat(findMessageContent.js): add utility function to find message content in JSON objects

* fix(PluginStoreDialog.tsx): update z-index value for the dialog container

The z-index value for the dialog container was updated to "102" to ensure it appears above other elements on the page.

* chore(web_pilot.json): add "params" field with "user_has_request" parameter set to true

* chore(eslintrc.js): update eslint rules
fix(Login.tsx): add missing semicolon after import statement

* fix(package-lock.json): update langchain dependency to version ^0.0.105

* fix(OpenAPIPlugin.js): change header key from 'id' to 'librechat_user_id' for consistency and clarity

feat(plugins): add documentation for using official ChatGPT Plugins with OpenAPI specs

This commit adds a new file `chatgpt_plugins_openapi.md` to the `docs/features/plugins` directory. The file provides detailed information on how to use official ChatGPT Plugins with OpenAPI specifications. It explains the components of a plugin, including the Plugin Manifest file and the OpenAPI spec. It also covers the process of adding a plugin, editing manifest files, and customizing OpenAPI spec files. Additionally, the commit includes disclaimers about the limitations and compatibility of plugins with LibreChat. The documentation also clarifies that the use of ChatGPT Plugins with LibreChat does not violate OpenAI's Terms of Service.

The purpose of this commit is to provide comprehensive documentation for developers who want to integrate ChatGPT Plugins into their projects using OpenAPI specs. It aims to guide them through the process of adding and configuring plugins, as well as addressing potential issues and

chore(introduction.md): update link to ChatGPT Plugins documentation
docs(introduction.md): clarify the purpose of the plugins endpoint and its capabilities

* fix(OpenAPIPlugin.js): update SUFFIX variable to provide a clearer description
docs(chatgpt_plugins_openapi.md): update information about adding plugins via url on the frontend

* feat(PluginsClient.js): sendIntermediateMessage on successful Agent load
fix(PluginsClient.js, server/index.js, gptPlugins.js): linting fixes
docs(chatgpt_plugins_openapi.md): update links and add additional information

* Update chatgpt_plugins_openapi.md

* chore: rebuild package-lock file

* chore: format/lint all files with new rules

* chore: format all files

* chore(README.md): update AI model selection list

The AI model selection list in the README.md file has been updated to reflect the current options available. The "Anthropic" model has been added as an alternative name for the "Claude" model.

* fix(Plugin.tsx): type issue

* feat(tools): add new tool WebPilot

feat(tools): remove tool Weather Report

feat(tools): add new tool Prompt Perfect

feat(tools): add new tool Scholarly Graph Link

* feat(OpenAPIPlugin.js): add getSpec and readSpecFile functions
feat(OpenAPIPlugin.spec.js): add tests for readSpecFile, getSpec, and createOpenAPIPlugin functions

* chore(agent-demo-1.js): remove unused code and dependencies
chore(agent-demo-2.js): remove unused code and dependencies
chore(demo.js): remove unused code and dependencies

* feat(addOpenAPISpecs): add function to transform OpenAPI specs into desired format
feat(addOpenAPISpecs.spec): add tests for transformSpec function
fix(loadSpecs): remove debugging code

* feat(loadSpecs.spec.js): add unit tests for ManifestDefinition, validateJson, and loadSpecs functions

* fix: package file resolution bug

* chore: move scholarly_graph_link manifest to 'has-issues'

* refactor(client/hooks): convert to TS and export from index

* Update introduction.md

* Update chatgpt_plugins_openapi.md

client/src/components/Input/BingAIOptions/index.jsx

@@ -18,8 +18,12 @@ function BingAIOptions({ show }) {
   const { endpoint, conversationId } = conversation;
   const { toneStyle, context, systemMessage, jailbreak } = conversation;
 
-  if (endpoint !== 'bingAI') return null;
-  if (conversationId !== 'new' && !show) return null;
+  if (endpoint !== 'bingAI') {
+    return null;
+  }
+  if (conversationId !== 'new' && !show) {
+    return null;
+  }
 
   const triggerAdvancedMode = () => setAdvancedMode((prev) => !prev);
 

client/src/components/Input/ChatGPTOptions/index.jsx

@@ -11,8 +11,12 @@ function ChatGPTOptions() {
 
   const endpointsConfig = useRecoilValue(store.endpointsConfig);
 
-  if (endpoint !== 'chatGPTBrowser') return null;
-  if (conversationId !== 'new') return null;
+  if (endpoint !== 'chatGPTBrowser') {
+    return null;
+  }
+  if (conversationId !== 'new') {
+    return null;
+  }
 
   const models = endpointsConfig?.['chatGPTBrowser']?.['availableModels'] || [];
 

client/src/components/Input/NewConversationMenu/EndpointItem.jsx

@@ -29,7 +29,7 @@ export default function ModelItem({ endpoint, value, isSelected }) {
         value={value}
         className={cn(
           'group dark:font-semibold dark:text-gray-100 dark:hover:bg-gray-800',
-          isSelected && 'dark:bg-gray-800 bg-gray-50 active',
+          isSelected && 'active bg-gray-50 dark:bg-gray-800',
         )}
         id={endpoint}
       >

client/src/components/Input/NewConversationMenu/FileUpload.tsx

@@ -26,7 +26,9 @@ const FileUpload: React.FC<FileUploadProps> = ({
 
   const handleFileChange = (event: React.ChangeEvent<HTMLInputElement>): void => {
     const file = event.target.files?.[0];
-    if (!file) return;
+    if (!file) {
+      return;
+    }
 
     const reader = new FileReader();
     reader.onload = (e) => {

client/src/components/Input/NewConversationMenu/PresetItem.jsx

@@ -19,22 +19,38 @@ export default function PresetItem({ preset = {}, value, onChangePreset, onDelet
 
     if (endpoint === 'azureOpenAI' || endpoint === 'openAI') {
       const { chatGptLabel, model } = preset;
-      if (model) _title += `: ${model}`;
-      if (chatGptLabel) _title += ` as ${chatGptLabel}`;
+      if (model) {
+        _title += `: ${model}`;
+      }
+      if (chatGptLabel) {
+        _title += ` as ${chatGptLabel}`;
+      }
     } else if (endpoint === 'google') {
       const { modelLabel, model } = preset;
-      if (model) _title += `: ${model}`;
-      if (modelLabel) _title += ` as ${modelLabel}`;
+      if (model) {
+        _title += `: ${model}`;
+      }
+      if (modelLabel) {
+        _title += ` as ${modelLabel}`;
+      }
     } else if (endpoint === 'bingAI') {
       const { jailbreak, toneStyle } = preset;
-      if (toneStyle) _title += `: ${toneStyle}`;
-      if (jailbreak) _title += ' as Sydney';
+      if (toneStyle) {
+        _title += `: ${toneStyle}`;
+      }
+      if (jailbreak) {
+        _title += ' as Sydney';
+      }
     } else if (endpoint === 'chatGPTBrowser') {
       const { model } = preset;
-      if (model) _title += `: ${model}`;
+      if (model) {
+        _title += `: ${model}`;
+      }
     } else if (endpoint === 'gptPlugins') {
       const { model } = preset;
-      if (model) _title += `: ${model}`;
+      if (model) {
+        _title += `: ${model}`;
+      }
     } else if (endpoint === null) {
       null;
     } else {

client/src/components/Input/NewConversationMenu/index.jsx

@@ -155,7 +155,9 @@ export default function NewConversationMenu() {
           <Button
             id="new-conversation-menu"
             variant="outline"
-            className={'group relative mb-[-12px] ml-0 mt-[-8px] items-center rounded-md border-0 p-1 outline-none focus:ring-0 focus:ring-offset-0 dark:data-[state=open]:bg-opacity-50 md:left-1 md:ml-[-12px] md:pl-1'}
+            className={
+              'group relative mb-[-12px] ml-0 mt-[-8px] items-center rounded-md border-0 p-1 outline-none focus:ring-0 focus:ring-offset-0 dark:data-[state=open]:bg-opacity-50 md:left-1 md:ml-[-12px] md:pl-1'
+            }
           >
             {icon}
             <span className="max-w-0 overflow-hidden whitespace-nowrap px-0 text-slate-600 transition-all group-hover:max-w-[80px] group-hover:px-2 group-data-[state=open]:max-w-[80px] group-data-[state=open]:px-2 dark:text-slate-300">

client/src/components/Input/PluginsOptions/index.jsx

@@ -59,9 +59,10 @@ function PluginsOptions() {
   const triggerAgentSettings = () => setShowAgentSettings((prev) => !prev);
   const { endpoint, agentOptions } = conversation;
 
-  if (endpoint !== 'gptPlugins') return null;
+  if (endpoint !== 'gptPlugins') {
+    return null;
+  }
   const models = endpointsConfig?.['gptPlugins']?.['availableModels'] || [];
-  // const availableTools = endpointsConfig?.['gptPlugins']?.['availableTools'] || [];
 
   const triggerAdvancedMode = () => setAdvancedMode((prev) => !prev);
 
@@ -74,7 +75,9 @@ function PluginsOptions() {
   };
 
   function checkIfSelected(value) {
-    if (!conversation.tools) return false;
+    if (!conversation.tools) {
+      return false;
+    }
     return conversation.tools.find((el) => el.pluginKey === value) ? true : false;
   }
 
@@ -130,12 +133,18 @@ function PluginsOptions() {
           (!advancedMode ? opacityClass : '')
         }
         onMouseEnter={() => {
-          if (advancedMode) return;
+          if (advancedMode) {
+            return;
+          }
           setOpacityClass('full-opacity');
         }}
         onMouseLeave={() => {
-          if (advancedMode) return;
-          if (!messagesTree || messagesTree.length === 0) return;
+          if (advancedMode) {
+            return;
+          }
+          if (!messagesTree || messagesTree.length === 0) {
+            return;
+          }
           setOpacityClass('show');
         }}
       >

client/src/components/Input/SetTokenDialog/GoogleConfig.tsx

@@ -1,7 +1,7 @@
 import React from 'react';
 import FileUpload from '../NewConversationMenu/FileUpload';
 
-const GoogleConfig = ({ setToken } : { setToken: React.Dispatch<React.SetStateAction<string>> }) => {
+const GoogleConfig = ({ setToken }: { setToken: React.Dispatch<React.SetStateAction<string>> }) => {
   return (
     <FileUpload
       id="googleKey"
@@ -16,24 +16,24 @@ const GoogleConfig = ({ setToken } : { setToken: React.Dispatch<React.SetStateAc
 
         if (
           !credentials.client_email ||
-        typeof credentials.client_email !== 'string' ||
-        credentials.client_email.length <= 2
+          typeof credentials.client_email !== 'string' ||
+          credentials.client_email.length <= 2
         ) {
           return false;
         }
 
         if (
           !credentials.project_id ||
-        typeof credentials.project_id !== 'string' ||
-        credentials.project_id.length <= 2
+          typeof credentials.project_id !== 'string' ||
+          credentials.project_id.length <= 2
         ) {
           return false;
         }
 
         if (
           !credentials.private_key ||
-        typeof credentials.private_key !== 'string' ||
-        credentials.private_key.length <= 600
+          typeof credentials.private_key !== 'string' ||
+          credentials.private_key.length <= 600
         ) {
           return false;
         }

client/src/components/Input/SetTokenDialog/HelpText.tsx

@@ -1,6 +1,6 @@
 import React from 'react';
 
-function HelpText({ endpoint } : { endpoint: string }) {
+function HelpText({ endpoint }: { endpoint: string }) {
   const textMap = {
     bingAI: (
       <small className="break-all text-gray-600">
@@ -11,7 +11,7 @@ function HelpText({ endpoint } : { endpoint: string }) {
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        https://www.bing.com
+          https://www.bing.com
         </a>
         {`. Use dev tools or an extension while logged into the site to copy the content of the _U cookie.
       If this fails, follow these `}
@@ -21,7 +21,7 @@ function HelpText({ endpoint } : { endpoint: string }) {
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        instructions
+          instructions
         </a>
         {' to provide the full cookie strings.'}
       </small>
@@ -35,48 +35,47 @@ function HelpText({ endpoint } : { endpoint: string }) {
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        https://chat.openai.com
+          https://chat.openai.com
         </a>
-      , then visit{' '}
+        , then visit{' '}
         <a
           target="_blank"
           href="https://chat.openai.com/api/auth/session"
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        https://chat.openai.com/api/auth/session
+          https://chat.openai.com/api/auth/session
         </a>
-      . Copy access token.
+        . Copy access token.
       </small>
     ),
     google: (
       <small className="break-all text-gray-600">
-      You need to{' '}
+        You need to{' '}
         <a
           target="_blank"
           href="https://console.cloud.google.com/vertex-ai"
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        Enable Vertex AI
+          Enable Vertex AI
         </a>{' '}
-      API on Google Cloud, then{' '}
+        API on Google Cloud, then{' '}
         <a
           target="_blank"
           href="https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?walkthrough_id=iam--create-service-account#step_index=1"
           rel="noreferrer"
           className="text-blue-600 underline"
         >
-        Create a Service Account
+          Create a Service Account
         </a>
         {`. Make sure to click 'Create and Continue' to give at least the 'Vertex AI User' role.
       Lastly, create a JSON key to import here.`}
       </small>
     ),
-
   };
 
   return textMap[endpoint] || null;
-};
+}
 
-export default React.memo(HelpText);
+export default React.memo(HelpText);

client/src/components/Input/SetTokenDialog/InputWithLabel.tsx

@@ -32,6 +32,6 @@ const InputWithLabel: FC<InputWithLabelProps> = ({ value, onChange, label, id })
       />
     </>
   );
-}
+};
 
 export default InputWithLabel;

client/src/components/Input/SetTokenDialog/OpenAIConfig.tsx

@@ -21,7 +21,7 @@ type OpenAIConfigProps = {
   endpoint: string;
 };
 
-const OpenAIConfig = ({ token, setToken, endpoint } : OpenAIConfigProps) => {
+const OpenAIConfig = ({ token, setToken, endpoint }: OpenAIConfigProps) => {
   const [showPanel, setShowPanel] = useState(endpoint === 'azureOpenAI');
   const { getToken } = store.useToken(endpoint);
 
@@ -64,7 +64,7 @@ const OpenAIConfig = ({ token, setToken, endpoint } : OpenAIConfigProps) => {
           <InputWithLabel
             id={'chatGPTLabel'}
             value={token || ''}
-            onChange={(e: { target: { value: any; }; }) => setToken(e.target.value || '')}
+            onChange={(e: { target: { value: any } }) => setToken(e.target.value || '')}
             label={'OpenAI API Key'}
           />
         </>
@@ -73,28 +73,36 @@ const OpenAIConfig = ({ token, setToken, endpoint } : OpenAIConfigProps) => {
           <InputWithLabel
             id={'instanceNameLabel'}
             value={getAzure('azureOpenAIApiInstanceName') || ''}
-            onChange={(e: { target: { value: any; }; }) => setAzure('azureOpenAIApiInstanceName', e.target.value || '')}
+            onChange={(e: { target: { value: any } }) =>
+              setAzure('azureOpenAIApiInstanceName', e.target.value || '')
+            }
             label={'Azure OpenAI Instance Name'}
           />
 
           <InputWithLabel
             id={'deploymentNameLabel'}
             value={getAzure('azureOpenAIApiDeploymentName') || ''}
-            onChange={(e: { target: { value: any; }; }) => setAzure('azureOpenAIApiDeploymentName', e.target.value || '')}
+            onChange={(e: { target: { value: any } }) =>
+              setAzure('azureOpenAIApiDeploymentName', e.target.value || '')
+            }
             label={'Azure OpenAI Deployment Name'}
           />
 
           <InputWithLabel
             id={'versionLabel'}
             value={getAzure('azureOpenAIApiVersion') || ''}
-            onChange={(e: { target: { value: any; }; }) => setAzure('azureOpenAIApiVersion', e.target.value || '')}
+            onChange={(e: { target: { value: any } }) =>
+              setAzure('azureOpenAIApiVersion', e.target.value || '')
+            }
             label={'Azure OpenAI API Version'}
           />
 
           <InputWithLabel
             id={'apiKeyLabel'}
             value={getAzure('azureOpenAIApiKey') || ''}
-            onChange={(e: { target: { value: any; }; }) => setAzure('azureOpenAIApiKey', e.target.value || '')}
+            onChange={(e: { target: { value: any } }) =>
+              setAzure('azureOpenAIApiKey', e.target.value || '')
+            }
             label={'Azure OpenAI API Key'}
           />
         </>

client/src/components/Input/SetTokenDialog/OtherConfig.tsx

@@ -6,7 +6,7 @@ type ConfigProps = {
   setToken: React.Dispatch<React.SetStateAction<string>>;
 };
 
-const OtherConfig = ({ token, setToken } : ConfigProps) => {
+const OtherConfig = ({ token, setToken }: ConfigProps) => {
   return (
     <InputWithLabel
       id={'chatGPTLabel'}

client/src/components/Input/SetTokenDialog/SetTokenDialog.tsx

@@ -17,11 +17,11 @@ const SetTokenDialog = ({ open, onOpenChange, endpoint }) => {
   };
 
   const endpointComponents = {
-    'google': GoogleConfig,
-    'openAI': OpenAIConfig,
-    'azureOpenAI': OpenAIConfig,
-    'gptPlugins': OpenAIConfig,
-    'default': OtherConfig,
+    google: GoogleConfig,
+    openAI: OpenAIConfig,
+    azureOpenAI: OpenAIConfig,
+    gptPlugins: OpenAIConfig,
+    default: OtherConfig,
   };
 
   const EndpointComponent = endpointComponents[endpoint] || endpointComponents['default'];
@@ -32,11 +32,11 @@ const SetTokenDialog = ({ open, onOpenChange, endpoint }) => {
         title={`Set Token for ${alternateName[endpoint] ?? endpoint}`}
         main={
           <div className="grid w-full items-center gap-2">
-            <EndpointComponent token={token} setToken={setToken} endpoint={endpoint}/>
+            <EndpointComponent token={token} setToken={setToken} endpoint={endpoint} />
             <small className="text-red-600">
-        Your token will be sent to the server, but not saved.
+              Your token will be sent to the server, but not saved.
             </small>
-            <HelpText endpoint={endpoint}/>
+            <HelpText endpoint={endpoint} />
           </div>
         }
         selection={{

client/src/components/Input/SetTokenDialog/index.ts

@@ -1 +1 @@
-export { default as SetTokenDialog } from './SetTokenDialog';
+export { default as SetTokenDialog } from './SetTokenDialog';

client/src/components/Input/SubmitButton.jsx

@@ -39,7 +39,7 @@ export default function SubmitButton({
         </div>
       </button>
     );
-  } else if (!isTokenProvided && (!endpointsToHideSetTokens.has(endpoint))) {
+  } else if (!isTokenProvided && !endpointsToHideSetTokens.has(endpoint)) {
     return (
       <>
         <button

client/src/components/Input/index.jsx

@@ -131,7 +131,9 @@ export default function TextChat({ isSearchView = false }) {
     setShowBingToneSetting((show) => !show);
   };
 
-  if (isSearchView) return <></>;
+  if (isSearchView) {
+    return <></>;
+  }
 
   return (
     <>