WIP: pre-granular-permissions commit
feat: Add category and support contact fields to Agent schema and UI components
Revert "feat: Add category and support contact fields to Agent schema and UI components"
This reverts commit c43a52b4c9.
Fix: Update import for renderHook in useAgentCategories.spec.tsx
fix: Update icon rendering in AgentCategoryDisplay tests to use empty spans
refactor: Improve category synchronization logic and clean up AgentConfig component
refactor: Remove unused UI flow translations from translation.json
feat: agent marketplace features
🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804)
* fix: social login provider crossover
* feat: Enhance OpenID login handling and add tests for provider validation
* refactor: authentication error handling to use ErrorTypes.AUTH_FAILED enum
* refactor: update authentication error handling in LDAP and SAML strategies to use ErrorTypes.AUTH_FAILED enum
* ci: Add validation for login with existing email and different provider in SAML strategy
chore: Add logging for existing users with different providers in LDAP, SAML, and Social Login strategies
* ✨ feat: Add OpenID audience parameter support in authorization requests
* Updated .env.example to include OPENID_AUDIENCE variable for configuration.
* Enhanced openidStrategy to set the audience parameter in authorization requests if specified, improving OpenID integration.
* Update .env.example
* Update openidStrategy.js
---------
Co-authored-by: Danny Avila <danacordially@gmail.com>
Now prioritizes preferred_username claim, then the nonstandard
username claim, then email.
Removed given_name as a possible username choice to avoid exposing users’ first names as
usernames.
Updated openidStrategy.spec.js to reflect the new claim order.
Fixed mock OpenID server behavior where preferred_username was always
hardcoded, causing test failures.
Adjusted OpenID setup test to align with new username parameter
behavior.
* Fixes https://github.com/danny-avila/LibreChat/issues/8099 in correctly setting up proxy support
- fixes the openid Strategy
- fixes the openid jwt strategy (jwksRsa fetching in a proxy environment)
Signed-off-by: Regli Daniel <daniel.regli1@sanitas.com>
* Fixes https://github.com/danny-avila/LibreChat/issues/8099 in correctly setting up proxy support
- properly formatted
Signed-off-by: Regli Daniel <1daniregli@gmail.com>
---------
Signed-off-by: Regli Daniel <daniel.regli1@sanitas.com>
Signed-off-by: Regli Daniel <1daniregli@gmail.com>
Co-authored-by: schnaker85 <1daniregligmail.com>
* feat: Enhance OpenID Strategy with Debug Logging and Header Management
- Added detailed logging for OpenID requests and responses when debug mode is enabled.
- Introduced helper functions for safely logging sensitive data and headers.
- Updated OpenID strategy to handle non-standard WWW-Authenticate headers in responses.
- Refactored proxy configuration handling for improved clarity and logging.
* refactor: MemoryViewer Layout with Conditional Justification
- Updated the MemoryViewer component to conditionally apply justification styles based on memory data and access permissions.
- Introduced utility function `cn` for cleaner class name management in the component.
* refactor: Update OpenID Strategy to use Global Fetch
* refactor: Add undici for customFetch request handling in OpenID strategy
* fix: Export 'files' module in utils index
* chore: Add node-fetch dependency for openid image download
* ci: Add comprehensive tests for multer configuration and file handling
- Introduced a new test suite for multer configuration, covering storage destination and filename generation.
- Implemented tests for file filtering, ensuring only valid JSON files are accepted.
- Added error handling tests for edge cases and vulnerabilities, including handling empty field names and malformed filenames.
- Integrated real configuration testing with actual fileConfig and custom endpoints.
- Enhanced UUID generation tests to ensure uniqueness and cryptographic security.
* chore: Improve proxy configuration logging in customFetch function
* fix: Improve logging for non-standard WWW-Authenticate header in customFetch function
* refactor: move model definitions and database-related methods to packages/data-schemas
* ci: update tests due to new DB structure
fix: disable mocking `librechat-data-provider`
feat: Add schema exports to data-schemas package
- Introduced a new schema module that exports various schemas including action, agent, and user schemas.
- Updated index.ts to include the new schema exports for better modularity and organization.
ci: fix appleStrategy tests
fix: Agent.spec.js
ci: refactor handleTools tests to use MongoMemoryServer for in-memory database
fix: getLogStores imports
ci: update banViolation tests to use MongoMemoryServer and improve session mocking
test: refactor samlStrategy tests to improve mock configurations and user handling
ci: fix crypto mock in handleText tests for improved accuracy
ci: refactor spendTokens tests to improve model imports and setup
ci: refactor Message model tests to use MongoMemoryServer and improve database interactions
* refactor: streamline IMessage interface and move feedback properties to types/message.ts
* refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen)
* refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity
* refactor: remove unused mongoose imports from model files for cleaner code
* refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code
* refactor: remove unused import in Transaction model for cleaner code
* ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch
* chore: cleanup imports
* fix: Add default random state parameter to OpenID auth request for providers that require it; ensure passport strategy uses it
* ⌚ refactor: debounce setUserContext to avoid race condition
* refactor: Update OpenID authentication to use randomState from openid-client
* chore: linting in presetSettings type definition
* chore: import order in ModelPanel
* refactor: remove `isLegacyOutput` property from AnthropicClient since only used where defined, add latest models to non-legacy patterns, and remove from client cleanup
* refactor: adjust grid layout in Parameters component for improved responsiveness
* refactor: adjust grid layout in ModelPanel for improved display of model parameters
* test: add cases for maxOutputTokens handling in Claude 4 Sonnet and Opus models
* ci: mock loadCustomConfig in server tests and refactor OpenID route for improved authentication handling
* feat: integrate OpenID Connect support with token reuse
- Added `jwks-rsa` and `new-openid-client` dependencies for OpenID Connect functionality.
- Implemented OpenID token refresh logic in `AuthController`.
- Enhanced `LogoutController` to handle OpenID logout and session termination.
- Updated JWT authentication middleware to support OpenID token provider.
- Modified OAuth routes to accommodate OpenID authentication and token management.
- Created `setOpenIDAuthTokens` function to manage OpenID tokens in cookies.
- Upgraded OpenID strategy with user info fetching and token exchange protocol.
- Introduced `openIdJwtLogin` strategy for handling OpenID JWT tokens.
- Added caching mechanism for exchanged OpenID tokens.
- Updated configuration to include OpenID exchanged tokens cache key.
- updated .env.example to include the new env variables needed for the feature.
* fix: update return type in downloadImage documentation for clarity and fixed openIdJwtLogin env variables
* fix: update Jest configuration and tests for OpenID strategy integration
* fix: update OpenID strategy to include callback URL in setup
* fix: fix optionalJwtAuth middleware to support OpenID token reuse and improve currentUrl method in CustomOpenIDStrategy to override the dynamic host issue related to proxy (e.g. cloudfront)
* fix: fixed code formatting
* Fix: Add mocks for openid-client and passport strategy in Jest configuration to fix unit tests
* fix eslint errors: Format mock file openid-client.
* ✨ feat: Add PKCE support for OpenID and default handling in strategy setup
---------
Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
Co-authored-by: Ruben Talstra <RubenTalstra1211@outlook.com>
* WIP: end session endpoint
* refactor: move useGetBannerQuery outside of package
* refactor: add queriesEnabled and move useGetEndpointsConfigQuery to data-provider (local)
* refactor: move useGetEndpointsQuery import to data-provider
* refactor: relocate useGetEndpointsQuery import to improve module organization
* refactor: move `useGetStartupConfig` from package to `~/data-provider`
* refactor: move useGetUserBalance to data-provider and update imports
* refactor: update query enabled conditions to include config check
* refactor: remove unused useConfigOverride import from useAppStartup
* refactor: integrate queriesEnabled state into file and search queries and move useGetSearchEnabledQuery to data-provider (local)
* refactor: move useGetUserQuery to data-provider and update imports
* refactor: enhance loginUser mutation with success and error handling as pass in options to hook
* refactor: update enabled condition in queries to handle undefined config
* refactor: enhance authentication mutations with queriesEnabled state management
* refactor: improve conditional rendering for error messages and feature flags in Login component
* refactor: remove unused queriesEnabled state from AuthContextProvider
* refactor: implement queriesEnabled state management in LoginLayout with timeout handling
* refactor: add conditional check for end session endpoint in OpenID strategy
* ci: fix tests after changes
* refactor: remove endSessionEndpoint from user schema and update logoutController to use OpenID issuer's end_session_endpoint
* refactor: update logoutController to use end_session_endpoint from issuer metadata
* oidc-changes Initial attempt at testing openidStrategy and adding OPENID_USERNAME_CLAIM setting
* oidc-changes Add OPENID_NAME_CLAIM
* oidc-changes cleanup oidc test code
* oidc-changes using mongo memory server for test
* oidc-changes Change tests to expect username all lowercase
* oidc-changes Add more tests
* chore: linting
* refactor: Simplify OpenID full name retrieval logic
* refactor: Simplify OpenID user info retrieval logic
* refactor: move helper to openidStrategy.js
---------
Co-authored-by: alihacks <alihacks@pm.me>
Related to #3261
Add proxy support to `downloadImage` function in `openidStrategy.js`
* Import `HttpsProxyAgent` from `https-proxy-agent`.
* Add `agent` property to the fetch options in `downloadImage` function if `process.env.PROXY` is set.
* Update the `fetch` call in `downloadImage` function to use the proxy agent if available.
* feat: verification email
* chore: email verification invalid; localize: update
* fix: redirect to login when signup: fix: save emailVerified correctly
* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true
* fix: user needs to be authenticated
* style: update
* fix: registration success message and redirect logic
* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN
* refactor: move checkEmailConfig to server/utils
* refactor: use req as param for verifyEmail function
* chore: jsdoc
* chore: remove console log
* refactor: rename `createNewUser` to `createSocialUser`
* refactor: update typing and add expiresAt field to userSchema
* refactor: begin use of user methods over direct model access for User
* refactor: initial email verification rewrite
* chore: typing
* refactor: registration flow rewrite
* chore: remove help center text
* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes
* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only
* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition
* refactor: logout on deleteUser mutatation
* refactor: Update openidStrategy login success message format
* refactor: Add emailVerified field to Discord and Facebook profile details
* refactor: move limiters to separate middleware dir
* refactor: Add limiters for email verification and password reset
* refactor: Remove getUserController and update routes and controllers accordingly
* refactor: Update getUserById method to exclude password and version fields
* refactor: move verification to user route, add resend verification option
* refactor: Improve email verification process and resend option
* refactor: remove more direct model access of User and remove unused code
* refactor: replace user authentication methods and token generation
* fix: add user.id to jwt user
* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter
* fix(updateUserPluginsService): ensure userPlugins variable is defined
* refactor: Delete all shared links for a specific user
* fix: remove use of direct User.save() in handleExistingUser
* fix(importLibreChatConvo): handle missing createdAt field in messages
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* Add fetch requirement
Fixes - error: [openidStrategy] downloadImage: Error downloading image at URL "https://graph.microsoft.com/v1.0/me/photo/$value": TypeError: response.buffer is not a function
* Update openidStrategy.js
---------
Co-authored-by: Danny Avila <danacordially@gmail.com>
* Patch for OpenID username
`username` is generally based on email, rather than `given_name`. The challenge with `given_name` is that it can be a multi-value array (ex: "Nick, Fullname"), which completely breaks the system with:
```
LibreChat | ValidationError: User validation failed: username: Cast to string failed for value "[ 'Nickname', 'Firstname' ]" (type Array) at path "username"
LibreChat | at Document.invalidate (/app/node_modules/mongoose/lib/document.js:3200:32)
LibreChat | at model.$set (/app/node_modules/mongoose/lib/document.js:1459:12)
LibreChat | at model.set [as username] (/app/node_modules/mongoose/lib/helpers/document/compile.js:205:19)
LibreChat | at OpenIDConnectStrategy._verify (/app/api/strategies/openidStrategy.js:127:27)
LibreChat | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
* Update openidStrategy.js
* refactor(openidStrategy): add helper function for stringy username
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example
* WIP: Refactor logging statements in code
* WIP: Refactor logging statements and import configurations
* WIP: Refactor logging statements and import configurations
* refactor: broadcast Redis initialization message with `info` not `debug`
* refactor: complete Refactor logging statements and import configurations
* chore: delete unused tools
* fix: circular dependencies due to accessing logger
* refactor(handleText): handle booleans and write tests
* refactor: redact sensitive values, better formatting
* chore: improve log formatting, avoid passing strings to 2nd arg
* fix(ci): fix jest tests due to logger changes
* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time
* chore: update docs
* chore: update docs
* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose
* chore: spread objects where they are commonly logged to allow string truncation
* chore: improve error log formatting
* Add a restart to melisearch in docker-compose.yml
* Oauth fixes for Cognito
* Use the username or email for full name from oath if not provided
---------
Co-authored-by: Donavan <snark@hey.com>
* wip: proof of concept for openapi chain
* chore(api): update langchain dependency to version 0.0.105
* feat(Plugins): use ChatGPT Plugins/OpenAPI specs (first pass)
* chore(manifest.json): update pluginKey for "Browser" tool to "web-browser"
chore(handleTools.js): update customConstructor key for "web-browser" tool
* fix(handleSubmit.js): set unfinished property to false for all endpoints
* fix(handlers.js): remove unnecessary capitalizeWords function and use action.tool directly
refactor(endpoints.js): rename availableTools to tools and transform it into a map
* feat(endpoints): add plugins selector to endpoints file
refactor(CodeBlock.tsx): refactor to typescript
refactor(Plugin.tsx): use recoil Map for plugin name and refactor to typescript
chore(Message.jsx): linting
chore(PluginsOptions/index.jsx): remove comment/linting
chore(svg): export Clipboard and CheckMark components from SVG index and refactor to typescript
* fix(OpenAPIPlugin.js): rename readYamlFile function to readSpecFile
fix(OpenAPIPlugin.js): handle JSON files in readSpecFile function
fix(OpenAPIPlugin.js): handle JSON URLs in getSpec function
fix(OpenAPIPlugin.js): handle JSON variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add description for variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(loadSpecs.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(Plugin.tsx): remove unnecessary callback parameter in getPluginName function
fix(getDefaultConversation.js): fix browser console error: handle null value for lastConversationSetup in getDefaultConversation function
* feat(api): add new tools
Add Ai PDF tool for super-fast, interactive chats with PDFs of any size, complete with page references for fact checking.
Add VoxScript tool for searching through YouTube transcripts, financial data sources, Google Search results, and more.
Add WebPilot tool for browsing and QA of webpages, PDFs, and data. Generate articles from one or more URLs.
feat(api): update OpenAPIPlugin.js
- Add support for bearer token authorization in the OpenAPIPlugin.
- Add support for custom headers in the OpenAPIPlugin.
fix(api): fix loadTools.js
- Pass the user parameter to the loadSpecs function.
* feat(PluginsClient.js): import findMessageContent function from utils
feat(PluginsClient.js): add message parameter to options object in initializeCustomAgent function
feat(PluginsClient.js): add content to errorMessage if message content is found
feat(PluginsClient.js): break out of loop if message content is found
feat(PluginsClient.js): add delay option with value of 8 to generateTextStream function
feat(PluginsClient.js): add support for process.env.PORT environment variable in app.listen function
feat(askyourpdf.json): add askyourpdf plugin configuration
feat(metar.json): add metar plugin configuration
feat(askyourpdf.yaml): add askyourpdf plugin OpenAPI specification
feat(OpenAPIPlugin.js): add message parameter to createOpenAPIPlugin function
feat(OpenAPIPlugin.js): add description_for_model to chain run message
feat(addOpenAPISpecs.js): remove verbose option from loadSpecs function call
fix(loadSpecs.js): add 'message' parameter to the loadSpecs function
feat(findMessageContent.js): add utility function to find message content in JSON objects
* fix(PluginStoreDialog.tsx): update z-index value for the dialog container
The z-index value for the dialog container was updated to "102" to ensure it appears above other elements on the page.
* chore(web_pilot.json): add "params" field with "user_has_request" parameter set to true
* chore(eslintrc.js): update eslint rules
fix(Login.tsx): add missing semicolon after import statement
* fix(package-lock.json): update langchain dependency to version ^0.0.105
* fix(OpenAPIPlugin.js): change header key from 'id' to 'librechat_user_id' for consistency and clarity
feat(plugins): add documentation for using official ChatGPT Plugins with OpenAPI specs
This commit adds a new file `chatgpt_plugins_openapi.md` to the `docs/features/plugins` directory. The file provides detailed information on how to use official ChatGPT Plugins with OpenAPI specifications. It explains the components of a plugin, including the Plugin Manifest file and the OpenAPI spec. It also covers the process of adding a plugin, editing manifest files, and customizing OpenAPI spec files. Additionally, the commit includes disclaimers about the limitations and compatibility of plugins with LibreChat. The documentation also clarifies that the use of ChatGPT Plugins with LibreChat does not violate OpenAI's Terms of Service.
The purpose of this commit is to provide comprehensive documentation for developers who want to integrate ChatGPT Plugins into their projects using OpenAPI specs. It aims to guide them through the process of adding and configuring plugins, as well as addressing potential issues and
chore(introduction.md): update link to ChatGPT Plugins documentation
docs(introduction.md): clarify the purpose of the plugins endpoint and its capabilities
* fix(OpenAPIPlugin.js): update SUFFIX variable to provide a clearer description
docs(chatgpt_plugins_openapi.md): update information about adding plugins via url on the frontend
* feat(PluginsClient.js): sendIntermediateMessage on successful Agent load
fix(PluginsClient.js, server/index.js, gptPlugins.js): linting fixes
docs(chatgpt_plugins_openapi.md): update links and add additional information
* Update chatgpt_plugins_openapi.md
* chore: rebuild package-lock file
* chore: format/lint all files with new rules
* chore: format all files
* chore(README.md): update AI model selection list
The AI model selection list in the README.md file has been updated to reflect the current options available. The "Anthropic" model has been added as an alternative name for the "Claude" model.
* fix(Plugin.tsx): type issue
* feat(tools): add new tool WebPilot
feat(tools): remove tool Weather Report
feat(tools): add new tool Prompt Perfect
feat(tools): add new tool Scholarly Graph Link
* feat(OpenAPIPlugin.js): add getSpec and readSpecFile functions
feat(OpenAPIPlugin.spec.js): add tests for readSpecFile, getSpec, and createOpenAPIPlugin functions
* chore(agent-demo-1.js): remove unused code and dependencies
chore(agent-demo-2.js): remove unused code and dependencies
chore(demo.js): remove unused code and dependencies
* feat(addOpenAPISpecs): add function to transform OpenAPI specs into desired format
feat(addOpenAPISpecs.spec): add tests for transformSpec function
fix(loadSpecs): remove debugging code
* feat(loadSpecs.spec.js): add unit tests for ManifestDefinition, validateJson, and loadSpecs functions
* fix: package file resolution bug
* chore: move scholarly_graph_link manifest to 'has-issues'
* refactor(client/hooks): convert to TS and export from index
* Update introduction.md
* Update chatgpt_plugins_openapi.md
* fix: increase username max length and accept '-' in regex
* fix: increase username max length and accept '-' in regex
* fix: increase username max length and accept '-' in regex
