* feat: Add granular role-based permissions system with Entra ID integration
- Implement RBAC with viewer/editor/owner roles using bitwise permissions
- Add AccessRole, AclEntry, and Group models for permission management
- Create PermissionService for core permission logic and validation
- Integrate Microsoft Graph API for Entra ID user/group search
- Add middleware for resource access validation with custom ID resolvers
- Implement bulk permission updates with transaction support
- Create permission management UI with people picker and role selection
- Add public sharing capabilities for resources
- Include database migration for existing agent ownership
- Support hybrid local/Entra ID identity management
- Add comprehensive test coverage for all new services
chore: Update @librechat/data-schemas to version 0.0.9 and export common module in index.ts
fix: Update userGroup tests to mock logger correctly and change principalId expectation from null to undefined
* fix(data-schemas): use partial index for group idOnTheSource uniqueness
Replace sparse index with partial filter expression to allow multiple local groups
while maintaining unique constraint for external source IDs. The sparse option
on compound indexes doesn't work as expected when one field is always present.
* fix: imports in migrate-agent-permissions.js
* chore(data-schemas): add comprehensive README for data schemas package
- Introduced a detailed README.md file outlining the structure, architecture patterns, and best practices for the LibreChat Data Schemas package.
- Included guidelines for creating new entities, type definitions, schema files, model factory functions, and database methods.
- Added examples and common patterns to enhance understanding and usage of the package.
* chore: remove unused translation keys from localization file
* ci: fix existing tests based off new permission handling
- Renamed test cases to reflect changes in permission checks being handled at the route level.
- Updated assertions to verify that agents are returned regardless of user permissions due to the new permission system.
- Adjusted mocks in AppService and PermissionService tests to ensure proper functionality without relying on actual implementations.
* ci: add unit tests for access control middleware
- Introduced tests for the `canAccessAgentResource` middleware to validate permission checks for agent resources.
- Implemented tests for various scenarios including user roles, ACL entries, and permission levels.
- Added tests for the `checkAccess` function to ensure proper permission handling based on user roles and permissions.
- Utilized MongoDB in-memory server for isolated test environments.
* refactor: remove unused mocks from GraphApiService tests
* ci: enhance AgentFooter tests with improved mocks and permission handling
- Updated mocks for `useWatch`, `useAuthContext`, `useHasAccess`, and `useResourcePermissions` to streamline test setup.
- Adjusted assertions to reflect changes in UI based on agent ID and user roles.
- Replaced `share-agent` component with `grant-access-dialog` in tests to align with recent UI updates.
- Added tests for handling null agent data and permissions loading scenarios.
* ci: enhance GraphApiService tests with MongoDB in-memory server
- Updated test setup to use MongoDB in-memory server for isolated testing.
- Refactored beforeEach to beforeAll for database connection management.
- Cleared database before each test to ensure a clean state.
- Retained existing mocks while improving test structure for better clarity.
* ci: enhance GraphApiService tests with additional logger mocks
- Added mock implementation for logger methods in GraphApiService tests to improve error and debug logging during test execution.
- Ensured existing mocks remain intact while enhancing test coverage and clarity.
* chore: address ESLint Warnings
* - add cursor-based pagination to getListAgentsByAccess and update handler
- add index on updatedAt and _id in agent schema for improved query performance
* refactor permission service with reuse of model methods from data-schema package
* - Fix ObjectId comparison in getListAgentsHandler using .equals() method instead of strict equality
- Add findPubliclyAccessibleResources function to PermissionService for bulk public resource queries
- Add hasPublicPermission function to PermissionService for individual resource public permission checks
- Update getAgentHandler to use hasPublicPermission for accurate individual agent public status
- Replace instanceProjectId-based global checks with isPublic property from backend in client code
- Add isPublic property to Agent type definition
- Add NODE_TLS_REJECT_UNAUTHORIZED debug setting to VS Code launch config
* feat: add check for People.Read scope in searchContacts
* fix: add roleId parameter to grantPermission and update tests for GraphApiService
* refactor: remove problematic projection pipelines in getResourcePermissions for document db aws compatibility
* feat: enhance agent permissions migration with DocumentDB compatibility and add dry-run script
* feat: add support for including Entra ID group owners as members in permissions management + fix Group members paging
* feat: enforce at least one owner requirement for permission updates and add corresponding localization messages
* refactor: remove German locale (must be added via i18n)
* chore: linting in `api/models/Agent.js` and removed unused variables
* chore: linting, remove unused vars, and remove project-related parameters from `updateAgentHandler`
* chore: address ESLint errors
* chore: revert removal of unused vars for versioning
---------
Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
* feat: Enhance OpenID Strategy with Debug Logging and Header Management
- Added detailed logging for OpenID requests and responses when debug mode is enabled.
- Introduced helper functions for safely logging sensitive data and headers.
- Updated OpenID strategy to handle non-standard WWW-Authenticate headers in responses.
- Refactored proxy configuration handling for improved clarity and logging.
* refactor: MemoryViewer Layout with Conditional Justification
- Updated the MemoryViewer component to conditionally apply justification styles based on memory data and access permissions.
- Introduced utility function `cn` for cleaner class name management in the component.
* refactor: Update OpenID Strategy to use Global Fetch
* refactor: Add undici for customFetch request handling in OpenID strategy
* fix: Export 'files' module in utils index
* chore: Add node-fetch dependency for openid image download
* ci: Add comprehensive tests for multer configuration and file handling
- Introduced a new test suite for multer configuration, covering storage destination and filename generation.
- Implemented tests for file filtering, ensuring only valid JSON files are accepted.
- Added error handling tests for edge cases and vulnerabilities, including handling empty field names and malformed filenames.
- Integrated real configuration testing with actual fileConfig and custom endpoints.
- Enhanced UUID generation tests to ensure uniqueness and cryptographic security.
* chore: Improve proxy configuration logging in customFetch function
* fix: Improve logging for non-standard WWW-Authenticate header in customFetch function
* refactor: move model definitions and database-related methods to packages/data-schemas
* ci: update tests due to new DB structure
fix: disable mocking `librechat-data-provider`
feat: Add schema exports to data-schemas package
- Introduced a new schema module that exports various schemas including action, agent, and user schemas.
- Updated index.ts to include the new schema exports for better modularity and organization.
ci: fix appleStrategy tests
fix: Agent.spec.js
ci: refactor handleTools tests to use MongoMemoryServer for in-memory database
fix: getLogStores imports
ci: update banViolation tests to use MongoMemoryServer and improve session mocking
test: refactor samlStrategy tests to improve mock configurations and user handling
ci: fix crypto mock in handleText tests for improved accuracy
ci: refactor spendTokens tests to improve model imports and setup
ci: refactor Message model tests to use MongoMemoryServer and improve database interactions
* refactor: streamline IMessage interface and move feedback properties to types/message.ts
* refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen)
* refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity
* refactor: remove unused mongoose imports from model files for cleaner code
* refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code
* refactor: remove unused import in Transaction model for cleaner code
* ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch
* chore: cleanup imports
* fix: Add default random state parameter to OpenID auth request for providers that require it; ensure passport strategy uses it
* ⌚ refactor: debounce setUserContext to avoid race condition
* refactor: Update OpenID authentication to use randomState from openid-client
* chore: linting in presetSettings type definition
* chore: import order in ModelPanel
* refactor: remove `isLegacyOutput` property from AnthropicClient since only used where defined, add latest models to non-legacy patterns, and remove from client cleanup
* refactor: adjust grid layout in Parameters component for improved responsiveness
* refactor: adjust grid layout in ModelPanel for improved display of model parameters
* test: add cases for maxOutputTokens handling in Claude 4 Sonnet and Opus models
* ci: mock loadCustomConfig in server tests and refactor OpenID route for improved authentication handling
* feat: integrate OpenID Connect support with token reuse
- Added `jwks-rsa` and `new-openid-client` dependencies for OpenID Connect functionality.
- Implemented OpenID token refresh logic in `AuthController`.
- Enhanced `LogoutController` to handle OpenID logout and session termination.
- Updated JWT authentication middleware to support OpenID token provider.
- Modified OAuth routes to accommodate OpenID authentication and token management.
- Created `setOpenIDAuthTokens` function to manage OpenID tokens in cookies.
- Upgraded OpenID strategy with user info fetching and token exchange protocol.
- Introduced `openIdJwtLogin` strategy for handling OpenID JWT tokens.
- Added caching mechanism for exchanged OpenID tokens.
- Updated configuration to include OpenID exchanged tokens cache key.
- updated .env.example to include the new env variables needed for the feature.
* fix: update return type in downloadImage documentation for clarity and fixed openIdJwtLogin env variables
* fix: update Jest configuration and tests for OpenID strategy integration
* fix: update OpenID strategy to include callback URL in setup
* fix: fix optionalJwtAuth middleware to support OpenID token reuse and improve currentUrl method in CustomOpenIDStrategy to override the dynamic host issue related to proxy (e.g. cloudfront)
* fix: fixed code formatting
* Fix: Add mocks for openid-client and passport strategy in Jest configuration to fix unit tests
* fix eslint errors: Format mock file openid-client.
* ✨ feat: Add PKCE support for OpenID and default handling in strategy setup
---------
Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
Co-authored-by: Ruben Talstra <RubenTalstra1211@outlook.com>
* WIP: end session endpoint
* refactor: move useGetBannerQuery outside of package
* refactor: add queriesEnabled and move useGetEndpointsConfigQuery to data-provider (local)
* refactor: move useGetEndpointsQuery import to data-provider
* refactor: relocate useGetEndpointsQuery import to improve module organization
* refactor: move `useGetStartupConfig` from package to `~/data-provider`
* refactor: move useGetUserBalance to data-provider and update imports
* refactor: update query enabled conditions to include config check
* refactor: remove unused useConfigOverride import from useAppStartup
* refactor: integrate queriesEnabled state into file and search queries and move useGetSearchEnabledQuery to data-provider (local)
* refactor: move useGetUserQuery to data-provider and update imports
* refactor: enhance loginUser mutation with success and error handling as pass in options to hook
* refactor: update enabled condition in queries to handle undefined config
* refactor: enhance authentication mutations with queriesEnabled state management
* refactor: improve conditional rendering for error messages and feature flags in Login component
* refactor: remove unused queriesEnabled state from AuthContextProvider
* refactor: implement queriesEnabled state management in LoginLayout with timeout handling
* refactor: add conditional check for end session endpoint in OpenID strategy
* ci: fix tests after changes
* refactor: remove endSessionEndpoint from user schema and update logoutController to use OpenID issuer's end_session_endpoint
* refactor: update logoutController to use end_session_endpoint from issuer metadata
* oidc-changes Initial attempt at testing openidStrategy and adding OPENID_USERNAME_CLAIM setting
* oidc-changes Add OPENID_NAME_CLAIM
* oidc-changes cleanup oidc test code
* oidc-changes using mongo memory server for test
* oidc-changes Change tests to expect username all lowercase
* oidc-changes Add more tests
* chore: linting
* refactor: Simplify OpenID full name retrieval logic
* refactor: Simplify OpenID user info retrieval logic
* refactor: move helper to openidStrategy.js
---------
Co-authored-by: alihacks <alihacks@pm.me>
Related to #3261
Add proxy support to `downloadImage` function in `openidStrategy.js`
* Import `HttpsProxyAgent` from `https-proxy-agent`.
* Add `agent` property to the fetch options in `downloadImage` function if `process.env.PROXY` is set.
* Update the `fetch` call in `downloadImage` function to use the proxy agent if available.
* feat: verification email
* chore: email verification invalid; localize: update
* fix: redirect to login when signup: fix: save emailVerified correctly
* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true
* fix: user needs to be authenticated
* style: update
* fix: registration success message and redirect logic
* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN
* refactor: move checkEmailConfig to server/utils
* refactor: use req as param for verifyEmail function
* chore: jsdoc
* chore: remove console log
* refactor: rename `createNewUser` to `createSocialUser`
* refactor: update typing and add expiresAt field to userSchema
* refactor: begin use of user methods over direct model access for User
* refactor: initial email verification rewrite
* chore: typing
* refactor: registration flow rewrite
* chore: remove help center text
* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes
* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only
* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition
* refactor: logout on deleteUser mutatation
* refactor: Update openidStrategy login success message format
* refactor: Add emailVerified field to Discord and Facebook profile details
* refactor: move limiters to separate middleware dir
* refactor: Add limiters for email verification and password reset
* refactor: Remove getUserController and update routes and controllers accordingly
* refactor: Update getUserById method to exclude password and version fields
* refactor: move verification to user route, add resend verification option
* refactor: Improve email verification process and resend option
* refactor: remove more direct model access of User and remove unused code
* refactor: replace user authentication methods and token generation
* fix: add user.id to jwt user
* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter
* fix(updateUserPluginsService): ensure userPlugins variable is defined
* refactor: Delete all shared links for a specific user
* fix: remove use of direct User.save() in handleExistingUser
* fix(importLibreChatConvo): handle missing createdAt field in messages
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* Add fetch requirement
Fixes - error: [openidStrategy] downloadImage: Error downloading image at URL "https://graph.microsoft.com/v1.0/me/photo/$value": TypeError: response.buffer is not a function
* Update openidStrategy.js
---------
Co-authored-by: Danny Avila <danacordially@gmail.com>
* Patch for OpenID username
`username` is generally based on email, rather than `given_name`. The challenge with `given_name` is that it can be a multi-value array (ex: "Nick, Fullname"), which completely breaks the system with:
```
LibreChat | ValidationError: User validation failed: username: Cast to string failed for value "[ 'Nickname', 'Firstname' ]" (type Array) at path "username"
LibreChat | at Document.invalidate (/app/node_modules/mongoose/lib/document.js:3200:32)
LibreChat | at model.$set (/app/node_modules/mongoose/lib/document.js:1459:12)
LibreChat | at model.set [as username] (/app/node_modules/mongoose/lib/helpers/document/compile.js:205:19)
LibreChat | at OpenIDConnectStrategy._verify (/app/api/strategies/openidStrategy.js:127:27)
LibreChat | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
* Update openidStrategy.js
* refactor(openidStrategy): add helper function for stringy username
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example
* WIP: Refactor logging statements in code
* WIP: Refactor logging statements and import configurations
* WIP: Refactor logging statements and import configurations
* refactor: broadcast Redis initialization message with `info` not `debug`
* refactor: complete Refactor logging statements and import configurations
* chore: delete unused tools
* fix: circular dependencies due to accessing logger
* refactor(handleText): handle booleans and write tests
* refactor: redact sensitive values, better formatting
* chore: improve log formatting, avoid passing strings to 2nd arg
* fix(ci): fix jest tests due to logger changes
* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time
* chore: update docs
* chore: update docs
* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose
* chore: spread objects where they are commonly logged to allow string truncation
* chore: improve error log formatting
* Add a restart to melisearch in docker-compose.yml
* Oauth fixes for Cognito
* Use the username or email for full name from oath if not provided
---------
Co-authored-by: Donavan <snark@hey.com>
* wip: proof of concept for openapi chain
* chore(api): update langchain dependency to version 0.0.105
* feat(Plugins): use ChatGPT Plugins/OpenAPI specs (first pass)
* chore(manifest.json): update pluginKey for "Browser" tool to "web-browser"
chore(handleTools.js): update customConstructor key for "web-browser" tool
* fix(handleSubmit.js): set unfinished property to false for all endpoints
* fix(handlers.js): remove unnecessary capitalizeWords function and use action.tool directly
refactor(endpoints.js): rename availableTools to tools and transform it into a map
* feat(endpoints): add plugins selector to endpoints file
refactor(CodeBlock.tsx): refactor to typescript
refactor(Plugin.tsx): use recoil Map for plugin name and refactor to typescript
chore(Message.jsx): linting
chore(PluginsOptions/index.jsx): remove comment/linting
chore(svg): export Clipboard and CheckMark components from SVG index and refactor to typescript
* fix(OpenAPIPlugin.js): rename readYamlFile function to readSpecFile
fix(OpenAPIPlugin.js): handle JSON files in readSpecFile function
fix(OpenAPIPlugin.js): handle JSON URLs in getSpec function
fix(OpenAPIPlugin.js): handle JSON variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add description for variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(loadSpecs.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(Plugin.tsx): remove unnecessary callback parameter in getPluginName function
fix(getDefaultConversation.js): fix browser console error: handle null value for lastConversationSetup in getDefaultConversation function
* feat(api): add new tools
Add Ai PDF tool for super-fast, interactive chats with PDFs of any size, complete with page references for fact checking.
Add VoxScript tool for searching through YouTube transcripts, financial data sources, Google Search results, and more.
Add WebPilot tool for browsing and QA of webpages, PDFs, and data. Generate articles from one or more URLs.
feat(api): update OpenAPIPlugin.js
- Add support for bearer token authorization in the OpenAPIPlugin.
- Add support for custom headers in the OpenAPIPlugin.
fix(api): fix loadTools.js
- Pass the user parameter to the loadSpecs function.
* feat(PluginsClient.js): import findMessageContent function from utils
feat(PluginsClient.js): add message parameter to options object in initializeCustomAgent function
feat(PluginsClient.js): add content to errorMessage if message content is found
feat(PluginsClient.js): break out of loop if message content is found
feat(PluginsClient.js): add delay option with value of 8 to generateTextStream function
feat(PluginsClient.js): add support for process.env.PORT environment variable in app.listen function
feat(askyourpdf.json): add askyourpdf plugin configuration
feat(metar.json): add metar plugin configuration
feat(askyourpdf.yaml): add askyourpdf plugin OpenAPI specification
feat(OpenAPIPlugin.js): add message parameter to createOpenAPIPlugin function
feat(OpenAPIPlugin.js): add description_for_model to chain run message
feat(addOpenAPISpecs.js): remove verbose option from loadSpecs function call
fix(loadSpecs.js): add 'message' parameter to the loadSpecs function
feat(findMessageContent.js): add utility function to find message content in JSON objects
* fix(PluginStoreDialog.tsx): update z-index value for the dialog container
The z-index value for the dialog container was updated to "102" to ensure it appears above other elements on the page.
* chore(web_pilot.json): add "params" field with "user_has_request" parameter set to true
* chore(eslintrc.js): update eslint rules
fix(Login.tsx): add missing semicolon after import statement
* fix(package-lock.json): update langchain dependency to version ^0.0.105
* fix(OpenAPIPlugin.js): change header key from 'id' to 'librechat_user_id' for consistency and clarity
feat(plugins): add documentation for using official ChatGPT Plugins with OpenAPI specs
This commit adds a new file `chatgpt_plugins_openapi.md` to the `docs/features/plugins` directory. The file provides detailed information on how to use official ChatGPT Plugins with OpenAPI specifications. It explains the components of a plugin, including the Plugin Manifest file and the OpenAPI spec. It also covers the process of adding a plugin, editing manifest files, and customizing OpenAPI spec files. Additionally, the commit includes disclaimers about the limitations and compatibility of plugins with LibreChat. The documentation also clarifies that the use of ChatGPT Plugins with LibreChat does not violate OpenAI's Terms of Service.
The purpose of this commit is to provide comprehensive documentation for developers who want to integrate ChatGPT Plugins into their projects using OpenAPI specs. It aims to guide them through the process of adding and configuring plugins, as well as addressing potential issues and
chore(introduction.md): update link to ChatGPT Plugins documentation
docs(introduction.md): clarify the purpose of the plugins endpoint and its capabilities
* fix(OpenAPIPlugin.js): update SUFFIX variable to provide a clearer description
docs(chatgpt_plugins_openapi.md): update information about adding plugins via url on the frontend
* feat(PluginsClient.js): sendIntermediateMessage on successful Agent load
fix(PluginsClient.js, server/index.js, gptPlugins.js): linting fixes
docs(chatgpt_plugins_openapi.md): update links and add additional information
* Update chatgpt_plugins_openapi.md
* chore: rebuild package-lock file
* chore: format/lint all files with new rules
* chore: format all files
* chore(README.md): update AI model selection list
The AI model selection list in the README.md file has been updated to reflect the current options available. The "Anthropic" model has been added as an alternative name for the "Claude" model.
* fix(Plugin.tsx): type issue
* feat(tools): add new tool WebPilot
feat(tools): remove tool Weather Report
feat(tools): add new tool Prompt Perfect
feat(tools): add new tool Scholarly Graph Link
* feat(OpenAPIPlugin.js): add getSpec and readSpecFile functions
feat(OpenAPIPlugin.spec.js): add tests for readSpecFile, getSpec, and createOpenAPIPlugin functions
* chore(agent-demo-1.js): remove unused code and dependencies
chore(agent-demo-2.js): remove unused code and dependencies
chore(demo.js): remove unused code and dependencies
* feat(addOpenAPISpecs): add function to transform OpenAPI specs into desired format
feat(addOpenAPISpecs.spec): add tests for transformSpec function
fix(loadSpecs): remove debugging code
* feat(loadSpecs.spec.js): add unit tests for ManifestDefinition, validateJson, and loadSpecs functions
* fix: package file resolution bug
* chore: move scholarly_graph_link manifest to 'has-issues'
* refactor(client/hooks): convert to TS and export from index
* Update introduction.md
* Update chatgpt_plugins_openapi.md
* fix: increase username max length and accept '-' in regex
* fix: increase username max length and accept '-' in regex
* fix: increase username max length and accept '-' in regex
