Commit graph

7 commits

Author SHA1 Message Date
Danny Avila
d012da0065
🛡️ fix: Enhance File Upload Security & Error Handling (#4705)
* fix: sanitize filename in multer storage callback

* fix: ensure temporary image upload file is deleted after processing

* fix: prevent cleanup flag from being set to false before actually deleted

* refactor: user avatar, typing, use 'file' for formData instead of 'input', add disk storage, use localization

* fix: update Avatar component to include image dimensions in formData and refactor editor reference type

* fix: refactor avatar upload handling to use fs for file reading and enhance file validation

* fix: ensure temporary image upload file is deleted after processing

* fix: refactor avatar upload routes and handlers for agents and assistants, improve file handling and validation

* fix: improve audio file validation and cleanup

* fix: add filename sanitization utility and integrate it into multer storage configuration

* fix: update group project ID check for null and refactor delete prompt group response type

* fix: invalid access control for deleting prompt groups

* fix: add error handling and logging to checkBan middleware

* fix: catch conversation parsing errors

* chore: revert unnecessary height and width parameters from avatar upload

* chore: update librechat-data-provider version to 0.7.55

* style: ensure KaTeX can spread across visible space
2024-11-12 16:41:04 -05:00
Marco Beretta
ee673d682e
📧 feat: email verification (#2344)
* feat: verification email

* chore: email verification invalid; localize: update

* fix: redirect to login when signup: fix: save emailVerified correctly

* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true

* fix: user needs to be authenticated

* style: update

* fix: registration success message and redirect logic

* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN

* refactor: move checkEmailConfig to server/utils

* refactor: use req as param for verifyEmail function

* chore: jsdoc

* chore: remove console log

* refactor: rename `createNewUser` to `createSocialUser`

* refactor: update typing and add expiresAt field to userSchema

* refactor: begin use of user methods over direct model access for User

* refactor: initial email verification rewrite

* chore: typing

* refactor: registration flow rewrite

* chore: remove help center text

* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes

* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only

* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition

* refactor: logout on deleteUser mutatation

* refactor: Update openidStrategy login success message format

* refactor: Add emailVerified field to Discord and Facebook profile details

* refactor: move limiters to separate middleware dir

* refactor: Add limiters for email verification and password reset

* refactor: Remove getUserController and update routes and controllers accordingly

* refactor: Update getUserById method to exclude password and version fields

* refactor: move verification to user route, add resend verification option

* refactor: Improve email verification process and resend option

* refactor: remove more direct model access of User and remove unused code

* refactor: replace user authentication methods and token generation

* fix: add user.id to jwt user

* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter

* fix(updateUserPluginsService): ensure userPlugins variable is defined

* refactor: Delete all shared links for a specific user

* fix: remove use of direct User.save() in handleExistingUser

* fix(importLibreChatConvo): handle missing createdAt field in messages

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-07 15:06:47 -04:00
Danny Avila
514a502b9c
⏯️ fix(tts): Resolve Voice Selection and Manual Playback Issues (#2845)
* fix: voice setting for autoplayback TTS

* fix(useTextToSpeechExternal): resolve stateful playback issues and consolidate state logic

* refactor: initialize tts voice and provider schema once per request

* fix(tts): edge case, longer text inputs. TODO: use continuous stream for longer text inputs

* fix(tts): pause global audio on conversation change

* refactor: keyvMongo ban cache to allow db updates for unbanning, to prevent server restart

* chore: eslint fix

* refactor: make ban cache exclusively keyvMongo
2024-05-23 16:27:36 -04:00
Danny Avila
8c22bb1d3d
🛠️ fix(Azure/Assistants): Handle Long Domain Names & Other Minor chores (#2475)
* chore: replace violation cache accessors with enum

* chore: fix test

* chore(fileSchema): index timestamps

* fix(ActionService): use encoding/caching strategy for handling assistant function character length limit

* refactor(actions): async `domainParser` also resolve retrieved model (which is deployment name) to user-defined model

* style(AssistantAction): add `whitespace-nowrap` for ellipsis

* refactor(ActionService): if domain is less than or equal to encoded domain fixed length, return domain with replacement of separator

* refactor(actions): use sessions/transactions for updating Assistant Action database records

* chore: remove TTL from ENCODED_DOMAINS cache

* refactor(domainParser): minor optimization and add tests

* fix(spendTokens): use txData.user for token usage logging

* refactor(actions): add helper function `withSession` for database operations with sessions/transactions

* fix(PluginsClient): logger debug `message` field edge case
2024-04-20 15:02:56 -04:00
Danny Avila
5f3ecef575
fix(config/scripts): Enhance User Creation and Ban Handling, Standardize Imports (#1144)
* chore: use relative imports for scripts

* fix(create-user): newUser.save() now properly awaited, double-check user creation, use relative imports, catch exception

* fix(ban-user): catch exception, handle case where IP is undefined, proper check of user ban on login
2023-11-06 09:19:43 -05:00
Danny Avila
5145121eb7
feat(api): initial Redis support; fix(SearchBar): proper debounce (#1039)
* refactor: use keyv for search caching with 1 min expirations

* feat: keyvRedis; chore: bump keyv, bun.lockb, add jsconfig for vscode file resolution

* feat: api/search redis support

* refactor(redis) use ioredis cluster for keyv
fix(OpenID): when redis is configured, use redis memory store for express-session

* fix: revert using uri for keyvredis

* fix(SearchBar): properly debounce search queries, fix weird render behaviors

* refactor: add authentication to search endpoint and show error messages in results

* feat: redis support for violation logs

* fix(logViolation): ensure a number is always being stored in cache

* feat(concurrentLimiter): uses clearPendingReq, clears pendingReq on abort, redis support

* fix(api/search/enable): query only when authenticated

* feat(ModelService): redis support

* feat(checkBan): redis support

* refactor(api/search): consolidate keyv logic

* fix(ci): add default empty value for REDIS_URI

* refactor(keyvRedis): use condition to initialize keyvRedis assignment

* refactor(connectDb): handle disconnected state (should create a new conn)

* fix(ci/e2e): handle case where cleanUp did not successfully run

* fix(getDefaultEndpoint): return endpoint from localStorage if defined and endpointsConfig is default

* ci(e2e): remove afterAll messages as startup/cleanUp will clear messages

* ci(e2e): remove teardown for CI until further notice

* chore: bump playwright/test

* ci(e2e): reinstate teardown as CI issue is specific to github env

* fix(ci): click settings menu trigger by testid
2023-10-11 17:05:47 -04:00
Danny Avila
7b2cedf5ff
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903)
* refactor: require Auth middleware in route index files

* feat: concurrent message limiter

* feat: complete concurrent message limiter with caching

* refactor: SSE response methods separated from handleText

* fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body

* chore: minor name changes

* refactor: add isUUID condition to saveMessage

* fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization

* chore: bump keyv and remove console.log from Message

* fix(concurrentLimiter): ensure messages are only saved in later message children

* refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future

* feat: add denyRequest function for error responses

* feat(utils): add isStringTruthy function

Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true'

* feat: add optional message rate limiters by IP and userId

* feat: add optional message rate limiters by IP and userId to edit route

* refactor: rename isStringTruthy to isTrue for brevity

* refactor(getError): use map to make code cleaner

* refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation

* feat: check if errorMessage is object, stringify if so

* chore: send object to denyRequest which will stringify it

* feat: log excessive requests

* fix(getError): correctly pluralize messages

* refactor(limiters): make type consistent between logs and errorMessage

* refactor(cache): move files out of lib/db into separate cache dir
>> feat: add getLogStores function so Keyv instance is not redundantly created on every violation
feat: separate violation logging to own function with logViolation

* fix: cache/index.js export, properly record userViolations

* refactor(messageLimiters): use new logging method, add logging to registrations

* refactor(logViolation): make userLogs an array of logs per user

* feat: add logging to login limiter

* refactor: pass req as first param to logViolation and record offending IP

* refactor: rename isTrue helper fn to isEnabled

* feat: add simple non_browser check and log violation

* fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch

* chore: adjust nodemon ignore paths to properly ignore logs

* feat: add math helper function for safe use of eval

* refactor(api/convos): use middleware at top of file to avoid redundancy

* feat: add delete all static method for Sessions

* fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case)

* refactor(getLogStores): adjust return type

* feat: add ban violation and check ban logic
refactor(logViolation): pass both req and res objects

* feat: add removePorts helper function

* refactor: rename getError to getMessageError and add getLoginError for displaying different login errors

* fix(AuthContext): fix type issue and remove unused code

* refactor(bans): ban by ip and user id, send response based on origin

* chore: add frontend ban messages

* refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy

* feat: add ban check to AI messaging routes

* feat: add ban check to login/registration

* fix(ci/api): mock KeyvMongo to avoid tests hanging

* docs: update .env.example
> refactor(banViolation): calculate interval rate crossover, early return if duration is invalid
ci(banViolation): add tests to ensure users are only banned when expected

* docs: improve wording for mod system

* feat: add configurable env variables for violation scores

* chore: add jsdoc for uaParser.js

* chore: improve ban text log

* chore: update bun test scripts

* refactor(math.js): add fallback values

* fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead
refactor(getLogStores): get a single log store by type

* fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0

* fix(banViolation.js): getLogStores import

* feat: handle 500 code error at login

* fix(middleware): handle case where user.id is _id and not just id

* ci: add ban secrets for backend unit tests

* refactor: logout user upon ban

* chore: log session delete message only if deletedCount > 0

* refactor: change default ban duration (2h) and make logic more clear in JSDOC

* fix: login and registration limiters will now return rate limiting error

* fix: userId not parsable as non ObjectId string

* feat: add useTimeout hook to properly clear timeouts when invoking functions within them
refactor(AuthContext): cleanup code by using new hook and defining types in ~/common

* fix: login error message for rate limits

* docs: add info for automated mod system and rate limiters, update other docs accordingly

* chore: bump data-provider version
2023-09-13 10:57:07 -04:00