Commit graph

21 commits

Author SHA1 Message Date
Danny Avila
7b2cedf5ff
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903)
* refactor: require Auth middleware in route index files

* feat: concurrent message limiter

* feat: complete concurrent message limiter with caching

* refactor: SSE response methods separated from handleText

* fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body

* chore: minor name changes

* refactor: add isUUID condition to saveMessage

* fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization

* chore: bump keyv and remove console.log from Message

* fix(concurrentLimiter): ensure messages are only saved in later message children

* refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future

* feat: add denyRequest function for error responses

* feat(utils): add isStringTruthy function

Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true'

* feat: add optional message rate limiters by IP and userId

* feat: add optional message rate limiters by IP and userId to edit route

* refactor: rename isStringTruthy to isTrue for brevity

* refactor(getError): use map to make code cleaner

* refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation

* feat: check if errorMessage is object, stringify if so

* chore: send object to denyRequest which will stringify it

* feat: log excessive requests

* fix(getError): correctly pluralize messages

* refactor(limiters): make type consistent between logs and errorMessage

* refactor(cache): move files out of lib/db into separate cache dir
>> feat: add getLogStores function so Keyv instance is not redundantly created on every violation
feat: separate violation logging to own function with logViolation

* fix: cache/index.js export, properly record userViolations

* refactor(messageLimiters): use new logging method, add logging to registrations

* refactor(logViolation): make userLogs an array of logs per user

* feat: add logging to login limiter

* refactor: pass req as first param to logViolation and record offending IP

* refactor: rename isTrue helper fn to isEnabled

* feat: add simple non_browser check and log violation

* fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch

* chore: adjust nodemon ignore paths to properly ignore logs

* feat: add math helper function for safe use of eval

* refactor(api/convos): use middleware at top of file to avoid redundancy

* feat: add delete all static method for Sessions

* fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case)

* refactor(getLogStores): adjust return type

* feat: add ban violation and check ban logic
refactor(logViolation): pass both req and res objects

* feat: add removePorts helper function

* refactor: rename getError to getMessageError and add getLoginError for displaying different login errors

* fix(AuthContext): fix type issue and remove unused code

* refactor(bans): ban by ip and user id, send response based on origin

* chore: add frontend ban messages

* refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy

* feat: add ban check to AI messaging routes

* feat: add ban check to login/registration

* fix(ci/api): mock KeyvMongo to avoid tests hanging

* docs: update .env.example
> refactor(banViolation): calculate interval rate crossover, early return if duration is invalid
ci(banViolation): add tests to ensure users are only banned when expected

* docs: improve wording for mod system

* feat: add configurable env variables for violation scores

* chore: add jsdoc for uaParser.js

* chore: improve ban text log

* chore: update bun test scripts

* refactor(math.js): add fallback values

* fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead
refactor(getLogStores): get a single log store by type

* fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0

* fix(banViolation.js): getLogStores import

* feat: handle 500 code error at login

* fix(middleware): handle case where user.id is _id and not just id

* ci: add ban secrets for backend unit tests

* refactor: logout user upon ban

* chore: log session delete message only if deletedCount > 0

* refactor: change default ban duration (2h) and make logic more clear in JSDOC

* fix: login and registration limiters will now return rate limiting error

* fix: userId not parsable as non ObjectId string

* feat: add useTimeout hook to properly clear timeouts when invoking functions within them
refactor(AuthContext): cleanup code by using new hook and defining types in ~/common

* fix: login error message for rate limits

* docs: add info for automated mod system and rate limiters, update other docs accordingly

* chore: bump data-provider version
2023-09-13 10:57:07 -04:00
Fuegovic
1cd0fd9d5a
doc: Hugging Face Deployment (#867)
* docs: update ToC

* docs: update ToC

* update huggingface.md

* update render.md

* update huggingface.md

* update mongodb.md

* update huggingface.md

* update README.md
2023-09-01 08:12:35 -04:00
Fuegovic
66b8580487
docs: third-party tools (#848)
* docs: third-party tools

* docs: third-party tools

* Update third-party.md

* Update third-party.md

---------

Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
2023-08-28 09:18:25 -04:00
Fuegovic
29d3640546
docs: updates (#841) 2023-08-26 19:36:25 -04:00
Danny Avila
9ef1686e18
Update mkdocs.yml 2023-08-24 20:24:47 -04:00
Naosuke Yokoe
61dcb4d307
feat: Azure Cognitive Search Plugin (#815)
* feat(AzureCognitiveSearchPlugin)

* feat(tools/AzureCognitiveSearch.js): Add a new plugin (not structured
  version)
* feat(tools/structured/AzureCognitiveSearch.js): Add a new plugin (structured version)
* feat(tools/manifest.json, tools/index.js, tools/util/handleTools.js):
  Add configurations for the plugin
* feat(api/package.json, package-lock.json): Installed a new package for the
  plugin (@azure/search-documents)
* feat(.env.example): Add new environment variables for the plugin

Here is the link to the corresponding discussion page:
https://github.com/danny-avila/LibreChat/discussions/567

* docs(AzureCognitiveSearchPlugin)

* docs(features/plugins/azure_cognitive_search.md): Add a new document
  for the plugin

* (fix:.env.example)

* reverted extra whitespaces removed by the editor

* docs(mkdocs.yml)

* Add the Azure Cognitive Search Plugin's documentation item to
mkdocs.yml.
2023-08-19 07:11:31 -04:00
Danny Avila
6ea6f967ce
docs: DigitalOcean Deployment (#783)
* docs: wip: digitalocean guide

* feat(deployed-update.js): add script for updating deployed instance

docs(deployment/digitalocean.md): update instructions for Digital Ocean deployment

* fix(deployed-update.js): change docker-compose pull command to only pull api image
fix(digitalocean.md): update instructions to add user to docker group and start docker before running installation/update script

* feat(package.json): add 'update:deployed' script for deployed updates
docs: wip: digitalocean

* chore(package.json): add start:deployed and stop:deployed scripts for deploying with docker-compose

* docs(deployment/digitalocean.md): add instructions for stopping and starting the docker container

* docs(deployment/digitalocean.md): add instructions for stopping and starting the docker container
docs(deployment/digitalocean.md): add command for checking active docker containers
docs(deployment/digitalocean.md): provide guidance for troubleshooting before creating a new issue

* fix(deployed-update.js): refactor code to use getCurrentBranch function
feat(deployed-update.js): add support for rebasing current branch onto main branch
docs(digitalocean.md): update instructions for deploying with Docker on remote Ubuntu server
package.json: add rebase:deployed script to run deployed-update.js with --rebase flag

* fix(deployed-update.js): fix variable scope issue in deployed-update.js
docs(digitalocean.md): fix grammar and clarify instructions for editing NGINX file

* docs(deployment): add warning about potential merge conflicts when editing branch

docs(deployment): clarify that code changes in environment won't be reflected

* docs: Update digitalocean.md with images and revised instructions

* docs(digitalocean.md): formatting

* docs(digitalocean.md): add ToC

* docs(digitalocean.md): fix ToC

* Update mkdocs.yml
2023-08-10 10:03:59 -04:00
Danny Avila
d6dbd56e33
Update mkdocs.yml 2023-08-04 14:41:34 -04:00
Marco Beretta
0630b54193
docs: add how to add a language (#732)
* Create language-contributions.md

* Update language-contributions.md

* Update README.md

* Update mkdocs.yml

* Update README.md

* Update language-contributions.md

* Create languages.md

* Update languages.md

* Update README.md

* Update mkdocs.yml

* fix space languages.md

fix space at line 61
2023-08-01 08:14:01 -04:00
Marco Beretta
b5353e2640
Organize the getting started menu (#708)
* Create installation.md

* Delete installation.md

* Create installation.md

* Update installation.md

* Update README.md

* Update mkdocs.yml

* Update apis_and_tokens.md

* Update README.md

* Delete installation.md

* Update README.md

* Update mkdocs.yml
2023-07-27 08:05:49 -04:00
Danny Avila
3b865fbc59
Update mkdocs.yml 2023-07-23 20:34:41 -04:00
Fuegovic
ad29d25396
docs: updates (#662)
* docs: updates

* docs: updates

* Update Settings.jsx
2023-07-19 08:35:41 -07:00
Fuegovic
0246f164b0
docs: add "chatgpt_plugins_openapi.md" to readme.md and mkdocs (#655)
* docs: add chatgpt_plugins_openapi.md to toc

* docs: add chatgpt_plugins_openapi.md to mkdocs toc
2023-07-16 13:14:07 -04:00
Fuegovic
1a5144be76
Docs: Instruction to deploy on render.com (#638)
* Create render.md

* Update render.md

* Update mkdocs.yml

* Update render.md

* Update README.md

* Update render.md

* Update apis_and_tokens.md

add basic instruction for Anthropic Claude
2023-07-14 09:40:41 -04:00
Fuegovic
69d192bac3
Docs: assets clean up (#598)
* Update ngrok.md

* Update linode.md

* Update cloudflare.md

* Update testing.md

* Update google_search.md

* Update introduction.md

* Update stable_diffusion.md

* Update wolfram.md

* docs: assets clean up
2023-07-06 17:41:22 -04:00
Marco Beretta
3c1aeab340
Ngrok Documentation (#586)
* Ngrok doc

* Add files via upload

* Update README.md

* Update mkdocs.yml
2023-07-05 09:20:23 -04:00
Fuegovic
b89d46dac0
Update mkdocs.yml (#569)
fix the "breaking changes" page 404 error
2023-07-03 01:08:29 -04:00
Fuegovic
df2a68e1e7
Docs: updates & enhancements for MKDocs (#555)
* Update documents for mkdocs compatibility

* documents update

* documents update

* Update README.md

* Update README.md

add link to "https://docs.librechat.ai" on the logo

* document updates

* docs - badge updates

* docs - badge updates

* docs - badge updates

* Update docker_install.md

* Update .env.example

update default MONGO_URI to port 27018 so local install can communicate with the docker db

* Update windows_install.md

fix typo
2023-07-01 20:11:37 -04:00
bsu3338
4a0cf11c90
Add social sites to MkDocs (#554) 2023-06-27 08:57:40 -04:00
Marco Beretta
abd1b10b46
Enhanced Documentation: Added Cloudflare and Linode Setup (#549)
* Add files via upload

* Create linode-setup.md

* Create cloudflare-setup.md

* Update cloudflare-setup.md

* Delete 4-linode.png

* Delete 3-linode.png

* Add files via upload

* Add files via upload

* Update cloudflare-setup.md

* Update linode-setup.md

* Rename cloudflare-setup.md to cloudflare.md

* Rename linode-setup.md to linode.md

* Update mkdocs.yml

* Update cloudflare.md

* Update linode.md

* Update README.md

* Update README.md

* Update linode.md

sentence in Italian
2023-06-26 09:23:50 -04:00
bsu3338
fdc5265f48
MkDocs for Material (#545)
* Create mkdocs.yaml

* Create mkdocs.yml

* Update mkdocs.yml

* Update mkdocs.yml

* Update mkdocs.yml

* Update mkdocs.yml

* Update mkdocs.yml

* Update README.md

* Update coding_conventions.md

* Update documentation_guidelines.md

* Update testing.md

* Update heroku.md

* Update hetzner_ubuntu.md

* Update google_search.md

* Update introduction.md

* Update make_your_own.md

* Update stable_diffusion.md

* Update wolfram.md

* Update proxy.md

* Update user_auth_system.md

* Update bing_jailbreak_info.md

* Update breaking_changes.md

* Update multilingual_information.md

* Update project_origin.md

* Update tech_stack.md

* Update apis_and_tokens.md

* Update docker_install.md

* Update linux_install.md

* Update mac_install.md

* Update windows_install.md

* Update mkdocs.yml

* Update mkdocs.yml

* Update documentation_guidelines.md

* Add files via upload

* Create temp.txt

* Add files via upload

* Delete logo.png

* Create index.md

* Update mkdocs.yml

* Update mkdocs.yml

* Delete temp.txt

* Update README.md

* Update README.md

---------

Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
2023-06-24 23:00:10 -04:00