refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)

* docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2025-12-17 08:50:15 +01:00 · 2023-09-06 10:46:27 -04:00 · 2023-09-06 10:46:27 -04:00 · 4ca43fb53d
commit 4ca43fb53d
parent 64f1557852
122 changed files with 1933 additions and 966 deletions
--- a/api/models/schema/userSchema.js
+++ b/api/models/schema/userSchema.js
@ -0,0 +1,88 @@
+const mongoose = require('mongoose');
+
+const Session = mongoose.Schema({
+  refreshToken: {
+    type: String,
+    default: '',
+  },
+});
+
+const userSchema = mongoose.Schema(
+  {
+    name: {
+      type: String,
+    },
+    username: {
+      type: String,
+      lowercase: true,
+      default: '',
+    },
+    email: {
+      type: String,
+      required: [true, 'can\'t be blank'],
+      lowercase: true,
+      unique: true,
+      match: [/\S+@\S+\.\S+/, 'is invalid'],
+      index: true,
+    },
+    emailVerified: {
+      type: Boolean,
+      required: true,
+      default: false,
+    },
+    password: {
+      type: String,
+      trim: true,
+      minlength: 8,
+      maxlength: 128,
+    },
+    avatar: {
+      type: String,
+      required: false,
+    },
+    provider: {
+      type: String,
+      required: true,
+      default: 'local',
+    },
+    role: {
+      type: String,
+      default: 'USER',
+    },
+    googleId: {
+      type: String,
+      unique: true,
+      sparse: true,
+    },
+    facebookId: {
+      type: String,
+      unique: true,
+      sparse: true,
+    },
+    openidId: {
+      type: String,
+      unique: true,
+      sparse: true,
+    },
+    githubId: {
+      type: String,
+      unique: true,
+      sparse: true,
+    },
+    discordId: {
+      type: String,
+      unique: true,
+      sparse: true,
+    },
+    plugins: {
+      type: Array,
+      default: [],
+    },
+    refreshToken: {
+      type: [Session],
+    },
+  },
+  { timestamps: true },
+);
+
+module.exports = userSchema;