refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
const mongoose = require('mongoose');
|
|
|
|
|
|
2024-06-07 21:06:47 +02:00
|
|
|
/**
|
|
|
|
|
* @typedef {Object} MongoSession
|
|
|
|
|
* @property {string} [refreshToken] - The refresh token
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @typedef {Object} MongoUser
|
|
|
|
|
* @property {ObjectId} [_id] - MongoDB Document ID
|
|
|
|
|
* @property {string} [name] - The user's name
|
|
|
|
|
* @property {string} [username] - The user's username, in lowercase
|
|
|
|
|
* @property {string} email - The user's email address
|
|
|
|
|
* @property {boolean} emailVerified - Whether the user's email is verified
|
|
|
|
|
* @property {string} [password] - The user's password, trimmed with 8-128 characters
|
|
|
|
|
* @property {string} [avatar] - The URL of the user's avatar
|
|
|
|
|
* @property {string} provider - The provider of the user's account (e.g., 'local', 'google')
|
|
|
|
|
* @property {string} [role='USER'] - The role of the user
|
|
|
|
|
* @property {string} [googleId] - Optional Google ID for the user
|
|
|
|
|
* @property {string} [facebookId] - Optional Facebook ID for the user
|
|
|
|
|
* @property {string} [openidId] - Optional OpenID ID for the user
|
|
|
|
|
* @property {string} [ldapId] - Optional LDAP ID for the user
|
|
|
|
|
* @property {string} [githubId] - Optional GitHub ID for the user
|
|
|
|
|
* @property {string} [discordId] - Optional Discord ID for the user
|
|
|
|
|
* @property {Array} [plugins=[]] - List of plugins used by the user
|
|
|
|
|
* @property {Array.<MongoSession>} [refreshToken] - List of sessions with refresh tokens
|
|
|
|
|
* @property {Date} [expiresAt] - Optional expiration date of the file
|
|
|
|
|
* @property {Date} [createdAt] - Date when the user was created (added by timestamps)
|
|
|
|
|
* @property {Date} [updatedAt] - Date when the user was last updated (added by timestamps)
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/** @type {MongooseSchema<MongoSession>} */
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
const Session = mongoose.Schema({
|
|
|
|
|
refreshToken: {
|
|
|
|
|
type: String,
|
|
|
|
|
default: '',
|
|
|
|
|
},
|
|
|
|
|
});
|
|
|
|
|
|
2024-06-07 21:06:47 +02:00
|
|
|
/** @type {MongooseSchema<MongoUser>} */
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
const userSchema = mongoose.Schema(
|
|
|
|
|
{
|
|
|
|
|
name: {
|
|
|
|
|
type: String,
|
|
|
|
|
},
|
|
|
|
|
username: {
|
|
|
|
|
type: String,
|
|
|
|
|
lowercase: true,
|
|
|
|
|
default: '',
|
|
|
|
|
},
|
|
|
|
|
email: {
|
|
|
|
|
type: String,
|
|
|
|
|
required: [true, 'can\'t be blank'],
|
|
|
|
|
lowercase: true,
|
|
|
|
|
unique: true,
|
|
|
|
|
match: [/\S+@\S+\.\S+/, 'is invalid'],
|
|
|
|
|
index: true,
|
|
|
|
|
},
|
|
|
|
|
emailVerified: {
|
|
|
|
|
type: Boolean,
|
|
|
|
|
required: true,
|
|
|
|
|
default: false,
|
|
|
|
|
},
|
|
|
|
|
password: {
|
|
|
|
|
type: String,
|
|
|
|
|
trim: true,
|
|
|
|
|
minlength: 8,
|
|
|
|
|
maxlength: 128,
|
|
|
|
|
},
|
|
|
|
|
avatar: {
|
|
|
|
|
type: String,
|
|
|
|
|
required: false,
|
|
|
|
|
},
|
|
|
|
|
provider: {
|
|
|
|
|
type: String,
|
|
|
|
|
required: true,
|
|
|
|
|
default: 'local',
|
|
|
|
|
},
|
|
|
|
|
role: {
|
|
|
|
|
type: String,
|
|
|
|
|
default: 'USER',
|
|
|
|
|
},
|
|
|
|
|
googleId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
|
|
|
|
facebookId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
|
|
|
|
openidId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
2024-05-29 14:46:20 -07:00
|
|
|
ldapId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
githubId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
|
|
|
|
discordId: {
|
|
|
|
|
type: String,
|
|
|
|
|
unique: true,
|
|
|
|
|
sparse: true,
|
|
|
|
|
},
|
|
|
|
|
plugins: {
|
|
|
|
|
type: Array,
|
|
|
|
|
default: [],
|
|
|
|
|
},
|
|
|
|
|
refreshToken: {
|
|
|
|
|
type: [Session],
|
|
|
|
|
},
|
2024-06-07 21:06:47 +02:00
|
|
|
expiresAt: {
|
|
|
|
|
type: Date,
|
|
|
|
|
expires: 604800, // 7 days in seconds
|
|
|
|
|
},
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
},
|
|
|
|
|
{ timestamps: true },
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
module.exports = userSchema;
|