Commit graph

45 commits

Author SHA1 Message Date
Danny Avila
ea1dd59ef4
refactor(api): Central Logging 📜 (#1348)
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example

* WIP: Refactor logging statements in code

* WIP: Refactor logging statements and import configurations

* WIP: Refactor logging statements and import configurations

* refactor: broadcast Redis initialization message with `info` not `debug`

* refactor: complete Refactor logging statements and import configurations

* chore: delete unused tools

* fix: circular dependencies due to accessing logger

* refactor(handleText): handle booleans and write tests

* refactor: redact sensitive values, better formatting

* chore: improve log formatting, avoid passing strings to 2nd arg

* fix(ci): fix jest tests due to logger changes

* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time

* chore: update docs

* chore: update docs

* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose

* chore: spread objects where they are commonly logged to allow string truncation

* chore: improve error log formatting
2023-12-14 07:49:27 -05:00
Danny Avila
6358383001
feat(db & e2e): Enhance DB Schemas/Controllers and Improve E2E Tests (#966)
* feat: add global teardown to remove test data and add registration/log-out to auth flow

* refactor(models/Conversation): index user field and add JSDoc to deleteConvos

* refactor: add user index to message schema and ensure user is saved to each Message

* refactor: add user to each saveMessage call

* fix: handle case where title is null in zod schema

* feat(e2e): ensure messages are deleted on cleanUp

* fix: set last convo for all endpoints on conversation update

* fix: enable registration for CI env
2023-09-18 15:19:50 -04:00
Danny Avila
a2b6e9a6a8 fix(meilisearch): results will now properly paginate 2023-07-28 13:42:39 -04:00
Danny Avila
19af2b06ce
feat: utitlize lean queries, remove migration script, index createdAt timestamps (#698)
* feat(mongoDb): utitlize lean queries and index createdAt timestamps for cosmosDB support

* fix: remove unnecessary lean() method from deleteMany calls

* fix: remove unnecessary lean() method from deleteMany calls

* fix: remove lean() from queries that need hydration

* chore(migrateDb.js): remove unused migration script
fix(Preset.js): return lean documents when retrieving presets
refactor(index.js): remove migration script from server initialization
refactor(convos.js): remove toObject() when sending conversation object
refactor(presets.js): remove toObject() when sending presets object
2023-07-25 19:27:55 -04:00
Danny Avila
e5336039fc
ci(backend-review.yml): add linter step to the backend review workflow (#625)
* ci(backend-review.yml): add linter step to the backend review workflow

* chore(backend-review.yml): remove prettier from lint-action configuration

* chore: apply new linting workflow

* chore(lint-staged.config.js): reorder lint-staged tasks for JavaScript and TypeScript files

* chore(eslint): update ignorePatterns in .eslintrc.js
chore(lint-action): remove prettier option in backend-review.yml
chore(package.json): add lint and lint:fix scripts

* chore(lint-staged.config.js): remove prettier --write command for js, jsx, ts, tsx files

* chore(titleConvo.js): remove unnecessary console.log statement
chore(titleConvo.js): add missing comma in options object

* chore: apply linting to all files

* chore(lint-staged.config.js): update lint-staged configuration to include prettier formatting
2023-07-14 09:36:49 -04:00
Dan Orlando
a17b878617
refactor: reformat files to require parens around params (#316) 2023-05-18 14:44:07 -04:00
Danny Avila
5d40396fb2
refactor(Conversation.js): change default pageSize from 12 to 14 in getConvosByPage and getConvosQueried functions. Remove unnecessary parentheses and curly braces in getConvosQueried function. Remove unnecessary parentheses in deleteConvos function. (#267) 2023-05-14 11:45:18 -04:00
Dan Orlando
3b4ed98c1d
fix browser refresh redirecting to /chat/new (#210) 2023-05-07 19:16:12 -04:00
Dan Orlando
dac19038a3
feat: Auth and User System (#205)
* server-side JWT auth implementation

* move oauth routes and strategies, fix bugs

* backend modifications for wiring up the frontend login and reg forms

* Add frontend data services for login and registration

* Add login and registration forms

* Implment auth context, functional client side auth

* protect routes with jwt auth

* finish local strategy (using local storage)

* Start setting up google auth

* disable token refresh, remove old auth middleware

* refactor client, add ApiErrorBoundary context

* disable google and facebook strategies

* fix: fix presets not displaying specific to user

* fix: fix issue with browser refresh

* fix: casing issue with User.js (#11)

* delete user.js to be renamed

* fix: fix casing issue with User.js

* comment out api error watcher temporarily

* fix: issue with api error watcher (#12)

* delete user.js to be renamed

* fix: fix casing issue with User.js

* comment out api error watcher temporarily

* feat: add google auth social login

* fix: make google login url dynamic based on dev/prod

* fix: bug where UI is briefly displayed before redirecting to login

* fix: fix cookie expires value for local auth

* Update README.md

* Update LOCAL_INSTALL structure

* Add local testing instructions

* Only load google strategy if client id and secret are provided

* Update .env.example files with new params

* fix issue with not redirecting to register form

* only show google login button if value is set in .env

* cleanup log messages

* Add label to button for google login on login form

* doc: fix client/server url values in .env.example

* feat: add error message details to registration failure

* Restore preventing paste on confirm password

* auto-login user after registering

* feat: forgot password (#24)

* make login/reg pages look like openai's

* add password reset data services

* new form designs similar to openai, add password reset pages

* add api's for password reset

* email utils for password reset

* remove bcrypt salt rounds from process.env

* refactor: restructure api auth code, consolidate routes (#25)

* add api's for password reset

* remove bcrypt salt rounds from process.env

* refactor: consolidate auth routes, use controller pattern

* refactor: code cleanup

* feat: migrate data to first user (#26)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes after refactor (#27)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: issue with auto-login when logging out then logging in with new browser window (#28)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: fix issue with auto-login in new tab

* doc: Update README and .env.example files with user system information (#29)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: fix issue with auto-login in new tab

* doc: update README and .env.example files

* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)

Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>

* feat: send user with completion to protect against abuse (#31)

* Fixup: LOCAL_INSTALL.md PS instructions (#200)

* server-side JWT auth implementation

* move oauth routes and strategies, fix bugs

* backend modifications for wiring up the frontend login and reg forms

* Add frontend data services for login and registration

* Add login and registration forms

* Implment auth context, functional client side auth

* protect routes with jwt auth

* finish local strategy (using local storage)

* Start setting up google auth

* disable token refresh, remove old auth middleware

* refactor client, add ApiErrorBoundary context

* disable google and facebook strategies

* fix: fix presets not displaying specific to user

* fix: fix issue with browser refresh

* fix: casing issue with User.js (#11)

* delete user.js to be renamed

* fix: fix casing issue with User.js

* comment out api error watcher temporarily

* feat: add google auth social login

* fix: make google login url dynamic based on dev/prod

* fix: bug where UI is briefly displayed before redirecting to login

* fix: fix cookie expires value for local auth

* Only load google strategy if client id and secret are provided

* Update .env.example files with new params

* fix issue with not redirecting to register form

* only show google login button if value is set in .env

* cleanup log messages

* Add label to button for google login on login form

* doc: fix client/server url values in .env.example

* feat: add error message details to registration failure

* Restore preventing paste on confirm password

* auto-login user after registering

* feat: forgot password (#24)

* make login/reg pages look like openai's

* add password reset data services

* new form designs similar to openai, add password reset pages

* add api's for password reset

* email utils for password reset

* remove bcrypt salt rounds from process.env

* refactor: restructure api auth code, consolidate routes (#25)

* add api's for password reset

* remove bcrypt salt rounds from process.env

* refactor: consolidate auth routes, use controller pattern

* refactor: code cleanup

* feat: migrate data to first user (#26)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes after refactor (#27)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: issue with auto-login when logging out then logging in with new browser window (#28)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: fix issue with auto-login in new tab

* doc: Update README and .env.example files with user system information (#29)

* refactor: use /api for auth routes

* fix: use user id instead of username

* feat: migrate data to first user on register

* fix: fix social login routes

* fix: fix issue with auto-login in new tab

* doc: update README and .env.example files

* Send user id to openai to protect against abuse

* add meilisearch to gitignore

* Remove webpack

---------

Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>

---------

Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
Wentao Lyu
a5a0eab7f7 merge from dannya
feat: support unfinished messages.
2023-04-11 03:26:38 +08:00
Wentao Lyu
adcc021c9e feat: feat: new endpoint-style submit 2023-03-31 03:22:57 +08:00
Wentao Lyu
e796a19136 fix: remove related messages when deleting conversations. 2023-03-29 13:32:01 +08:00
Daniel Avila
0d7300be9b fix: chatgptBrowser handling and ask.js refactor 2023-03-28 22:28:43 -04:00
Wentao Lyu
d8ccc5b870 fix: clearConvo will remove all messages 2023-03-28 01:19:44 +08:00
Daniel Avila
34bef48e84 fix: correctly searches bing messages 2023-03-24 18:26:59 -04:00
Daniel Avila
97634865eb feat: syncs across document deletions 2023-03-22 19:52:38 -04:00
Danny Avila
8f58c95452 feat: main styling/ui/ux final changes 2023-03-22 16:06:11 -04:00
Daniel Avila
a5cf2f9148 search: correctly register/export schema/models for mongoMeili 2023-03-22 01:31:01 -04:00
Daniel Avila
194051e424 feat: api will plugin mongoMeili without a valid connection 2023-03-21 20:06:27 -04:00
Daniel Avila
0b75d5d6fe chore: another hotfix patch on getConvoTitle 2023-03-20 02:30:08 -04:00
Daniel Avila
80ca3bc375 chore: hotfix for browser client 2023-03-20 01:35:02 -04:00
Daniel Avila
b0936fa322 chore: unplug meilisearch, add leading option to throttle 2023-03-20 00:48:16 -04:00
Daniel Avila
d56aa2edef loads up to 20 messages, debugging markdown issue 2023-03-18 23:18:36 -04:00
Daniel Avila
4197a92609 feat: search working as expected 2023-03-18 17:49:24 -04:00
Daniel Avila
b97594c000 fix: conflicting fetch with /api/convos 2023-03-18 14:28:10 -04:00
Daniel Avila
0f54ffd8b4 feat: search bar working, still in progress 2023-03-18 01:40:49 -04:00
Daniel Avila
610cba4a60 backend logic drafted, moving to frontend 2023-03-17 22:20:36 -04:00
Daniel Avila
4f5ee8b198 move db functions 2023-03-17 19:58:13 -04:00
Danny Avila
915cda70ef fix: user:user redundancies 2023-03-16 13:32:04 -04:00
Wentao Lyu
b0284b6974 sync updates and merge with feat-resubmit 2023-03-16 13:20:54 +08:00
Wentao Lyu
62d88380e0 feat: add sample multi-user support
feat: update README
2023-03-16 13:20:34 +08:00
Daniel Avila
6192c2964e fix: validation to avoid saving customGpt params to non-custom models 2023-03-14 20:14:38 -04:00
Daniel Avila
626a8fbd8e fix: if db is empty, will never show new convos until refresh 2023-03-14 18:53:46 -04:00
Danny Avila
c230fe41f4 Merge branch 'feat-resubmit' into refactors 2023-03-14 16:09:25 -04:00
Wentao Lyu
8289558d94 feat: pagination in nav 2023-03-15 04:05:14 +08:00
Danny Avila
9a17e94f8f fix: refactor migration and sort old convos correctly 2023-03-14 14:51:26 -04:00
Wentao Lyu
644f3f716f feat: re-orginazed three ask api. To provide ability to reproduce message.
feat: bing and sydney come to work again, [need more test]
2023-03-15 02:15:46 +08:00
Wentao Lyu
7168498543 fix: jailbreakConversationId=false response will be saved jailbreakConversationId='false' in database. 2023-03-15 02:15:46 +08:00
Daniel Avila
4fd05e15b4 fix: migrate old schema to new 2023-03-15 02:15:46 +08:00
Wentao Lyu
0ed8a40a41 feat: merge all message.id into message.messageId
feat: the first message will have a parentMessageId as 00000000-0000-0000-0000-000000000000 (in order not to create new convo when resubmit)
feat: ask will return the userMessage as well, to send back the messageId

TODO: comment out the title generation.
TODO: bing version need to be test

fix: never use the same messageId
fix: never delete exist messages
fix: connect response.parentMessageId to the userMessage.messageId
fix: set default convo title as new Chat
2023-03-15 02:15:46 +08:00
Wentao Lyu
dd1f74da72 replace all created to timestamps: true in db 2023-03-15 02:15:46 +08:00
Daniel Avila
67054e7504 working sydney, need to test the other models 2023-03-08 22:30:29 -05:00
Daniel Avila
69b3edc52c feat: sydney is functional 2023-03-08 19:47:23 -05:00
Danny Avila
da61a2841f fix: customGpt persistence blindspot, also remove free gpt 2023-03-07 14:22:33 -05:00
Danny Avila
fca546af63 reorganize dirs for dockerize 2023-03-06 10:15:07 -05:00
Renamed from models/Conversation.js (Browse further)