Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-21 19:00:13 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)

Browse source 
* Basic implementation of ChatGPT conversation import

* remove debug code

* Handle citations

* Fix updatedAt in import

* update default model

* Use job scheduler to handle import requests

* import job status endpoint

* Add wrapper around Agenda

* Rate limits for import endpoint

* rename import api path

* Batch save import to mongo

* Improve naming

* Add documenting comments

* Test for importers

* Change button for importing conversations

* Frontend changes

* Import job status endpoint

* Import endpoint response

* Add translations to new phrases

* Fix conversations refreshing

* cleanup unused functions

* set timeout for import job status polling

* Add documentation

* get extra spaces back

* Improve error message

* Fix translation files after merge

* fix translation files 2

* Add zh translation for import functionality

* Sync mailisearch index after import

* chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests

* docs: fix links

* docs: fix conversationsImport section

* fix: user role issue for librechat imports

* refactor: import conversations from json
- organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route

* fix: undefined metadata edge case and replace ChatGtp -> ChatGpt

* Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt

* fix: chatgpt importer

* feat: maintain tree relationship for librechat messages

* chore: use enum

* refactor: saveMessage to use single object arg, replace console logs, add userId to log message

* chore: additional comment

* chore: multer edge case

* feat: first pass, maintain tree relationship

* chore: organize

* chore: remove log

* ci: add heirarchy test for chatgpt

* ci: test maintaining of heirarchy for librechat

* wip: allow non-text content type messages

* refactor: import content part object json string

* refactor: more content types to format

* chore: consolidate messageText formatting

* docs: update on changes, bump data-provider/config versions, update readme

* refactor(indexSync): singleton pattern for MeiliSearchClient

* refactor: debug log after batch is done

* chore: add back indexSync error handling

---------

Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
This commit is contained in:
Denis Palnitsky 2024-05-02 08:48:26 +02:00 • committed by GitHub
parent 3b44741cf9
commit ab6fbe48f1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
64 changed files with 3795 additions and 98 deletions
Download patch file Download diff file Expand all files Collapse all files

27
docs/features/conversations_import.md Normal file
View file

@ -0,0 +1,27 @@
---
title: 📥 Import conversations from other chats
description: Conversations Import allows user to import conversations exported from other GPT chat applications. Currently, we support importing conversations from ChatGPT, ChatbotUI v1, and LibreChat itself.
weight: -1
---
Conversations Import allows user to import conversations exported from other GPT chat applications. Currently, we support importing conversations from ChatGPT, [ChatbotUI v1](https://github.com/mckaywrigley/chatbot-ui/tree/b865b0555f53957e96727bc0bbb369c9eaecd83b?tab=readme-ov-file#legacy-code), and LibreChat itself.
Import functionality is available in the "Settings" -> "Data Controls" section.
![image](https://github.com/danny-avila/LibreChat/assets/154290/205d70fd-8fbd-4ae4-a1f6-8baf97553dbe)
# How to import conversations from Chat GPT
1. Follow the [ChatGPT export instructions](https://help.openai.com/en/articles/7260999-how-do-i-export-my-chatgpt-history-and-data) to export your conversations.
2. You should get a link to download archive in you email.
3. Download the archive. It should be a zip file with random name like: _d119d98bb3711aff7a2c73bcc7ea53d96c984650d8f7e033faef78386a9907-2024-01-01-10-30-00.zip_
4. Extract the content of the zip file.
5. Navigate to LibreChat Settings -> Data Controls
![image](https://github.com/danny-avila/LibreChat/assets/154290/205d70fd-8fbd-4ae4-a1f6-8baf97553dbe)
6. Click on the "Import" button and select `conversations.json` file from the extracted archive. It will start importing the conversations.
7. Shortly you will get a notification that the import is complete.
![image](https://github.com/danny-avila/LibreChat/assets/154290/7d3b6766-db76-41d0-aa26-4fab8577353d)
## Sharing on Discord
Join us on [discord](https://discord.librechat.ai) and see our **[#presets ](https://discord.com/channels/1086345563026489514/1093249324797935746)** channel where thousands of presets are shared by users worldwide. Check out pinned posts for popular presets!

1
docs/features/index.md
View file

@ -11,6 +11,7 @@ weight: 2
* 🤖[Custom Endpoints](../install/configuration/custom_config.md)
* 🗃️ [RAG API (Chat with Files)](./rag_api.md)
* 🔖 [Presets](./presets.md)
* 📥 [Import conversations from other chats](./conversations_import.md)
* 🔌[Plugins](./plugins/index.md)
* 🔌 [Introduction](./plugins/introduction.md)
* 🛠️ [Make Your Own](./plugins/make_your_own.md)

4
docs/features/plugins/make_your_own.md
View file

@ -30,7 +30,7 @@ Here are the key takeaways for creating your own plugin:
**3.** [**Define Helper Methods:**](make_your_own.md#step-3-define-helper-methods) Define helper methods within your class to handle specific tasks if needed.
**4.** [**Implement the `_call` Method:**](make_your_own.md#step-4-implement-the-_call-method) Implement the `_call` method where the main functionality of your plugin is defined. This method is called when the language model decides to use your plugin. It should take an `input` parameter and return a result. If an error occurs, the function should return a string representing an error, rather than throwing an error. If your plugin requires multiple inputs from the LLM, read the [StructuredTools](#StructuredTools) section.
**4.** [**Implement the `_call` Method:**](make_your_own.md#step-4-implement-the-_call-method) Implement the `_call` method where the main functionality of your plugin is defined. This method is called when the language model decides to use your plugin. It should take an `input` parameter and return a result. If an error occurs, the function should return a string representing an error, rather than throwing an error. If your plugin requires multiple inputs from the LLM, read the [StructuredTools](#structuredtools) section.
**5.** [**Export Your Plugin and Import into handleTools.js:**](make_your_own.md#step-5-export-your-plugin-and-import-into-handletoolsjs) Export your plugin and import it into `handleTools.js`. Add your plugin to the `toolConstructors` object in the `loadTools` function. If your plugin requires more advanced initialization, add it to the `customConstructors` object.
@ -132,7 +132,7 @@ class StableDiffusionAPI extends Tool {
The `_call` method is where the main functionality of your plugin is implemented. This method is called when the language model decides to use your plugin. It should take an `input` parameter and return a result.
> In a basic Tool, the LLM will generate one string value as an input. If your plugin requires multiple inputs from the LLM, read the **[StructuredTools](#StructuredTools)** section.
> In a basic Tool, the LLM will generate one string value as an input. If your plugin requires multiple inputs from the LLM, read the **[StructuredTools](#structuredtools)** section.
```javascript
class StableDiffusionAPI extends Tool {