Merge branch 'main' into added-codeql

This commit is contained in:
Ruben Talstra 2025-05-14 21:34:53 +02:00 committed by GitHub
commit c80507d1a6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
752 changed files with 54455 additions and 19896 deletions

View file

@ -24,22 +24,40 @@ Project maintainers have the right and responsibility to remove, edit, or reject
## To contribute to this project, please adhere to the following guidelines:
## 1. Development notes
## 1. Development Setup
1. Before starting work, make sure your main branch has the latest commits with `npm run update`
2. Run linting command to find errors: `npm run lint`. Alternatively, ensure husky pre-commit checks are functioning.
1. Use Node.JS 20.x.
2. Install typescript globally: `npm i -g typescript`.
3. Run `npm ci` to install dependencies.
4. Build the data provider: `npm run build:data-provider`.
5. Build MCP: `npm run build:mcp`.
6. Build data schemas: `npm run build:data-schemas`.
7. Setup and run unit tests:
- Copy `.env.test`: `cp api/test/.env.test.example api/test/.env.test`.
- Run backend unit tests: `npm run test:api`.
- Run frontend unit tests: `npm run test:client`.
8. Setup and run integration tests:
- Build client: `cd client && npm run build`.
- Create `.env`: `cp .env.example .env`.
- Install [MongoDB Community Edition](https://www.mongodb.com/docs/manual/administration/install-community/), ensure that `mongosh` connects to your local instance.
- Run: `npx install playwright`, then `npx playwright install`.
- Copy `config.local`: `cp e2e/config.local.example.ts e2e/config.local.ts`.
- Copy `librechat.yaml`: `cp librechat.example.yaml librechat.yaml`.
- Run: `npm run e2e`.
## 2. Development Notes
1. Before starting work, make sure your main branch has the latest commits with `npm run update`.
3. Run linting command to find errors: `npm run lint`. Alternatively, ensure husky pre-commit checks are functioning.
3. After your changes, reinstall packages in your current branch using `npm run reinstall` and ensure everything still works.
- Restart the ESLint server ("ESLint: Restart ESLint Server" in VS Code command bar) and your IDE after reinstalling or updating.
4. Clear web app localStorage and cookies before and after changes.
5. For frontend changes:
- Install typescript globally: `npm i -g typescript`.
- Compile typescript before and after changes to check for introduced errors: `cd client && tsc --noEmit`.
6. Run tests locally:
- Backend unit tests: `npm run test:api`
- Frontend unit tests: `npm run test:client`
- Integration tests: `npm run e2e` (requires playwright installed, `npx install playwright`)
5. For frontend changes, compile typescript before and after changes to check for introduced errors: `cd client && npm run build`.
6. Run backend unit tests: `npm run test:api`.
7. Run frontend unit tests: `npm run test:client`.
8. Run integration tests: `npm run e2e`.
## 2. Git Workflow
## 3. Git Workflow
We utilize a GitFlow workflow to manage changes to this project's codebase. Follow these general steps when contributing code:
@ -49,7 +67,7 @@ We utilize a GitFlow workflow to manage changes to this project's codebase. Foll
4. Submit a pull request with a clear and concise description of your changes and the reasons behind them.
5. We will review your pull request, provide feedback as needed, and eventually merge the approved changes into the main branch.
## 3. Commit Message Format
## 4. Commit Message Format
We follow the [semantic format](https://gist.github.com/joshbuchea/6f47e86d2510bce28f8e7f42ae84c716) for commit messages.
@ -76,7 +94,7 @@ feat: add hat wobble
```
## 4. Pull Request Process
## 5. Pull Request Process
When submitting a pull request, please follow these guidelines:
@ -91,7 +109,7 @@ Ensure that your changes meet the following criteria:
- The commit history is clean and easy to follow. You can use `git rebase` or `git merge --squash` to clean your commit history before submitting the pull request.
- The pull request description clearly outlines the changes and the reasons behind them. Be sure to include the steps to test the pull request.
## 5. Naming Conventions
## 6. Naming Conventions
Apply the following naming conventions to branches, labels, and other Git-related entities:
@ -100,7 +118,7 @@ Apply the following naming conventions to branches, labels, and other Git-relate
- **JS/TS:** Directories and file names: Descriptive and camelCase. First letter uppercased for React files (e.g., `helperFunction.ts, ReactComponent.tsx`).
- **Docs:** Directories and file names: Descriptive and snake_case (e.g., `config_files.md`).
## 6. TypeScript Conversion
## 7. TypeScript Conversion
1. **Original State**: The project was initially developed entirely in JavaScript (JS).
@ -126,7 +144,7 @@ Apply the following naming conventions to branches, labels, and other Git-relate
- **Current Stance**: At present, this backend transition is of lower priority and might not be pursued.
## 7. Module Import Conventions
## 8. Module Import Conventions
- `npm` packages first,
- from shortest line (top) to longest (bottom)

View file

@ -79,6 +79,8 @@ body:
For UI-related issues, browser console logs can be very helpful. You can provide these as screenshots or paste the text here.
render: shell
validations:
required: true
- type: textarea
id: screenshots
attributes:

View file

@ -0,0 +1,42 @@
name: Locize Translation Access Request
description: Request access to an additional language in Locize for LibreChat translations.
title: "Locize Access Request: "
labels: ["🌍 i18n", "🔑 access request"]
body:
- type: markdown
attributes:
value: |
Thank you for your interest in contributing to LibreChat translations!
Please fill out the form below to request access to an additional language in **Locize**.
**🔗 Available Languages:** [View the list here](https://www.librechat.ai/docs/translation)
**📌 Note:** Ensure that the requested language is supported before submitting your request.
- type: input
id: account_name
attributes:
label: Locize Account Name
description: Please provide your Locize account name (e.g., John Doe).
placeholder: e.g., John Doe
validations:
required: true
- type: input
id: language_requested
attributes:
label: Language Code (ISO 639-1)
description: |
Enter the **ISO 639-1** language code for the language you want to translate into.
Example: `es` for Spanish, `zh-Hant` for Traditional Chinese.
**🔗 Reference:** [Available Languages](https://www.librechat.ai/docs/translation)
placeholder: e.g., es
validations:
required: true
- type: checkboxes
id: agreement
attributes:
label: Agreement
description: By submitting this request, you confirm that you will contribute responsibly and adhere to the project guidelines.
options:
- label: I agree to use my access solely for contributing to LibreChat translations.
required: true

View file

@ -1,50 +0,0 @@
name: Question
description: Ask your question
title: "[Question]: "
labels: ["❓ question"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill this!
- type: textarea
id: what-is-your-question
attributes:
label: What is your question?
description: Please give as many details as possible
placeholder: Please give as many details as possible
validations:
required: true
- type: textarea
id: more-details
attributes:
label: More Details
description: Please provide more details if needed.
placeholder: Please provide more details if needed.
validations:
required: true
- type: dropdown
id: browsers
attributes:
label: What is the main subject of your question?
multiple: true
options:
- Documentation
- Installation
- UI
- Endpoints
- User System/OAuth
- Other
- type: textarea
id: screenshots
attributes:
label: Screenshots
description: If applicable, add screenshots to help explain your problem. You can drag and drop, paste images directly here or link to them.
- type: checkboxes
id: terms
attributes:
label: Code of Conduct
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/danny-avila/LibreChat/blob/main/.github/CODE_OF_CONDUCT.md)
options:
- label: I agree to follow this project's Code of Conduct
required: true

60
.github/configuration-release.json vendored Normal file
View file

@ -0,0 +1,60 @@
{
"categories": [
{
"title": "### ✨ New Features",
"labels": ["feat"]
},
{
"title": "### 🌍 Internationalization",
"labels": ["i18n"]
},
{
"title": "### 👐 Accessibility",
"labels": ["a11y"]
},
{
"title": "### 🔧 Fixes",
"labels": ["Fix", "fix"]
},
{
"title": "### ⚙️ Other Changes",
"labels": ["ci", "style", "docs", "refactor", "chore"]
}
],
"ignore_labels": [
"🔁 duplicate",
"📊 analytics",
"🌱 good first issue",
"🔍 investigation",
"🙏 help wanted",
"❌ invalid",
"❓ question",
"🚫 wontfix",
"🚀 release",
"version"
],
"base_branches": ["main"],
"sort": {
"order": "ASC",
"on_property": "mergedAt"
},
"label_extractor": [
{
"pattern": "^(?:[^A-Za-z0-9]*)(feat|fix|chore|docs|refactor|ci|style|a11y|i18n)\\s*:",
"target": "$1",
"flags": "i",
"on_property": "title",
"method": "match"
},
{
"pattern": "^(?:[^A-Za-z0-9]*)(v\\d+\\.\\d+\\.\\d+(?:-rc\\d+)?).*",
"target": "version",
"flags": "i",
"on_property": "title",
"method": "match"
}
],
"template": "## [#{{TO_TAG}}] - #{{TO_TAG_DATE}}\n\nChanges from #{{FROM_TAG}} to #{{TO_TAG}}.\n\n#{{CHANGELOG}}\n\n[See full release details][release-#{{TO_TAG}}]\n\n[release-#{{TO_TAG}}]: https://github.com/#{{OWNER}}/#{{REPO}}/releases/tag/#{{TO_TAG}}\n\n---",
"pr_template": "- #{{TITLE}} by **@#{{AUTHOR}}** in [##{{NUMBER}}](#{{URL}})",
"empty_template": "- no changes"
}

68
.github/configuration-unreleased.json vendored Normal file
View file

@ -0,0 +1,68 @@
{
"categories": [
{
"title": "### ✨ New Features",
"labels": ["feat"]
},
{
"title": "### 🌍 Internationalization",
"labels": ["i18n"]
},
{
"title": "### 👐 Accessibility",
"labels": ["a11y"]
},
{
"title": "### 🔧 Fixes",
"labels": ["Fix", "fix"]
},
{
"title": "### ⚙️ Other Changes",
"labels": ["ci", "style", "docs", "refactor", "chore"]
}
],
"ignore_labels": [
"🔁 duplicate",
"📊 analytics",
"🌱 good first issue",
"🔍 investigation",
"🙏 help wanted",
"❌ invalid",
"❓ question",
"🚫 wontfix",
"🚀 release",
"version",
"action"
],
"base_branches": ["main"],
"sort": {
"order": "ASC",
"on_property": "mergedAt"
},
"label_extractor": [
{
"pattern": "^(?:[^A-Za-z0-9]*)(feat|fix|chore|docs|refactor|ci|style|a11y|i18n)\\s*:",
"target": "$1",
"flags": "i",
"on_property": "title",
"method": "match"
},
{
"pattern": "^(?:[^A-Za-z0-9]*)(v\\d+\\.\\d+\\.\\d+(?:-rc\\d+)?).*",
"target": "version",
"flags": "i",
"on_property": "title",
"method": "match"
},
{
"pattern": "^(?:[^A-Za-z0-9]*)(action)\\b.*",
"target": "action",
"flags": "i",
"on_property": "title",
"method": "match"
}
],
"template": "## [Unreleased]\n\n#{{CHANGELOG}}\n\n---",
"pr_template": "- #{{TITLE}} by **@#{{AUTHOR}}** in [##{{NUMBER}}](#{{URL}})",
"empty_template": "- no changes"
}

View file

@ -39,6 +39,9 @@ jobs:
- name: Install MCP Package
run: npm run build:mcp
- name: Install Data Schemas Package
run: npm run build:data-schemas
- name: Create empty auth.json file
run: |
mkdir -p api/data
@ -61,4 +64,7 @@ jobs:
run: cd api && npm run test:ci
- name: Run librechat-data-provider unit tests
run: cd packages/data-provider && npm run test:ci
run: cd packages/data-provider && npm run test:ci
- name: Run librechat-mcp unit tests
run: cd packages/mcp && npm run test:ci

58
.github/workflows/data-schemas.yml vendored Normal file
View file

@ -0,0 +1,58 @@
name: Publish `@librechat/data-schemas` to NPM
on:
push:
branches:
- main
paths:
- 'packages/data-schemas/package.json'
workflow_dispatch:
inputs:
reason:
description: 'Reason for manual trigger'
required: false
default: 'Manual publish requested'
jobs:
build-and-publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
- name: Install dependencies
run: cd packages/data-schemas && npm ci
- name: Build
run: cd packages/data-schemas && npm run build
- name: Set up npm authentication
run: echo "//registry.npmjs.org/:_authToken=${{ secrets.PUBLISH_NPM_TOKEN }}" > ~/.npmrc
- name: Check version change
id: check
working-directory: packages/data-schemas
run: |
PACKAGE_VERSION=$(node -p "require('./package.json').version")
PUBLISHED_VERSION=$(npm view @librechat/data-schemas version 2>/dev/null || echo "0.0.0")
if [ "$PACKAGE_VERSION" = "$PUBLISHED_VERSION" ]; then
echo "No version change, skipping publish"
echo "skip=true" >> $GITHUB_OUTPUT
else
echo "Version changed, proceeding with publish"
echo "skip=false" >> $GITHUB_OUTPUT
fi
- name: Pack package
if: steps.check.outputs.skip != 'true'
working-directory: packages/data-schemas
run: npm pack
- name: Publish
if: steps.check.outputs.skip != 'true'
working-directory: packages/data-schemas
run: npm publish *.tgz --access public

View file

@ -0,0 +1,95 @@
name: Generate Release Changelog PR
on:
push:
tags:
- 'v*.*.*'
workflow_dispatch:
jobs:
generate-release-changelog-pr:
permissions:
contents: write # Needed for pushing commits and creating branches.
pull-requests: write
runs-on: ubuntu-latest
steps:
# 1. Checkout the repository (with full history).
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0
# 2. Generate the release changelog using our custom configuration.
- name: Generate Release Changelog
id: generate_release
uses: mikepenz/release-changelog-builder-action@v5.1.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
configuration: ".github/configuration-release.json"
owner: ${{ github.repository_owner }}
repo: ${{ github.event.repository.name }}
outputFile: CHANGELOG-release.md
# 3. Update the main CHANGELOG.md:
# - If it doesn't exist, create it with a basic header.
# - Remove the "Unreleased" section (if present).
# - Prepend the new release changelog above previous releases.
# - Remove all temporary files before committing.
- name: Update CHANGELOG.md
run: |
# Determine the release tag, e.g. "v1.2.3"
TAG=${GITHUB_REF##*/}
echo "Using release tag: $TAG"
# Ensure CHANGELOG.md exists; if not, create a basic header.
if [ ! -f CHANGELOG.md ]; then
echo "# Changelog" > CHANGELOG.md
echo "" >> CHANGELOG.md
echo "All notable changes to this project will be documented in this file." >> CHANGELOG.md
echo "" >> CHANGELOG.md
fi
echo "Updating CHANGELOG.md…"
# Remove the "Unreleased" section (from "## [Unreleased]" until the first occurrence of '---') if it exists.
if grep -q "^## \[Unreleased\]" CHANGELOG.md; then
awk '/^## \[Unreleased\]/{flag=1} flag && /^---/{flag=0; next} !flag' CHANGELOG.md > CHANGELOG.cleaned
else
cp CHANGELOG.md CHANGELOG.cleaned
fi
# Split the cleaned file into:
# - header.md: content before the first release header ("## [v...").
# - tail.md: content from the first release header onward.
awk '/^## \[v/{exit} {print}' CHANGELOG.cleaned > header.md
awk 'f{print} /^## \[v/{f=1; print}' CHANGELOG.cleaned > tail.md
# Combine header, the new release changelog, and the tail.
echo "Combining updated changelog parts..."
cat header.md CHANGELOG-release.md > CHANGELOG.md.new
echo "" >> CHANGELOG.md.new
cat tail.md >> CHANGELOG.md.new
mv CHANGELOG.md.new CHANGELOG.md
# Remove temporary files.
rm -f CHANGELOG.cleaned header.md tail.md CHANGELOG-release.md
echo "Final CHANGELOG.md content:"
cat CHANGELOG.md
# 4. Create (or update) the Pull Request with the updated CHANGELOG.md.
- name: Create Pull Request
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
sign-commits: true
commit-message: "chore: update CHANGELOG for release ${{ github.ref_name }}"
base: main
branch: "changelog/${{ github.ref_name }}"
reviewers: danny-avila
title: "📜 docs: Changelog for release ${{ github.ref_name }}"
body: |
**Description**:
- This PR updates the CHANGELOG.md by removing the "Unreleased" section and adding new release notes for release ${{ github.ref_name }} above previous releases.

View file

@ -0,0 +1,107 @@
name: Generate Unreleased Changelog PR
on:
schedule:
- cron: "0 0 * * 1" # Runs every Monday at 00:00 UTC
workflow_dispatch:
jobs:
generate-unreleased-changelog-pr:
permissions:
contents: write # Needed for pushing commits and creating branches.
pull-requests: write
runs-on: ubuntu-latest
steps:
# 1. Checkout the repository on main.
- name: Checkout Repository on Main
uses: actions/checkout@v4
with:
ref: main
fetch-depth: 0
# 4. Get the latest version tag.
- name: Get Latest Tag
id: get_latest_tag
run: |
LATEST_TAG=$(git describe --tags $(git rev-list --tags --max-count=1) || echo "none")
echo "Latest tag: $LATEST_TAG"
echo "tag=$LATEST_TAG" >> $GITHUB_OUTPUT
# 5. Generate the Unreleased changelog.
- name: Generate Unreleased Changelog
id: generate_unreleased
uses: mikepenz/release-changelog-builder-action@v5.1.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
configuration: ".github/configuration-unreleased.json"
owner: ${{ github.repository_owner }}
repo: ${{ github.event.repository.name }}
outputFile: CHANGELOG-unreleased.md
fromTag: ${{ steps.get_latest_tag.outputs.tag }}
toTag: main
# 7. Update CHANGELOG.md with the new Unreleased section.
- name: Update CHANGELOG.md
id: update_changelog
run: |
# Create CHANGELOG.md if it doesn't exist.
if [ ! -f CHANGELOG.md ]; then
echo "# Changelog" > CHANGELOG.md
echo "" >> CHANGELOG.md
echo "All notable changes to this project will be documented in this file." >> CHANGELOG.md
echo "" >> CHANGELOG.md
fi
echo "Updating CHANGELOG.md…"
# Extract content before the "## [Unreleased]" (or first version header if missing).
if grep -q "^## \[Unreleased\]" CHANGELOG.md; then
awk '/^## \[Unreleased\]/{exit} {print}' CHANGELOG.md > CHANGELOG_TMP.md
else
awk '/^## \[v/{exit} {print}' CHANGELOG.md > CHANGELOG_TMP.md
fi
# Append the generated Unreleased changelog.
echo "" >> CHANGELOG_TMP.md
cat CHANGELOG-unreleased.md >> CHANGELOG_TMP.md
echo "" >> CHANGELOG_TMP.md
# Append the remainder of the original changelog (starting from the first version header).
awk 'f{print} /^## \[v/{f=1; print}' CHANGELOG.md >> CHANGELOG_TMP.md
# Replace the old file with the updated file.
mv CHANGELOG_TMP.md CHANGELOG.md
# Remove the temporary generated file.
rm -f CHANGELOG-unreleased.md
echo "Final CHANGELOG.md:"
cat CHANGELOG.md
# 8. Check if CHANGELOG.md has any updates.
- name: Check for CHANGELOG.md changes
id: changelog_changes
run: |
if git diff --quiet CHANGELOG.md; then
echo "has_changes=false" >> $GITHUB_OUTPUT
else
echo "has_changes=true" >> $GITHUB_OUTPUT
fi
# 9. Create (or update) the Pull Request only if there are changes.
- name: Create Pull Request
if: steps.changelog_changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
base: main
branch: "changelog/unreleased-update"
sign-commits: true
commit-message: "action: update Unreleased changelog"
title: "📜 docs: Unreleased Changelog"
body: |
**Description**:
- This PR updates the Unreleased section in CHANGELOG.md.
- It compares the current main branch with the latest version tag (determined as ${{ steps.get_latest_tag.outputs.tag }}),
regenerates the Unreleased changelog, removes any old Unreleased block, and inserts the new content.

View file

@ -4,6 +4,7 @@ on:
pull_request:
paths:
- "client/src/**"
- "api/**"
jobs:
detect-unused-i18n-keys:
@ -21,7 +22,7 @@ jobs:
# Define paths
I18N_FILE="client/src/locales/en/translation.json"
SOURCE_DIR="client/src"
SOURCE_DIRS=("client/src" "api")
# Check if translation file exists
if [[ ! -f "$I18N_FILE" ]]; then
@ -37,7 +38,38 @@ jobs:
# Check if each key is used in the source code
for KEY in $KEYS; do
if ! grep -r --include=\*.{js,jsx,ts,tsx} -q "$KEY" "$SOURCE_DIR"; then
FOUND=false
# Special case for dynamically constructed special variable keys
if [[ "$KEY" == com_ui_special_var_* ]]; then
# Check if TSpecialVarLabel is used in the codebase
for DIR in "${SOURCE_DIRS[@]}"; do
if grep -r --include=\*.{js,jsx,ts,tsx} -q "TSpecialVarLabel" "$DIR"; then
FOUND=true
break
fi
done
# Also check if the key is directly used somewhere
if [[ "$FOUND" == false ]]; then
for DIR in "${SOURCE_DIRS[@]}"; do
if grep -r --include=\*.{js,jsx,ts,tsx} -q "$KEY" "$DIR"; then
FOUND=true
break
fi
done
fi
else
# Regular check for other keys
for DIR in "${SOURCE_DIRS[@]}"; do
if grep -r --include=\*.{js,jsx,ts,tsx} -q "$KEY" "$DIR"; then
FOUND=true
break
fi
done
fi
if [[ "$FOUND" == false ]]; then
UNUSED_KEYS+=("$KEY")
fi
done
@ -59,8 +91,8 @@ jobs:
run: |
PR_NUMBER=$(jq --raw-output .pull_request.number "$GITHUB_EVENT_PATH")
# Format the unused keys list correctly, filtering out empty entries
FILTERED_KEYS=$(echo "$unused_keys" | jq -r '.[]' | grep -v '^\s*$' | sed 's/^/- `/;s/$/`/' )
# Format the unused keys list as checkboxes for easy manual checking.
FILTERED_KEYS=$(echo "$unused_keys" | jq -r '.[]' | grep -v '^\s*$' | sed 's/^/- [ ] `/;s/$/`/' )
COMMENT_BODY=$(cat <<EOF
### 🚨 Unused i18next Keys Detected
@ -81,4 +113,4 @@ jobs:
- name: Fail workflow if unused keys found
if: env.unused_keys != '[]'
run: exit 1 # This makes the PR fail if unused keys exist
run: exit 1

View file

@ -1,6 +1,12 @@
name: Detect Unused NPM Packages
on: [pull_request]
on:
pull_request:
paths:
- 'package.json'
- 'package-lock.json'
- 'client/**'
- 'api/**'
jobs:
detect-unused-packages: