From c3da148fa0fb2d5e7c6ff11ab79d1bf1c36de916 Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Thu, 19 Feb 2026 16:33:43 -0500 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D=20docs:=20Add=20AGENTS.md=20for=20?= =?UTF-8?q?Project=20Structure=20and=20Coding=20Standards=20(#11866)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 📝 docs: Add AGENTS.md for project structure and coding standards - Introduced AGENTS.md to outline project workspaces, coding standards, and development commands. - Defined workspace boundaries for backend and frontend development, emphasizing TypeScript usage. - Established guidelines for code style, iteration performance, type safety, and import order. - Updated CONTRIBUTING.md to reference AGENTS.md for coding standards and project conventions. - Modified package.json to streamline build commands, consolidating frontend and backend build processes. * chore: Update build commands and improve smart reinstall process - Modified AGENTS.md to clarify the purpose of `npm run smart-reinstall` and other build commands, emphasizing Turborepo's role in dependency management and builds. - Updated package.json to streamline build commands, replacing the legacy frontend build with a Turborepo-based approach for improved performance. - Enhanced the smart reinstall script to fully delegate build processes to Turborepo, including cache management and dependency checks, ensuring a more efficient build workflow. --- .github/CONTRIBUTING.md | 79 +++++++++---------- AGENTS.md | 158 ++++++++++++++++++++++++++++++++++++++ config/smart-reinstall.js | 100 ++++++------------------ package.json | 1 + 4 files changed, 217 insertions(+), 121 deletions(-) create mode 100644 AGENTS.md diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index ad0a75ab9b..ae9e6d8e4b 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -26,18 +26,14 @@ Project maintainers have the right and responsibility to remove, edit, or reject ## 1. Development Setup -1. Use Node.JS 20.x. -2. Install typescript globally: `npm i -g typescript`. -3. Run `npm ci` to install dependencies. -4. Build the data provider: `npm run build:data-provider`. -5. Build data schemas: `npm run build:data-schemas`. -6. Build API methods: `npm run build:api`. -7. Setup and run unit tests: +1. Use Node.js v20.19.0+ or ^22.12.0 or >= 23.0.0. +2. Run `npm run smart-reinstall` to install dependencies (uses Turborepo). Use `npm run reinstall` for a clean install, or `npm ci` for a fresh lockfile-based install. +3. Build all compiled code: `npm run build`. +4. Setup and run unit tests: - Copy `.env.test`: `cp api/test/.env.test.example api/test/.env.test`. - Run backend unit tests: `npm run test:api`. - Run frontend unit tests: `npm run test:client`. -8. Setup and run integration tests: - - Build client: `cd client && npm run build`. +5. Setup and run integration tests: - Create `.env`: `cp .env.example .env`. - Install [MongoDB Community Edition](https://www.mongodb.com/docs/manual/administration/install-community/), ensure that `mongosh` connects to your local instance. - Run: `npx install playwright`, then `npx playwright install`. @@ -48,11 +44,11 @@ Project maintainers have the right and responsibility to remove, edit, or reject ## 2. Development Notes 1. Before starting work, make sure your main branch has the latest commits with `npm run update`. -3. Run linting command to find errors: `npm run lint`. Alternatively, ensure husky pre-commit checks are functioning. +2. Run linting command to find errors: `npm run lint`. Alternatively, ensure husky pre-commit checks are functioning. 3. After your changes, reinstall packages in your current branch using `npm run reinstall` and ensure everything still works. - Restart the ESLint server ("ESLint: Restart ESLint Server" in VS Code command bar) and your IDE after reinstalling or updating. 4. Clear web app localStorage and cookies before and after changes. -5. For frontend changes, compile typescript before and after changes to check for introduced errors: `cd client && npm run build`. +5. To check for introduced errors, build all compiled code: `npm run build`. 6. Run backend unit tests: `npm run test:api`. 7. Run frontend unit tests: `npm run test:client`. 8. Run integration tests: `npm run e2e`. @@ -118,50 +114,45 @@ Apply the following naming conventions to branches, labels, and other Git-relate - **JS/TS:** Directories and file names: Descriptive and camelCase. First letter uppercased for React files (e.g., `helperFunction.ts, ReactComponent.tsx`). - **Docs:** Directories and file names: Descriptive and snake_case (e.g., `config_files.md`). -## 7. TypeScript Conversion +## 7. Coding Standards + +For detailed coding conventions, workspace boundaries, and architecture guidance, refer to the [`AGENTS.md`](../AGENTS.md) file at the project root. It covers code style, type safety, import ordering, iteration/performance expectations, frontend rules, testing, and development commands. + +## 8. TypeScript Conversion 1. **Original State**: The project was initially developed entirely in JavaScript (JS). -2. **Frontend Transition**: - - We are in the process of transitioning the frontend from JS to TypeScript (TS). - - The transition is nearing completion. - - This conversion is feasible due to React's capability to intermix JS and TS prior to code compilation. It's standard practice to compile/bundle the code in such scenarios. +2. **Frontend**: Fully transitioned to TypeScript. -3. **Backend Considerations**: - - Transitioning the backend to TypeScript would be a more intricate process, especially for an established Express.js server. - - - **Options for Transition**: - - **Single Phase Overhaul**: This involves converting the entire backend to TypeScript in one go. It's the most straightforward approach but can be disruptive, especially for larger codebases. - - - **Incremental Transition**: Convert parts of the backend progressively. This can be done by: - - Maintaining a separate directory for TypeScript files. - - Gradually migrating and testing individual modules or routes. - - Using a build tool like `tsc` to compile TypeScript files independently until the entire transition is complete. - - - **Compilation Considerations**: - - Introducing a compilation step for the server is an option. This would involve using tools like `ts-node` for development and `tsc` for production builds. - - However, this is not a conventional approach for Express.js servers and could introduce added complexity, especially in terms of build and deployment processes. - - - **Current Stance**: At present, this backend transition is of lower priority and might not be pursued. +3. **Backend**: + - The legacy Express.js server remains in `/api` as JavaScript. + - All new backend code is written in TypeScript under `/packages/api`, which is compiled and consumed by `/api`. + - Shared database logic lives in `/packages/data-schemas` (TypeScript). + - Shared frontend/backend API types and services live in `/packages/data-provider` (TypeScript). + - Minimize direct changes to `/api`; prefer adding TypeScript code to `/packages/api` and importing it. -## 8. Module Import Conventions +## 9. Module Import Conventions -- `npm` packages first, - - from longest line (top) to shortest (bottom) +Imports are organized into three sections (in order): -- Followed by typescript types (pertains to data-provider and client workspaces) - - longest line (top) to shortest (bottom) - - types from package come first +1. **Package imports** — sorted from shortest to longest line length. + - `react` is always the first import. + - Multi-line (stacked) imports count their total character length across all lines for sorting. -- Lastly, local imports - - longest line (top) to shortest (bottom) - - imports with alias `~` treated the same as relative import with respect to line length +2. **`import type` imports** — sorted from longest to shortest line length. + - Package type imports come first, then local type imports. + - Line length sorting resets between the package and local sub-groups. + +3. **Local/project imports** — sorted from longest to shortest line length. + - Multi-line (stacked) imports count their total character length across all lines for sorting. + - Imports with alias `~` are treated the same as relative imports with respect to line length. + +- Consolidate value imports from the same module as much as possible. +- Always use standalone `import type { ... }` for type imports; never use inline `type` keyword inside value imports (e.g., `import { Foo, type Bar }` is wrong). **Note:** ESLint will automatically enforce these import conventions when you run `npm run lint --fix` or through pre-commit hooks. ---- - -Please ensure that you adapt this summary to fit the specific context and nuances of your project. +For the full set of coding standards, see [`AGENTS.md`](../AGENTS.md). --- diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..23b5fc0fbb --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,158 @@ +# LibreChat + +## Project Overview + +LibreChat is a monorepo with the following key workspaces: + +| Workspace | Language | Side | Dependency | Purpose | +|---|---|---|---|---| +| `/api` | JS (legacy) | Backend | `packages/api`, `packages/data-schemas`, `packages/data-provider`, `@librechat/agents` | Express server — minimize changes here | +| `/packages/api` | **TypeScript** | Backend | `packages/data-schemas`, `packages/data-provider` | New backend code lives here (TS only, consumed by `/api`) | +| `/packages/data-schemas` | TypeScript | Backend | `packages/data-provider` | Database models/schemas, shareable across backend projects | +| `/packages/data-provider` | TypeScript | Shared | — | Shared API types, endpoints, data-service — used by both frontend and backend | +| `/client` | TypeScript/React | Frontend | `packages/data-provider`, `packages/client` | Frontend SPA | +| `/packages/client` | TypeScript | Frontend | `packages/data-provider` | Shared frontend utilities | + +The source code for `@librechat/agents` (major backend dependency, same team) is at `/home/danny/agentus`. + +--- + +## Workspace Boundaries + +- **All new backend code must be TypeScript** in `/packages/api`. +- Keep `/api` changes to the absolute minimum (thin JS wrappers calling into `/packages/api`). +- Database-specific shared logic goes in `/packages/data-schemas`. +- Frontend/backend shared API logic (endpoints, types, data-service) goes in `/packages/data-provider`. +- Build data-provider from project root: `npm run build:data-provider`. + +--- + +## Code Style + +### Structure and Clarity + +- **Never-nesting**: early returns, flat code, minimal indentation. Break complex operations into well-named helpers. +- **Functional first**: pure functions, immutable data, `map`/`filter`/`reduce` over imperative loops. Only reach for OOP when it clearly improves domain modeling or state encapsulation. +- **No dynamic imports** unless absolutely necessary. + +### DRY + +- Extract repeated logic into utility functions. +- Reusable hooks / higher-order components for UI patterns. +- Parameterized helpers instead of near-duplicate functions. +- Constants for repeated values; configuration objects over duplicated init code. +- Shared validators, centralized error handling, single source of truth for business rules. +- Shared typing system with interfaces/types extending common base definitions. +- Abstraction layers for external API interactions. + +### Iteration and Performance + +- **Minimize looping** — especially over shared data structures like message arrays, which are iterated frequently throughout the codebase. Every additional pass adds up at scale. +- Consolidate sequential O(n) operations into a single pass whenever possible; never loop over the same collection twice if the work can be combined. +- Choose data structures that reduce the need to iterate (e.g., `Map`/`Set` for lookups instead of `Array.find`/`Array.includes`). +- Avoid unnecessary object creation; consider space-time tradeoffs. +- Prevent memory leaks: careful with closures, dispose resources/event listeners, no circular references. + +### Type Safety + +- **Never use `any`**. Explicit types for all parameters, return values, and variables. +- **Limit `unknown`** — avoid `unknown`, `Record`, and `as unknown as T` assertions. A `Record` almost always signals a missing explicit type definition. +- **Don't duplicate types** — before defining a new type, check whether it already exists in the project (especially `packages/data-provider`). Reuse and extend existing types rather than creating redundant definitions. +- Use union types, generics, and interfaces appropriately. +- All TypeScript and ESLint warnings/errors must be addressed — do not leave unresolved diagnostics. + +### Comments and Documentation + +- Write self-documenting code; no inline comments narrating what code does. +- JSDoc only for complex/non-obvious logic or intellisense on public APIs. +- Single-line JSDoc for brief docs, multi-line for complex cases. +- Avoid standalone `//` comments unless absolutely necessary. + +### Import Order + +Imports are organized into three sections: + +1. **Package imports** — sorted shortest to longest line length (`react` always first). +2. **`import type` imports** — sorted longest to shortest (package types first, then local types; length resets between sub-groups). +3. **Local/project imports** — sorted longest to shortest. + +Multi-line imports count total character length across all lines. Consolidate value imports from the same module. Always use standalone `import type { ... }` — never inline `type` inside value imports. + +### JS/TS Loop Preferences + +- **Limit looping as much as possible.** Prefer single-pass transformations and avoid re-iterating the same data. +- `for (let i = 0; ...)` for performance-critical or index-dependent operations. +- `for...of` for simple array iteration. +- `for...in` only for object property enumeration. + +--- + +## Frontend Rules (`client/src/**/*`) + +### Localization + +- All user-facing text must use `useLocalize()`. +- Only update English keys in `client/src/locales/en/translation.json` (other languages are automated externally). +- Semantic key prefixes: `com_ui_`, `com_assistants_`, etc. + +### Components + +- TypeScript for all React components with proper type imports. +- Semantic HTML with ARIA labels (`role`, `aria-label`) for accessibility. +- Group related components in feature directories (e.g., `SidePanel/Memories/`). +- Use index files for clean exports. + +### Data Management + +- Feature hooks: `client/src/data-provider/[Feature]/queries.ts` → `[Feature]/index.ts` → `client/src/data-provider/index.ts`. +- React Query (`@tanstack/react-query`) for all API interactions; proper query invalidation on mutations. +- QueryKeys and MutationKeys in `packages/data-provider/src/keys.ts`. + +### Data-Provider Integration + +- Endpoints: `packages/data-provider/src/api-endpoints.ts` +- Data service: `packages/data-provider/src/data-service.ts` +- Types: `packages/data-provider/src/types/queries.ts` +- Use `encodeURIComponent` for dynamic URL parameters. + +### Performance + +- Prioritize memory and speed efficiency at scale. +- Cursor pagination for large datasets. +- Proper dependency arrays to avoid unnecessary re-renders. +- Leverage React Query caching and background refetching. + +--- + +## Development Commands + +| Command | Purpose | +|---|---| +| `npm run smart-reinstall` | Install deps (if lockfile changed) + build via Turborepo | +| `npm run reinstall` | Clean install — wipe `node_modules` and reinstall from scratch | +| `npm run backend` | Start the backend server | +| `npm run backend:dev` | Start backend with file watching (development) | +| `npm run build` | Build all compiled code via Turborepo (parallel, cached) | +| `npm run frontend` | Build all compiled code sequentially (legacy fallback) | +| `npm run frontend:dev` | Start frontend dev server with HMR (port 3090, requires backend running) | +| `npm run build:data-provider` | Rebuild `packages/data-provider` after changes | + +- Node.js: v20.19.0+ or ^22.12.0 or >= 23.0.0 +- Database: MongoDB +- Backend runs on `http://localhost:3080/`; frontend dev server on `http://localhost:3090/` + +--- + +## Testing + +- Framework: **Jest**, run per-workspace. +- Run tests from their workspace directory: `cd api && npx jest `, `cd packages/api && npx jest `, etc. +- Frontend tests: `__tests__` directories alongside components; use `test/layout-test-utils` for rendering. +- Cover loading, success, and error states for UI/data flows. +- Mock data-provider hooks and external dependencies. + +--- + +## Formatting + +Fix all formatting lint errors (trailing spaces, tabs, newlines, indentation) using auto-fix when available. All TypeScript/ESLint warnings and errors **must** be resolved. diff --git a/config/smart-reinstall.js b/config/smart-reinstall.js index 18fe689127..f22bb25151 100644 --- a/config/smart-reinstall.js +++ b/config/smart-reinstall.js @@ -9,10 +9,9 @@ * Skips npm ci entirely when the lockfile hasn't changed. * * Package builds (Turborepo): - * Turbo hashes each package's source/config inputs, caches build - * outputs (dist/), and restores from cache when inputs match. - * Turbo v2 uses a global cache (~/.cache/turbo) that survives - * npm ci and is shared across worktrees. + * Turbo hashes each package's source/config inputs (including the + * lockfile), caches build outputs (dist/), and restores from cache + * when inputs match. This script delegates entirely to turbo for builds. * * Usage: * npm run smart-reinstall # Smart cached mode @@ -27,11 +26,8 @@ const fs = require('fs'); const path = require('path'); const { execSync } = require('child_process'); -// Adds console.green, console.purple, etc. require('./helpers'); -// ─── Configuration ─────────────────────────────────────────────────────────── - const ROOT_DIR = path.resolve(__dirname, '..'); const DEPS_HASH_MARKER = path.join(ROOT_DIR, 'node_modules', '.librechat-deps-hash'); @@ -42,7 +38,6 @@ const flags = { verbose: process.argv.includes('--verbose'), }; -// Workspace directories whose node_modules should be cleaned during reinstall const NODE_MODULES_DIRS = [ ROOT_DIR, path.join(ROOT_DIR, 'packages', 'data-provider'), @@ -53,8 +48,6 @@ const NODE_MODULES_DIRS = [ path.join(ROOT_DIR, 'api'), ]; -// ─── Helpers ───────────────────────────────────────────────────────────────── - function hashFile(filePath) { return crypto.createHash('sha256').update(fs.readFileSync(filePath)).digest('hex').slice(0, 16); } @@ -63,8 +56,6 @@ function exec(cmd, opts = {}) { execSync(cmd, { cwd: ROOT_DIR, stdio: 'inherit', ...opts }); } -// ─── Dependency Installation ───────────────────────────────────────────────── - function checkDeps() { const lockfile = path.join(ROOT_DIR, 'package-lock.json'); if (!fs.existsSync(lockfile)) { @@ -97,19 +88,15 @@ function installDeps(hash) { fs.writeFileSync(DEPS_HASH_MARKER, hash, 'utf-8'); } -// ─── Turbo Build ───────────────────────────────────────────────────────────── - function runTurboBuild() { const args = ['npx', 'turbo', 'run', 'build']; if (flags.skipClient) { args.push('--filter=!@librechat/frontend'); } - if (flags.force) { args.push('--force'); } - if (flags.verbose) { args.push('--verbosity=2'); } @@ -119,76 +106,41 @@ function runTurboBuild() { exec(cmd); } -/** - * Fallback for when turbo is not installed (e.g., first run before npm ci). - * Runs the same sequential build as the original `npm run frontend`. - */ -function runFallbackBuild() { - console.orange(' turbo not found — using sequential fallback build\n'); - - const scripts = [ - 'build:data-provider', - 'build:data-schemas', - 'build:api', - 'build:client-package', - ]; - - if (!flags.skipClient) { - scripts.push('build:client'); +function cleanTurboCache() { + console.purple('Clearing Turborepo cache...'); + try { + exec('npx turbo daemon stop', { stdio: 'pipe' }); + } catch { + // daemon may not be running } - for (const script of scripts) { - console.purple(` Running ${script}...`); - exec(`npm run ${script}`); + const localTurboCache = path.join(ROOT_DIR, '.turbo'); + if (fs.existsSync(localTurboCache)) { + fs.rmSync(localTurboCache, { recursive: true }); + } + + try { + exec('npx turbo clean', { stdio: 'pipe' }); + console.green('Turbo cache cleared.'); + } catch { + console.gray('Could not clear global turbo cache (may not exist yet).'); } } -function hasTurbo() { - const binDir = path.join(ROOT_DIR, 'node_modules', '.bin'); - return ['turbo', 'turbo.cmd', 'turbo.ps1'].some((name) => fs.existsSync(path.join(binDir, name))); -} - -// ─── Main ──────────────────────────────────────────────────────────────────── - (async () => { const startTime = Date.now(); console.green('\n Smart Reinstall — LibreChat'); console.green('─'.repeat(45)); - // ── Handle --clean-cache ─────────────────────────────────────────────── if (flags.cleanCache) { - console.purple('Clearing Turborepo cache...'); - if (hasTurbo()) { - try { - exec('npx turbo daemon stop', { stdio: 'pipe' }); - } catch { - // ignore — daemon may not be running - } - } - // Clear local .turbo cache dir - const localTurboCache = path.join(ROOT_DIR, '.turbo'); - if (fs.existsSync(localTurboCache)) { - fs.rmSync(localTurboCache, { recursive: true }); - } - // Clear global turbo cache - if (hasTurbo()) { - try { - exec('npx turbo clean', { stdio: 'pipe' }); - console.green('Turbo cache cleared.'); - } catch { - console.gray('Could not clear global turbo cache (may not exist yet).'); - } - } else { - console.gray('turbo not installed — nothing to clear.'); - } - + cleanTurboCache(); if (!flags.force) { return; } } - // ── Step 1: Dependencies ─────────────────────────────────────────────── + // Step 1: Dependencies console.purple('\n[1/2] Checking dependencies...'); if (flags.force) { @@ -208,16 +160,10 @@ function hasTurbo() { } } - // ── Step 2: Build packages ───────────────────────────────────────────── + // Step 2: Build via Turborepo console.purple('\n[2/2] Building packages...'); + runTurboBuild(); - if (hasTurbo()) { - runTurboBuild(); - } else { - runFallbackBuild(); - } - - // ── Done ─────────────────────────────────────────────────────────────── const elapsed = ((Date.now() - startTime) / 1000).toFixed(1); console.log(''); console.green('─'.repeat(45)); diff --git a/package.json b/package.json index 6c8e715a1a..9c27a7ea99 100644 --- a/package.json +++ b/package.json @@ -48,6 +48,7 @@ "build:client": "cd client && npm run build", "build:client-package": "cd packages/client && npm run build", "build:packages": "npm run build:data-provider && npm run build:data-schemas && npm run build:api && npm run build:client-package", + "build": "npx turbo run build", "frontend": "npm run build:data-provider && npm run build:data-schemas && npm run build:api && npm run build:client-package && cd client && npm run build", "frontend:ci": "npm run build:data-provider && npm run build:client-package && cd client && npm run build:ci", "frontend:dev": "cd client && npm run dev",