LibreChat/packages/data-schemas
Danny Avila c324a8d9e4
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
refactor: Parallelize CI Workflows with Isolated Caching and Fan-Out Test Jobs (#12088)
* refactor: CI Workflow for Backend with Build and Test Jobs

- Updated the GitHub Actions workflow to include a new build job that compiles packages and uploads build artifacts.
- Added separate test jobs for each package (`api`, `data-provider`, and `data-schemas`) to run unit tests after the build process.
- Introduced caching for build artifacts to optimize build times.
- Configured Jest to utilize 50% of available workers for improved test performance across all Jest configurations in the `api`, `data-schemas`, and `packages/api` directories.

* refactor: Update CI Workflow for Backend with Enhanced Build and Cache Management

- Modified the GitHub Actions workflow to improve the build process by separating build and cache steps for `data-provider`, `data-schemas`, and `api` packages.
- Updated artifact upload and download steps to reflect the new naming conventions for better clarity.
- Enhanced caching strategies to optimize build times and ensure efficient artifact management.

* chore: Node Modules Caching in CI Workflow

- Updated the GitHub Actions workflow to implement caching for the `node_modules` directory, improving build efficiency by restoring cached dependencies.
- Adjusted the installation step to conditionally run based on cache availability, optimizing the overall CI process.

* refactor: Enhance CI Workflow for Frontend with Build and Test Jobs

- Updated the GitHub Actions workflow to introduce a structured build process for frontend packages, including separate jobs for building and testing on both Ubuntu and Windows environments.
- Implemented caching strategies for `node_modules` and build artifacts to optimize build times and improve efficiency.
- Added artifact upload and download steps for `data-provider` and `client-package` builds, ensuring that builds are reused across jobs.
- Adjusted Node.js version specification for consistency and reliability across different jobs.

* refactor: Update CI Workflows for Backend and Frontend with Node.js 20.19 and Enhanced Caching

- Updated Node.js version to 20.19 across all jobs in both backend and frontend workflows for consistency.
- Enhanced caching strategies for build artifacts and `node_modules`, increasing retention days from 1 to 2 for better efficiency.
- Adjusted cache keys to include additional files for improved cache hit rates during builds.
- Added conditional installation of dependencies to optimize the CI process.

* chore: Configure Jest to Use 50% of Available Workers Across Client and Data Provider

- Added `maxWorkers: '50%'` setting to Jest configuration files for the client and data provider packages to optimize test performance by utilizing half of the available CPU cores during test execution.

* chore: Enhance Node Modules Caching in CI Workflows

- Updated caching paths in both backend and frontend GitHub Actions workflows to include additional `node_modules` directories for improved dependency management.
- This change optimizes the caching strategy, ensuring that all relevant modules are cached, which can lead to faster build times and more efficient CI processes.

* chore: Update Node Modules Cache Keys in CI Workflows

- Modified cache keys in both backend and frontend GitHub Actions workflows to include the Node.js version (20.19) for improved cache management.
- This change ensures that the caching mechanism is more specific, potentially enhancing cache hit rates and build efficiency.

* chore: Refactor Node Modules Cache Keys in CI Workflows

- Updated cache keys in backend and frontend GitHub Actions workflows to be more specific, distinguishing between frontend and backend caches.
- Removed references to `client/node_modules` in backend workflows to streamline caching paths and improve cache management.
2026-03-05 13:56:07 -05:00
..
src 🕰️ fix: Preserve updatedAt Timestamps During Meilisearch Batch Sync (#12084) 2026-03-05 10:40:43 -05:00
.gitignore 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00
babel.config.cjs 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00
jest.config.mjs refactor: Parallelize CI Workflows with Isolated Caching and Fan-Out Test Jobs (#12088) 2026-03-05 13:56:07 -05:00
LICENSE 🔏 fix: Enhance Two-Factor Authentication (#6247) 2025-03-08 15:28:27 -05:00
package.json v0.8.3-rc2 (#12027) 2026-03-04 09:28:20 -05:00
README.md 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-08-13 16:24:17 -04:00
rollup.config.js 🚀 feat: Refactor schema exports and update package version to 0.0.4 (#6455) 2025-03-21 08:20:23 -04:00
tsconfig.json 🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650) 2025-05-30 22:18:13 -04:00
tsconfig.spec.json 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00

LibreChat Data Schemas Package

This package provides the database schemas, models, types, and methods for LibreChat using Mongoose ODM.

📁 Package Structure

packages/data-schemas/
├── src/
│   ├── schema/         # Mongoose schema definitions
│   ├── models/         # Model factory functions
│   ├── types/          # TypeScript type definitions
│   ├── methods/        # Database operation methods
│   ├── common/         # Shared constants and enums
│   ├── config/         # Configuration files (winston, etc.)
│   └── index.ts        # Main package exports

🏗️ Architecture Patterns

1. Schema Files (src/schema/)

Schema files define the Mongoose schema structure. They follow these conventions:

  • Naming: Use lowercase filenames (e.g., user.ts, accessRole.ts)
  • Imports: Import types from ~/types for TypeScript support
  • Exports: Export only the schema as default

Example:

import { Schema } from 'mongoose';
import type { IUser } from '~/types';

const userSchema = new Schema<IUser>(
  {
    name: { type: String },
    email: { type: String, required: true },
    // ... other fields
  },
  { timestamps: true }
);

export default userSchema;

2. Type Definitions (src/types/)

Type files define TypeScript interfaces and types. They follow these conventions:

  • Base Type: Define a plain type without Mongoose Document properties
  • Document Interface: Extend the base type with Document and _id
  • Enums/Constants: Place related enums in the type file or common/ if shared

Example:

import type { Document, Types } from 'mongoose';

export type User = {
  name?: string;
  email: string;
  // ... other fields
};

export type IUser = User &
  Document & {
    _id: Types.ObjectId;
  };

3. Model Factory Functions (src/models/)

Model files create Mongoose models using factory functions. They follow these conventions:

  • Function Name: create[EntityName]Model
  • Singleton Pattern: Check if model exists before creating
  • Type Safety: Use the corresponding interface from types

Example:

import userSchema from '~/schema/user';
import type * as t from '~/types';

export function createUserModel(mongoose: typeof import('mongoose')) {
  return mongoose.models.User || mongoose.model<t.IUser>('User', userSchema);
}

4. Database Methods (src/methods/)

Method files contain database operations for each entity. They follow these conventions:

  • Function Name: create[EntityName]Methods
  • Return Type: Export a type for the methods object
  • Operations: Include CRUD operations and entity-specific queries

Example:

import type { Model } from 'mongoose';
import type { IUser } from '~/types';

export function createUserMethods(mongoose: typeof import('mongoose')) {
  async function findUserById(userId: string): Promise<IUser | null> {
    const User = mongoose.models.User as Model<IUser>;
    return await User.findById(userId).lean();
  }

  async function createUser(userData: Partial<IUser>): Promise<IUser> {
    const User = mongoose.models.User as Model<IUser>;
    return await User.create(userData);
  }

  return {
    findUserById,
    createUser,
    // ... other methods
  };
}

export type UserMethods = ReturnType<typeof createUserMethods>;

5. Main Exports (src/index.ts)

The main index file exports:

  • createModels() - Factory function for all models
  • createMethods() - Factory function for all methods
  • Type exports from ~/types
  • Shared utilities and constants

🚀 Adding a New Entity

To add a new entity to the data-schemas package, follow these steps:

Step 1: Create the Type Definition

Create src/types/[entityName].ts:

import type { Document, Types } from 'mongoose';

export type EntityName = {
  /** Field description */
  fieldName: string;
  // ... other fields
};

export type IEntityName = EntityName &
  Document & {
    _id: Types.ObjectId;
  };

Step 2: Update Types Index

Add to src/types/index.ts:

export * from './entityName';

Step 3: Create the Schema

Create src/schema/[entityName].ts:

import { Schema } from 'mongoose';
import type { IEntityName } from '~/types';

const entityNameSchema = new Schema<IEntityName>(
  {
    fieldName: { type: String, required: true },
    // ... other fields
  },
  { timestamps: true }
);

export default entityNameSchema;

Step 4: Create the Model Factory

Create src/models/[entityName].ts:

import entityNameSchema from '~/schema/entityName';
import type * as t from '~/types';

export function createEntityNameModel(mongoose: typeof import('mongoose')) {
  return (
    mongoose.models.EntityName || 
    mongoose.model<t.IEntityName>('EntityName', entityNameSchema)
  );
}

Step 5: Update Models Index

Add to src/models/index.ts:

  1. Import the factory function:
import { createEntityNameModel } from './entityName';
  1. Add to the return object in createModels():
EntityName: createEntityNameModel(mongoose),

Step 6: Create Database Methods

Create src/methods/[entityName].ts:

import type { Model, Types } from 'mongoose';
import type { IEntityName } from '~/types';

export function createEntityNameMethods(mongoose: typeof import('mongoose')) {
  async function findEntityById(id: string | Types.ObjectId): Promise<IEntityName | null> {
    const EntityName = mongoose.models.EntityName as Model<IEntityName>;
    return await EntityName.findById(id).lean();
  }

  // ... other methods

  return {
    findEntityById,
    // ... other methods
  };
}

export type EntityNameMethods = ReturnType<typeof createEntityNameMethods>;

Step 7: Update Methods Index

Add to src/methods/index.ts:

  1. Import the methods:
import { createEntityNameMethods, type EntityNameMethods } from './entityName';
  1. Add to the return object in createMethods():
...createEntityNameMethods(mongoose),
  1. Add to the AllMethods type:
export type AllMethods = UserMethods &
  // ... other methods
  EntityNameMethods;

📝 Best Practices

  1. Consistent Naming: Use lowercase for filenames, PascalCase for types/interfaces
  2. Type Safety: Always use TypeScript types, avoid any
  3. JSDoc Comments: Document complex fields and methods
  4. Indexes: Define database indexes in schema files for query performance
  5. Validation: Use Mongoose schema validation for data integrity
  6. Lean Queries: Use .lean() for read operations when you don't need Mongoose document methods

🔧 Common Patterns

Enums and Constants

Place shared enums in src/common/:

// src/common/permissions.ts
export enum PermissionBits {
  VIEW = 1,
  EDIT = 2,
  DELETE = 4,
  SHARE = 8,
}

Compound Indexes

For complex queries, add compound indexes:

schema.index({ field1: 1, field2: 1 });
schema.index(
  { uniqueField: 1 },
  { 
    unique: true, 
    partialFilterExpression: { uniqueField: { $exists: true } }
  }
);

Virtual Properties

Add computed properties using virtuals:

schema.virtual('fullName').get(function() {
  return `${this.firstName} ${this.lastName}`;
});

🧪 Testing

When adding new entities, ensure:

  • Types compile without errors
  • Models can be created successfully
  • Methods handle edge cases (null checks, validation)
  • Indexes are properly defined for query patterns

📚 Resources