LibreChat/packages/data-schemas
Danny Avila 65c81955f0
🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804)
* feat: Add granular role-based permissions system with Entra ID integration

      - Implement RBAC with viewer/editor/owner roles using bitwise permissions
      - Add AccessRole, AclEntry, and Group models for permission management
      - Create PermissionService for core permission logic and validation
      - Integrate Microsoft Graph API for Entra ID user/group search
      - Add middleware for resource access validation with custom ID resolvers
      - Implement bulk permission updates with transaction support
      - Create permission management UI with people picker and role selection
      - Add public sharing capabilities for resources
      - Include database migration for existing agent ownership
      - Support hybrid local/Entra ID identity management
      - Add comprehensive test coverage for all new services

chore: Update @librechat/data-schemas to version 0.0.9 and export common module in index.ts

fix: Update userGroup tests to mock logger correctly and change principalId expectation from null to undefined

* fix(data-schemas): use partial index for group idOnTheSource uniqueness

Replace sparse index with partial filter expression to allow multiple local groups
while maintaining unique constraint for external source IDs. The sparse option
on compound indexes doesn't work as expected when one field is always present.

* fix: imports in migrate-agent-permissions.js

* chore(data-schemas): add comprehensive README for data schemas package

- Introduced a detailed README.md file outlining the structure, architecture patterns, and best practices for the LibreChat Data Schemas package.
- Included guidelines for creating new entities, type definitions, schema files, model factory functions, and database methods.
- Added examples and common patterns to enhance understanding and usage of the package.

* chore: remove unused translation keys from localization file

* ci: fix existing tests based off new permission handling

- Renamed test cases to reflect changes in permission checks being handled at the route level.
- Updated assertions to verify that agents are returned regardless of user permissions due to the new permission system.
- Adjusted mocks in AppService and PermissionService tests to ensure proper functionality without relying on actual implementations.

* ci: add unit tests for access control middleware

- Introduced tests for the `canAccessAgentResource` middleware to validate permission checks for agent resources.
- Implemented tests for various scenarios including user roles, ACL entries, and permission levels.
- Added tests for the `checkAccess` function to ensure proper permission handling based on user roles and permissions.
- Utilized MongoDB in-memory server for isolated test environments.

* refactor: remove unused mocks from GraphApiService tests

* ci: enhance AgentFooter tests with improved mocks and permission handling

- Updated mocks for `useWatch`, `useAuthContext`, `useHasAccess`, and `useResourcePermissions` to streamline test setup.
- Adjusted assertions to reflect changes in UI based on agent ID and user roles.
- Replaced `share-agent` component with `grant-access-dialog` in tests to align with recent UI updates.
- Added tests for handling null agent data and permissions loading scenarios.

* ci: enhance GraphApiService tests with MongoDB in-memory server

- Updated test setup to use MongoDB in-memory server for isolated testing.
- Refactored beforeEach to beforeAll for database connection management.
- Cleared database before each test to ensure a clean state.
- Retained existing mocks while improving test structure for better clarity.

* ci: enhance GraphApiService tests with additional logger mocks

- Added mock implementation for logger methods in GraphApiService tests to improve error and debug logging during test execution.
- Ensured existing mocks remain intact while enhancing test coverage and clarity.

* chore: address ESLint Warnings

* - add cursor-based pagination to getListAgentsByAccess and update handler
- add index on updatedAt and _id in agent schema for improved query performance

* refactor permission service with reuse of model methods from data-schema package

* - Fix ObjectId comparison in getListAgentsHandler using .equals() method instead of strict equality
  - Add findPubliclyAccessibleResources function to PermissionService for bulk public resource queries
  - Add hasPublicPermission function to PermissionService for individual resource public permission checks
  - Update getAgentHandler to use hasPublicPermission for accurate individual agent public status
  - Replace instanceProjectId-based global checks with isPublic property from backend in client code
  - Add isPublic property to Agent type definition
  - Add NODE_TLS_REJECT_UNAUTHORIZED debug setting to VS Code launch config

* feat: add check for People.Read scope in searchContacts

* fix: add roleId parameter to grantPermission and update tests for GraphApiService

* refactor: remove problematic projection pipelines in getResourcePermissions for document db aws compatibility

* feat: enhance agent permissions migration with DocumentDB compatibility and add dry-run script

* feat: add support for including Entra ID group owners as members in permissions management + fix Group members  paging

* feat: enforce at least one owner requirement for permission updates and add corresponding localization messages

* refactor: remove German locale (must be added via i18n)

* chore: linting in `api/models/Agent.js` and removed unused variables

* chore: linting, remove unused vars, and remove project-related parameters from `updateAgentHandler`

* chore: address ESLint errors

* chore: revert removal of unused vars for versioning

---------

Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
2025-06-23 10:50:07 -04:00
..
src 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-06-23 10:50:07 -04:00
.gitignore 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00
babel.config.cjs 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00
jest.config.mjs 📧 feat: Mailgun API Email Configuration (#7742) 2025-06-04 13:12:37 -04:00
LICENSE 🔏 fix: Enhance Two-Factor Authentication (#6247) 2025-03-08 15:28:27 -05:00
package.json 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-06-23 10:50:07 -04:00
README.md 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-06-23 10:50:07 -04:00
rollup.config.js 🚀 feat: Refactor schema exports and update package version to 0.0.4 (#6455) 2025-03-21 08:20:23 -04:00
tsconfig.json 🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650) 2025-05-30 22:18:13 -04:00
tsconfig.spec.json 📦 refactor: Move DB Models to @librechat/data-schemas (#6210) 2025-03-07 11:55:44 -05:00

LibreChat Data Schemas Package

This package provides the database schemas, models, types, and methods for LibreChat using Mongoose ODM.

📁 Package Structure

packages/data-schemas/
├── src/
│   ├── schema/         # Mongoose schema definitions
│   ├── models/         # Model factory functions
│   ├── types/          # TypeScript type definitions
│   ├── methods/        # Database operation methods
│   ├── common/         # Shared constants and enums
│   ├── config/         # Configuration files (winston, etc.)
│   └── index.ts        # Main package exports

🏗️ Architecture Patterns

1. Schema Files (src/schema/)

Schema files define the Mongoose schema structure. They follow these conventions:

  • Naming: Use lowercase filenames (e.g., user.ts, accessRole.ts)
  • Imports: Import types from ~/types for TypeScript support
  • Exports: Export only the schema as default

Example:

import { Schema } from 'mongoose';
import type { IUser } from '~/types';

const userSchema = new Schema<IUser>(
  {
    name: { type: String },
    email: { type: String, required: true },
    // ... other fields
  },
  { timestamps: true }
);

export default userSchema;

2. Type Definitions (src/types/)

Type files define TypeScript interfaces and types. They follow these conventions:

  • Base Type: Define a plain type without Mongoose Document properties
  • Document Interface: Extend the base type with Document and _id
  • Enums/Constants: Place related enums in the type file or common/ if shared

Example:

import type { Document, Types } from 'mongoose';

export type User = {
  name?: string;
  email: string;
  // ... other fields
};

export type IUser = User &
  Document & {
    _id: Types.ObjectId;
  };

3. Model Factory Functions (src/models/)

Model files create Mongoose models using factory functions. They follow these conventions:

  • Function Name: create[EntityName]Model
  • Singleton Pattern: Check if model exists before creating
  • Type Safety: Use the corresponding interface from types

Example:

import userSchema from '~/schema/user';
import type * as t from '~/types';

export function createUserModel(mongoose: typeof import('mongoose')) {
  return mongoose.models.User || mongoose.model<t.IUser>('User', userSchema);
}

4. Database Methods (src/methods/)

Method files contain database operations for each entity. They follow these conventions:

  • Function Name: create[EntityName]Methods
  • Return Type: Export a type for the methods object
  • Operations: Include CRUD operations and entity-specific queries

Example:

import type { Model } from 'mongoose';
import type { IUser } from '~/types';

export function createUserMethods(mongoose: typeof import('mongoose')) {
  async function findUserById(userId: string): Promise<IUser | null> {
    const User = mongoose.models.User as Model<IUser>;
    return await User.findById(userId).lean();
  }

  async function createUser(userData: Partial<IUser>): Promise<IUser> {
    const User = mongoose.models.User as Model<IUser>;
    return await User.create(userData);
  }

  return {
    findUserById,
    createUser,
    // ... other methods
  };
}

export type UserMethods = ReturnType<typeof createUserMethods>;

5. Main Exports (src/index.ts)

The main index file exports:

  • createModels() - Factory function for all models
  • createMethods() - Factory function for all methods
  • Type exports from ~/types
  • Shared utilities and constants

🚀 Adding a New Entity

To add a new entity to the data-schemas package, follow these steps:

Step 1: Create the Type Definition

Create src/types/[entityName].ts:

import type { Document, Types } from 'mongoose';

export type EntityName = {
  /** Field description */
  fieldName: string;
  // ... other fields
};

export type IEntityName = EntityName &
  Document & {
    _id: Types.ObjectId;
  };

Step 2: Update Types Index

Add to src/types/index.ts:

export * from './entityName';

Step 3: Create the Schema

Create src/schema/[entityName].ts:

import { Schema } from 'mongoose';
import type { IEntityName } from '~/types';

const entityNameSchema = new Schema<IEntityName>(
  {
    fieldName: { type: String, required: true },
    // ... other fields
  },
  { timestamps: true }
);

export default entityNameSchema;

Step 4: Create the Model Factory

Create src/models/[entityName].ts:

import entityNameSchema from '~/schema/entityName';
import type * as t from '~/types';

export function createEntityNameModel(mongoose: typeof import('mongoose')) {
  return (
    mongoose.models.EntityName || 
    mongoose.model<t.IEntityName>('EntityName', entityNameSchema)
  );
}

Step 5: Update Models Index

Add to src/models/index.ts:

  1. Import the factory function:
import { createEntityNameModel } from './entityName';
  1. Add to the return object in createModels():
EntityName: createEntityNameModel(mongoose),

Step 6: Create Database Methods

Create src/methods/[entityName].ts:

import type { Model, Types } from 'mongoose';
import type { IEntityName } from '~/types';

export function createEntityNameMethods(mongoose: typeof import('mongoose')) {
  async function findEntityById(id: string | Types.ObjectId): Promise<IEntityName | null> {
    const EntityName = mongoose.models.EntityName as Model<IEntityName>;
    return await EntityName.findById(id).lean();
  }

  // ... other methods

  return {
    findEntityById,
    // ... other methods
  };
}

export type EntityNameMethods = ReturnType<typeof createEntityNameMethods>;

Step 7: Update Methods Index

Add to src/methods/index.ts:

  1. Import the methods:
import { createEntityNameMethods, type EntityNameMethods } from './entityName';
  1. Add to the return object in createMethods():
...createEntityNameMethods(mongoose),
  1. Add to the AllMethods type:
export type AllMethods = UserMethods &
  // ... other methods
  EntityNameMethods;

📝 Best Practices

  1. Consistent Naming: Use lowercase for filenames, PascalCase for types/interfaces
  2. Type Safety: Always use TypeScript types, avoid any
  3. JSDoc Comments: Document complex fields and methods
  4. Indexes: Define database indexes in schema files for query performance
  5. Validation: Use Mongoose schema validation for data integrity
  6. Lean Queries: Use .lean() for read operations when you don't need Mongoose document methods

🔧 Common Patterns

Enums and Constants

Place shared enums in src/common/:

// src/common/permissions.ts
export enum PermissionBits {
  VIEW = 1,
  EDIT = 2,
  DELETE = 4,
  SHARE = 8,
}

Compound Indexes

For complex queries, add compound indexes:

schema.index({ field1: 1, field2: 1 });
schema.index(
  { uniqueField: 1 },
  { 
    unique: true, 
    partialFilterExpression: { uniqueField: { $exists: true } }
  }
);

Virtual Properties

Add computed properties using virtuals:

schema.virtual('fullName').get(function() {
  return `${this.firstName} ${this.lastName}`;
});

🧪 Testing

When adding new entities, ensure:

  • Types compile without errors
  • Models can be created successfully
  • Methods handle edge cases (null checks, validation)
  • Indexes are properly defined for query patterns

📚 Resources