mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-02-23 19:04:10 +01:00
🪣 fix: S3 path-style URL support for MinIO, R2, and custom endpoints (#11894)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Has been cancelled
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Has been cancelled
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* 🪣 fix: S3 path-style URL support for MinIO, R2, and custom endpoints `extractKeyFromS3Url` now uses `AWS_BUCKET_NAME` to automatically detect and strip the bucket prefix from path-style URLs, fixing `NoSuchKey` errors on URL refresh for any S3-compatible provider using a custom endpoint (MinIO, Cloudflare R2, Hetzner, Backblaze B2, etc.). No additional configuration required — the bucket name is already a required env var for S3 to function. `initializeS3` now passes `forcePathStyle: true` to the S3Client constructor when `AWS_FORCE_PATH_STYLE=true` is set. Required for providers whose SSL certificates do not support virtual-hosted-style bucket subdomains (e.g. Hetzner Object Storage), which previously caused 401 / SignatureDoesNotMatch on upload. Additional fixes: - Suppress error log noise in `extractKeyFromS3Url` catch path: plain S3 keys no longer log as errors, only inputs that start with http(s):// do - Fix test env var ordering so module-level constants pick up `AWS_BUCKET_NAME` and `S3_URL_EXPIRY_SECONDS` correctly before the module is required - Add missing `deleteRagFile` mock and assertion in `deleteFileFromS3` tests - Add `AWS_BUCKET_NAME` cleanup to `afterEach` to prevent cross-test pollution - Add `initializeS3` unit tests covering endpoint, forcePathStyle, credentials, singleton, and IRSA code paths - Document `AWS_FORCE_PATH_STYLE` in `.env.example`, `dotenv.mdx`, and `s3.mdx` * 🪣 fix: Enhance S3 URL key extraction for custom endpoints Updated `extractKeyFromS3Url` to support precise key extraction when using custom endpoints with path-style URLs. The logic now accounts for the `AWS_ENDPOINT_URL` and `AWS_FORCE_PATH_STYLE` environment variables, ensuring correct key handling for various S3-compatible providers. Added unit tests to verify the new functionality, including scenarios for endpoints with base paths. This improves compatibility and reduces potential errors when interacting with S3-like services.
This commit is contained in:
parent
b7bfdfa8b2
commit
7692fa837e
5 changed files with 182 additions and 2 deletions
|
|
@ -19,6 +19,7 @@ jest.mock('@aws-sdk/client-s3');
|
|||
jest.mock('@librechat/api', () => ({
|
||||
initializeS3: jest.fn(),
|
||||
deleteRagFile: jest.fn().mockResolvedValue(undefined),
|
||||
isEnabled: jest.fn((val) => val === 'true'),
|
||||
}));
|
||||
|
||||
jest.mock('@librechat/data-schemas', () => ({
|
||||
|
|
@ -841,5 +842,35 @@ describe('S3 CRUD Operations', () => {
|
|||
),
|
||||
).toBe('images/user123/avatar.png');
|
||||
});
|
||||
|
||||
it('should use endpoint base path when AWS_ENDPOINT_URL and AWS_FORCE_PATH_STYLE are set', () => {
|
||||
process.env.AWS_BUCKET_NAME = 'test-bucket';
|
||||
process.env.AWS_ENDPOINT_URL = 'https://minio.example.com';
|
||||
process.env.AWS_FORCE_PATH_STYLE = 'true';
|
||||
jest.resetModules();
|
||||
const { extractKeyFromS3Url: fn } = require('~/server/services/Files/S3/crud');
|
||||
|
||||
expect(fn('https://minio.example.com/test-bucket/images/user123/file.jpg')).toBe(
|
||||
'images/user123/file.jpg',
|
||||
);
|
||||
|
||||
delete process.env.AWS_ENDPOINT_URL;
|
||||
delete process.env.AWS_FORCE_PATH_STYLE;
|
||||
});
|
||||
|
||||
it('should handle endpoint with a base path', () => {
|
||||
process.env.AWS_BUCKET_NAME = 'test-bucket';
|
||||
process.env.AWS_ENDPOINT_URL = 'https://example.com/storage/';
|
||||
process.env.AWS_FORCE_PATH_STYLE = 'true';
|
||||
jest.resetModules();
|
||||
const { extractKeyFromS3Url: fn } = require('~/server/services/Files/S3/crud');
|
||||
|
||||
expect(fn('https://example.com/storage/test-bucket/images/user123/file.jpg')).toBe(
|
||||
'images/user123/file.jpg',
|
||||
);
|
||||
|
||||
delete process.env.AWS_ENDPOINT_URL;
|
||||
delete process.env.AWS_FORCE_PATH_STYLE;
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue