diff --git a/deploy/aws-sam/.gitignore b/deploy/aws-sam/.gitignore new file mode 100644 index 0000000000..3408fb475e --- /dev/null +++ b/deploy/aws-sam/.gitignore @@ -0,0 +1,77 @@ +# SAM build artifacts +.aws-sam/ +samconfig.toml.bak + +# Environment files +.env +.env.local +.env.production +.env.staging +.env.development +.librechat-deploy-config* + +# AWS credentials +.aws/ + +# Logs +*.log +logs/ + +# OS generated files +.DS_Store +.DS_Store? +._* +.Spotlight-V100 +.Trashes +ehthumbs.db +Thumbs.db + +# IDE files +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# Temporary files +*.tmp +*.temp +.cache/ +donotcommit.txt +repo/ + +# Node modules (if any) +node_modules/ + +# Python +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +*.egg-info/ +.installed.cfg +*.egg + +# Secrets and sensitive data +secrets.yaml +secrets.json +*.pem +*.key +*.crt + +# Backup files +*.bak +*.backup \ No newline at end of file diff --git a/deploy/aws-sam/README.md b/deploy/aws-sam/README.md new file mode 100644 index 0000000000..02cd4ab309 --- /dev/null +++ b/deploy/aws-sam/README.md @@ -0,0 +1,724 @@ +# LibreChat AWS SAM Deployment + +This repository contains AWS SAM templates and scripts to deploy LibreChat on AWS with maximum scalability and high availability. + +## What is LibreChat? + +LibreChat is an enhanced, open-source ChatGPT clone that provides: +- **Multi-AI Provider Support**: OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, and more +- **Advanced Features**: Agents, function calling, file uploads, conversation search, code interpreter +- **Secure Multi-User**: Authentication, user management, conversation privacy +- **Extensible**: Plugin system, custom endpoints, RAG integration +- **Self-Hosted**: Complete control over your data and infrastructure + +## Architecture Overview + +This deployment creates a highly scalable, production-ready LibreChat environment optimized for enterprise use: + +### Core Infrastructure (Scalability-First Design) +- **ECS Fargate**: Serverless container orchestration with auto-scaling (2-20 instances) +- **Application Load Balancer**: High availability with health checks and SSL termination +- **VPC**: Multi-AZ setup with public/private subnets and flexible internet connectivity options +- **Internet Connectivity**: Choose between NAT Gateways (standard AWS pattern) or Transit Gateway (existing infrastructure) +- **Auto Scaling**: CPU-based scaling with target tracking (70% CPU utilization) + +### Data & Storage Layer +- **DocumentDB**: MongoDB-compatible database with multi-AZ deployment and automatic failover +- **ElastiCache Redis**: In-memory caching, session storage, and conversation search with failover +- **S3**: Encrypted file storage for user uploads, avatars, documents, and static assets + + + +### Internet Connectivity Options + +The deployment supports two network connectivity patterns: + +**Option 1: NAT Gateway (Standard AWS Pattern)** +- **High Availability**: NAT Gateways in each AZ with automatic failover +- **Enterprise Performance**: Up to 45 Gbps bandwidth per gateway +- **Zero Maintenance**: Fully managed by AWS with 99.95% SLA +- **Cost**: ~$90/month for 2 NAT Gateways + data processing fees +- **Use Case**: New deployments or when maximum reliability is required + +**Option 2: Transit Gateway (Existing Infrastructure)** +- **Cost Optimization**: No NAT Gateway costs (~$90/month savings) +- **Existing Infrastructure**: Leverages existing Transit Gateway setup +- **Controlled Routing**: Uses existing network policies and routing +- **Use Case**: Organizations with existing Transit Gateway infrastructure + +### Security & Monitoring +- **Secrets Manager**: Secure storage for database passwords, JWT secrets, and API keys +- **CloudWatch**: Centralized logging, monitoring, and alerting +- **Security Groups**: Network-level security with least privilege access +- **IAM Roles**: Fine-grained permissions for ECS tasks and AWS service access + +### Advanced Scalability Features +- **Fargate Spot Integration**: 80% Spot instances + 20% On-Demand for cost optimization +- **Multi-AZ High Availability**: Automatic failover across multiple availability zones +- **Horizontal Auto Scaling**: Scales from 2-20 instances based on CPU utilization +- **Load Balancing**: Intelligent traffic distribution across healthy instances +- **Container Health Checks**: Automatic replacement of unhealthy containers +- **Database Read Replicas**: DocumentDB supports read scaling for high-traffic scenarios +- **Redis Clustering**: ElastiCache supports cluster mode for memory scaling + +## Prerequisites + +1. **AWS CLI** - [Installation Guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) +2. **SAM CLI** - [Installation Guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +3. **AWS Account** with appropriate permissions and network topology +4. **Domain & SSL Certificate** (for custom domain) +5. **AWS Cognito User Pool** (optional - for SSO authentication) + +### SSO Prerequisites (Optional) +If you plan to use SSO authentication: +- **AWS Cognito User Pool** with configured identity providers +- **App Client** created in the Cognito User Pool with appropriate settings +- **Identity Provider** (SAML, OIDC, or social) configured in Cognito +- **Attribute mappings** configured in Cognito for user claims (name, email) + +### Required AWS Permissions + +Your AWS user/role needs permissions for: +- CloudFormation (full access) +- ECS (full access) +- EC2 (VPC, Security Groups, Load Balancers) +- DocumentDB (full access) +- ElastiCache (full access) +- S3 (bucket creation and management) +- IAM (role creation) +- Secrets Manager (secret creation) +- CloudWatch (log groups) +- STS (checking caller identity) + +## Quick Start + +### Interactive Deployment (Recommended) + +1. **Clone and configure:** + ```bash + git clone + cd librechat-aws-sam + + # Configure AWS credentials + aws configure + ``` + +2. **Run interactive deployment:** + ```bash + ./deploy-clean.sh + ``` + + The script will interactively prompt for: + - Environment (dev/staging/prod) + - AWS region + - Stack name + - Internet connectivity option (NAT Gateway vs Transit Gateway) + - VPC ID (with helpful VPC listing) + - Public subnet IDs (for load balancer) + - Private subnet IDs (for ECS tasks and databases) + - AWS Bedrock credentials for AI model access + - Optional SSO configuration with AWS Cognito + - Optional domain name and SSL certificate + +3. **Save configuration for future deployments:** + The script automatically offers to save your configuration to `.librechat-deploy-config` + +4. **Redeploy with saved configuration:** + ```bash + ./deploy-clean.sh --load-config + ``` + +5. **Update YAML config file only option:** + To update yaml config file and restart containers only + ```bash + ./deploy-clean.sh --update-config + ``` + + +## Deployment Options + +### Interactive Deployment (Recommended) +```bash +# First-time deployment +./deploy-clean.sh + +# Redeploy with saved configuration +./deploy-clean.sh --load-config + +# Reset saved configuration +./deploy-clean.sh --reset-config + +# Update yaml config file and restart containers only +./deploy-clean.sh --update-config +``` + +The interactive deployment provides: +- **Guided Setup**: Step-by-step prompts for all parameters +- **AWS Resource Discovery**: Lists available VPCs and subnets +- **Validation**: Checks VPC and subnet accessibility +- **Configuration Persistence**: Saves settings for future deployments +- **Smart Defaults**: Remembers previous choices + +## Configuration + +### Deploy script configuration (`.librechat-deploy-config`) + +The deploy script saves your choices to `.librechat-deploy-config` and reloads them with `--load-config`. You can also edit this file to set or change options without re-prompting. + +**Optional: Custom container image (`LIBRECHAT_IMAGE`)** + +By default, the stack uses the container image defined in the template (e.g. the official `librechat/librechat:latest` or a template default). To use a custom image (e.g. your own ECR build), set `LIBRECHAT_IMAGE` in your deploy config: + +```bash +LIBRECHAT_IMAGE=".dkr.ecr..amazonaws.com/:" +``` + +Then deploy with the config loaded so the parameter is applied: + +```bash +./deploy-clean.sh --load-config +``` + +If `LIBRECHAT_IMAGE` is unset or empty, the template’s default image is used. + +### Environment Variables + +The deployment automatically configures these environment variables for LibreChat: + +**Core Application Settings:** +- `NODE_ENV`: Set to "production" +- `MONGO_URI`: DocumentDB connection string with SSL and authentication +- `REDIS_URI`: ElastiCache Redis connection string +- `NODE_TLS_REJECT_UNAUTHORIZED`: Set to "0" for DocumentDB SSL compatibility +- `ALLOW_REGISTRATION`: Set to "false" (configure SAML post-deployment) + +**Security & Authentication:** +- `JWT_SECRET`: Auto-generated secure JWT secret (stored in Secrets Manager) +- `JWT_REFRESH_SECRET`: Auto-generated refresh token secret (stored in Secrets Manager) +- `CREDS_KEY`: Auto-generated credentials encryption key (stored in Secrets Manager) +- `CREDS_IV`: Auto-generated encryption IV (stored in Secrets Manager) + +**SSO Authentication (Optional):** +- `ENABLE_SSO`: Set to "true" to enable SSO authentication +- `COGNITO_USER_POOL_ID`: AWS Cognito User Pool ID +- `OPENID_CLIENT_ID`: App Client ID from Cognito User Pool +- `OPENID_CLIENT_SECRET`: App Client Secret from Cognito User Pool +- `OPENID_SCOPE`: OpenID scope for authentication (default: `openid profile email`) +- `OPENID_BUTTON_LABEL`: Login button text (default: `Sign in with SSO`) +- `OPENID_NAME_CLAIM`: Name attribute mapping (default: `name`) +- `OPENID_EMAIL_CLAIM`: Email attribute mapping (default: `email`) +- `OPENID_SESSION_SECRET`: Auto-generated session secret (stored in Secrets Manager) +- `OPENID_ISSUER`: Auto-configured Cognito issuer URL +- `OPENID_CALLBACK_URL`: Auto-configured callback URL (`/oauth/openid/callback`) + +**AWS Bedrock Configuration:** +- `AWS_REGION`: Deployment region for AWS services +- `BEDROCK_AWS_DEFAULT_REGION`: AWS region for Bedrock API calls +- `BEDROCK_AWS_ACCESS_KEY_ID`: AWS access key for Bedrock access (from deployment parameters) +- `BEDROCK_AWS_SECRET_ACCESS_KEY`: AWS secret key for Bedrock access (from deployment parameters) +- `BEDROCK_AWS_MODELS`: Pre-configured Bedrock models including: + - `us.anthropic.claude-3-7-sonnet-20250219-v1:0` + - `us.anthropic.claude-opus-4-20250514-v1:0` + - `us.anthropic.claude-sonnet-4-20250514-v1:0` + - `us.anthropic.claude-3-5-haiku-20241022-v1:0` + - `us.meta.llama3-3-70b-instruct-v1:0` + - `us.amazon.nova-pro-v1:0` + +**Configuration Management:** +- `CONFIG_PATH`: Set to "/app/config/librechat.yaml" (mounted from EFS) +- `CACHE`: Set to "false" to disable prompt caching (avoids Bedrock caching issues) + +### EFS Configuration System: + +The deployment includes an EFS-based configuration management system: +- **Real-time Updates**: Configuration changes without container rebuilds +- **S3 β†’ EFS Pipeline**: Automated sync from S3 to EFS via Lambda +- **Container Mounting**: EFS volume mounted at `/app/config/librechat.yaml` and CONFIG_PATH environmental variable set to match it +- **Update Commands**: Use `./deploy-clean.sh --update-config` for config-only updates + +### Scaling Configuration + +Default scaling settings: +- **Min Capacity**: 2 instances +- **Max Capacity**: 20 instances +- **Target CPU**: 70% utilization +- **Scale Out Cooldown**: 5 minutes +- **Scale In Cooldown**: 5 minutes + +To modify scaling, edit the `ECSAutoScalingTarget` and `ECSAutoScalingPolicy` resources in `template.yaml`. + +### Database Configuration + +**DocumentDB (MongoDB-compatible):** +- Instance Class: `db.t3.medium` (2 instances) +- Backup Retention: 7 days +- Encryption: Enabled +- Multi-AZ: Yes + +**ElastiCache Redis:** +- Node Type: `cache.t3.micro` (2 nodes) +- Engine Version: 7.0 +- Encryption: At-rest and in-transit +- Multi-AZ: Yes with automatic failover + +## LibreChat Dependencies & Features + +### Core Dependencies Deployed +- **MongoDB/DocumentDB**: Primary database for conversations, users, and metadata +- **Redis/ElastiCache**: Session management, caching, and real-time features +- **S3**: File storage with support for multiple strategies: + - **Avatars**: User and agent profile images + - **Images**: Chat image uploads and generations + - **Documents**: PDF uploads, text files, and attachments + - **Static Assets**: CSS, JavaScript, and other static content + +### Optional Components (Can Be Added) +- **Meilisearch**: Full-text search for conversation history with typo tolerance +- **Vector Database**: For RAG (Retrieval-Augmented Generation) functionality +- **CDN**: CloudFront integration for global content delivery + +### File Storage Strategies +LibreChat supports multiple storage strategies that can be mixed: +- **S3**: Scalable cloud storage (configured in this deployment) + + +## Post-Deployment Setup + +### 1. Access LibreChat +After deployment completes (15-20 minutes), access LibreChat using the Load Balancer URL: + +```bash +# Get the application URL +aws cloudformation describe-stacks \ + --stack-name librechat \ + --query 'Stacks[0].Outputs[?OutputKey==`LoadBalancerURL`].OutputValue' \ + --output text +``` + +The application will be available at: `http://your-load-balancer-url` (or `https://` if you configured SSL) + +### 2. Initial Admin Setup +1. **First User Registration**: The first user to register becomes the admin + + +### 3. Configure SSO Authentication (Optional) + +**Prerequisites:** +- AWS Cognito User Pool created and configured +- App Client created in the User Pool with appropriate settings +- Identity Provider configured in Cognito (SAML, OIDC, or social providers) +- Attribute mappings configured in Cognito + +**SSO Configuration Options:** + +The deployment supports optional SSO authentication through AWS Cognito with OpenID Connect: + +**Required SSO Settings:** +- `ENABLE_SSO`: Set to "true" to enable SSO authentication +- `COGNITO_USER_POOL_ID`: Your AWS Cognito User Pool ID (e.g., `us-east-1_8o9DM3lHZ`) +- `OPENID_CLIENT_ID`: App Client ID from your Cognito User Pool +- `OPENID_CLIENT_SECRET`: App Client Secret from your Cognito User Pool + +**Optional SSO Settings:** +- `OPENID_SCOPE`: OpenID scope for authentication (default: `openid profile email`) +- `OPENID_BUTTON_LABEL`: Login button text (default: `Sign in with SSO`) +- `OPENID_NAME_CLAIM`: Name attribute mapping (default: `name`) +- `OPENID_EMAIL_CLAIM`: Email attribute mapping (default: `email`) + +**Automatic Configuration:** +The deployment automatically configures: +- `OPENID_ISSUER`: Cognito issuer URL (`https://cognito-idp.{region}.amazonaws.com/{user-pool-id}`) +- `OPENID_CALLBACK_URL`: OAuth callback URL (`/oauth/openid/callback`) +- `OPENID_SESSION_SECRET`: Secure session secret (auto-generated and stored in Secrets Manager) + +**Configuration Methods:** + +1. **During Deployment**: The interactive deployment script will prompt for SSO settings +2. **Post-Deployment**: Update the CloudFormation stack with SSO parameters +3. **Environment Variables**: Configure directly in the ECS task definition + +**SSO Setup Steps:** + +1. **Create AWS Cognito User Pool**: + - Create a new User Pool in AWS Cognito + - Configure sign-in options (email, username, etc.) + - Set up password policies and MFA if desired + - Configure attribute mappings for name and email + +2. **Create App Client**: + - Create an App Client in your User Pool + - Enable "Generate client secret" + - Configure OAuth 2.0 settings: + - Allowed OAuth Flows: Authorization code grant + - Allowed OAuth Scopes: openid, profile, email + - Callback URLs: `https://your-domain/oauth/openid/callback` + - Sign out URLs: `https://your-domain` + +3. **Configure Identity Provider (Optional)**: + - Add SAML, OIDC, or social identity providers to Cognito + - Configure attribute mappings between IdP and Cognito + - Test the identity provider integration + +4. **Deploy with SSO**: + ```bash + ./deploy-clean.sh + # Choose "y" when prompted for SSO configuration + # Provide the required Cognito User Pool ID, Client ID, and Client Secret + ``` + +5. **Verify SSO Integration**: + - Access LibreChat URL + - Click the SSO login button (customizable label) + - Complete authentication flow through Cognito + - Verify user attributes are mapped correctly + +**Important Notes:** +- SSO configuration is completely optional +- If SSO is not configured, LibreChat uses standard email/password authentication +- SSO settings can be added or modified after initial deployment +- Ensure Cognito User Pool and App Client configuration is complete before enabling SSO +- The callback URL must match exactly what's configured in your Cognito App Client + +**Adding SSO After Initial Deployment:** + +If you deployed without SSO initially, you can add it later: + +1. **Update CloudFormation Stack**: + ```bash + aws cloudformation update-stack \ + --stack-name your-stack-name \ + --use-previous-template \ + --parameters ParameterKey=EnableSSO,ParameterValue="true" \ + ParameterKey=CognitoUserPoolId,ParameterValue="your-user-pool-id" \ + ParameterKey=OpenIdClientId,ParameterValue="your-client-id" \ + ParameterKey=OpenIdClientSecret,ParameterValue="your-client-secret" \ + --capabilities CAPABILITY_IAM + ``` + +2. **Or Re-run Deployment Script**: + ```bash + ./deploy-clean.sh --load-config + # Choose "y" for SSO configuration when prompted + ``` + +**Supported Identity Providers:** +Through AWS Cognito, you can integrate with: +- **SAML 2.0**: Enterprise identity providers (Active Directory, Okta, etc.) +- **OpenID Connect**: OIDC-compliant providers +- **Social Providers**: Google, Facebook, Amazon, Apple +- **Custom Providers**: Any OAuth 2.0 or SAML 2.0 compliant system + +### 4. Set Up AI Provider API Keys +Configure your AI providers in the LibreChat interface: + +**Supported Providers:** +- **OpenAI**: GPT-4, GPT-3.5, DALL-E, Whisper +- **Anthropic**: Claude 3.5 Sonnet, Claude 3 Opus/Haiku +- **Google**: Gemini Pro, Gemini Vision +- **Azure OpenAI**: Enterprise OpenAI models +- **AWS Bedrock**: Claude, Titan, Llama models +- **Groq**: Fast inference for Llama, Mixtral +- **OpenRouter**: Access to multiple model providers +- **Custom Endpoints**: Any OpenAI-compatible API + +**Configuration Methods:** +- **Environment Variables**: Pre-configure in deployment (more secure) +- **YAML FILE**: Certain configuration options are configured via librechat.yaml + + + +### 5. Advanced Configuration Options + + + +### 6. Monitoring & Maintenance + +**CloudWatch Dashboards:** +- ECS service metrics (CPU, memory, task count) +- Load balancer performance (response time, error rates) +- Database metrics (DocumentDB and Redis) +- Application logs and error tracking + +**Automated Scaling:** +- Monitors CPU utilization (target: 70%) +- Scales from 2-20 instances automatically +- Uses 80% Spot instances for cost optimization + +**Health Checks:** +- Application-level health checks +- Database connectivity monitoring +- Automatic unhealthy task replacement + +## Monitoring and Maintenance + +### CloudWatch Logs +View application logs: +```bash +aws logs tail /ecs/librechat --follow +``` + +### ECS Service Status +Check service health: +```bash +aws ecs describe-services --cluster librechat-cluster --services librechat-service +``` + +### Database Monitoring +- DocumentDB metrics available in CloudWatch +- ElastiCache Redis metrics and performance insights +- Set up CloudWatch alarms for critical metrics + +### Cost Optimization +- Monitor Fargate Spot vs On-Demand usage +- Review DocumentDB and ElastiCache instance sizes +- Set up billing alerts + + +## Scaling Considerations + +### Horizontal Scaling (Automatic) +The deployment automatically handles horizontal scaling: + +**ECS Auto Scaling:** +- **Minimum**: 2 instances (high availability) +- **Maximum**: 20 instances (configurable) +- **Trigger**: 70% CPU utilization average +- **Scale Out**: Add instances when CPU > 70% for 5 minutes +- **Scale In**: Remove instances when CPU < 70% for 5 minutes +- **Cooldown**: 5-minute intervals between scaling actions + +**Database Scaling:** +- **DocumentDB**: Supports up to 15 read replicas for read scaling +- **ElastiCache Redis**: Supports cluster mode for memory scaling +- **Connection Pooling**: Efficient database connection management + +### Vertical Scaling (Manual) +For higher per-instance performance: + +**ECS Task Scaling:** +```yaml +# In template.yaml, modify: +Cpu: 2048 # Double CPU (1024 -> 2048) +Memory: 4096 # Double memory (2048 -> 4096) +``` + +**Database Scaling:** +```yaml +# Upgrade DocumentDB instances: +DBInstanceClass: db.r5.large # From db.t3.medium +DBInstanceClass: db.r5.xlarge # For heavy workloads + +# Upgrade Redis instances: +NodeType: cache.r6g.large # From cache.t3.micro +``` + +### Global Scaling (Multi-Region) +For worldwide deployment: + + + +### Load Testing +Before production deployment, perform load testing: + +```bash +# Example load test with Apache Bench +ab -n 10000 -c 100 http://your-load-balancer-url/ + +# Or use more sophisticated tools: +# - Artillery.io for API testing +# - JMeter for comprehensive testing +# - Locust for Python-based testing +``` + +### Capacity Planning +Plan for growth with these guidelines: + +**User Scaling:** +- **Light Users**: 1 instance per 100 concurrent users +- **Medium Users**: 1 instance per 50 concurrent users +- **Heavy Users**: 1 instance per 25 concurrent users + +**Database Scaling:** +- **DocumentDB**: 1000 connections per db.t3.medium +- **Redis**: 65,000 connections per cache.t3.micro +- **Storage**: Plan 1GB per 1000 conversations + +## Security Best Practices + +### Network Security +- All databases in private subnets +- Security groups with minimal required access +- Optional NAT gateways or Transit Gateway for outbound internet access +- Flexible internet connectivity based on existing infrastructure + +### Data Security +- Encryption at rest for all data stores +- Encryption in transit for Redis +- S3 bucket encryption and versioning +- Secrets Manager for sensitive data + +### Access Control +- IAM roles with least privilege +- ECS task roles for service-specific permissions +- No hardcoded credentials + +## Troubleshooting + +### Common Issues + +**Deployment Fails:** +```bash +# Check CloudFormation events +aws cloudformation describe-stack-events --stack-name librechat + +# Check SAM logs +sam logs -n ECSService --stack-name librechat +``` + +**Service Won't Start:** +```bash +# Check ECS task logs +aws ecs describe-tasks --cluster librechat-cluster --tasks + +# Check CloudWatch logs +aws logs tail /ecs/librechat --follow +``` + +**Database Connection Issues:** +- Verify security group rules +- Check DocumentDB cluster status +- Validate connection strings in Secrets Manager + +### Performance Issues +- Monitor ECS service CPU/memory utilization +- Check DocumentDB performance insights +- Review ElastiCache Redis metrics +- Analyze ALB target group health + +## Cleanup + +To remove all resources: +```bash +aws cloudformation delete-stack --stack-name librechat +``` + +**Note:** This will delete all data. Ensure you have backups if needed. + +## Cost Optimization & Estimation + +### Cost Optimization Features +This deployment is optimized for cost efficiency while maintaining high availability: + +**Fargate Spot Integration:** +- **80% Spot Instances**: Up to 70% cost savings on compute +- **20% On-Demand**: Ensures availability during Spot interruptions +- **Automatic Failover**: Seamless transition between Spot and On-Demand + +**Right-Sizing Strategy:** +- **Auto Scaling**: Only pay for resources you need (2-20 instances) +- **Efficient Instance Types**: Optimized CPU/memory ratios +- **Database Optimization**: DocumentDB and Redis sized for typical workloads + +**Storage Optimization:** +- **S3 Intelligent Tiering**: Automatic cost optimization for file storage +- **Lifecycle Policies**: Automatic cleanup of incomplete uploads +- **Compression**: Efficient storage of conversation data + +### Monthly Cost Estimation (US-East-1) + +**Base Infrastructure (Minimum 2 instances):** +- **ECS Fargate (2 instances)**: ~$30-50/month + - 80% Spot pricing: ~$24-40/month + - 20% On-Demand: ~$6-10/month +- **DocumentDB (2x db.t3.medium)**: ~$100-120/month +- **ElastiCache Redis (2x cache.t3.micro)**: ~$30-40/month +- **Application Load Balancer**: ~$20/month +- **NAT Gateway (2 AZs) - Optional**: ~$90/month + - **Base cost**: $45/month per NAT Gateway Γ— 2 = $90/month + - **Data processing**: $0.045 per GB processed + - **High availability**: Automatic failover between AZs + - **Performance**: Up to 45 Gbps bandwidth per gateway +- **S3 Storage**: ~$5-25/month (depending on usage) +- **Data Transfer**: ~$10-30/month (depending on traffic) + +**Total Monthly Cost Ranges:** + +**With NAT Gateways (Standard AWS Pattern):** +- **Light Usage (2-3 instances)**: ~$285-335/month +- **Medium Usage (5-8 instances)**: ~$380-480/month +- **Heavy Usage (10-20 instances)**: ~$530-830/month + +**Without NAT Gateways (Transit Gateway Pattern):** +- **Light Usage (2-3 instances)**: ~$195-245/month +- **Medium Usage (5-8 instances)**: ~$290-390/month +- **Heavy Usage (10-20 instances)**: ~$440-740/month + +**NAT Gateway vs Transit Gateway Comparison:** +- **NAT Gateway Benefits**: 99.95% SLA, zero maintenance, 45 Gbps performance, built-in DDoS protection +- **Transit Gateway Benefits**: ~$90/month cost savings, leverages existing infrastructure, centralized routing +- **Cost Difference**: ~$90/month for NAT Gateway option +- **Performance**: NAT Gateway typically faster for internet access, Transit Gateway may have additional latency + +**Cost Comparison:** +- **Traditional EC2**: 40-60% more expensive +- **Managed Services**: 70-80% more expensive than self-managed +- **Multi-Cloud**: This deployment is 50-70% cheaper than equivalent GCP/Azure + +### Cost Monitoring & Alerts +- **AWS Cost Explorer**: Track spending by service +- **Billing Alerts**: Set up budget notifications +- **Resource Tagging**: Track costs by environment/team +- **Spot Instance Savings**: Monitor Spot vs On-Demand usage + +### Additional Cost Optimization Tips +1. **Use Reserved Instances**: For DocumentDB if usage is predictable +2. **Enable S3 Intelligent Tiering**: Automatic storage class optimization +3. **Monitor Data Transfer**: Optimize between AZs and regions +4. **Regular Cleanup**: Remove unused resources and old backups +5. **Right-Size Databases**: Monitor and adjust instance types based on usage + +## Support + +For issues related to: +- **LibreChat**: [LibreChat GitHub](https://github.com/danny-avila/LibreChat) +- **AWS SAM**: [AWS SAM Documentation](https://docs.aws.amazon.com/serverless-application-model/) +- **This deployment**: Create an issue in this repository + +## License + +This deployment template is provided under the MIT License. LibreChat itself is licensed under the MIT License. \ No newline at end of file diff --git a/deploy/aws-sam/deploy-clean.sh b/deploy/aws-sam/deploy-clean.sh new file mode 100644 index 0000000000..0aeb96fe42 --- /dev/null +++ b/deploy/aws-sam/deploy-clean.sh @@ -0,0 +1,1009 @@ +#!/bin/bash + +# LibreChat AWS SAM Deployment Script +# Interactive deployment for existing VPC infrastructure with Transit Gateway + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +MAGENTA='\033[0;35m' +NC='\033[0m' # No Color + +# Configuration file +CONFIG_FILE=".librechat-deploy-config" + +# Default values +ENVIRONMENT="prod" +REGION="us-east-1" +STACK_NAME="librechat" +DOMAIN_NAME="" +CERTIFICATE_ARN="" +VPC_ID="" +PUBLIC_SUBNETS="" +PRIVATE_SUBNETS="" +AWS_ACCESS_KEY_ID="" +AWS_SECRET_ACCESS_KEY="" +ENABLE_SSO="false" +COGNITO_USER_POOL_ID="" +OPENID_CLIENT_ID="" +OPENID_CLIENT_SECRET="" +OPENID_SCOPE="openid profile email" +OPENID_BUTTON_LABEL="Sign in with SSO" +OPENID_IMAGE_URL="" +OPENID_NAME_CLAIM="name" +OPENID_EMAIL_CLAIM="email" +HELP_AND_FAQ_URL="" +CREATE_NAT_GATEWAY="false" +# Set to false when deploying staging/prod in same VPC as dev (dev already created Secrets Manager VPC endpoint with private DNS) +CREATE_SECRETS_MANAGER_VPC_ENDPOINT="true" +# Optional: set to override LibreChatImage (e.g. custom ECR image); when unset, template default is used +LIBRECHAT_IMAGE="" +# MCP secrets (optional; one per MCP server) +MCP_CONGRESS_TOKEN="" +MCP_EASTERN_TIME_TOKEN="" + +# Function to generate random string +generate_random_string() { + local length="$1" + openssl rand -base64 "$length" | tr -d "=+/" | cut -c1-"$length" +} + +# Function to print colored output +print_status() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +print_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}[WARNING]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +print_prompt() { + echo -e "${CYAN}[INPUT]${NC} $1" +} + +print_important() { + echo -e "${MAGENTA}[IMPORTANT]${NC} $1" +} + +# Function to show usage +usage() { + echo "LibreChat AWS SAM Interactive Deployment" + echo "" + echo "Usage: $0 [OPTIONS]" + echo "" + echo "Options:" + echo " --load-config Load configuration from saved file" + echo " --reset-config Delete saved configuration and start fresh" + echo " --update-config Update config file and restart containers (no full deployment)" + echo " -h, --help Show this help message" + echo "" + echo "The script will interactively prompt for all required parameters" + echo "and save them to '$CONFIG_FILE' for future deployments." +} + +# Function to save configuration +save_config() { + cat > "$CONFIG_FILE" << EOF +# LibreChat Deployment Configuration +# Generated on $(date) +ENVIRONMENT="$ENVIRONMENT" +REGION="$REGION" +STACK_NAME="$STACK_NAME" +DOMAIN_NAME="$DOMAIN_NAME" +CERTIFICATE_ARN="$CERTIFICATE_ARN" +VPC_ID="$VPC_ID" +PUBLIC_SUBNETS="$PUBLIC_SUBNETS" +PRIVATE_SUBNETS="$PRIVATE_SUBNETS" +AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" +AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" +ENABLE_SSO="$ENABLE_SSO" +COGNITO_USER_POOL_ID="$COGNITO_USER_POOL_ID" +OPENID_CLIENT_ID="$OPENID_CLIENT_ID" +OPENID_CLIENT_SECRET="$OPENID_CLIENT_SECRET" +OPENID_SCOPE="$OPENID_SCOPE" +OPENID_BUTTON_LABEL="$OPENID_BUTTON_LABEL" +OPENID_IMAGE_URL="$OPENID_IMAGE_URL" +OPENID_NAME_CLAIM="$OPENID_NAME_CLAIM" +OPENID_EMAIL_CLAIM="$OPENID_EMAIL_CLAIM" +HELP_AND_FAQ_URL="$HELP_AND_FAQ_URL" +CREATE_NAT_GATEWAY="$CREATE_NAT_GATEWAY" +CREATE_SECRETS_MANAGER_VPC_ENDPOINT="$CREATE_SECRETS_MANAGER_VPC_ENDPOINT" +LIBRECHAT_IMAGE="$LIBRECHAT_IMAGE" +MCP_CONGRESS_TOKEN="$MCP_CONGRESS_TOKEN" +MCP_EASTERN_TIME_TOKEN="$MCP_EASTERN_TIME_TOKEN" +EOF + print_success "Configuration saved to $CONFIG_FILE" +} + +# Function to load configuration +load_config() { + if [[ -f "$CONFIG_FILE" ]]; then + print_status "Loading configuration from $CONFIG_FILE" + source "$CONFIG_FILE" + print_success "Configuration loaded successfully" + return 0 + else + print_warning "No configuration file found at $CONFIG_FILE" + return 1 + fi +} + +# Function to prompt for input with default value +prompt_input() { + local prompt="$1" + local default="$2" + local var_name="$3" + local current_value="${!var_name}" + + if [[ -n "$current_value" ]]; then + default="$current_value" + fi + + if [[ -n "$default" ]]; then + print_prompt "$prompt [$default]: " + read -r input + if [[ -z "$input" ]]; then + input="$default" + fi + else + print_prompt "$prompt: " + read -r input + while [[ -z "$input" ]]; do + print_error "This field is required!" + print_prompt "$prompt: " + read -r input + done + fi + + eval "$var_name=\"$input\"" +} + +# Function to validate AWS CLI and credentials +validate_aws() { + if ! command -v aws &> /dev/null; then + print_error "AWS CLI is not installed. Please install it first." + exit 1 + fi + + if ! aws sts get-caller-identity &> /dev/null; then + print_error "AWS credentials not configured. Please run 'aws configure' first." + exit 1 + fi +} + +# Function to ensure ECS service-linked role exists +ensure_ecs_service_role() { + local region="$1" + + print_status "Checking ECS service-linked role..." + + # Check if the service-linked role exists + if aws iam get-role --role-name AWSServiceRoleForECS --region "$region" &>/dev/null; then + print_success "ECS service-linked role already exists" + return 0 + fi + + print_status "Creating ECS service-linked role..." + + # Create the service-linked role + if aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com --region "$region" &>/dev/null; then + print_success "ECS service-linked role created successfully" + + # Wait a moment for the role to propagate + print_status "Waiting for role to propagate..." + sleep 10 + + return 0 + else + # Check if it failed because the role already exists + if aws iam get-role --role-name AWSServiceRoleForECS --region "$region" &>/dev/null; then + print_success "ECS service-linked role already exists (created by another process)" + return 0 + else + print_error "Failed to create ECS service-linked role" + print_error "Please create it manually with: aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com" + return 1 + fi + fi +} + +# Function to validate AWS Bedrock access +validate_bedrock_access() { + local access_key="$1" + local secret_key="$2" + local region="$3" + + print_status "Validating AWS Bedrock access..." + + # Test Bedrock access with provided credentials + AWS_ACCESS_KEY_ID="$access_key" AWS_SECRET_ACCESS_KEY="$secret_key" \ + aws bedrock list-foundation-models --region "$region" &>/dev/null + + if [[ $? -eq 0 ]]; then + print_success "AWS Bedrock access validated successfully" + return 0 + else + print_warning "Could not validate Bedrock access. Please ensure:" + print_warning "1. The credentials have Bedrock permissions" + print_warning "2. Bedrock model access is enabled in your AWS account" + print_warning "3. The region supports Bedrock services" + return 1 + fi +} + +# Function to validate SAM CLI +validate_sam() { + if ! command -v sam &> /dev/null; then + print_error "SAM CLI is not installed. Please install it first." + exit 1 + fi +} + +# Function to list VPCs +list_vpcs() { + print_status "Available VPCs in region $REGION:" + AWS_PAGER="" aws ec2 describe-vpcs \ + --region "$REGION" \ + --query 'Vpcs[*].[VpcId,CidrBlock,Tags[?Key==`Name`].Value|[0]]' \ + --output table 2>/dev/null || print_warning "Could not list VPCs" +} + +# Function to list subnets for a VPC +list_subnets() { + local vpc_id="$1" + if [[ -n "$vpc_id" ]]; then + print_status "Available subnets in VPC $vpc_id:" + AWS_PAGER="" aws ec2 describe-subnets \ + --region "$REGION" \ + --filters "Name=vpc-id,Values=$vpc_id" \ + --query 'Subnets[*].[SubnetId,CidrBlock,AvailabilityZone,Tags[?Key==`Name`].Value|[0]]' \ + --output table 2>/dev/null || print_warning "Could not list subnets" + fi +} + +# Function to upload config to S3 +upload_config() { + local stack_name="$1" + local region="$2" + + # Get S3 bucket name from CloudFormation outputs + local bucket_name=$(aws cloudformation describe-stacks \ + --stack-name "$stack_name" \ + --region "$region" \ + --query 'Stacks[0].Outputs[?OutputKey==`S3BucketName`].OutputValue' \ + --output text 2>/dev/null) + + if [[ -z "$bucket_name" ]]; then + print_warning "Could not find S3 bucket name from stack outputs" + return 1 + fi + + if [[ -f "librechat.yaml" ]]; then + print_status "Uploading librechat.yaml to S3..." + aws s3 cp librechat.yaml "s3://$bucket_name/configs/librechat.yaml" \ + --content-type "application/x-yaml" \ + --region "$region" + + if [[ $? -eq 0 ]]; then + print_success "Configuration uploaded to s3://$bucket_name/configs/librechat.yaml" + return 0 + else + print_error "Failed to upload configuration to S3" + return 1 + fi + else + print_warning "librechat.yaml not found - skipping config upload" + print_status "A default configuration will be used" + return 0 + fi +} + +# Function to trigger config update and container restart +update_config() { + local stack_name="$1" + local region="$2" + + print_status "Updating configuration and restarting containers..." + + # Upload config to S3 + if ! upload_config "$stack_name" "$region"; then + return 1 + fi + + # Trigger Config Manager Lambda to copy S3 β†’ EFS + local lambda_name="${stack_name}-config-manager" + print_status "Triggering config manager Lambda: $lambda_name" + + aws lambda invoke \ + --function-name "$lambda_name" \ + --region "$region" \ + --payload '{}' \ + /tmp/lambda-response.json >/dev/null 2>&1 + + if [[ $? -eq 0 ]]; then + print_success "Config manager Lambda executed successfully" + else + print_warning "Could not invoke config manager Lambda (may not exist yet)" + fi + + # Force ECS service to restart containers + local cluster_name=$(aws cloudformation describe-stacks \ + --stack-name "$stack_name" \ + --region "$region" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSClusterName`].OutputValue' \ + --output text 2>/dev/null) + + local service_name=$(aws cloudformation describe-stacks \ + --stack-name "$stack_name" \ + --region "$region" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSServiceName`].OutputValue' \ + --output text 2>/dev/null) + + if [[ -n "$cluster_name" && -n "$service_name" ]]; then + print_status "Restarting ECS containers to pick up new config..." + aws ecs update-service \ + --cluster "$cluster_name" \ + --service "$service_name" \ + --region "$region" \ + --force-new-deployment >/dev/null 2>&1 + + if [[ $? -eq 0 ]]; then + print_success "ECS service restart initiated" + print_status "Containers will restart with the new configuration" + else + print_warning "Could not restart ECS service" + fi + else + print_warning "Could not find ECS cluster/service information" + fi +} + +encode_certificate() { + local cert="$1" + # Remove any existing escaping and re-encode properly + echo "$cert" | sed 's/\\n/\n/g' | tr '\n' '\\n' | sed 's/\\n$//' +} + +validate_subnets() { + local subnet_list="$1" + local subnet_type="$2" + + if [[ -z "$subnet_list" ]]; then + print_error "$subnet_type subnets cannot be empty" + return 1 + fi + + # Convert comma-separated list to array + IFS=',' read -ra subnets <<< "$subnet_list" + + if [[ ${#subnets[@]} -lt 2 ]]; then + print_error "$subnet_type subnets must include at least 2 subnets in different AZs" + return 1 + fi + + print_status "Validating $subnet_type subnets..." + for subnet in "${subnets[@]}"; do + subnet=$(echo "$subnet" | xargs) # trim whitespace + if ! aws ec2 describe-subnets --region "$REGION" --subnet-ids "$subnet" &>/dev/null; then + print_error "Subnet $subnet not found or not accessible" + return 1 + fi + done + + print_success "$subnet_type subnets validated successfully" + return 0 +} + +# Create or update a Secrets Manager secret with a string value; output the secret ARN +ensure_secret_string() { + local name="$1" + local value="$2" + local region="$3" + if aws secretsmanager describe-secret --secret-id "$name" --region "$region" &>/dev/null; then + aws secretsmanager put-secret-value --secret-id "$name" --secret-string "$value" --region "$region" >/dev/null + aws secretsmanager describe-secret --secret-id "$name" --region "$region" --query ARN --output text + else + aws secretsmanager create-secret --name "$name" --secret-string "$value" --region "$region" --query ARN --output text + fi +} + +# Create or update Bedrock credentials secret (JSON with accessKeyId, secretAccessKey); output the secret ARN +ensure_bedrock_credentials_secret() { + local name="$1" + local access_key="$2" + local secret_key="$3" + local region="$4" + [[ -z "$access_key" || -z "$secret_key" ]] && { print_error "Bedrock credentials (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY) are required."; exit 1; } + local json + # Pass AK/SK as env vars so Python can read them (AK="$x" SK="$y" command sets env for command only) + json=$(AK="$access_key" SK="$secret_key" python3 -c 'import json,os; print(json.dumps({"accessKeyId":os.environ.get("AK",""),"secretAccessKey":os.environ.get("SK","")}))') + ensure_secret_string "$name" "$json" "$region" +} + +# Interactive configuration function +interactive_config() { + echo "" + echo "==============================================" + echo " LibreChat AWS SAM Interactive Deployment" + echo "==============================================" + echo "" + + print_important "🌐 NETWORK ARCHITECTURE NOTICE 🌐" + print_important "This deployment supports two network connectivity options:" + print_important "" + print_important "Option 1: Transit Gateway (Recommended for existing infrastructure)" + print_important "βœ… NO NAT GATEWAYS created (saves ~\$90/month)" + print_important "βœ… Uses existing Transit Gateway for internet access" + print_important "βœ… Private subnets remain secure with controlled routing" + print_important "" + print_important "Option 2: NAT Gateway (Standard AWS pattern)" + print_important "β€’ Creates NAT Gateways in each AZ (~\$90/month cost)" + print_important "β€’ Provides direct internet access for private subnets" + print_important "β€’ Higher availability and performance guarantees" + print_important "β€’ No dependency on existing Transit Gateway" + echo "" + + # Environment + print_status "Step 1: Environment Configuration" + echo "Available environments: dev, staging, prod" + prompt_input "Environment" "$ENVIRONMENT" "ENVIRONMENT" + + if [[ ! "$ENVIRONMENT" =~ ^(dev|staging|prod)$ ]]; then + print_error "Invalid environment: $ENVIRONMENT. Must be dev, staging, or prod." + exit 1 + fi + + # Region + prompt_input "AWS Region" "$REGION" "REGION" + + # Stack name + prompt_input "CloudFormation Stack Name" "$STACK_NAME" "STACK_NAME" + + echo "" + print_status "Step 2: Network Configuration" + + # NAT Gateway option + print_status "Internet Connectivity Options:" + print_status "1. Transit Gateway (existing infrastructure) - No additional cost" + print_status "2. NAT Gateway (standard AWS pattern) - ~\$90/month" + echo "" + print_prompt "Create NAT Gateways for internet connectivity? (y/N): (IF YOU ALREADY HAVE A NAT GATEWAY, THE ANSWER IS NO.) " + read -r nat_choice + if [[ "$nat_choice" =~ ^[Yy]$ ]]; then + CREATE_NAT_GATEWAY="true" + print_success "NAT Gateways will be created for internet connectivity" + print_warning "This will add approximately \$90/month to your AWS bill" + else + CREATE_NAT_GATEWAY="false" + print_success "Using existing Transit Gateway infrastructure (no NAT Gateway cost)" + print_important "Ensure your private subnets have routes to Transit Gateway for internet access" + fi + + # VPC ID + list_vpcs + echo "" + prompt_input "VPC ID" "$VPC_ID" "VPC_ID" + + # Validate VPC + if ! aws ec2 describe-vpcs --region "$REGION" --vpc-ids "$VPC_ID" &>/dev/null; then + print_error "VPC $VPC_ID not found or not accessible" + exit 1 + fi + + # Public Subnets + echo "" + list_subnets "$VPC_ID" + echo "" + print_status "Public subnets will be used for the Application Load Balancer" + prompt_input "Public Subnet IDs (comma-separated, minimum 2)" "$PUBLIC_SUBNETS" "PUBLIC_SUBNETS" + + if ! validate_subnets "$PUBLIC_SUBNETS" "Public"; then + exit 1 + fi + + # Private Subnets + echo "" + if [[ "$CREATE_NAT_GATEWAY" == "true" ]]; then + print_status "Private subnets will be used for ECS tasks and databases" + print_status "NAT Gateways will provide internet access for these subnets" + else + print_status "Private subnets will be used for ECS tasks and databases" + print_status "These subnets should have internet access via Transit Gateway" + fi + prompt_input "Private Subnet IDs (comma-separated, minimum 2)" "$PRIVATE_SUBNETS" "PRIVATE_SUBNETS" + + if ! validate_subnets "$PRIVATE_SUBNETS" "Private"; then + exit 1 + fi + + # Secrets Manager VPC endpoint: only one per VPC can have private DNS; skip if another stack (e.g. dev) already created it + echo "" + print_prompt "Does another stack in this VPC already have a Secrets Manager VPC endpoint (e.g. dev)? (y/N): " + read -r sm_ep_choice + if [[ "$sm_ep_choice" =~ ^[Yy]$ ]]; then + CREATE_SECRETS_MANAGER_VPC_ENDPOINT="false" + print_success "Will not create Secrets Manager VPC endpoint (reusing existing in VPC)" + else + CREATE_SECRETS_MANAGER_VPC_ENDPOINT="true" + print_success "This stack will create the Secrets Manager VPC endpoint (with private DNS)" + fi + + echo "" + print_status "Step 3: AWS Bedrock Credentials" + print_important "LibreChat needs AWS credentials to access Bedrock models." + print_important "These credentials will be securely stored in AWS Secrets Manager." + echo "" + + # AWS Access Key ID + prompt_input "AWS Access Key ID for Bedrock access" "$AWS_ACCESS_KEY_ID" "AWS_ACCESS_KEY_ID" + + # AWS Secret Access Key + prompt_input "AWS Secret Access Key for Bedrock access" "$AWS_SECRET_ACCESS_KEY" "AWS_SECRET_ACCESS_KEY" + + # Validate Bedrock access + validate_bedrock_access "$AWS_ACCESS_KEY_ID" "$AWS_SECRET_ACCESS_KEY" "$REGION" + + echo "" + print_status "Step 4: SSO Configuration (Optional)" + print_important "Configure Single Sign-On with AWS Cognito for user authentication." + print_important "" + print_important "πŸ“‹ BEFORE ENABLING SSO, ENSURE YOU HAVE:" + print_important "β€’ Created a Cognito User Pool" + print_important "β€’ Created an App Client in the User Pool" + print_important "β€’ Configured the App Client with appropriate callback URLs" + print_important "β€’ Set up attribute mappings (name, email) in the User Pool" + echo "" + + # SSO Enable + print_prompt "Enable SSO authentication? (y/N): " + read -r sso_choice + if [[ "$sso_choice" =~ ^[Yy]$ ]]; then + ENABLE_SSO="true" + + # Cognito User Pool ID + prompt_input "Cognito User Pool ID" "$COGNITO_USER_POOL_ID" "COGNITO_USER_POOL_ID" + + # OpenID Client ID + prompt_input "OpenID Client ID (from Cognito App Client)" "$OPENID_CLIENT_ID" "OPENID_CLIENT_ID" + + # OpenID Client Secret + prompt_input "OpenID Client Secret (from Cognito App Client)" "$OPENID_CLIENT_SECRET" "OPENID_CLIENT_SECRET" + + # OpenID Scope + prompt_input "OpenID Scope" "$OPENID_SCOPE" "OPENID_SCOPE" + + # OpenID Button Label + prompt_input "SSO Button Label" "$OPENID_BUTTON_LABEL" "OPENID_BUTTON_LABEL" + + # OpenID Image URL + prompt_input "SSO Button Image URL" "$OPENID_IMAGE_URL" "OPENID_IMAGE_URL" + + # OpenID Name Claim + prompt_input "Name Claim Attribute" "$OPENID_NAME_CLAIM" "OPENID_NAME_CLAIM" + + # OpenID Email Claim + prompt_input "Email Claim Attribute" "$OPENID_EMAIL_CLAIM" "OPENID_EMAIL_CLAIM" + + # Validate required SSO fields + if [[ -z "$COGNITO_USER_POOL_ID" || -z "$OPENID_CLIENT_ID" || -z "$OPENID_CLIENT_SECRET" ]]; then + print_error "All SSO fields are required when SSO is enabled" + exit 1 + fi + + print_success "SSO configuration completed" + else + ENABLE_SSO="false" + print_status "SSO authentication disabled - users will use email/password login" + fi + + echo "" + print_status "Step 5: Application Configuration" + + # Help and FAQ URL + prompt_input "Help and FAQ URL (use '/' to disable button)" "$HELP_AND_FAQ_URL" "HELP_AND_FAQ_URL" + + echo "" + print_status "Step 5b: MCP Server Secrets" + print_status "Enter Bearer tokens for MCP servers that require auth. Press Enter to skip or use existing secret." + prompt_input "MCP Congress token" "$MCP_CONGRESS_TOKEN" "MCP_CONGRESS_TOKEN" + prompt_input "MCP Eastern Time token" "$MCP_EASTERN_TIME_TOKEN" "MCP_EASTERN_TIME_TOKEN" + + echo "" + print_status "Step 6: SSL/Domain Configuration" + + # Domain name + prompt_input "Domain Name (librechatchat.example.com)" "$DOMAIN_NAME" "DOMAIN_NAME" + + # Certificate ARN + if [[ -n "$DOMAIN_NAME" ]]; then + prompt_input "ACM Certificate ARN (required for HTTPS)" "$CERTIFICATE_ARN" "CERTIFICATE_ARN" + fi + + echo "" + print_status "Configuration Summary:" + echo " Environment: $ENVIRONMENT" + echo " Region: $REGION" + echo " Stack Name: $STACK_NAME" + echo " VPC ID: $VPC_ID" + echo " Public Subnets: $PUBLIC_SUBNETS" + echo " Private Subnets: $PRIVATE_SUBNETS" + echo " Create NAT Gateway: $CREATE_NAT_GATEWAY" + echo " Create Secrets Manager VPC Endpoint: $CREATE_SECRETS_MANAGER_VPC_ENDPOINT" + echo " AWS Access Key: ${AWS_ACCESS_KEY_ID:0:8}..." + echo " SSO Enabled: $ENABLE_SSO" + if [[ "$ENABLE_SSO" == "true" ]]; then + echo " Cognito User Pool: $COGNITO_USER_POOL_ID" + echo " OpenID Client ID: ${OPENID_CLIENT_ID:0:8}..." + echo " SSO Button Label: $OPENID_BUTTON_LABEL" + if [[ -n "$OPENID_IMAGE_URL" ]]; then + echo " SSO Button Image: $OPENID_IMAGE_URL" + fi + fi + if [[ -n "$HELP_AND_FAQ_URL" ]]; then + echo " Help & FAQ URL: $HELP_AND_FAQ_URL" + fi + if [[ -n "$DOMAIN_NAME" ]]; then + echo " Domain: $DOMAIN_NAME" + echo " Certificate: $CERTIFICATE_ARN" + fi + + echo "" + print_important "πŸš€ DEPLOYMENT FEATURES:" + print_important "β€’ ECS Fargate with auto-scaling (2-20 instances)" + print_important "β€’ DocumentDB (MongoDB-compatible) with multi-AZ" + print_important "β€’ ElastiCache Redis with failover" + print_important "β€’ S3 for file storage with encryption" + print_important "β€’ VPC endpoints for AWS services (reduced internet traffic)" + if [[ "$CREATE_NAT_GATEWAY" == "true" ]]; then + print_important "β€’ NAT Gateways for reliable internet connectivity (~\$90/month)" + else + print_important "β€’ Transit Gateway routing (no NAT Gateway costs)" + fi + + echo "" + print_prompt "Save this configuration for future deployments? (y/N): " + read -r save_choice + if [[ "$save_choice" =~ ^[Yy]$ ]]; then + save_config + fi + + echo "" + print_prompt "Proceed with deployment? (y/N): " + read -r deploy_choice + if [[ ! "$deploy_choice" =~ ^[Yy]$ ]]; then + print_status "Deployment cancelled" + exit 0 + fi +} + +# Parse command line arguments +LOAD_CONFIG=false +RESET_CONFIG=false +UPDATE_CONFIG_ONLY=false + +while [[ $# -gt 0 ]]; do + case $1 in + --load-config) + LOAD_CONFIG=true + shift + ;; + --reset-config) + RESET_CONFIG=true + shift + ;; + --update-config) + UPDATE_CONFIG_ONLY=true + LOAD_CONFIG=true # Auto-load config for update-only mode + shift + ;; + -h|--help) + usage + exit 0 + ;; + *) + print_error "Unknown option: $1" + usage + exit 1 + ;; + esac +done + +# Handle reset config +if [[ "$RESET_CONFIG" == true ]]; then + if [[ -f "$CONFIG_FILE" ]]; then + rm "$CONFIG_FILE" + print_success "Configuration file deleted" + else + print_warning "No configuration file to delete" + fi + exit 0 +fi + +# Validate prerequisites +validate_aws +validate_sam + +# Load existing config if requested +if [[ "$LOAD_CONFIG" == true ]]; then + if ! load_config; then + if [[ "$UPDATE_CONFIG_ONLY" == true ]]; then + print_error "Cannot update config without existing configuration. Run full deployment first." + exit 1 + fi + print_status "Starting fresh configuration..." + fi +fi + +# Handle config-only update +if [[ "$UPDATE_CONFIG_ONLY" == true ]]; then + print_status "Config-only update mode - updating configuration and restarting containers" + + if [[ -z "$STACK_NAME" || -z "$REGION" ]]; then + print_error "Missing stack name or region in configuration" + exit 1 + fi + + # Verify stack exists + if ! aws cloudformation describe-stacks --stack-name "$STACK_NAME" --region "$REGION" &>/dev/null; then + print_error "Stack $STACK_NAME not found in region $REGION" + exit 1 + fi + + print_status "Updating configuration for stack: $STACK_NAME" + if update_config "$STACK_NAME" "$REGION"; then + print_success "Configuration updated successfully!" + print_status "Containers are restarting with the new configuration" + else + print_error "Configuration update failed" + exit 1 + fi + exit 0 +fi + +# Run interactive configuration +interactive_config + +# Ensure ECS service-linked role exists now that we have the region +if ! ensure_ecs_service_role "$REGION"; then + exit 1 +fi + +print_status "Starting LibreChat deployment..." + +# Build the SAM application +print_status "Building SAM application..." +sam build + +# Create or update MCP secrets in Secrets Manager when tokens are provided +MCP_CONGRESS_SECRET_ARN="" +MCP_EASTERN_TIME_SECRET_ARN="" +if [[ -n "$MCP_CONGRESS_TOKEN" ]]; then + print_status "Creating/updating MCP Congress secret in Secrets Manager..." + MCP_CONGRESS_SECRET_ARN=$(ensure_secret_string "${STACK_NAME}/mcp/congress" "$MCP_CONGRESS_TOKEN" "$REGION") +fi +if [[ -n "$MCP_EASTERN_TIME_TOKEN" ]]; then + print_status "Creating/updating MCP Eastern Time secret in Secrets Manager..." + MCP_EASTERN_TIME_SECRET_ARN=$(ensure_secret_string "${STACK_NAME}/mcp/eastern-time" "$MCP_EASTERN_TIME_TOKEN" "$REGION") +fi + + +# Create Bedrock credentials secret and OpenID client secret so we pass only ARNs to CloudFormation +BEDROCK_CREDENTIALS_SECRET_ARN="" +OPENID_CLIENT_SECRET_ARN="" +print_status "Creating/updating Bedrock credentials secret in Secrets Manager..." +BEDROCK_CREDENTIALS_SECRET_ARN=$(ensure_bedrock_credentials_secret "${STACK_NAME}/bedrock/credentials" "$AWS_ACCESS_KEY_ID" "$AWS_SECRET_ACCESS_KEY" "$REGION") +if [[ "$ENABLE_SSO" == "true" && -n "$OPENID_CLIENT_SECRET" ]]; then + print_status "Creating/updating OpenID client secret in Secrets Manager..." + OPENID_CLIENT_SECRET_ARN=$(ensure_secret_string "${STACK_NAME}/openid/client-secret" "$OPENID_CLIENT_SECRET" "$REGION") +fi + +# Prepare deployment parameters with proper quoting (only ARNs, no raw credentials) +DEPLOY_PARAMS=( + "Environment=$ENVIRONMENT" + "VpcId=$VPC_ID" + "PublicSubnetIds=$PUBLIC_SUBNETS" + "PrivateSubnetIds=$PRIVATE_SUBNETS" + "CreateNATGateway=$CREATE_NAT_GATEWAY" + "CreateSecretsManagerVPCEndpoint=$CREATE_SECRETS_MANAGER_VPC_ENDPOINT" + "BedrockCredentialsSecretArn=$BEDROCK_CREDENTIALS_SECRET_ARN" + "EnableSSO=$ENABLE_SSO" +) +if [[ -n "$LIBRECHAT_IMAGE" ]]; then + DEPLOY_PARAMS+=("LibreChatImage=$LIBRECHAT_IMAGE") +fi + +if [[ "$ENABLE_SSO" == "true" ]]; then + DEPLOY_PARAMS+=( + "CognitoUserPoolId=$COGNITO_USER_POOL_ID" + "OpenIdClientId=$OPENID_CLIENT_ID" + "OpenIdScope=$OPENID_SCOPE" + "OpenIdButtonLabel=\"$OPENID_BUTTON_LABEL\"" + "OpenIdNameClaim=$OPENID_NAME_CLAIM" + "OpenIdEmailClaim=$OPENID_EMAIL_CLAIM" + ) + if [[ -n "$OPENID_CLIENT_SECRET_ARN" ]]; then + DEPLOY_PARAMS+=("OpenIdClientSecretArn=$OPENID_CLIENT_SECRET_ARN") + else + DEPLOY_PARAMS+=("OpenIdClientSecret=$OPENID_CLIENT_SECRET") + fi + + # Add image URL if provided + if [[ -n "$OPENID_IMAGE_URL" ]]; then + DEPLOY_PARAMS+=("OpenIdImageUrl=\"$OPENID_IMAGE_URL\"") + fi +fi + +if [[ -n "$HELP_AND_FAQ_URL" ]]; then + DEPLOY_PARAMS+=("HelpAndFaqUrl=$HELP_AND_FAQ_URL") +fi + +if [[ -n "$DOMAIN_NAME" ]]; then + DEPLOY_PARAMS+=("DomainName=$DOMAIN_NAME") +fi + +if [[ -n "$CERTIFICATE_ARN" ]]; then + DEPLOY_PARAMS+=("CertificateArn=$CERTIFICATE_ARN") +fi + +if [[ -n "$MCP_CONGRESS_SECRET_ARN" ]]; then + DEPLOY_PARAMS+=("MCPCongressSecretArn=$MCP_CONGRESS_SECRET_ARN") +fi +if [[ -n "$MCP_EASTERN_TIME_SECRET_ARN" ]]; then + DEPLOY_PARAMS+=("MCPEasternTimeSecretArn=$MCP_EASTERN_TIME_SECRET_ARN") +fi + +# When reusing existing Secrets Manager VPC endpoint, discover its SG so the stack can add this stack's ECS SG before ECS Service starts +if [[ "$CREATE_SECRETS_MANAGER_VPC_ENDPOINT" == "false" ]]; then + print_status "Discovering existing Secrets Manager VPC endpoint security group..." + EXISTING_SM_ENDPOINT_SG_ID=$(aws ec2 describe-vpc-endpoints --region "$REGION" \ + --filters \ + "Name=service-name,Values=com.amazonaws.${REGION}.secretsmanager" \ + "Name=vpc-id,Values=$VPC_ID" \ + "Name=vpc-endpoint-type,Values=Interface" \ + --query 'VpcEndpoints[0].Groups[0].GroupId' --output text 2>/dev/null) + if [[ -z "$EXISTING_SM_ENDPOINT_SG_ID" || "$EXISTING_SM_ENDPOINT_SG_ID" == "None" ]]; then + print_error "CreateSecretsManagerVPCEndpoint is false but no Secrets Manager VPC endpoint found in VPC $VPC_ID. Create an endpoint in this VPC first, or set CreateSecretsManagerVPCEndpoint=true for the first stack." + exit 1 + fi + DEPLOY_PARAMS+=("ExistingSecretsManagerEndpointSecurityGroupId=$EXISTING_SM_ENDPOINT_SG_ID") +fi + +# Deploy the application +print_status "Deploying to AWS..." +sam deploy \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --capabilities CAPABILITY_IAM \ + --parameter-overrides "${DEPLOY_PARAMS[@]}" \ + --confirm-changeset \ + --resolve-s3 + +if [[ $? -eq 0 ]]; then + print_success "Deployment completed successfully!" + + # Upload config to S3 and trigger EFS sync + print_status "Uploading configuration and syncing to EFS..." + if upload_config "$STACK_NAME" "$REGION"; then + print_success "Configuration deployed successfully" + else + print_warning "Configuration upload failed, but deployment succeeded" + fi + + # Get the load balancer URL + LB_URL=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`LoadBalancerURL`].OutputValue' \ + --output text) + + if [[ -n "$LB_URL" ]]; then + print_success "LibreChat is available at: $LB_URL" + print_status "Note: It may take a few minutes for the service to be fully available." + fi + + # Show other important outputs + print_status "Getting deployment information..." + aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[*].[OutputKey,OutputValue]' \ + --output table + + # Show SSO-specific information if enabled + if [[ "$ENABLE_SSO" == "true" ]]; then + echo "" + print_important "πŸ” SSO CONFIGURATION SUMMARY:" + print_important "β€’ SSO Authentication: ENABLED" + print_important "β€’ Cognito User Pool: $COGNITO_USER_POOL_ID" + print_important "β€’ OpenID Issuer: https://cognito-idp.$REGION.amazonaws.com/$COGNITO_USER_POOL_ID" + + # Get callback URL from outputs + CALLBACK_URL=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`CallbackURL`].OutputValue' \ + --output text 2>/dev/null) + + if [[ -n "$CALLBACK_URL" ]]; then + print_important "β€’ Callback URL: $CALLBACK_URL" + print_important "" + print_important "πŸ“ COGNITO APP CLIENT CONFIGURATION:" + print_important "β€’ Add this callback URL to your Cognito App Client settings" + print_important "β€’ Ensure 'Authorization code grant' is enabled" + print_important "β€’ Configure attribute mappings for 'name' and 'email'" + fi + else + echo "" + print_important "πŸ” AUTHENTICATION: Email/Password login enabled" + fi + +else + print_error "Deployment failed!" + exit 1 +fi + +print_status "Deployment complete!" +echo "" +print_important "πŸŽ‰ DEPLOYMENT SUMMARY:" +print_important "β€’ Environment: $ENVIRONMENT" +print_important "β€’ Region: $REGION" +print_important "β€’ Stack: $STACK_NAME" +if [[ -n "$LB_URL" ]]; then + print_important "β€’ Application URL: $LB_URL" +fi +if [[ "$ENABLE_SSO" == "true" ]]; then + print_important "β€’ Authentication: SSO (Cognito) + Email/Password disabled" +else + print_important "β€’ Authentication: Email/Password login" +fi +if [[ -n "$HELP_AND_FAQ_URL" && "$HELP_AND_FAQ_URL" != "/" ]]; then + print_important "β€’ Help & FAQ: $HELP_AND_FAQ_URL" +elif [[ "$HELP_AND_FAQ_URL" == "/" ]]; then + print_important "β€’ Help & FAQ: Disabled" +fi +echo "" +print_important "" +print_important "πŸ€– PARTIAL LIST OF AVAILABLE BEDROCK MODELS:" +print_important "β€’ Claude Opus 4 (us.anthropic.claude-opus-4-20250514-v1:0)" +print_important "β€’ Claude Sonnet 4 (us.anthropic.claude-sonnet-4-20250514-v1:0)" +print_important "β€’ Claude 3.7 Sonnet (us.anthropic.claude-3-7-sonnet-20250219-v1:0)" +print_important "β€’ Claude 3.5 Haiku (us.anthropic.claude-3-5-haiku-20241022-v1:0)" +print_important "β€’ Llama 3.3 70B (us.meta.llama3-3-70b-instruct-v1:0)" +print_important "" +print_important "πŸ’° COST SAVINGS:" +if [[ "$CREATE_NAT_GATEWAY" == "true" ]]; then + print_important "β€’ NAT Gateways provide reliable internet connectivity (~\$90/month)" + print_important "β€’ High availability with automatic failover" + print_important "β€’ Enterprise-grade performance and security" +else + print_important "β€’ No NAT Gateway costs (~\$90/month saved)" + print_important "β€’ Using existing Transit Gateway infrastructure" + print_important " SHOULD BE REVIEWED, might be too slow" +fi +print_important "β€’ VPC endpoints reduce data transfer costs" + +echo "" +print_status "To redeploy with the same configuration, run:" +print_status " $0 --load-config" +print_status "" +print_status "To update config file and restart containers only, run:" +print_status " $0 --update-config" +print_status "" +print_status "To start with fresh configuration, run:" +print_status " $0 --reset-config" + +# Clean up temporary files +rm -f /tmp/lambda-response.json 2>/dev/null || true \ No newline at end of file diff --git a/deploy/aws-sam/librechat.example.yaml b/deploy/aws-sam/librechat.example.yaml new file mode 100644 index 0000000000..49799061e9 --- /dev/null +++ b/deploy/aws-sam/librechat.example.yaml @@ -0,0 +1,108 @@ +# Minimal LibreChat config for AWS SAM deploy +# Copy this file to librechat.yaml and customize for your deployment. +# For full options, see: https://www.librechat.ai/docs/configuration/librechat_yaml + +# Configuration version (required) +version: 1.2.8 + +# Cache settings +cache: true + +# File storage configuration +fileStrategy: "s3" + +# Transaction settings +transactions: + enabled: true + +interface: + mcpServers: + placeholder: "Select MCP Servers" + use: true + create: true + share: true + trustCheckbox: + label: "I trust this server" + subLabel: "Only enable servers you trust" + privacyPolicy: + externalUrl: "https://example.com/privacy" + openNewTab: true + termsOfService: + externalUrl: "https://example.com/terms" + openNewTab: true + modalAcceptance: true + modalTitle: "Terms of Service" + modalContent: | + # Terms of Service + ## Introduction + Welcome to LibreChat! + modelSelect: true + parameters: true + sidePanel: true + presets: false + prompts: false + bookmarks: false + multiConvo: true + agents: true + customWelcome: "Welcome to LibreChat!" + runCode: true + webSearch: true + fileSearch: true + fileCitations: true + +# MCP Servers Configuration (customize or add your own) +# Use env var placeholders for secrets, e.g. ${MCP_SOME_TOKEN} +mcpServers: + # Example: third-party MCP + # Deepwiki: + # url: "https://mcp.deepwiki.com/mcp" + # name: "DeepWiki" + # description: "DeepWiki MCP Server..." + # type: "streamable-http" + # Example: your own MCP (replace with your API URL and token env var) + # MyMcp: + # name: "My MCP Server" + # description: "Description of the server" + # url: "https://YOUR_API_ID.execute-api.YOUR_REGION.amazonaws.com/dev/mcp/your_mcp" + # type: "streamable-http" + # headers: + # Authorization: "Bearer ${MCP_MY_TOKEN}" + +# Registration (optional) +# registration: +# socialLogins: ['saml', 'github', 'google', 'openid', ...] +registration: + socialLogins: + - "saml" + - "openid" + # allowedDomains: + # - "example.edu" + # - "*.example.edu" + +# Balance settings (optional) +balance: + enabled: true + startBalance: 650000 + autoRefillEnabled: true + refillIntervalValue: 1440 + refillIntervalUnit: "minutes" + refillAmount: 250000 + +# Custom endpoints (e.g. Bedrock) +endpoints: + # bedrock: + # cache: true + # promptCache: true + # titleModel: "us.anthropic.claude-3-7-sonnet-20250219-v1:0" + +# Model specs – default model selection for new users +# modelSpecs: +# prioritize: true +# list: +# - name: "my-default" +# label: "My Default Model" +# description: "Default model for new conversations" +# default: true +# preset: +# endpoint: "bedrock" +# model: "us.anthropic.claude-sonnet-4-5-20250929-v1:0" diff --git a/deploy/aws-sam/scripts/README.md b/deploy/aws-sam/scripts/README.md new file mode 100644 index 0000000000..0d3af1de5b --- /dev/null +++ b/deploy/aws-sam/scripts/README.md @@ -0,0 +1,235 @@ +# LibreChat Admin Scripts + +This directory contains utility scripts for managing your LibreChat deployment. + +## Managing Admin Users + +### Grant Admin Permissions + +To grant admin permissions to a user: + +```bash +./scripts/make-admin.sh user@domain.edu +``` + +### Remove Admin Permissions + +To remove admin permissions from a user (demote to regular user): + +```bash +./scripts/make-admin.sh user@domain.edu --remove +``` + +### How It Works + +The script: +1. Spins up a one-off ECS task using your existing task definition +2. Connects to MongoDB using the same credentials as your running application +3. Updates the user's role to ADMIN or USER +4. Waits for completion and reports success/failure +5. Automatically cleans up the task + +The user will need to log out and log back in for changes to take effect. + +## Managing User Balance + +### Add Balance to a User + +To add tokens to a user's balance: + +```bash +./scripts/add-balance.sh user@domain.edu 1000 +``` + +This will add 1000 tokens to the user's account. + +### Requirements + +- Balance must be enabled in `librechat.yaml`: + ```yaml + balance: + enabled: true + startBalance: 600000 + autoRefillEnabled: true + refillIntervalValue: 1440 + refillIntervalUnit: 'minutes' + refillAmount: 100000 + ``` + +### How It Works + +The script: +1. Validates that balance is enabled in your configuration +2. Finds the user by email +3. Creates a transaction record with the specified amount +4. Updates the user's balance +5. Reports the new balance + +### Common Use Cases + +```bash +# Give a new user initial credits +./scripts/add-balance.sh newuser@domain.edu 5000 + +# Top up a user who ran out +./scripts/add-balance.sh user@domain.edu 10000 + +# Grant bonus credits +./scripts/add-balance.sh poweruser@domain.edu 50000 +``` + +## Manual AWS CLI Commands + +If you prefer to run commands manually or need to troubleshoot: + +### 1. Get your cluster and network configuration + +```bash +# Load your deployment config +source .librechat-deploy-config + +CLUSTER_NAME="${STACK_NAME}-cluster" +REGION="${REGION:-us-east-1}" + +# Get network configuration from existing service +aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "${STACK_NAME}-service" \ + --region "$REGION" \ + --query 'services[0].networkConfiguration.awsvpcConfiguration' +``` + +### 2. Run a one-off task to manage admin role + +```bash +# Set the user email and action +USER_EMAIL="user@domain.edu" +TARGET_ROLE="ADMIN" # or "USER" to remove admin + +# Get task definition +TASK_DEF=$(aws ecs describe-task-definition \ + --task-definition "${STACK_NAME}-task" \ + --region "$REGION" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) + +# Create the command +SHELL_CMD="cd /app/api && cat > manage-admin.js << 'EOFSCRIPT' +const path = require('path'); +require('module-alias')({ base: path.resolve(__dirname) }); +const mongoose = require('mongoose'); +const { updateUser, findUser } = require('~/models'); + +(async () => { + try { + await mongoose.connect(process.env.MONGO_URI); + const user = await findUser({ email: '$USER_EMAIL' }); + if (!user) { + console.error('User not found'); + process.exit(1); + } + await updateUser(user._id, { role: '$TARGET_ROLE' }); + console.log('User role updated to $TARGET_ROLE'); + await mongoose.connection.close(); + } catch (err) { + console.error('Error:', err.message); + process.exit(1); + } +})(); +EOFSCRIPT +node manage-admin.js" + +# Build JSON with jq +OVERRIDES=$(jq -n --arg cmd "$SHELL_CMD" '{ + containerOverrides: [{ + name: "librechat", + command: ["sh", "-c", $cmd] + }] +}') + +# Run the task (replace SUBNETS and SECURITY_GROUPS with values from step 1) +aws ecs run-task \ + --cluster "$CLUSTER_NAME" \ + --task-definition "$TASK_DEF" \ + --launch-type FARGATE \ + --network-configuration "awsvpcConfiguration={subnets=[subnet-xxx,subnet-yyy],securityGroups=[sg-xxxxx],assignPublicIp=DISABLED}" \ + --overrides "$OVERRIDES" \ + --region "$REGION" +``` + +## Troubleshooting + +### Task fails to start +- Check that your ECS service is running +- Verify network configuration (subnets, security groups) +- Check CloudWatch Logs: `/aws/ecs/${STACK_NAME}` + +### User not found error +- Verify the email address is correct +- Check that the user has logged in at least once +- Email addresses are case-sensitive + +### MongoDB connection fails +- Verify the MONGO_URI environment variable is set correctly in the task +- Check that the security group allows access to DocumentDB (port 27017) +- Ensure the task is running in the same VPC as DocumentDB + +### Changes don't take effect +- User must log out and log back in for role changes to apply +- Check CloudWatch Logs to confirm the update was successful +- Verify the exit code was 0 (success) + +### Balance not enabled error +- Ensure `balance.enabled: true` is set in `librechat.yaml` +- Restart your ECS service after updating the configuration +- Verify the config file is properly mounted in the container + +### Invalid amount error +- Amount must be a positive integer +- Do not use decimals or negative numbers +- Example: `1000` not `1000.5` or `-1000` + +## Security Notes + +- These scripts use your existing task definition with all environment variables +- The MongoDB connection uses the same credentials as your running application +- Tasks run in your private subnets with no public IP +- All commands are logged to CloudWatch Logs +- One-off tasks automatically stop after completion + +## Alternative: Use OpenID Groups (Recommended for Production) + +Instead of manually managing admin users, consider using OpenID groups for automatic role assignment: + +### Setup + +1. **In AWS Cognito**, create a group called "admin" +2. **Add users** to that group through the Cognito console +3. **Configure LibreChat** (already done in `.env.local`): + ```bash + OPENID_ADMIN_ROLE=admin + OPENID_ADMIN_ROLE_PARAMETER_PATH=cognito:groups + OPENID_ADMIN_ROLE_TOKEN_KIND=id_token + ``` +4. **Users automatically get admin permissions** on their next login + +### Benefits + +- No database access required +- Centralized user management in Cognito +- Automatic role assignment on login +- Easier to audit and manage at scale +- Role changes take effect immediately on next login + +### When to Use the Script vs OpenID Groups + +**Use the script when:** +- You need to quickly grant/revoke admin access +- You're troubleshooting or testing +- You have a one-time admin setup need + +**Use OpenID groups when:** +- Managing multiple admins +- You want centralized access control +- You need audit trails through Cognito +- You want automatic role management diff --git a/deploy/aws-sam/scripts/add-balance.sh b/deploy/aws-sam/scripts/add-balance.sh new file mode 100644 index 0000000000..3cb74a1ec9 --- /dev/null +++ b/deploy/aws-sam/scripts/add-balance.sh @@ -0,0 +1,208 @@ +#!/bin/bash +# Script to add balance to a user by running a one-off ECS task +# Usage: ./scripts/add-balance.sh + +set -e + +# Check if arguments are provided +if [ -z "$1" ] || [ -z "$2" ]; then + echo "Usage: $0 " + echo "" + echo "Examples:" + echo " Add 1000 tokens: $0 user@domain.com 1000" + echo " Add 5000 tokens: $0 user@domain.com 5000" + echo "" + echo "Note: Balance must be enabled in librechat.yaml" + exit 1 +fi + +USER_EMAIL="$1" +AMOUNT="$2" + +# Validate amount is a number +if ! [[ "$AMOUNT" =~ ^[0-9]+$ ]]; then + echo "Error: Amount must be a positive number" + exit 1 +fi + +# Load configuration +if [ ! -f .librechat-deploy-config ]; then + echo "Error: .librechat-deploy-config not found" + exit 1 +fi + +source .librechat-deploy-config + +# Set variables +CLUSTER_NAME="${STACK_NAME}-cluster" +TASK_FAMILY="${STACK_NAME}-task" +REGION="${REGION:-us-east-1}" + +echo "==========================================" +echo "Adding balance to user: $USER_EMAIL" +echo "Amount: $AMOUNT tokens" +echo "Stack: $STACK_NAME" +echo "Region: $REGION" +echo "==========================================" + +# Get VPC configuration from the existing service +echo "Getting network configuration from existing service..." +SERVICE_INFO=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "${STACK_NAME}-service" \ + --region "$REGION" \ + --query 'services[0].networkConfiguration.awsvpcConfiguration' \ + --output json) + +SUBNETS=$(echo "$SERVICE_INFO" | jq -r '.subnets | join(",")') +SECURITY_GROUPS=$(echo "$SERVICE_INFO" | jq -r '.securityGroups | join(",")') + +echo "Subnets: $SUBNETS" +echo "Security Groups: $SECURITY_GROUPS" + +# Get the task definition +echo "Getting task definition..." +TASK_DEF=$(aws ecs describe-task-definition \ + --task-definition "$TASK_FAMILY" \ + --region "$REGION" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) + +echo "Task Definition: $TASK_DEF" + +# Run the one-off task +echo "Starting ECS task to add balance..." + +# Create a Node.js script that mimics the add-balance.js functionality +SHELL_CMD="cd /app/api && cat > add-balance-task.js << 'EOFSCRIPT' +// Setup module-alias like LibreChat does +const path = require('path'); +require('module-alias')({ base: path.resolve(__dirname) }); + +const mongoose = require('mongoose'); +const { getBalanceConfig } = require('@librechat/api'); +const { User } = require('@librechat/data-schemas').createModels(mongoose); +const { createTransaction } = require('~/models/Transaction'); +const { getAppConfig } = require('~/server/services/Config'); + +const email = '$USER_EMAIL'; +const amount = $AMOUNT; + +(async () => { + try { + // Connect to MongoDB + console.log('Connecting to MongoDB...'); + await mongoose.connect(process.env.MONGO_URI); + console.log('Connected to MongoDB'); + + // Get app config and balance config + console.log('Loading configuration...'); + const appConfig = await getAppConfig(); + const balanceConfig = getBalanceConfig(appConfig); + + if (!balanceConfig?.enabled) { + console.error('Error: Balance is not enabled. Use librechat.yaml to enable it'); + await mongoose.connection.close(); + process.exit(1); + } + + // Find the user + console.log('Looking for user:', email); + const user = await User.findOne({ email }).lean(); + + if (!user) { + console.error('Error: No user with that email was found!'); + await mongoose.connection.close(); + process.exit(1); + } + + console.log('Found user:', user.email); + + // Create transaction and update balance + console.log('Creating transaction for', amount, 'tokens...'); + const result = await createTransaction({ + user: user._id, + tokenType: 'credits', + context: 'admin', + rawAmount: +amount, + balance: balanceConfig, + }); + + if (!result?.balance) { + console.error('Error: Something went wrong while updating the balance!'); + await mongoose.connection.close(); + process.exit(1); + } + + // Success! + console.log('βœ… Transaction created successfully!'); + console.log('Amount added:', amount); + console.log('New balance:', result.balance); + + await mongoose.connection.close(); + process.exit(0); + } catch (err) { + console.error('Error:', err.message); + console.error(err.stack); + if (mongoose.connection.readyState === 1) { + await mongoose.connection.close(); + } + process.exit(1); + } +})(); +EOFSCRIPT +node add-balance-task.js" + +# Build the overrides JSON using jq for proper escaping +OVERRIDES=$(jq -n \ + --arg cmd "$SHELL_CMD" \ + '{ + containerOverrides: [{ + name: "librechat", + command: ["sh", "-c", $cmd] + }] + }') + +echo "Running command in container..." +TASK_ARN=$(aws ecs run-task \ + --cluster "$CLUSTER_NAME" \ + --task-definition "$TASK_DEF" \ + --launch-type FARGATE \ + --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUPS],assignPublicIp=DISABLED}" \ + --overrides "$OVERRIDES" \ + --region "$REGION" \ + --query 'tasks[0].taskArn' \ + --output text) + +echo "Task started: $TASK_ARN" +echo "" +echo "Waiting for task to complete..." +echo "You can monitor the task with:" +echo " aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $TASK_ARN --region $REGION" +echo "" +echo "Or view logs in CloudWatch Logs:" +echo " Log Group: /aws/ecs/${STACK_NAME}" +echo "" + +# Wait for task to complete +aws ecs wait tasks-stopped \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" + +# Check task exit code +EXIT_CODE=$(aws ecs describe-tasks \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" \ + --query 'tasks[0].containers[0].exitCode' \ + --output text) + +if [ "$EXIT_CODE" = "0" ]; then + echo "βœ… Success! Added $AMOUNT tokens to $USER_EMAIL" + echo "Check CloudWatch Logs for the new balance." +else + echo "❌ Task failed with exit code: $EXIT_CODE" + echo "Check CloudWatch Logs for details." + exit 1 +fi diff --git a/deploy/aws-sam/scripts/flush-redis-cache.sh b/deploy/aws-sam/scripts/flush-redis-cache.sh new file mode 100644 index 0000000000..6f2a066619 --- /dev/null +++ b/deploy/aws-sam/scripts/flush-redis-cache.sh @@ -0,0 +1,201 @@ +#!/bin/bash +# Script to flush Redis cache by running a one-off ECS task +# Usage: ./scripts/flush-redis-cache.sh + +set -e + +# Load configuration +if [ ! -f .librechat-deploy-config ]; then + echo "Error: .librechat-deploy-config not found" + exit 1 +fi + +source .librechat-deploy-config + +# Set variables +CLUSTER_NAME="${STACK_NAME}-cluster" +TASK_FAMILY="${STACK_NAME}-task" +REGION="${REGION:-us-east-1}" + +echo "==========================================" +echo "Flushing Redis Cache" +echo "Stack: $STACK_NAME" +echo "Region: $REGION" +echo "==========================================" + +# Get VPC configuration from the existing service +echo "Getting network configuration from existing service..." +SERVICE_INFO=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "${STACK_NAME}-service" \ + --region "$REGION" \ + --query 'services[0].networkConfiguration.awsvpcConfiguration' \ + --output json) + +SUBNETS=$(echo "$SERVICE_INFO" | jq -r '.subnets | join(",")') +SECURITY_GROUPS=$(echo "$SERVICE_INFO" | jq -r '.securityGroups | join(",")') + +echo "Subnets: $SUBNETS" +echo "Security Groups: $SECURITY_GROUPS" + +# Get the task definition +echo "Getting task definition..." +TASK_DEF=$(aws ecs describe-task-definition \ + --task-definition "$TASK_FAMILY" \ + --region "$REGION" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) + +echo "Task Definition: $TASK_DEF" + +# Run the one-off task +echo "Starting ECS task to flush Redis cache..." + +# Inline Node.js script to flush Redis cache +FLUSH_SCRIPT=' +const IoRedis = require("ioredis"); + +const isEnabled = (value) => value === "true" || value === true; + +async function flushRedis() { + try { + console.log("πŸ” Connecting to Redis..."); + + const urls = (process.env.REDIS_URI || "").split(",").map((uri) => new URL(uri)); + const username = urls[0]?.username || process.env.REDIS_USERNAME; + const password = urls[0]?.password || process.env.REDIS_PASSWORD; + + const redisOptions = { + username: username, + password: password, + connectTimeout: 10000, + maxRetriesPerRequest: 3, + enableOfflineQueue: true, + lazyConnect: false, + }; + + const useCluster = urls.length > 1 || isEnabled(process.env.USE_REDIS_CLUSTER); + let redis; + + if (useCluster) { + const clusterOptions = { + redisOptions, + enableOfflineQueue: true, + }; + + if (isEnabled(process.env.REDIS_USE_ALTERNATIVE_DNS_LOOKUP)) { + clusterOptions.dnsLookup = (address, callback) => callback(null, address); + } + + redis = new IoRedis.Cluster( + urls.map((url) => ({ host: url.hostname, port: parseInt(url.port, 10) || 6379 })), + clusterOptions, + ); + } else { + redis = new IoRedis(process.env.REDIS_URI, redisOptions); + } + + await new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error("Connection timeout")), 10000); + redis.once("ready", () => { clearTimeout(timeout); resolve(); }); + redis.once("error", (err) => { clearTimeout(timeout); reject(err); }); + }); + + console.log("βœ… Connected to Redis"); + + let keyCount = 0; + try { + if (useCluster) { + const nodes = redis.nodes("master"); + for (const node of nodes) { + const keys = await node.keys("*"); + keyCount += keys.length; + } + } else { + const keys = await redis.keys("*"); + keyCount = keys.length; + } + } catch (_error) {} + + if (useCluster) { + const nodes = redis.nodes("master"); + await Promise.all(nodes.map((node) => node.flushdb())); + console.log(`βœ… Redis cluster cache flushed successfully (${nodes.length} master nodes)`); + } else { + await redis.flushdb(); + console.log("βœ… Redis cache flushed successfully"); + } + + if (keyCount > 0) { + console.log(` Deleted ${keyCount} keys`); + } + + await redis.disconnect(); + console.log("⚠️ Note: All users will need to re-authenticate"); + process.exit(0); + } catch (error) { + console.error("❌ Error flushing Redis cache:", error.message); + process.exit(1); + } +} + +flushRedis(); +' + +SHELL_CMD="cd /app && node -e '$FLUSH_SCRIPT'" + +# Build the overrides JSON using jq for proper escaping +OVERRIDES=$(jq -n \ + --arg cmd "$SHELL_CMD" \ + '{ + containerOverrides: [{ + name: "librechat", + command: ["sh", "-c", $cmd] + }] + }') + +echo "Running command in container..." +TASK_ARN=$(aws ecs run-task \ + --cluster "$CLUSTER_NAME" \ + --task-definition "$TASK_DEF" \ + --launch-type FARGATE \ + --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUPS],assignPublicIp=DISABLED}" \ + --overrides "$OVERRIDES" \ + --region "$REGION" \ + --query 'tasks[0].taskArn' \ + --output text) + +echo "Task started: $TASK_ARN" +echo "" +echo "Waiting for task to complete..." +echo "You can monitor the task with:" +echo " aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $TASK_ARN --region $REGION" +echo "" +echo "Or view logs in CloudWatch Logs:" +echo " Log Group: /ecs/${STACK_NAME}-task" +echo "" + +# Wait for task to complete +aws ecs wait tasks-stopped \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" + +# Check task exit code +EXIT_CODE=$(aws ecs describe-tasks \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" \ + --query 'tasks[0].containers[0].exitCode' \ + --output text) + +if [ "$EXIT_CODE" = "0" ]; then + echo "βœ… Success! Redis cache has been flushed." + echo "" + echo "⚠️ Note: All users will need to re-authenticate." +else + echo "❌ Task failed with exit code: $EXIT_CODE" + echo "Check CloudWatch Logs for details:" + echo " aws logs tail /ecs/${STACK_NAME}-task --follow --region $REGION" + exit 1 +fi diff --git a/deploy/aws-sam/scripts/make-admin.sh b/deploy/aws-sam/scripts/make-admin.sh new file mode 100644 index 0000000000..13f1cd1c72 --- /dev/null +++ b/deploy/aws-sam/scripts/make-admin.sh @@ -0,0 +1,212 @@ +#!/bin/bash +# Script to manage user admin role by running a one-off ECS task +# Usage: ./scripts/make-admin.sh [--remove] + +set -e + +# Parse arguments +REMOVE_ADMIN=false +USER_EMAIL="" + +while [[ $# -gt 0 ]]; do + case $1 in + --remove|-r) + REMOVE_ADMIN=true + shift + ;; + *) + USER_EMAIL="$1" + shift + ;; + esac +done + +# Check if email is provided +if [ -z "$USER_EMAIL" ]; then + echo "Usage: $0 [--remove]" + echo "" + echo "Examples:" + echo " Grant admin: $0 user@domain.com" + echo " Remove admin: $0 user@domain.com --remove" + exit 1 +fi + +# Load configuration +if [ ! -f .librechat-deploy-config ]; then + echo "Error: .librechat-deploy-config not found" + exit 1 +fi + +source .librechat-deploy-config + +# Set variables +CLUSTER_NAME="${STACK_NAME}-cluster" +TASK_FAMILY="${STACK_NAME}-task" +REGION="${REGION:-us-east-1}" + +if [ "$REMOVE_ADMIN" = true ]; then + ACTION="Removing admin role from" + TARGET_ROLE="USER" +else + ACTION="Granting admin role to" + TARGET_ROLE="ADMIN" +fi + +echo "==========================================" +echo "$ACTION: $USER_EMAIL" +echo "Stack: $STACK_NAME" +echo "Region: $REGION" +echo "==========================================" + +# Get VPC configuration from the existing service +echo "Getting network configuration from existing service..." +SERVICE_INFO=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "${STACK_NAME}-service" \ + --region "$REGION" \ + --query 'services[0].networkConfiguration.awsvpcConfiguration' \ + --output json) + +SUBNETS=$(echo "$SERVICE_INFO" | jq -r '.subnets | join(",")') +SECURITY_GROUPS=$(echo "$SERVICE_INFO" | jq -r '.securityGroups | join(",")') + +echo "Subnets: $SUBNETS" +echo "Security Groups: $SECURITY_GROUPS" + +# Get the task definition +echo "Getting task definition..." +TASK_DEF=$(aws ecs describe-task-definition \ + --task-definition "$TASK_FAMILY" \ + --region "$REGION" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) + +echo "Task Definition: $TASK_DEF" + +# Run the one-off task +echo "Starting ECS task to update user role..." + +# Create a Node.js script that uses LibreChat's models with proper module-alias setup +SHELL_CMD="cd /app/api && cat > manage-admin.js << 'EOFSCRIPT' +// Setup module-alias like LibreChat does +const path = require('path'); +require('module-alias')({ base: path.resolve(__dirname) }); + +const mongoose = require('mongoose'); +const { updateUser, findUser } = require('~/models'); +const { SystemRoles } = require('librechat-data-provider'); + +const targetRole = '$TARGET_ROLE'; + +(async () => { + try { + // Connect to MongoDB + console.log('Connecting to MongoDB...'); + await mongoose.connect(process.env.MONGO_URI); + console.log('Connected to MongoDB'); + + // Find the user by email + console.log('Looking for user: $USER_EMAIL'); + const user = await findUser({ email: '$USER_EMAIL' }); + + if (!user) { + console.error('User not found: $USER_EMAIL'); + await mongoose.connection.close(); + process.exit(1); + } + + console.log('Found user:', user.email, 'Current role:', user.role); + + // Check if already has target role + if (user.role === targetRole) { + console.log('User already has ' + targetRole + ' role'); + await mongoose.connection.close(); + process.exit(0); + } + + // Update the user role + console.log('Updating user role to ' + targetRole + '...'); + const result = await updateUser(user._id, { role: targetRole }); + + if (result) { + if (targetRole === 'ADMIN') { + console.log('βœ… User $USER_EMAIL granted ADMIN role successfully'); + } else { + console.log('βœ… User $USER_EMAIL removed from ADMIN role successfully'); + } + await mongoose.connection.close(); + process.exit(0); + } else { + console.error('Failed to update user role'); + await mongoose.connection.close(); + process.exit(1); + } + } catch (err) { + console.error('Error:', err.message); + console.error(err.stack); + if (mongoose.connection.readyState === 1) { + await mongoose.connection.close(); + } + process.exit(1); + } +})(); +EOFSCRIPT +node manage-admin.js" + +# Build the overrides JSON using jq for proper escaping +OVERRIDES=$(jq -n \ + --arg cmd "$SHELL_CMD" \ + '{ + containerOverrides: [{ + name: "librechat", + command: ["sh", "-c", $cmd] + }] + }') + +echo "Running command in container..." +TASK_ARN=$(aws ecs run-task \ + --cluster "$CLUSTER_NAME" \ + --task-definition "$TASK_DEF" \ + --launch-type FARGATE \ + --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUPS],assignPublicIp=DISABLED}" \ + --overrides "$OVERRIDES" \ + --region "$REGION" \ + --query 'tasks[0].taskArn' \ + --output text) + +echo "Task started: $TASK_ARN" +echo "" +echo "Waiting for task to complete..." +echo "You can monitor the task with:" +echo " aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $TASK_ARN --region $REGION" +echo "" +echo "Or view logs in CloudWatch Logs:" +echo " Log Group: /aws/ecs/${STACK_NAME}" +echo "" + +# Wait for task to complete +aws ecs wait tasks-stopped \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" + +# Check task exit code +EXIT_CODE=$(aws ecs describe-tasks \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --region "$REGION" \ + --query 'tasks[0].containers[0].exitCode' \ + --output text) + +if [ "$EXIT_CODE" = "0" ]; then + if [ "$REMOVE_ADMIN" = true ]; then + echo "βœ… Success! User $USER_EMAIL has been removed from admin role." + else + echo "βœ… Success! User $USER_EMAIL has been granted admin permissions." + fi + echo "The user will need to log out and log back in for changes to take effect." +else + echo "❌ Task failed with exit code: $EXIT_CODE" + echo "Check CloudWatch Logs for details." + exit 1 +fi diff --git a/deploy/aws-sam/scripts/scale-service.sh b/deploy/aws-sam/scripts/scale-service.sh new file mode 100755 index 0000000000..9675c631c2 --- /dev/null +++ b/deploy/aws-sam/scripts/scale-service.sh @@ -0,0 +1,153 @@ +#!/bin/bash + +# Script to manually scale LibreChat ECS service +# Usage: ./scale-service.sh [stack-name] [desired-count] + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Default values +STACK_NAME="${1:-librechat}" +DESIRED_COUNT="${2}" +REGION="${AWS_DEFAULT_REGION:-us-east-1}" + +# Function to print colored output +print_status() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +print_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}[WARNING]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +# Show usage if desired count not provided +if [[ -z "$DESIRED_COUNT" ]]; then + echo "Usage: $0 [stack-name] [desired-count]" + echo "" + echo "Examples:" + echo " $0 librechat 5 # Scale to 5 instances" + echo " $0 librechat-dev 1 # Scale dev environment to 1 instance" + exit 1 +fi + +# Validate desired count is a number +if ! [[ "$DESIRED_COUNT" =~ ^[0-9]+$ ]]; then + print_error "Desired count must be a number" + exit 1 +fi + +# Check if AWS CLI is available +if ! command -v aws &> /dev/null; then + print_error "AWS CLI is not installed" + exit 1 +fi + +# Check AWS credentials +if ! aws sts get-caller-identity &> /dev/null; then + print_error "AWS credentials not configured" + exit 1 +fi + +print_status "Scaling LibreChat service..." +print_status "Stack: $STACK_NAME" +print_status "Desired Count: $DESIRED_COUNT" +print_status "Region: $REGION" + +# Get cluster and service names from CloudFormation +CLUSTER_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSClusterName`].OutputValue' \ + --output text) + +SERVICE_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSServiceName`].OutputValue' \ + --output text) + +if [[ -z "$CLUSTER_NAME" || -z "$SERVICE_NAME" ]]; then + print_error "Could not find ECS cluster or service in stack $STACK_NAME" + exit 1 +fi + +print_status "Cluster: $CLUSTER_NAME" +print_status "Service: $SERVICE_NAME" + +# Get current service status +CURRENT_STATUS=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" \ + --query 'services[0].{ + RunningCount: runningCount, + PendingCount: pendingCount, + DesiredCount: desiredCount + }') + +print_status "Current service status:" +echo "$CURRENT_STATUS" | jq . + +CURRENT_DESIRED=$(echo "$CURRENT_STATUS" | jq -r '.DesiredCount') + +if [[ "$CURRENT_DESIRED" == "$DESIRED_COUNT" ]]; then + print_warning "Service is already scaled to $DESIRED_COUNT instances" + exit 0 +fi + +# Update the service desired count +print_status "Scaling service from $CURRENT_DESIRED to $DESIRED_COUNT instances..." +aws ecs update-service \ + --cluster "$CLUSTER_NAME" \ + --service "$SERVICE_NAME" \ + --desired-count "$DESIRED_COUNT" \ + --region "$REGION" \ + --query 'service.serviceName' \ + --output text + +# Wait for deployment to stabilize +print_status "Waiting for service to stabilize..." +aws ecs wait services-stable \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" + +print_success "Service scaling completed successfully!" + +# Show final service status +print_status "Final service status:" +aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" \ + --query 'services[0].{ + ServiceName: serviceName, + Status: status, + RunningCount: runningCount, + PendingCount: pendingCount, + DesiredCount: desiredCount + }' \ + --output table + +# Show running tasks +print_status "Running tasks:" +aws ecs list-tasks \ + --cluster "$CLUSTER_NAME" \ + --service-name "$SERVICE_NAME" \ + --region "$REGION" \ + --query 'taskArns' \ + --output table \ No newline at end of file diff --git a/deploy/aws-sam/scripts/simple-config-upload.sh b/deploy/aws-sam/scripts/simple-config-upload.sh new file mode 100755 index 0000000000..c08cd9e058 --- /dev/null +++ b/deploy/aws-sam/scripts/simple-config-upload.sh @@ -0,0 +1,130 @@ +#!/bin/bash + +# Simple config upload script - replaces the complex Python approach +# Usage: ./simple-config-upload.sh [region] [config-file] + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +print_status() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +print_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +# Parameters +STACK_NAME="$1" +REGION="${2:-us-east-1}" +CONFIG_FILE="${3:-librechat.yaml}" + +if [[ -z "$STACK_NAME" ]]; then + print_error "Usage: $0 [region] [config-file]" + exit 1 +fi + +print_status "Uploading config for stack: $STACK_NAME" +print_status "Region: $REGION" +print_status "Config file: $CONFIG_FILE" + +# Get S3 bucket name from CloudFormation outputs +print_status "Getting S3 bucket name from stack outputs..." +BUCKET_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`S3BucketName`].OutputValue' \ + --output text 2>/dev/null) + +if [[ -z "$BUCKET_NAME" ]]; then + print_error "Could not find S3BucketName in stack outputs" + exit 1 +fi + +print_success "Found S3 bucket: $BUCKET_NAME" + +# Upload config file to S3 +if [[ -f "$CONFIG_FILE" ]]; then + print_status "Uploading $CONFIG_FILE to S3..." + aws s3 cp "$CONFIG_FILE" "s3://$BUCKET_NAME/configs/librechat.yaml" \ + --content-type "application/x-yaml" \ + --region "$REGION" + + if [[ $? -eq 0 ]]; then + print_success "Configuration uploaded to s3://$BUCKET_NAME/configs/librechat.yaml" + else + print_error "Failed to upload configuration to S3" + exit 1 + fi +else + print_error "Config file not found: $CONFIG_FILE" + exit 1 +fi + +# Trigger Config Manager Lambda to copy S3 β†’ EFS +LAMBDA_NAME="${STACK_NAME}-config-manager" +print_status "Triggering config manager Lambda: $LAMBDA_NAME" + +aws lambda invoke \ + --function-name "$LAMBDA_NAME" \ + --region "$REGION" \ + --payload '{}' \ + /tmp/lambda-response.json >/dev/null 2>&1 + +if [[ $? -eq 0 ]]; then + print_success "Config manager Lambda executed successfully" + print_status "Configuration has been copied to EFS" +else + print_error "Could not invoke config manager Lambda" + exit 1 +fi + +# Force ECS service to restart containers +print_status "Getting ECS cluster and service information..." +CLUSTER_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSClusterName`].OutputValue' \ + --output text 2>/dev/null) + +SERVICE_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSServiceName`].OutputValue' \ + --output text 2>/dev/null) + +if [[ -n "$CLUSTER_NAME" && -n "$SERVICE_NAME" ]]; then + print_status "Restarting ECS containers to pick up new config..." + print_status "Cluster: $CLUSTER_NAME" + print_status "Service: $SERVICE_NAME" + + aws ecs update-service \ + --cluster "$CLUSTER_NAME" \ + --service "$SERVICE_NAME" \ + --region "$REGION" \ + --force-new-deployment >/dev/null 2>&1 + + if [[ $? -eq 0 ]]; then + print_success "ECS service restart initiated" + print_status "Containers will restart with the new configuration" + print_status "This may take a few minutes to complete" + else + print_error "Could not restart ECS service" + exit 1 + fi +else + print_error "Could not find ECS cluster/service information" + exit 1 +fi + +print_success "Configuration update completed successfully!" \ No newline at end of file diff --git a/deploy/aws-sam/scripts/update-service.sh b/deploy/aws-sam/scripts/update-service.sh new file mode 100755 index 0000000000..8fbb17cff3 --- /dev/null +++ b/deploy/aws-sam/scripts/update-service.sh @@ -0,0 +1,142 @@ +#!/bin/bash + +# Script to update LibreChat ECS service with new image version +# Usage: ./update-service.sh [stack-name] [image-tag] + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Default values +STACK_NAME="${1:-librechat}" +IMAGE_TAG="${2:-latest}" +REGION="${AWS_DEFAULT_REGION:-us-east-1}" + +# Function to print colored output +print_status() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +print_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}[WARNING]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +# Check if AWS CLI is available +if ! command -v aws &> /dev/null; then + print_error "AWS CLI is not installed" + exit 1 +fi + +# Check AWS credentials +if ! aws sts get-caller-identity &> /dev/null; then + print_error "AWS credentials not configured" + exit 1 +fi + +print_status "Updating LibreChat service..." +print_status "Stack: $STACK_NAME" +print_status "Image Tag: $IMAGE_TAG" +print_status "Region: $REGION" + +# Get cluster and service names from CloudFormation +CLUSTER_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSClusterName`].OutputValue' \ + --output text) + +SERVICE_NAME=$(aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query 'Stacks[0].Outputs[?OutputKey==`ECSServiceName`].OutputValue' \ + --output text) + +if [[ -z "$CLUSTER_NAME" || -z "$SERVICE_NAME" ]]; then + print_error "Could not find ECS cluster or service in stack $STACK_NAME" + exit 1 +fi + +print_status "Cluster: $CLUSTER_NAME" +print_status "Service: $SERVICE_NAME" + +# Get current task definition +TASK_DEF_ARN=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" \ + --query 'services[0].taskDefinition' \ + --output text) + +print_status "Current task definition: $TASK_DEF_ARN" + +# Get task definition details +TASK_DEF=$(aws ecs describe-task-definition \ + --task-definition "$TASK_DEF_ARN" \ + --region "$REGION" \ + --query 'taskDefinition') + +# Update the image in the task definition +NEW_IMAGE="ghcr.io/danny-avila/librechat:$IMAGE_TAG" +UPDATED_TASK_DEF=$(echo "$TASK_DEF" | jq --arg image "$NEW_IMAGE" ' + .containerDefinitions[0].image = $image | + del(.taskDefinitionArn, .revision, .status, .requiresAttributes, .placementConstraints, .compatibilities, .registeredAt, .registeredBy) +') + +print_status "Updating image to: $NEW_IMAGE" + +# Register new task definition +NEW_TASK_DEF_ARN=$(echo "$UPDATED_TASK_DEF" | aws ecs register-task-definition \ + --region "$REGION" \ + --cli-input-json file:///dev/stdin \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) + +print_status "New task definition: $NEW_TASK_DEF_ARN" + +# Update the service +print_status "Updating ECS service..." +aws ecs update-service \ + --cluster "$CLUSTER_NAME" \ + --service "$SERVICE_NAME" \ + --task-definition "$NEW_TASK_DEF_ARN" \ + --region "$REGION" \ + --query 'service.serviceName' \ + --output text + +# Wait for deployment to complete +print_status "Waiting for deployment to complete..." +aws ecs wait services-stable \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" + +print_success "Service update completed successfully!" + +# Show service status +print_status "Service status:" +aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "$SERVICE_NAME" \ + --region "$REGION" \ + --query 'services[0].{ + ServiceName: serviceName, + Status: status, + RunningCount: runningCount, + PendingCount: pendingCount, + DesiredCount: desiredCount, + TaskDefinition: taskDefinition + }' \ + --output table \ No newline at end of file diff --git a/deploy/aws-sam/src/config_manager/app.py b/deploy/aws-sam/src/config_manager/app.py new file mode 100644 index 0000000000..0caa18f907 --- /dev/null +++ b/deploy/aws-sam/src/config_manager/app.py @@ -0,0 +1,239 @@ +import json +import logging +import os +import boto3 +import urllib3 +from botocore.exceptions import ClientError + +# Configure logging +logger = logging.getLogger() +logger.setLevel(logging.INFO) + +# Initialize AWS clients +s3_client = boto3.client('s3') + +def lambda_handler(event, context): + """ + Lambda function to copy configuration files from S3 to EFS. + Handles both CloudFormation custom resource lifecycle events and direct invocations. + """ + logger.info(f"Received event: {json.dumps(event, default=str)}") + + # Check if this is a CloudFormation custom resource call or direct invocation + is_cloudformation = 'RequestType' in event and 'ResourceProperties' in event + + if is_cloudformation: + # CloudFormation custom resource call + request_type = event.get('RequestType') + resource_properties = event.get('ResourceProperties', {}) + s3_bucket = resource_properties.get('S3Bucket') + s3_key = resource_properties.get('S3Key', 'configs/librechat.yaml') + else: + # Direct invocation - get parameters from environment or event + logger.info("Direct invocation detected - processing config update") + request_type = 'Update' # Treat direct calls as updates + s3_bucket = event.get('S3Bucket') or get_s3_bucket_from_environment() + s3_key = event.get('S3Key', 'configs/librechat.yaml') + + # Configuration + efs_mount_path = os.environ.get('EFS_MOUNT_PATH', '/mnt/efs') + efs_file_path = os.path.join(efs_mount_path, 'librechat.yaml') + + response_data = {} + + try: + if request_type in ['Create', 'Update']: + logger.info(f"Processing {request_type} request") + + # Validate required parameters + if not s3_bucket: + raise ValueError("S3Bucket is required - either in ResourceProperties or environment") + + # Ensure EFS mount directory exists + os.makedirs(efs_mount_path, exist_ok=True) + logger.info(f"EFS mount path ready: {efs_mount_path}") + + # Download file from S3 + logger.info(f"Downloading s3://{s3_bucket}/{s3_key}") + used_default_config = False + try: + s3_response = s3_client.get_object(Bucket=s3_bucket, Key=s3_key) + file_content = s3_response['Body'].read() + logger.info(f"Successfully downloaded {len(file_content)} bytes from S3") + except ClientError as e: + error_code = e.response['Error']['Code'] + if error_code == 'NoSuchKey': + logger.warning(f"Configuration file not found: s3://{s3_bucket}/{s3_key}") + logger.info("Creating default configuration file on EFS") + used_default_config = True + # Create a minimal default config if the file doesn't exist + file_content = b"""# Default LibreChat Configuration +# This file was created automatically because no custom config was found +version: 1.2.8 +cache: false +interface: + customWelcome: "" +""" + elif error_code == 'NoSuchBucket': + logger.warning(f"S3 bucket not found: {s3_bucket}") + logger.info("Creating default configuration file on EFS") + used_default_config = True + # Create a minimal default config if the bucket doesn't exist + file_content = b"""# Default LibreChat Configuration +# This file was created automatically because S3 bucket was not accessible +version: 1.2.8 +cache: false +interface: + customWelcome: "Welcome to LibreChat! (Using Default Config - S3 Bucket Not Found)" +""" + elif error_code == 'AccessDenied': + logger.warning(f"Access denied to S3: s3://{s3_bucket}/{s3_key}") + logger.info("Creating default configuration file on EFS") + used_default_config = True + # Create a minimal default config if access is denied + file_content = b"""# Default LibreChat Configuration +# This file was created automatically because S3 access was denied +version: 1.2.8 +cache: false +interface: + customWelcome: "Welcome to LibreChat! (Using Default Config - S3 Access Denied)" +""" + else: + raise ValueError(f"Failed to download from S3: {str(e)}") + + # Write file to EFS + logger.info(f"Writing file to EFS: {efs_file_path}") + with open(efs_file_path, 'wb') as f: + f.write(file_content) + + # Set appropriate file permissions (readable by all, writable by owner) + os.chmod(efs_file_path, 0o644) + logger.info(f"Set file permissions to 644 for {efs_file_path}") + + # Verify file was written correctly + if os.path.exists(efs_file_path): + file_size = os.path.getsize(efs_file_path) + logger.info(f"File successfully written to EFS: {file_size} bytes") + response_data['FileSize'] = file_size + response_data['EFSPath'] = efs_file_path + response_data['UsedDefaultConfig'] = used_default_config + + # For direct invocations, return success immediately + if not is_cloudformation: + logger.info("Direct invocation completed successfully") + return { + 'statusCode': 200, + 'body': json.dumps({ + 'message': 'Configuration updated successfully', + 'fileSize': file_size, + 'efsPath': efs_file_path, + 'usedDefaultConfig': used_default_config + }) + } + else: + raise RuntimeError("File was not created on EFS") + + elif request_type == 'Delete': + logger.info("Processing Delete request") + # For delete operations, we could optionally remove the file + # but it's safer to leave it in place for potential rollbacks + if os.path.exists(efs_file_path): + logger.info(f"Configuration file exists at {efs_file_path} (leaving in place)") + else: + logger.info("Configuration file not found (already removed or never created)") + + # Send success response to CloudFormation (only for CF calls) + if is_cloudformation: + send_response(event, context, 'SUCCESS', response_data) + + except Exception as e: + logger.error(f"Error processing request: {str(e)}", exc_info=True) + + # Handle errors differently for CF vs direct calls + if is_cloudformation: + send_response(event, context, 'FAILED', {'Error': str(e)}) + else: + # For direct invocations, return error response + return { + 'statusCode': 500, + 'body': json.dumps({ + 'error': str(e), + 'message': 'Configuration update failed' + }) + } + raise + + +def get_s3_bucket_from_environment(): + """ + Try to determine the S3 bucket name from the Lambda function's environment. + This is used for direct invocations when the bucket isn't provided in the event. + Prefers S3_BUCKET_NAME (set by the template) to avoid needing CloudFormation permissions. + """ + # Prefer environment variable (set by CloudFormation template; no extra IAM needed) + bucket_name = os.environ.get('S3_BUCKET_NAME') + if bucket_name: + logger.info(f"Found S3 bucket from environment: {bucket_name}") + return bucket_name + + # Fallback: try to get from CloudFormation stack outputs (requires cloudformation:DescribeStacks) + function_name = os.environ.get('AWS_LAMBDA_FUNCTION_NAME', '') + if function_name.endswith('-config-manager'): + stack_name = function_name[:-15] # Remove '-config-manager' + try: + cf_client = boto3.client('cloudformation') + response = cf_client.describe_stacks(StackName=stack_name) + outputs = response['Stacks'][0].get('Outputs', []) + for output in outputs: + if output['OutputKey'] == 'S3BucketName': + bucket_name = output['OutputValue'] + logger.info(f"Found S3 bucket from CloudFormation: {bucket_name}") + return bucket_name + except Exception as e: + logger.warning(f"Could not get S3 bucket from CloudFormation: {str(e)}") + + logger.warning("Could not determine S3 bucket name") + return None + + +def send_response(event, context, response_status, response_data): + """ + Send response to CloudFormation custom resource. + """ + response_url = event.get('ResponseURL') + if not response_url: + logger.warning("No ResponseURL provided - this may be a test invocation") + return + + # Prepare response payload + response_body = { + 'Status': response_status, + 'Reason': f'See CloudWatch Log Stream: {context.log_stream_name}', + 'PhysicalResourceId': event.get('LogicalResourceId', 'ConfigManagerResource'), + 'StackId': event.get('StackId'), + 'RequestId': event.get('RequestId'), + 'LogicalResourceId': event.get('LogicalResourceId'), + 'Data': response_data + } + + json_response_body = json.dumps(response_body) + logger.info(f"Sending response to CloudFormation: {response_status}") + logger.debug(f"Response body: {json_response_body}") + + try: + # Send HTTP PUT request to CloudFormation + http = urllib3.PoolManager() + response = http.request( + 'PUT', + response_url, + body=json_response_body, + headers={ + 'Content-Type': 'application/json', + 'Content-Length': str(len(json_response_body)) + } + ) + logger.info(f"CloudFormation response status: {response.status}") + + except Exception as e: + logger.error(f"Failed to send response to CloudFormation: {str(e)}") + raise \ No newline at end of file diff --git a/deploy/aws-sam/src/config_manager/requirements.txt b/deploy/aws-sam/src/config_manager/requirements.txt new file mode 100644 index 0000000000..732ae6bb80 --- /dev/null +++ b/deploy/aws-sam/src/config_manager/requirements.txt @@ -0,0 +1,2 @@ +boto3>=1.26.0 +urllib3>=1.26.0 \ No newline at end of file diff --git a/deploy/aws-sam/src/mount_target_waiter/app.py b/deploy/aws-sam/src/mount_target_waiter/app.py new file mode 100644 index 0000000000..bf569246a7 --- /dev/null +++ b/deploy/aws-sam/src/mount_target_waiter/app.py @@ -0,0 +1,135 @@ +import json +import logging +import boto3 +import urllib3 +import time +from botocore.exceptions import ClientError + +# Configure logging +logger = logging.getLogger() +logger.setLevel(logging.INFO) + +# Initialize AWS clients +efs_client = boto3.client('efs') + +def lambda_handler(event, context): + """ + Lambda function to wait for EFS mount targets to be available. + This ensures mount targets are ready before other resources try to use them. + """ + logger.info(f"Received event: {json.dumps(event, default=str)}") + + # Extract CloudFormation custom resource properties + request_type = event.get('RequestType') + resource_properties = event.get('ResourceProperties', {}) + + # Configuration + file_system_id = resource_properties.get('FileSystemId') + + response_data = {} + + try: + if request_type in ['Create', 'Update']: + logger.info(f"Processing {request_type} request") + + # Validate required parameters + if not file_system_id: + raise ValueError("FileSystemId is required in ResourceProperties") + + # Wait for mount targets to be available + logger.info(f"Waiting for mount targets to be available for EFS: {file_system_id}") + + max_wait_time = 300 # 5 minutes + start_time = time.time() + + while time.time() - start_time < max_wait_time: + try: + # Get mount targets for the file system + response = efs_client.describe_mount_targets(FileSystemId=file_system_id) + mount_targets = response.get('MountTargets', []) + + if not mount_targets: + logger.info("No mount targets found yet, waiting...") + time.sleep(10) + continue + + # Check if all mount targets are available + all_available = True + for mt in mount_targets: + state = mt.get('LifeCycleState') + logger.info(f"Mount target {mt.get('MountTargetId')} state: {state}") + if state != 'available': + all_available = False + break + + if all_available: + logger.info("All mount targets are available!") + response_data['MountTargetsReady'] = True + response_data['MountTargetCount'] = len(mount_targets) + break + else: + logger.info("Some mount targets are not ready yet, waiting...") + time.sleep(10) + + except ClientError as e: + logger.warning(f"Error checking mount targets: {e}") + time.sleep(10) + else: + # Timeout reached + raise RuntimeError(f"Mount targets did not become available within {max_wait_time} seconds") + + elif request_type == 'Delete': + logger.info("Processing Delete request - nothing to do") + response_data['Status'] = 'Deleted' + + # Send success response to CloudFormation + send_response(event, context, 'SUCCESS', response_data) + + except Exception as e: + logger.error(f"Error processing request: {str(e)}", exc_info=True) + # Send failure response to CloudFormation + send_response(event, context, 'FAILED', {'Error': str(e)}) + raise + + +def send_response(event, context, response_status, response_data): + """ + Send response to CloudFormation custom resource. + """ + response_url = event.get('ResponseURL') + if not response_url: + logger.warning("No ResponseURL provided - this may be a test invocation") + return + + # Prepare response payload + response_body = { + 'Status': response_status, + 'Reason': f'See CloudWatch Log Stream: {context.log_stream_name}', + 'PhysicalResourceId': event.get('LogicalResourceId', 'MountTargetWaiterResource'), + 'StackId': event.get('StackId'), + 'RequestId': event.get('RequestId'), + 'LogicalResourceId': event.get('LogicalResourceId'), + 'Data': response_data + } + + json_response_body = json.dumps(response_body) + logger.info(f"Sending response to CloudFormation: {response_status}") + logger.debug(f"Response body: {json_response_body}") + + try: + # Send HTTP PUT request to CloudFormation + http = urllib3.PoolManager() + response = http.request( + 'PUT', + response_url, + body=json_response_body, + headers={ + 'Content-Type': 'application/json', + 'Content-Length': str(len(json_response_body)) + } + ) + logger.info(f"CloudFormation response status: {response.status}") + + except Exception as e: + logger.error(f"Failed to send response to CloudFormation: {str(e)}") + raise \ No newline at end of file diff --git a/deploy/aws-sam/src/mount_target_waiter/requirements.txt b/deploy/aws-sam/src/mount_target_waiter/requirements.txt new file mode 100644 index 0000000000..732ae6bb80 --- /dev/null +++ b/deploy/aws-sam/src/mount_target_waiter/requirements.txt @@ -0,0 +1,2 @@ +boto3>=1.26.0 +urllib3>=1.26.0 \ No newline at end of file diff --git a/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/app.py b/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/app.py new file mode 100644 index 0000000000..beaa4a161e --- /dev/null +++ b/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/app.py @@ -0,0 +1,97 @@ +""" +CloudFormation custom resource: add this stack's ECS security group to an existing +Secrets Manager VPC endpoint's security group so ECS tasks can pull secrets. +Runs during stack create/update (after ECSSecurityGroup exists, before ECS Service). +""" +import json +import logging +import urllib3 +import boto3 +from botocore.exceptions import ClientError + +logger = logging.getLogger() +logger.setLevel(logging.INFO) +ec2 = boto3.client('ec2') + + +def lambda_handler(event, context): + request_type = event.get('RequestType') + props = event.get('ResourceProperties', {}) + endpoint_sg_id = (props.get('EndpointSecurityGroupId') or '').strip() + ecs_sg_id = (props.get('EcsSecurityGroupId') or '').strip() + response_data = {} + + try: + if request_type in ('Create', 'Update'): + if endpoint_sg_id and ecs_sg_id: + logger.info( + "Adding ingress to endpoint SG %s: TCP 443 from ECS SG %s", + endpoint_sg_id, ecs_sg_id + ) + try: + ec2.authorize_security_group_ingress( + GroupId=endpoint_sg_id, + IpPermissions=[{ + 'IpProtocol': 'tcp', + 'FromPort': 443, + 'ToPort': 443, + 'UserIdGroupPairs': [{'GroupId': ecs_sg_id}], + }], + ) + response_data['RuleAdded'] = 'true' + except ClientError as e: + if e.response['Error']['Code'] == 'InvalidPermission.Duplicate': + logger.info("Rule already exists, no change") + response_data['RuleAdded'] = 'already_exists' + else: + raise + else: + logger.info( + "EndpointSecurityGroupId or EcsSecurityGroupId empty; skipping (no-op)" + ) + elif request_type == 'Delete': + if endpoint_sg_id and ecs_sg_id: + try: + ec2.revoke_security_group_ingress( + GroupId=endpoint_sg_id, + IpPermissions=[{ + 'IpProtocol': 'tcp', + 'FromPort': 443, + 'ToPort': 443, + 'UserIdGroupPairs': [{'GroupId': ecs_sg_id}], + }], + ) + response_data['RuleRevoked'] = 'true' + except ClientError as e: + if e.response['Error']['Code'] in ( + 'InvalidPermission.NotFound', 'InvalidGroup.NotFound' + ): + logger.info("Rule or group already gone, ignoring") + else: + logger.warning("Revoke failed (non-fatal): %s", e) + send_response(event, context, 'SUCCESS', response_data) + except Exception as e: + logger.error("Error: %s", e, exc_info=True) + send_response(event, context, 'FAILED', {'Error': str(e)}) + raise + + +def send_response(event, context, response_status, response_data): + response_url = event.get('ResponseURL') + if not response_url: + return + body = { + 'Status': response_status, + 'Reason': f'See CloudWatch Log Stream: {context.log_stream_name}', + 'PhysicalResourceId': event.get('LogicalResourceId', 'SecretsManagerEndpointEcsAccess'), + 'StackId': event.get('StackId'), + 'RequestId': event.get('RequestId'), + 'LogicalResourceId': event.get('LogicalResourceId'), + 'Data': response_data, + } + http = urllib3.PoolManager() + http.request( + 'PUT', response_url, + body=json.dumps(body), + headers={'Content-Type': 'application/json'}, + ) diff --git a/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/requirements.txt b/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/requirements.txt new file mode 100644 index 0000000000..9e90444cb0 --- /dev/null +++ b/deploy/aws-sam/src/secretsmanager_endpoint_ecs_access/requirements.txt @@ -0,0 +1,2 @@ +boto3>=1.26.0 +urllib3>=1.26.0 diff --git a/deploy/aws-sam/template.yaml b/deploy/aws-sam/template.yaml new file mode 100644 index 0000000000..5fa42eb2e6 --- /dev/null +++ b/deploy/aws-sam/template.yaml @@ -0,0 +1,1401 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: 'Scalable LibreChat deployment on AWS using ECS Fargate' + +Parameters: + Environment: + Type: String + Default: prod + AllowedValues: [dev, staging, prod] + Description: Environment name + + VpcId: + Type: AWS::EC2::VPC::Id + Description: Existing VPC ID to deploy into + + PublicSubnetIds: + Type: List + Description: List of existing public subnet IDs for the load balancer (minimum 2 in different AZs) + + PrivateSubnetIds: + Type: List + Description: List of existing private subnet IDs for ECS tasks and databases (minimum 2 in different AZs) + + LibreChatImage: + Type: String + Default: librechat/librechat:latest + Description: LibreChat Docker image custom build + + DomainName: + Type: String + Default: "" + Description: Domain name for LibreChat (optional) + + CertificateArn: + Type: String + Default: "" + Description: ACM certificate ARN for HTTPS (optional) + + BedrockAccessKeyId: + Type: String + NoEcho: true + Default: "" + Description: "(Deprecated) AWS Access Key ID for Bedrock; use BedrockCredentialsSecretArn instead" + + BedrockSecretAccessKey: + Type: String + NoEcho: true + Default: "" + Description: "(Deprecated) AWS Secret Access Key for Bedrock; use BedrockCredentialsSecretArn instead" + + BedrockCredentialsSecretArn: + Type: String + Default: "" + Description: Secrets Manager ARN for Bedrock credentials JSON (accessKeyId, secretAccessKey); preferred over BedrockAccessKeyId/BedrockSecretAccessKey + + EnableSSO: + Type: String + Default: "false" + AllowedValues: ["true", "false"] + Description: Enable SSO authentication with Cognito + + CognitoUserPoolId: + Type: String + Default: "" + Description: Cognito User Pool ID for SSO (required if EnableSSO is true) + + OpenIdClientId: + Type: String + Default: "" + Description: OpenID Client ID from Cognito App Client (required if EnableSSO is true) + + OpenIdClientSecret: + Type: String + Default: "" + NoEcho: true + Description: "(Deprecated) OpenID Client Secret; use OpenIdClientSecretArn instead when EnableSSO is true" + + OpenIdClientSecretArn: + Type: String + Default: "" + Description: Secrets Manager ARN for OpenID client secret string (preferred when EnableSSO is true) + + OpenIdScope: + Type: String + Default: "openid profile email" + Description: OpenID scope for authentication + + OpenIdButtonLabel: + Type: String + Default: "Sign in with SSO" + Description: Label for the SSO login button + + OpenIdImageUrl: + Type: String + Default: "" + Description: Image URL for the SSO login button (optional) + + OpenIdNameClaim: + Type: String + Default: "name" + Description: Claim attribute for user name + + OpenIdEmailClaim: + Type: String + Default: "email" + Description: Claim attribute for user email + + HelpAndFaqUrl: + Type: String + Default: "" + Description: Help and FAQ URL (use '/' to disable button) + + CreateNATGateway: + Type: String + Default: "false" + AllowedValues: ["true", "false"] + Description: Create NAT Gateways for internet connectivity in private subnets + + CreateSecretsManagerVPCEndpoint: + Type: String + Default: "true" + AllowedValues: ["true", "false"] + Description: Create Secrets Manager VPC endpoint in this stack. Set to false when deploying multiple stacks (dev/staging/prod) in the same VPC so the first stack's endpoint (with private DNS) is reused. + + ExistingSecretsManagerEndpointSecurityGroupId: + Type: String + Default: "" + Description: When CreateSecretsManagerVPCEndpoint is false, the security group ID of the existing Secrets Manager VPC endpoint (so this stack's ECS tasks can be allowed to reach it). The deploy script sets this automatically. + + # Optional MCP secrets (one secret per MCP; created by deploy script) + MCPCongressSecretArn: + Type: String + Default: "" + Description: Secrets Manager ARN for MCP Congress token (optional) + MCPEasternTimeSecretArn: + Type: String + Default: "" + Description: Secrets Manager ARN for MCP Eastern Time token (optional) + MCPPennkeyLookupSecretArn: + Type: String + Default: "" + Description: Secrets Manager ARN for MCP Pennkey Lookup token (optional) + +Conditions: + HasDomain: !Not [!Equals [!Ref DomainName, ""]] + HasCertificate: !Not [!Equals [!Ref CertificateArn, ""]] + EnableSSO: !Equals [!Ref EnableSSO, "true"] + HasHelpUrl: !Not [!Equals [!Ref HelpAndFaqUrl, ""]] + CreateNATGateway: !Equals [!Ref CreateNATGateway, "true"] + CreateSecretsManagerVPCEndpoint: !Equals [!Ref CreateSecretsManagerVPCEndpoint, "true"] + HasMCPCongressSecret: !Not [!Equals [!Ref MCPCongressSecretArn, ""]] + HasMCPEasternTimeSecret: !Not [!Equals [!Ref MCPEasternTimeSecretArn, ""]] + HasMCPPennkeyLookupSecret: !Not [!Equals [!Ref MCPPennkeyLookupSecretArn, ""]] + UseBedrockCredentialsSecret: !Not [!Equals [!Ref BedrockCredentialsSecretArn, ""]] + UseOpenIdClientSecretArn: !And [!Equals [!Ref EnableSSO, "true"], !Not [!Equals [!Ref OpenIdClientSecretArn, ""]]] + +Globals: + Function: + Timeout: 30 + MemorySize: 512 + Runtime: python3.11 + +Resources: + # NAT Gateway Resources (Conditional) + NATGateway1EIP: + Type: AWS::EC2::EIP + Condition: CreateNATGateway + Properties: + Domain: vpc + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-nat-eip-1 + + NATGateway2EIP: + Type: AWS::EC2::EIP + Condition: CreateNATGateway + Properties: + Domain: vpc + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-nat-eip-2 + + NATGateway1: + Type: AWS::EC2::NatGateway + Condition: CreateNATGateway + Properties: + AllocationId: !GetAtt NATGateway1EIP.AllocationId + SubnetId: !Select [0, !Ref PublicSubnetIds] + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-nat-gateway-1 + + NATGateway2: + Type: AWS::EC2::NatGateway + Condition: CreateNATGateway + Properties: + AllocationId: !GetAtt NATGateway2EIP.AllocationId + SubnetId: !Select [1, !Ref PublicSubnetIds] + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-nat-gateway-2 + + # Route Tables for Private Subnets (Conditional) + PrivateRouteTable1: + Type: AWS::EC2::RouteTable + Condition: CreateNATGateway + Properties: + VpcId: !Ref VpcId + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-private-rt-1 + + PrivateRouteTable2: + Type: AWS::EC2::RouteTable + Condition: CreateNATGateway + Properties: + VpcId: !Ref VpcId + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-private-rt-2 + + # Default Routes to NAT Gateways + PrivateRoute1: + Type: AWS::EC2::Route + Condition: CreateNATGateway + Properties: + RouteTableId: !Ref PrivateRouteTable1 + DestinationCidrBlock: 0.0.0.0/0 + NatGatewayId: !Ref NATGateway1 + + PrivateRoute2: + Type: AWS::EC2::Route + Condition: CreateNATGateway + Properties: + RouteTableId: !Ref PrivateRouteTable2 + DestinationCidrBlock: 0.0.0.0/0 + NatGatewayId: !Ref NATGateway2 + + # Associate Route Tables with Private Subnets + PrivateSubnetRouteTableAssociation1: + Type: AWS::EC2::SubnetRouteTableAssociation + Condition: CreateNATGateway + Properties: + SubnetId: !Select [0, !Ref PrivateSubnetIds] + RouteTableId: !Ref PrivateRouteTable1 + + PrivateSubnetRouteTableAssociation2: + Type: AWS::EC2::SubnetRouteTableAssociation + Condition: CreateNATGateway + Properties: + SubnetId: !Select [1, !Ref PrivateSubnetIds] + RouteTableId: !Ref PrivateRouteTable2 + + # Security Groups + ALBSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for Application Load Balancer + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 80 + ToPort: 80 + CidrIp: 0.0.0.0/0 + - IpProtocol: tcp + FromPort: 443 + ToPort: 443 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-alb-sg + + ECSSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for ECS tasks + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 3080 + ToPort: 3080 + SourceSecurityGroupId: !Ref ALBSecurityGroup + SecurityGroupEgress: + - IpProtocol: -1 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-ecs-sg + + DatabaseSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for databases + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 27017 + ToPort: 27017 + SourceSecurityGroupId: !Ref ECSSecurityGroup + - IpProtocol: tcp + FromPort: 6379 + ToPort: 6379 + SourceSecurityGroupId: !Ref ECSSecurityGroup + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-db-sg + + LambdaSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for Lambda functions + VpcId: !Ref VpcId + SecurityGroupEgress: + - IpProtocol: tcp + FromPort: 2049 + ToPort: 2049 + CidrIp: 10.0.0.0/8 + Description: NFS access to EFS (private subnets) + - IpProtocol: tcp + FromPort: 443 + ToPort: 443 + CidrIp: 0.0.0.0/0 + Description: HTTPS access for S3 API calls + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-lambda-sg + + EFSSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for EFS file system + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 2049 + ToPort: 2049 + SourceSecurityGroupId: !Ref ECSSecurityGroup + Description: NFS access from ECS tasks + - IpProtocol: tcp + FromPort: 2049 + ToPort: 2049 + SourceSecurityGroupId: !Ref LambdaSecurityGroup + Description: NFS access from Lambda functions + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-efs-sg + + # VPC endpoint security group for Secrets Manager (so ECS tasks in private subnet can inject secrets) + # Only created when this stack owns the Secrets Manager VPC endpoint (first stack in VPC). + SecretsManagerEndpointSecurityGroup: + Type: AWS::EC2::SecurityGroup + Condition: CreateSecretsManagerVPCEndpoint + Properties: + GroupDescription: Security group for Secrets Manager VPC endpoint + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 443 + ToPort: 443 + SourceSecurityGroupId: !Ref ECSSecurityGroup + Description: HTTPS from ECS tasks for secret injection + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-secretsmanager-endpoint-sg + + # VPC interface endpoint for Secrets Manager (required for ECS secret injection in private subnets). + # Only one endpoint per VPC can have private DNS; set CreateSecretsManagerVPCEndpoint=false for additional stacks in the same VPC. + SecretsManagerVPCEndpoint: + Type: AWS::EC2::VPCEndpoint + Condition: CreateSecretsManagerVPCEndpoint + Properties: + VpcId: !Ref VpcId + ServiceName: !Sub 'com.amazonaws.${AWS::Region}.secretsmanager' + VpcEndpointType: Interface + PrivateDnsEnabled: true + SubnetIds: !Ref PrivateSubnetIds + SecurityGroupIds: + - !Ref SecretsManagerEndpointSecurityGroup + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-secretsmanager-endpoint + + # When reusing an existing Secrets Manager VPC endpoint, add this stack's ECS SG to its SG before ECS Service starts + SecretsManagerEndpointEcsAccessRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole + Policies: + - PolicyName: EC2SecurityGroupIngress + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - ec2:AuthorizeSecurityGroupIngress + - ec2:RevokeSecurityGroupIngress + - ec2:DescribeSecurityGroups + Resource: "*" + + SecretsManagerEndpointEcsAccessFunction: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub ${AWS::StackName}-secretsmanager-endpoint-ecs-access + CodeUri: src/secretsmanager_endpoint_ecs_access/ + Handler: app.lambda_handler + Runtime: python3.11 + Timeout: 60 + MemorySize: 128 + Role: !GetAtt SecretsManagerEndpointEcsAccessRole.Arn + + SecretsManagerEndpointEcsAccessPermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !Ref SecretsManagerEndpointEcsAccessFunction + Action: lambda:InvokeFunction + Principal: cloudformation.amazonaws.com + SourceAccount: !Ref AWS::AccountId + + SecretsManagerEndpointEcsAccess: + Type: AWS::CloudFormation::CustomResource + DependsOn: ECSSecurityGroup + Properties: + ServiceToken: !GetAtt SecretsManagerEndpointEcsAccessFunction.Arn + EndpointSecurityGroupId: !Ref ExistingSecretsManagerEndpointSecurityGroupId + EcsSecurityGroupId: !Ref ECSSecurityGroup + + # Application Load Balancer + ApplicationLoadBalancer: + Type: AWS::ElasticLoadBalancingV2::LoadBalancer + Properties: + Name: !Sub ${AWS::StackName}-alb + Scheme: internet-facing + Type: application + Subnets: !Ref PublicSubnetIds + SecurityGroups: + - !Ref ALBSecurityGroup + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-alb + + ALBTargetGroup: + Type: AWS::ElasticLoadBalancingV2::TargetGroup + Properties: + Name: !Sub ${AWS::StackName}-tg + Port: 3080 + Protocol: HTTP + VpcId: !Ref VpcId + TargetType: ip + HealthCheckPath: /health + HealthCheckProtocol: HTTP + HealthCheckIntervalSeconds: 60 + HealthCheckTimeoutSeconds: 30 + HealthyThresholdCount: 2 + UnhealthyThresholdCount: 6 + + ALBListener: + Type: AWS::ElasticLoadBalancingV2::Listener + Properties: + DefaultActions: + - Type: forward + TargetGroupArn: !Ref ALBTargetGroup + LoadBalancerArn: !Ref ApplicationLoadBalancer + Port: !If [HasCertificate, 443, 80] + Protocol: !If [HasCertificate, HTTPS, HTTP] + Certificates: !If + - HasCertificate + - - CertificateArn: !Ref CertificateArn + - !Ref AWS::NoValue + + # ECS Cluster + ECSCluster: + Type: AWS::ECS::Cluster + Properties: + ClusterName: !Sub ${AWS::StackName}-cluster + CapacityProviders: + - FARGATE + - FARGATE_SPOT + DefaultCapacityProviderStrategy: + - CapacityProvider: FARGATE + Weight: 1 + - CapacityProvider: FARGATE_SPOT + Weight: 4 + ClusterSettings: + - Name: containerInsights + Value: enabled + + # ECS Task Definition + ECSTaskDefinition: + Type: AWS::ECS::TaskDefinition + Properties: + Family: !Sub ${AWS::StackName}-task + NetworkMode: awsvpc + RequiresCompatibilities: + - FARGATE + Cpu: 1024 + Memory: 2048 + ExecutionRoleArn: !Ref ECSTaskExecutionRole + TaskRoleArn: !Ref ECSTaskRole + Volumes: + - Name: config-volume + EFSVolumeConfiguration: + FilesystemId: !Ref EFSFileSystem + RootDirectory: /lambda + TransitEncryption: ENABLED + AuthorizationConfig: + IAM: ENABLED + ContainerDefinitions: + - Name: librechat + Image: !Ref LibreChatImage + PortMappings: + - ContainerPort: 3080 + Protocol: tcp + MountPoints: + - SourceVolume: config-volume + ContainerPath: /app/config + ReadOnly: true + Environment: + - Name: NODE_ENV + Value: production + - Name: MONGO_URI + Value: !Sub + - mongodb://librechat:{{resolve:secretsmanager:${DocumentDBPassword}:SecretString:password}}@${MongoHost}:27017/LibreChat?ssl=true&tlsAllowInvalidCertificates=true&authSource=admin&retryWrites=false + - MongoHost: !GetAtt DocumentDBCluster.Endpoint + DocumentDBPassword: !Ref DocumentDBPassword + - Name: USE_REDIS + Value: "true" + - Name: REDIS_URI + Value: !Sub + - rediss://${RedisHost}:6379 # Changed from redis:// to rediss:// + - RedisHost: !GetAtt ElastiCacheReplicationGroup.PrimaryEndPoint.Address + # For ElastiCache with TLS + - Name: REDIS_USE_ALTERNATIVE_DNS_LOOKUP + Value: "true" + - Name: REDIS_KEY_PREFIX_VAR + Value: "AWS_EXECUTION_ENV" # or use static prefix + - Name: NODE_TLS_REJECT_UNAUTHORIZED + Value: "1" + # AWS Configuration (uses credentials from Secrets Manager) + - Name: AWS_REGION + Value: !Ref AWS::Region + # AWS Bedrock Configuration (keys in Secrets when BedrockCredentialsSecretArn set; else plain below) + - Name: BEDROCK_AWS_DEFAULT_REGION + Value: !Ref AWS::Region + - !If + - UseBedrockCredentialsSecret + - !Ref AWS::NoValue + - Name: BEDROCK_AWS_ACCESS_KEY_ID + Value: !Ref BedrockAccessKeyId + - !If + - UseBedrockCredentialsSecret + - !Ref AWS::NoValue + - Name: BEDROCK_AWS_SECRET_ACCESS_KEY + Value: !Ref BedrockSecretAccessKey + - Name: BEDROCK_AWS_MODELS + Value: "us.anthropic.claude-opus-4-20250514-v1:0,us.anthropic.claude-3-7-sonnet-20250219-v1:0,us.cohere.embed-v4:0,us.anthropic.claude-haiku-4-5-20251001-v1:0,us.anthropic.claude-sonnet-4-20250514-v1:0,us.anthropic.claude-opus-4-5-20251101-v1:0,us.anthropic.claude-sonnet-4-5-20250929-v1:0" + # Value: "us.anthropic.claude-opus-4-20250514-v1:0,us.anthropic.claude-3-7-sonnet-20250219-v1:0,us.cohere.embed-v4:0,us.anthropic.claude-haiku-4-5-20251001-v1:0,us.anthropic.claude-sonnet-4-20250514-v1:0,us.anthropic.claude-opus-4-5-20251101-v1:0,us.anthropic.claude-sonnet-4-5-20250929-v1:0,us.anthropic.claude-opus-4-1-20250805-v1:0" + # S3 Storage Credentials (in Secrets when Bedrock secret; else plain below) + - !If + - UseBedrockCredentialsSecret + - !Ref AWS::NoValue + - Name: AWS_ACCESS_KEY_ID + Value: !Ref BedrockAccessKeyId + - !If + - UseBedrockCredentialsSecret + - !Ref AWS::NoValue + - Name: AWS_SECRET_ACCESS_KEY + Value: !Ref BedrockSecretAccessKey + - Name: AWS_BUCKET_NAME + Value: !Sub ${AWS::StackName}-files-${AWS::AccountId} + # LibreChat config file path + - Name: CONFIG_PATH + Value: "/app/config/librechat.yaml" + #### APP CONFIG SETTINGS #### + # Allow new user registration + - Name: ALLOW_REGISTRATION + Value: "false" + # Help and Faq URL - If empty or commented, the button is enabled. To disable the Help and FAQ button, set to "/". + - Name: HELP_AND_FAQ_URL + Value: !If [HasHelpUrl, !Ref HelpAndFaqUrl, "https://www.google.edu"] + # Cache settings + - Name: CACHE + Value: "true" + - Name: ALLOW_SOCIAL_LOGIN + Value: !If [EnableSSO, "true", "false"] + - Name: ALLOW_SOCIAL_REGISTRATION + Value: !If [EnableSSO, "true", "false"] + - Name: DOMAIN_CLIENT + Value: !If [HasDomain, !Sub "https://${DomainName}", !Sub "https://${ApplicationLoadBalancer.DNSName}"] + - Name: DOMAIN_SERVER + Value: !If [HasDomain, !Sub "https://${DomainName}", !Sub "https://${ApplicationLoadBalancer.DNSName}"] + - Name: DEBUG_OPENID_REQUESTS + Value: "false" + - Name: DEBUG_OPENID + Value: "false" + - Name: DEBUG_DOMAIN_VALIDATION + Value: "false" + - Name: DEBUG_OAUTH_ERRORS + Value: "false" + - Name: DEBUG_SESSION_MESSAGES + Value: "false" + - Name: BYPASS_DOMAIN_CHECK + Value: "false" + # SSO Configuration (conditional) + - !If + - EnableSSO + - Name: OPENID_CLIENT_ID + Value: !Ref OpenIdClientId + - !Ref AWS::NoValue + # OPENID_CLIENT_SECRET in Secrets section when UseOpenIdClientSecretArn; else plain in Environment below + - !If + - EnableSSO + - Name: OPENID_ISSUER + Value: !Sub "https://cognito-idp.${AWS::Region}.amazonaws.com/${CognitoUserPoolId}" + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_SCOPE + Value: !Ref OpenIdScope + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_CALLBACK_URL + Value: "/oauth/openid/callback" + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_BUTTON_LABEL + Value: !Ref OpenIdButtonLabel + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_IMAGE_URL + Value: !Ref OpenIdImageUrl + - !Ref AWS::NoValue + # OPENID_SESSION_SECRET in Secrets section + - !If + - EnableSSO + - Name: OPENID_GENERATE_NONCE + Value: "true" + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_NAME_CLAIM + Value: !Ref OpenIdNameClaim + - !Ref AWS::NoValue + - !If + - EnableSSO + - Name: OPENID_EMAIL_CLAIM + Value: !Ref OpenIdEmailClaim + - !Ref AWS::NoValue + - Name: ALLOW_EMAIL_LOGIN + Value: !If [EnableSSO, "false", "true"] + # Session configuration to prevent conflicts + - Name: SESSION_EXPIRY + Value: "900000" + - Name: REFRESH_TOKEN_EXPIRY + Value: "604800000" + # Redis session configuration for multi-container deployments + - Name: SESSION_STORE + Value: "redis" + - Name: REDIS_SESSION_STORE + Value: "true" + # Debug (production-safe defaults) + - Name: DEBUG_LOGGING + Value: "false" + - Name: DEBUG_CONSOLE + Value: "false" + - Name: CONSOLE_JSON + Value: "true" + # OPENID_CLIENT_SECRET (plain only when not using secret ARN) + - !If + - EnableSSO + - !If + - UseOpenIdClientSecretArn + - !Ref AWS::NoValue + - Name: OPENID_CLIENT_SECRET + Value: !Ref OpenIdClientSecret + - !Ref AWS::NoValue + Secrets: + - Name: JWT_SECRET + ValueFrom: !Ref JWTSecret + - Name: JWT_REFRESH_SECRET + ValueFrom: !Ref JWTRefreshSecret + - Name: CREDS_KEY + ValueFrom: !Ref CredsKey + - Name: CREDS_IV + ValueFrom: !Ref CredsIV + - !If + - EnableSSO + - Name: OPENID_SESSION_SECRET + ValueFrom: !Ref OpenIdSessionSecret + - !Ref AWS::NoValue + - !If + - UseBedrockCredentialsSecret + - Name: BEDROCK_AWS_ACCESS_KEY_ID + ValueFrom: !Sub '${BedrockCredentialsSecretArn}:accessKeyId::' + - !Ref AWS::NoValue + - !If + - UseBedrockCredentialsSecret + - Name: BEDROCK_AWS_SECRET_ACCESS_KEY + ValueFrom: !Sub '${BedrockCredentialsSecretArn}:secretAccessKey::' + - !Ref AWS::NoValue + - !If + - UseBedrockCredentialsSecret + - Name: AWS_ACCESS_KEY_ID + ValueFrom: !Sub '${BedrockCredentialsSecretArn}:accessKeyId::' + - !Ref AWS::NoValue + - !If + - UseBedrockCredentialsSecret + - Name: AWS_SECRET_ACCESS_KEY + ValueFrom: !Sub '${BedrockCredentialsSecretArn}:secretAccessKey::' + - !Ref AWS::NoValue + - !If + - UseOpenIdClientSecretArn + - Name: OPENID_CLIENT_SECRET + ValueFrom: !Ref OpenIdClientSecretArn + - !Ref AWS::NoValue + - !If + - HasMCPCongressSecret + - Name: MCP_CONGRESS_TOKEN + ValueFrom: !Ref MCPCongressSecretArn + - !Ref AWS::NoValue + - !If + - HasMCPEasternTimeSecret + - Name: MCP_EASTERN_TIME_TOKEN + ValueFrom: !Ref MCPEasternTimeSecretArn + - !Ref AWS::NoValue + - !If + - HasMCPPennkeyLookupSecret + - Name: MCP_PENNKEY_LOOKUP_TOKEN + ValueFrom: !Ref MCPPennkeyLookupSecretArn + - !Ref AWS::NoValue + LogConfiguration: + LogDriver: awslogs + Options: + awslogs-group: !Ref CloudWatchLogGroup + awslogs-region: !Ref AWS::Region + awslogs-stream-prefix: librechat + + # ECS Service + ECSService: + Type: AWS::ECS::Service + DependsOn: + - ALBListener + - SecretsManagerEndpointEcsAccess + Properties: + ServiceName: !Sub ${AWS::StackName}-service + Cluster: !Ref ECSCluster + TaskDefinition: !Ref ECSTaskDefinition + LaunchType: FARGATE + DesiredCount: 2 + NetworkConfiguration: + AwsvpcConfiguration: + SecurityGroups: + - !Ref ECSSecurityGroup + Subnets: !Ref PrivateSubnetIds + AssignPublicIp: DISABLED + LoadBalancers: + - ContainerName: librechat + ContainerPort: 3080 + TargetGroupArn: !Ref ALBTargetGroup + HealthCheckGracePeriodSeconds: 300 + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 50 + DeploymentCircuitBreaker: + Enable: true + Rollback: true + EnableExecuteCommand: true + + # Auto Scaling + ECSAutoScalingTarget: + Type: AWS::ApplicationAutoScaling::ScalableTarget + Properties: + MaxCapacity: 20 + MinCapacity: 1 + ResourceId: !Sub service/${ECSCluster}/${ECSService.Name} + RoleARN: !Sub arn:aws:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService + ScalableDimension: ecs:service:DesiredCount + ServiceNamespace: ecs + + ECSAutoScalingPolicy: + Type: AWS::ApplicationAutoScaling::ScalingPolicy + Properties: + PolicyName: !Sub ${AWS::StackName}-scaling-policy + PolicyType: TargetTrackingScaling + ScalingTargetId: !Ref ECSAutoScalingTarget + TargetTrackingScalingPolicyConfiguration: + PredefinedMetricSpecification: + PredefinedMetricType: ECSServiceAverageCPUUtilization + TargetValue: 70.0 + ScaleOutCooldown: 300 + ScaleInCooldown: 300 + + # DocumentDB (MongoDB-compatible) + DocumentDBSubnetGroup: + Type: AWS::DocDB::DBSubnetGroup + Properties: + DBSubnetGroupDescription: Subnet group for DocumentDB + SubnetIds: !Ref PrivateSubnetIds + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-docdb-subnet-group + + DocumentDBCluster: + Type: AWS::DocDB::DBCluster + Properties: + DBClusterIdentifier: !Sub ${AWS::StackName}-docdb + MasterUsername: librechat + MasterUserPassword: !Sub '{{resolve:secretsmanager:${DocumentDBPassword}:SecretString:password}}' + BackupRetentionPeriod: 7 + PreferredBackupWindow: "03:00-04:00" + PreferredMaintenanceWindow: "sun:04:00-sun:05:00" + DBSubnetGroupName: !Ref DocumentDBSubnetGroup + VpcSecurityGroupIds: + - !Ref DatabaseSecurityGroup + StorageEncrypted: true + DeletionProtection: true + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-docdb + + DocumentDBInstance1: + Type: AWS::DocDB::DBInstance + Properties: + DBClusterIdentifier: !Ref DocumentDBCluster + DBInstanceIdentifier: !Sub ${AWS::StackName}-docdb-1 + DBInstanceClass: db.t3.medium + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-docdb-1 + + DocumentDBInstance2: + Type: AWS::DocDB::DBInstance + Properties: + DBClusterIdentifier: !Ref DocumentDBCluster + DBInstanceIdentifier: !Sub ${AWS::StackName}-docdb-2 + DBInstanceClass: db.t3.medium + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-docdb-2 + + # ElastiCache Redis + ElastiCacheSubnetGroup: + Type: AWS::ElastiCache::SubnetGroup + Properties: + Description: Subnet group for ElastiCache + SubnetIds: !Ref PrivateSubnetIds + + ElastiCacheReplicationGroup: + Type: AWS::ElastiCache::ReplicationGroup + Properties: + ReplicationGroupId: !Sub ${AWS::StackName}-redis + ReplicationGroupDescription: Redis cluster for LibreChat + CacheNodeType: cache.t3.micro + NumCacheClusters: 2 + Engine: redis + EngineVersion: 7.0 + Port: 6379 + CacheSubnetGroupName: !Ref ElastiCacheSubnetGroup + SecurityGroupIds: + - !Ref DatabaseSecurityGroup + AtRestEncryptionEnabled: true + TransitEncryptionEnabled: true + MultiAZEnabled: true + AutomaticFailoverEnabled: true + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-redis + + # EFS File System for Configuration Files + EFSFileSystem: + Type: AWS::EFS::FileSystem + Properties: + Encrypted: true + PerformanceMode: generalPurpose + ThroughputMode: bursting + FileSystemTags: + - Key: Name + Value: !Sub ${AWS::StackName}-config-efs + + EFSMountTarget1: + Type: AWS::EFS::MountTarget + Properties: + FileSystemId: !Ref EFSFileSystem + SubnetId: !Select [0, !Ref PrivateSubnetIds] + SecurityGroups: + - !Ref EFSSecurityGroup + + EFSMountTarget2: + Type: AWS::EFS::MountTarget + Properties: + FileSystemId: !Ref EFSFileSystem + SubnetId: !Select [1, !Ref PrivateSubnetIds] + SecurityGroups: + - !Ref EFSSecurityGroup + + # EFS Access Point for Lambda function + EFSAccessPoint: + Type: AWS::EFS::AccessPoint + Properties: + FileSystemId: !Ref EFSFileSystem + PosixUser: + Uid: 1000 + Gid: 1000 + RootDirectory: + Path: /lambda + CreationInfo: + OwnerUid: 1000 + OwnerGid: 1000 + Permissions: 755 + AccessPointTags: + - Key: Name + Value: !Sub ${AWS::StackName}-lambda-access-point + + # Secrets Manager + DocumentDBPassword: + Type: AWS::SecretsManager::Secret + Properties: + Name: !Sub ${AWS::StackName}/docdb/password + Description: DocumentDB master password + GenerateSecretString: + SecretStringTemplate: '{"username": "librechat"}' + GenerateStringKey: 'password' + PasswordLength: 32 + ExcludeCharacters: '"@/\ ''`~!#$%^&*()+={}[]|:;<>?,' + + JWTSecret: + Type: AWS::SecretsManager::Secret + Properties: + Name: !Sub ${AWS::StackName}/jwt/secret + Description: JWT secret for LibreChat + GenerateSecretString: + PasswordLength: 64 + ExcludeCharacters: '"@/\' + + JWTRefreshSecret: + Type: AWS::SecretsManager::Secret + Properties: + Name: !Sub ${AWS::StackName}/jwt/refresh-secret + Description: JWT refresh secret for LibreChat + GenerateSecretString: + PasswordLength: 64 + ExcludeCharacters: '"@/\' + + CredsKey: + Type: AWS::SecretsManager::Secret + Properties: + Name: !Sub ${AWS::StackName}/creds/key + Description: Credentials encryption key for LibreChat + GenerateSecretString: + PasswordLength: 32 + ExcludeCharacters: '"@/\' + + CredsIV: + Type: AWS::SecretsManager::Secret + Properties: + Name: !Sub ${AWS::StackName}/creds/iv + Description: Credentials encryption IV for LibreChat + GenerateSecretString: + PasswordLength: 16 + ExcludeCharacters: '"@/\' + + OpenIdSessionSecret: + Type: AWS::SecretsManager::Secret + Condition: EnableSSO + Properties: + Name: !Sub ${AWS::StackName}/openid/session-secret + Description: OpenID session secret for LibreChat SSO + GenerateSecretString: + PasswordLength: 64 + ExcludeCharacters: '"@/\' + + + # IAM Roles + ECSTaskExecutionRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy + Policies: + - PolicyName: SecretsManagerAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - secretsmanager:GetSecretValue + Resource: + - !Ref DocumentDBPassword + - !Ref JWTSecret + - !Ref JWTRefreshSecret + - !Ref CredsKey + - !Ref CredsIV + - !If [HasMCPCongressSecret, !Ref MCPCongressSecretArn, !Ref AWS::NoValue] + - !If [HasMCPEasternTimeSecret, !Ref MCPEasternTimeSecretArn, !Ref AWS::NoValue] + - !If [HasMCPPennkeyLookupSecret, !Ref MCPPennkeyLookupSecretArn, !Ref AWS::NoValue] + - !If [UseBedrockCredentialsSecret, !Ref BedrockCredentialsSecretArn, !Ref AWS::NoValue] + - !If [UseOpenIdClientSecretArn, !Ref OpenIdClientSecretArn, !Ref AWS::NoValue] + - !If + - EnableSSO + - PolicyName: SSOSecretsManagerAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - secretsmanager:GetSecretValue + Resource: + - !Ref OpenIdSessionSecret + - !Ref AWS::NoValue + + ECSTaskRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + Policies: + - PolicyName: S3Access + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - s3:GetObject + - s3:PutObject + - s3:DeleteObject + Resource: !Sub + - "${BucketArn}/*" + - BucketArn: !GetAtt S3Bucket.Arn + - Effect: Allow + Action: + - s3:ListBucket + Resource: !GetAtt S3Bucket.Arn + - PolicyName: BedrockAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - bedrock:InvokeModel + - bedrock:ListFoundationModels + - bedrock:GetFoundationModel + Resource: "*" + - PolicyName: EFSAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - elasticfilesystem:ClientMount + - elasticfilesystem:ClientWrite + Resource: !Sub + - "arn:aws:elasticfilesystem:${AWS::Region}:${AWS::AccountId}:file-system/${FileSystemId}" + - FileSystemId: !Ref EFSFileSystem + + # Mount Target Waiter Lambda Role + MountTargetWaiterLambdaRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole + Policies: + - PolicyName: EFSDescribeAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - elasticfilesystem:DescribeMountTargets + Resource: "*" + + # Config Manager Lambda Role + ConfigManagerLambdaRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole + Policies: + - PolicyName: S3ConfigAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - s3:GetObject + Resource: !Sub + - "${BucketArn}/configs/*" + - BucketArn: !GetAtt S3Bucket.Arn + - Effect: Allow + Action: + - s3:ListBucket + Resource: !GetAtt S3Bucket.Arn + Condition: + StringLike: + s3:prefix: "configs/*" + - PolicyName: EFSAccess + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - elasticfilesystem:ClientMount + - elasticfilesystem:ClientWrite + Resource: !Sub + - "arn:aws:elasticfilesystem:${AWS::Region}:${AWS::AccountId}:file-system/${FileSystemId}" + - FileSystemId: !Ref EFSFileSystem + + # Lambda permission for CloudFormation to invoke Config Manager + ConfigManagerLambdaPermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !Ref ConfigManagerFunction + Action: lambda:InvokeFunction + Principal: cloudformation.amazonaws.com + SourceAccount: !Ref AWS::AccountId + + # S3 Bucket for file storage and configuration + S3Bucket: + Type: AWS::S3::Bucket + Properties: + BucketName: !Sub ${AWS::StackName}-files-${AWS::AccountId} + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: AES256 + PublicAccessBlockConfiguration: + BlockPublicAcls: true + BlockPublicPolicy: true + IgnorePublicAcls: true + RestrictPublicBuckets: true + VersioningConfiguration: + Status: Enabled + LifecycleConfiguration: + Rules: + - Id: DeleteIncompleteMultipartUploads + Status: Enabled + AbortIncompleteMultipartUpload: + DaysAfterInitiation: 7 + + # Mount Target Waiter Lambda Function + MountTargetWaiterFunction: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub ${AWS::StackName}-mount-target-waiter + CodeUri: src/mount_target_waiter/ + Handler: app.lambda_handler + Runtime: python3.11 + Timeout: 360 + MemorySize: 256 + Role: !GetAtt MountTargetWaiterLambdaRole.Arn + LoggingConfig: + LogGroup: !Ref MountTargetWaiterLogGroup + + # Config Manager Lambda Function + ConfigManagerFunction: + Type: AWS::Serverless::Function + DependsOn: + - MountTargetWaiterTrigger + Properties: + FunctionName: !Sub ${AWS::StackName}-config-manager + CodeUri: src/config_manager/ + Handler: app.lambda_handler + Runtime: python3.11 + Timeout: 300 + MemorySize: 512 + Role: !GetAtt ConfigManagerLambdaRole.Arn + VpcConfig: + SecurityGroupIds: + - !Ref LambdaSecurityGroup + SubnetIds: !Ref PrivateSubnetIds + FileSystemConfigs: + - Arn: !Sub + - "arn:aws:elasticfilesystem:${AWS::Region}:${AWS::AccountId}:access-point/${AccessPointId}" + - AccessPointId: !Ref EFSAccessPoint + LocalMountPath: /mnt/efs + Environment: + Variables: + EFS_MOUNT_PATH: /mnt/efs + S3_BUCKET_NAME: !Ref S3Bucket + Events: + # This function will be invoked by CloudFormation custom resource + # No direct events needed here + LoggingConfig: + LogGroup: !Ref ConfigManagerLogGroup + + # CloudWatch Log Group for Mount Target Waiter Lambda + MountTargetWaiterLogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub /aws/lambda/${AWS::StackName}-mount-target-waiter + RetentionInDays: 30 + + # CloudWatch Log Group for Config Manager Lambda + ConfigManagerLogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub /aws/lambda/${AWS::StackName}-config-manager + RetentionInDays: 30 + + # Custom Resource to wait for mount targets to be ready + MountTargetWaiterTrigger: + Type: AWS::CloudFormation::CustomResource + DependsOn: + - EFSMountTarget1 + - EFSMountTarget2 + - EFSAccessPoint + Properties: + ServiceToken: !GetAtt MountTargetWaiterFunction.Arn + FileSystemId: !Ref EFSFileSystem + + # Lambda permission for CloudFormation to invoke Mount Target Waiter + MountTargetWaiterLambdaPermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !Ref MountTargetWaiterFunction + Action: lambda:InvokeFunction + Principal: cloudformation.amazonaws.com + SourceAccount: !Ref AWS::AccountId + + # Custom Resource to trigger Config Manager Lambda + ConfigManagerTrigger: + Type: AWS::CloudFormation::CustomResource + DependsOn: + - EFSMountTarget1 + - EFSMountTarget2 + - EFSAccessPoint + Properties: + ServiceToken: !GetAtt ConfigManagerFunction.Arn + S3Bucket: !Ref S3Bucket + S3Key: configs/librechat.yaml + EFSFileSystemId: !Ref EFSFileSystem + # Force update on every deployment by including timestamp-like value + DeploymentId: !Sub "${AWS::StackName}-${AWS::Region}-config" + + # CloudWatch Log Group + CloudWatchLogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub /ecs/${AWS::StackName} + RetentionInDays: 30 + +Outputs: + LoadBalancerURL: + Description: URL of the Application Load Balancer + Value: !Sub + - ${Protocol}://${DNSName} + - Protocol: !If [HasCertificate, https, http] + DNSName: !GetAtt ApplicationLoadBalancer.DNSName + Export: + Name: !Sub ${AWS::StackName}-LoadBalancerURL + + DocumentDBEndpoint: + Description: DocumentDB cluster endpoint + Value: !GetAtt DocumentDBCluster.Endpoint + Export: + Name: !Sub ${AWS::StackName}-DocumentDBEndpoint + + RedisEndpoint: + Description: Redis cluster endpoint + Value: !GetAtt ElastiCacheReplicationGroup.PrimaryEndPoint.Address + Export: + Name: !Sub ${AWS::StackName}-RedisEndpoint + + S3BucketName: + Description: S3 bucket for file storage + Value: !Ref S3Bucket + Export: + Name: !Sub ${AWS::StackName}-S3Bucket + + ECSClusterName: + Description: ECS cluster name + Value: !Ref ECSCluster + Export: + Name: !Sub ${AWS::StackName}-ECSCluster + + ECSServiceName: + Description: ECS service name + Value: !GetAtt ECSService.Name + Export: + Name: !Sub ${AWS::StackName}-ECSService + + EFSFileSystemId: + Description: EFS file system ID for configuration files + Value: !Ref EFSFileSystem + Export: + Name: !Sub ${AWS::StackName}-EFSFileSystemId + + EFSAccessPointId: + Description: EFS Access Point ID for Lambda function + Value: !Ref EFSAccessPoint + Export: + Name: !Sub ${AWS::StackName}-EFSAccessPointId + + ConfigManagerFunctionArn: + Description: Config Manager Lambda function ARN + Value: !GetAtt ConfigManagerFunction.Arn + Export: + Name: !Sub ${AWS::StackName}-ConfigManagerFunctionArn + + ConfigDeploymentStatus: + Description: Status of configuration file deployment to EFS + Value: !GetAtt ConfigManagerTrigger.FileSize + Export: + Name: !Sub ${AWS::StackName}-ConfigDeploymentStatus + + SSOEnabled: + Description: Whether SSO authentication is enabled + Value: !Ref EnableSSO + Export: + Name: !Sub ${AWS::StackName}-SSOEnabled + + CognitoIssuerURL: + Condition: EnableSSO + Description: Cognito OpenID Issuer URL + Value: !Sub "https://cognito-idp.${AWS::Region}.amazonaws.com/${CognitoUserPoolId}" + Export: + Name: !Sub ${AWS::StackName}-CognitoIssuerURL + + CallbackURL: + Condition: EnableSSO + Description: OAuth callback URL for Cognito configuration + Value: !Sub + - "${BaseURL}/oauth/openid/callback" + - BaseURL: !If + - HasDomain + - !Sub "https://${DomainName}" + - !Sub "https://${ApplicationLoadBalancer.DNSName}" + Export: + Name: !Sub ${AWS::StackName}-CallbackURL + + NATGatewayEnabled: + Description: Whether NAT Gateways were created for internet connectivity + Value: !Ref CreateNATGateway + Export: + Name: !Sub ${AWS::StackName}-NATGatewayEnabled + + NATGateway1Id: + Condition: CreateNATGateway + Description: NAT Gateway 1 ID + Value: !Ref NATGateway1 + Export: + Name: !Sub ${AWS::StackName}-NATGateway1Id + + NATGateway2Id: + Condition: CreateNATGateway + Description: NAT Gateway 2 ID + Value: !Ref NATGateway2 + Export: + Name: !Sub ${AWS::StackName}-NATGateway2Id + + NATGateway1EIP: + Condition: CreateNATGateway + Description: NAT Gateway 1 Elastic IP + Value: !Ref NATGateway1EIP + Export: + Name: !Sub ${AWS::StackName}-NATGateway1EIP + + NATGateway2EIP: + Condition: CreateNATGateway + Description: NAT Gateway 2 Elastic IP + Value: !Ref NATGateway2EIP + Export: + Name: !Sub ${AWS::StackName}-NATGateway2EIP + + foo: + Value: "bar bar" \ No newline at end of file