Andreas/wekan
1
0
Fork 0
mirror of https://github.com/wekan/wekan.git synced 2025-12-16 15:30:13 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Snap MongoDB Server 3-8 support. Part 1.

Browse source 
Thanks to xet7 !
This commit is contained in:
Lauri Ojansivu 2025-10-11 11:53:47 +03:00
parent 8e71b68dbf
commit b949357e66
8 changed files with 1035 additions and 1493 deletions
Download patch file Download diff file Expand all files Collapse all files

189
docs/Databases/MongoDB-Version-Management.md Normal file
View file

@ -0,0 +1,189 @@
# MongoDB Version Management System
This document describes the MongoDB version management system that automatically detects MongoDB server versions and switches to the appropriate binary and Node.js driver.
## Overview
The system provides:
- **Automatic version detection** based on connection attempts and wire protocol errors
- **Automatic binary switching** to the correct MongoDB server version
- **Automatic driver selection** using the appropriate Node.js MongoDB driver
- **Support for MongoDB versions 3-8** with fallback mechanisms
## Current Status
### Available MongoDB Server Binaries
- ✅ **MongoDB 3.2.22** - Available at `/snap/wekan/current/migratemongo/bin/`
- ❌ **MongoDB 4.4.28** - Not available (needs to be added to snap package)
- ❌ **MongoDB 5.0.28** - Not available (needs to be added to snap package)
- ❌ **MongoDB 6.0.15** - Not available (needs to be added to snap package)
- ✅ **MongoDB 7.0.25** - Available at `/snap/wekan/current/bin/`
- ❌ **MongoDB 8.0.4** - Not available (needs to be added to snap package)
### Available MongoDB Node.js Drivers
- ✅ **MongoDB 3.7.4** - Available as `mongodb3legacy`
- ✅ **MongoDB 4.17.2** - Available as `mongodb4legacy`
- ✅ **MongoDB 5.9.2** - Available as `mongodb5legacy`
- ✅ **MongoDB 6.3.0** - Available as `mongodb6legacy`
- ✅ **MongoDB 7.0.1** - Available as `mongodb7legacy`
- ✅ **MongoDB 8.0** - Available as `mongodb8legacy`
## How It Works
### 1. Version Detection
The system attempts to connect to MongoDB using the latest available binary (MongoDB 7.x) and analyzes the response:
- **Success**: If connection succeeds, it's MongoDB 7.x
- **Wire Protocol Error**: If connection fails with wire protocol errors, it detects the version based on the protocol version number:
- Protocol 0-3: MongoDB 3.x
- Protocol 4: MongoDB 4.x
- Protocol 5: MongoDB 5.x
- Protocol 6: MongoDB 6.x
- Protocol 7: MongoDB 7.x
- Protocol 8: MongoDB 8.x
### 2. Binary Switching
Based on the detected version, the system switches to the appropriate MongoDB server binary:
```bash
# MongoDB 3.x
export PATH="/snap/wekan/current/migratemongo/bin:$PATH"
export LD_LIBRARY_PATH="/snap/wekan/current/migratemongo/lib:$LD_LIBRARY_PATH"
# MongoDB 7.x (default)
export PATH="/snap/wekan/current/bin:$PATH"
export LD_LIBRARY_PATH="/snap/wekan/current/usr/lib:$LD_LIBRARY_PATH"
```
### 3. Driver Selection
The Node.js application automatically selects the appropriate MongoDB driver based on the detected version using the `mongodbDriverManager` system.
## Files Modified
### Removed Migration Files
- ❌ `snap-src/bin/mongodb-migrate` - Removed
- ❌ `snap-src/bin/mongodb-migration-web` - Removed
- ❌ `snap-src/bin/mongodb-migration-status` - Removed
### Updated Files
- ✅ `snap-src/bin/mongodb-control` - Completely rewritten with version detection
- ✅ `snap-src/bin/mongodb-version-manager` - New utility for managing versions
### Node.js Driver System
- ✅ `models/lib/mongodbDriverManager.js` - Manages multiple MongoDB drivers
- ✅ `models/lib/mongodbConnectionManager.js` - Handles connections with version detection
- ✅ `models/lib/meteorMongoIntegration.js` - Integrates with Meteor's Mongo.Collection
- ✅ `server/mongodb-driver-startup.js` - Initializes the driver system
## Adding Missing MongoDB Server Binaries
To add MongoDB versions 4, 5, 6, and 8 to the snap package, you need to:
### 1. Download MongoDB Server Binaries
Download the appropriate MongoDB server binaries for each version:
```bash
# MongoDB 4.4.28
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.4.28.tgz
# MongoDB 5.0.28
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.28.tgz
# MongoDB 6.0.15
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-6.0.15.tgz
# MongoDB 8.0.4
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-8.0.4.tgz
```
### 2. Extract and Install in Snap Package
Extract each version to the appropriate directory in the snap package:
```bash
# Extract MongoDB 4.x
tar -xzf mongodb-linux-x86_64-4.4.28.tgz
mkdir -p /snap/wekan/current/mongodb4/bin
cp mongodb-linux-x86_64-4.4.28/bin/* /snap/wekan/current/mongodb4/bin/
# Extract MongoDB 5.x
tar -xzf mongodb-linux-x86_64-5.0.28.tgz
mkdir -p /snap/wekan/current/mongodb5/bin
cp mongodb-linux-x86_64-5.0.28/bin/* /snap/wekan/current/mongodb5/bin/
# Extract MongoDB 6.x
tar -xzf mongodb-linux-x86_64-6.0.15.tgz
mkdir -p /snap/wekan/current/mongodb6/bin
cp mongodb-linux-x86_64-6.0.15/bin/* /snap/wekan/current/mongodb6/bin/
# Extract MongoDB 8.x
tar -xzf mongodb-linux-x86_64-8.0.4.tgz
mkdir -p /snap/wekan/current/mongodb8/bin
cp mongodb-linux-x86_64-8.0.4/bin/* /snap/wekan/current/mongodb8/bin/
```
### 3. Update Snap Package Build
Modify the snap package build process to include these binaries in the final package.
## Usage
### Check Available Versions
```bash
$SNAP/bin/mongodb-version-manager versions
```
### Check Current Active Version
```bash
$SNAP/bin/mongodb-version-manager active
```
### Force Version Detection
```bash
$SNAP/bin/mongodb-version-manager detect
```
### View Detection Log
```bash
$SNAP/bin/mongodb-version-manager log
```
## Configuration
### Environment Variables
- `MONGO_URL` - MongoDB connection URL (default: `mongodb://127.0.0.1:27017/wekan`)
- `SNAP_COMMON` - Snap common directory for logs and version files
### Version Files
- `${SNAP_COMMON}/mongodb-active-version` - Caches the currently active version
- `${SNAP_COMMON}/mongodb-version-detection.log` - Logs version detection events
## Benefits
1. **No Downtime**: Automatic version detection and switching without manual intervention
2. **Backward Compatibility**: Works with existing MongoDB 3.x databases
3. **Forward Compatibility**: Ready for MongoDB 8.x when available
4. **Automatic Fallback**: Falls back to MongoDB 7.x if detection fails
5. **Comprehensive Logging**: Detailed logs for troubleshooting
6. **No Migrations**: Eliminates the need for database migrations
## Troubleshooting
### Version Detection Not Working
1. Check the detection log: `$SNAP/bin/mongodb-version-manager log`
2. Force re-detection: `$SNAP/bin/mongodb-version-manager detect`
3. Restart MongoDB service
### Wrong Version Selected
1. Clear the cached version: `rm ${SNAP_COMMON}/mongodb-active-version`
2. Restart MongoDB service
### Connection Issues
1. Verify MongoDB server is running
2. Check MONGO_URL environment variable
3. Review MongoDB server logs
## Future Enhancements
1. **Add Missing Binaries**: Include MongoDB 4, 5, 6, and 8 server binaries in snap package
2. **Health Checks**: Add health monitoring for different MongoDB versions
3. **Performance Metrics**: Track performance across different versions
4. **Auto-Upgrade**: Automatically upgrade to newer versions when available

476
snap-src/bin/mongodb-control
View file

@ -1,8 +1,8 @@
#!/bin/bash
# MongoDB Control Script
# IMPORTANT: Migration is only triggered when MIGRATE_MONGODB=true environment variable is set
# IMPORTANT: No snap settings or channel changes are made during migration
# Handles MongoDB server startup with automatic version detection and switching
# Supports MongoDB versions 3-8 with automatic binary selection
# get wekan/mongo settings
source $SNAP/bin/wekan-read-settings
@ -13,117 +13,177 @@ if [ "true" == "${DISABLE_MONGODB}" ]; then
exit 0
fi
# Migration checking and handling - controlled by MIGRATE_MONGODB environment variable
MIGRATION_STATUS="${SNAP_COMMON}/mongodb-migration-status.json"
MIGRATION_LOG="${SNAP_COMMON}/mongodb-migration-log.txt"
REVERT_FILE="${SNAP_COMMON}/revert-mongodb-migration.txt"
# MongoDB Version Detection and Auto-Switching System
# Detects MongoDB server version from connection attempts and switches to correct binary
# Check if migration is needed
check_migration_needed() {
if [ -f "$MIGRATION_STATUS" ]; then
local status=$(jq -r '.status' "$MIGRATION_STATUS" 2>/dev/null || echo "unknown")
if [ "$status" = "completed" ]; then
return 1
elif [ "$status" = "running" ]; then
return 0
fi
fi
# MongoDB binary paths for different versions
# Note: Currently only versions 3 and 7 are available in the snap package
# Versions 4, 5, 6, 8 would need to be added to the snap package
MONGO3_BIN="/snap/${SNAP_NAME}/current/migratemongo/bin"
MONGO7_BIN="/snap/${SNAP_NAME}/current/bin"
MONGO3_LIB="/snap/${SNAP_NAME}/current/migratemongo/lib"
MONGO7_LIB="/snap/${SNAP_NAME}/current/usr/lib"
# Check if we have MongoDB 3 data
if [ -d "${SNAP_COMMON}/wekan" ] && [ ! -f "${SNAP_COMMON}/mongodb-version-7" ]; then
return 0
fi
# Future paths for additional versions (when added to snap package)
MONGO4_BIN="/snap/${SNAP_NAME}/current/mongodb4/bin"
MONGO5_BIN="/snap/${SNAP_NAME}/current/mongodb5/bin"
MONGO6_BIN="/snap/${SNAP_NAME}/current/mongodb6/bin"
MONGO8_BIN="/snap/${SNAP_NAME}/current/mongodb8/bin"
return 1
# Version detection log
VERSION_DETECTION_LOG="${SNAP_COMMON}/mongodb-version-detection.log"
# Log version detection events
log_version_detection() {
echo "$(date): $1" >> "$VERSION_DETECTION_LOG"
}
# Handle migration - only if MIGRATE_MONGODB=true
handle_migration() {
echo "MongoDB migration needed, starting migration process..."
# Detect MongoDB server version by attempting connection
detect_mongodb_version() {
local mongo_url="${MONGO_URL:-mongodb://127.0.0.1:27017/wekan}"
local detected_version=""
# Stop Wekan (meteor) process before migration
echo "Stopping Wekan (meteor) process for migration..."
snapctl stop --disable ${SNAP_NAME}.wekan || true
snapctl stop --disable ${SNAP_NAME} || true
log_version_detection "Starting MongoDB version detection for: $mongo_url"
# Wait a moment for processes to stop
sleep 2
# Kill any remaining meteor/node processes
pkill -f "node.*main.js" || true
pkill -f "meteor" || true
sleep 1
# Start migration web interface in background
$SNAP/bin/mongodb-migration-web &
local web_pid=$!
echo "$web_pid" > "${SNAP_COMMON}/migration-web.pid"
# Run migration
if $SNAP/bin/mongodb-migrate; then
echo "MongoDB migration completed successfully"
# Kill migration web interface
if [ -f "${SNAP_COMMON}/migration-web.pid" ]; then
local web_pid=$(cat "${SNAP_COMMON}/migration-web.pid")
kill "$web_pid" 2>/dev/null || true
rm -f "${SNAP_COMMON}/migration-web.pid"
fi
# Clean up temporary Node.js server file
rm -f "${SNAP_COMMON}/migration-web-server.js"
echo "Migration completed. Wekan will be restarted automatically."
# Try to connect with MongoDB 7 first (latest available)
log_version_detection "Attempting connection with MongoDB 7 binary"
if timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>/dev/null | grep -q "7\."; then
detected_version="7"
log_version_detection "Detected MongoDB 7.x server"
elif timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>&1 | grep -q "protocol version\|wire protocol"; then
# Check for wire protocol errors that indicate older version
local error_output=$(timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>&1)
if echo "$error_output" | grep -q "protocol version 0\|wire protocol version 0"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 0)"
elif echo "$error_output" | grep -q "protocol version 1\|wire protocol version 1"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 1)"
elif echo "$error_output" | grep -q "protocol version 2\|wire protocol version 2"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 2)"
elif echo "$error_output" | grep -q "protocol version 3\|wire protocol version 3"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 3)"
elif echo "$error_output" | grep -q "protocol version 4\|wire protocol version 4"; then
detected_version="4"
log_version_detection "Detected MongoDB 4.x server (wire protocol 4)"
elif echo "$error_output" | grep -q "protocol version 5\|wire protocol version 5"; then
detected_version="5"
log_version_detection "Detected MongoDB 5.x server (wire protocol 5)"
elif echo "$error_output" | grep -q "protocol version 6\|wire protocol version 6"; then
detected_version="6"
log_version_detection "Detected MongoDB 6.x server (wire protocol 6)"
elif echo "$error_output" | grep -q "protocol version 7\|wire protocol version 7"; then
detected_version="7"
log_version_detection "Detected MongoDB 7.x server (wire protocol 7)"
elif echo "$error_output" | grep -q "protocol version 8\|wire protocol version 8"; then
detected_version="8"
log_version_detection "Detected MongoDB 8.x server (wire protocol 8)"
else
echo "MongoDB migration failed"
# Kill migration web interface
if [ -f "${SNAP_COMMON}/migration-web.pid" ]; then
local web_pid=$(cat "${SNAP_COMMON}/migration-web.pid")
kill "$web_pid" 2>/dev/null || true
rm -f "${SNAP_COMMON}/migration-web.pid"
log_version_detection "Unknown wire protocol error: $error_output"
fi
else
log_version_detection "No MongoDB server running or connection failed"
fi
# Clean up temporary Node.js server file
rm -f "${SNAP_COMMON}/migration-web-server.js"
echo "$detected_version"
}
exit 1
# Switch to appropriate MongoDB binary based on detected version
switch_mongodb_binary() {
local version="$1"
case "$version" in
"3")
if [ -f "/snap/${SNAP_NAME}/current/migratemongo/bin/mongod" ]; then
log_version_detection "Switching to MongoDB 3.x binary"
export PATH="${MONGO3_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO3_LIB}:${MONGO3_LIB}/x86_64-linux-gnu:${LD_LIBRARY_PATH}"
echo "3" > "${SNAP_COMMON}/mongodb-active-version"
else
log_version_detection "MongoDB 3.x binary not found, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
;;
"4")
if [ -f "/snap/${SNAP_NAME}/current/mongodb4/bin/mongod" ]; then
log_version_detection "Switching to MongoDB 4.x binary"
export PATH="${MONGO4_BIN}:${PATH}"
echo "4" > "${SNAP_COMMON}/mongodb-active-version"
else
log_version_detection "MongoDB 4.x binary not found, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
;;
"5")
if [ -f "/snap/${SNAP_NAME}/current/mongodb5/bin/mongod" ]; then
log_version_detection "Switching to MongoDB 5.x binary"
export PATH="${MONGO5_BIN}:${PATH}"
echo "5" > "${SNAP_COMMON}/mongodb-active-version"
else
log_version_detection "MongoDB 5.x binary not found, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
;;
"6")
if [ -f "/snap/${SNAP_NAME}/current/mongodb6/bin/mongod" ]; then
log_version_detection "Switching to MongoDB 6.x binary"
export PATH="${MONGO6_BIN}:${PATH}"
echo "6" > "${SNAP_COMMON}/mongodb-active-version"
else
log_version_detection "MongoDB 6.x binary not found, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
;;
"7"|"")
log_version_detection "Using MongoDB 7.x binary (default)"
export PATH="${MONGO7_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO7_LIB}:${LD_LIBRARY_PATH}"
echo "7" > "${SNAP_COMMON}/mongodb-active-version"
;;
"8")
if [ -f "/snap/${SNAP_NAME}/current/mongodb8/bin/mongod" ]; then
log_version_detection "Switching to MongoDB 8.x binary"
export PATH="${MONGO8_BIN}:${PATH}"
echo "8" > "${SNAP_COMMON}/mongodb-active-version"
else
log_version_detection "MongoDB 8.x binary not found, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
;;
*)
log_version_detection "Unknown version $version, using default MongoDB 7.x"
export PATH="${MONGO7_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO7_LIB}:${LD_LIBRARY_PATH}"
echo "7" > "${SNAP_COMMON}/mongodb-active-version"
;;
esac
}
# Main version detection and switching logic
setup_mongodb_version() {
# Check if we have a cached version
if [ -f "${SNAP_COMMON}/mongodb-active-version" ]; then
local cached_version=$(cat "${SNAP_COMMON}/mongodb-active-version")
log_version_detection "Using cached MongoDB version: $cached_version"
switch_mongodb_binary "$cached_version"
return
fi
# Detect version and switch
local detected_version=$(detect_mongodb_version)
if [ -n "$detected_version" ]; then
switch_mongodb_binary "$detected_version"
else
# Default to MongoDB 7 if detection fails
log_version_detection "Version detection failed, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
}
# Check if revert is requested
if [ -f "$REVERT_FILE" ]; then
echo "Revert requested, stopping Wekan and MongoDB, then reverting migration..."
# Stop Wekan (meteor) process before revert
echo "Stopping Wekan (meteor) process for revert..."
snapctl stop --disable ${SNAP_NAME}.wekan || true
snapctl stop --disable ${SNAP_NAME} || true
# Wait a moment for processes to stop
sleep 2
# Kill any remaining meteor/node processes
pkill -f "node.*main.js" || true
pkill -f "meteor" || true
sleep 1
# Stop MongoDB
snapctl stop --disable ${SNAP_NAME}.mongodb
# Run migration (which will handle revert)
$SNAP/bin/mongodb-migrate
exit $?
fi
# Check if migration is needed - only if MIGRATE_MONGODB=true
if [ "$MIGRATE_MONGODB" = "true" ]; then
if check_migration_needed; then
handle_migration
else
echo "MIGRATE_MONGODB=true but no migration needed"
fi
else
echo "MIGRATE_MONGODB not set to 'true' - skipping migration check"
fi
# Run version detection and setup
setup_mongodb_version
# make sure we have set minimum env variables for locale
if [ -z "${LANG}" ]; then
@ -143,145 +203,93 @@ if [ -f ${SNAP_COMMON}/settings.log ]; then
rm ${SNAP_COMMON}/settings.log
fi
#if test -f "$SNAP_COMMON/01-migrate-mongo3-to-mongo5.txt"; then
# touch "$SNAP_COMMON/01-migrate-mongo3-to-mongo5.txt"
# # Stop MongoDB
# touch "$SNAP_COMMON/02-disable-mongo.txt"
# snapctl stop --disable ${SNAP_NAME}.mongodb
# touch "$SNAP_COMMON/03-eval-stop-mongo.txt"
# mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# # Start MongoDB 4.4
# touch "$SNAP_COMMON/04-start-mongo44.txt"
# $SNAP/mongo44bin/mongod --dbpath $SNAP_COMMON --logpath $SNAP_COMMON/02_mongodb_log_while_migrate.txt --logappend --journal $MONGO_URL --quiet
# # Wait MongoDB 4.4 to start
# touch "$SNAP_COMMON/05-wait-2s-mongo44-start.txt"
# sleep 2s
# # Dump Old MongoDB 3.x database
# touch "$SNAP_COMMON/06-dump-database.txt"
# (cd $SNAP_COMMON && mongodump --port ${MONGODB_PORT})
# # Stop MongoDB 4.4
# touch "$SNAP_COMMON/07-stop-mongo44.txt"
# $SNAP/mongo44bin/mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# # Wait MongoDB 4.4 to stop
# touch "$SNAP_COMMON/08-wait-2s-mongo44-stop.txt"
# sleep 2s
# # Start MongoDB 5
# touch "$SNAP_COMMON/09-start-mongo5.txt"
# mongod --dbpath $SNAP_COMMON --logpath $SNAP_COMMON/10_mongodb_log_while_migrate.txt --logappend --journal $MONGO_URL --quiet
# # Restore database
# touch "$SNAP_COMMON/11-mongorestore-to-mongo5.txt"
# (cd $SNAP_COMMON && mongorestore --port ${MONGODB_PORT})
# # Wait 5s
# touch "$SNAP_COMMON/12-wait-5s-after-restore.txt"
# sleep 5s
# # Shutdown mongodb
# touch "$SNAP_COMMON/13-shutdown-mongodb.txt"
# mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# touch "$SNAP_COMMON/14-wait-5s-after-mongo5-shutdown.txt"
# sleep 5s
# # Enable MongoDB 5
# touch "$SNAP_COMMON/15-enable-mongo-5.txt"
# snapctl start --enable ${SNAP_NAME}.mongodb
#fi
# When starting MongoDB, if logfile exist, delete it, because now uses syslog instead of logfile,
# because syslog usually already has log rotation.
# https://github.com/wekan/wekan-snap/issues/92
#if test -f "$SNAP_COMMON/mongodb.log"; then
# rm -f "$SNAP_COMMON/mongodb.log"
#fi
# Set MongoDB log destination to snapcommon for log file detection
export MONGO_LOG_DESTINATION="snapcommon"
# Alternative: When starting MongoDB, and using logfile, truncate log to last 1000 lines of text.
# 1) If file exists:
#if test -f "$SNAP_COMMON/mongodb.log"; then
# # 2) Copy last 1000 lines to variable loglast1000lines.
# loglast1000lines=$(tail -1000 "$SNAP_COMMON/mongodb.log")
# # 3) Copy variable to replace original MongoDB log.
# echo "$loglast1000lines" > "$SNAP_COMMON/mongodb.log"
# # 4) Set variable to be empty.
# loglast1000lines=""
#fi
# Set MongoDB data directory
export MONGO_DATA_DIR="${SNAP_COMMON}/wekan"
if [ -z "${MONGO_URL}" ]; then
# start mongo deamon
BIND_OPTIONS=""
if [ "nill" != "${MONGODB_BIND_UNIX_SOCKET}" ] && [ "x" != "x${MONGODB_BIND_UNIX_SOCKET}" ]; then
BIND_OPTIONS+=" --unixSocketPrefix ${MONGODB_BIND_UNIX_SOCKET}"
fi
# Newest MongoDB uses --host or --bind_ip
if [ "x" != "x${MONGODB_BIND_IP}" ]; then
BIND_OPTIONS+=" --bind_ip $MONGODB_BIND_IP"
fi
if [ "x" != "x${MONGODB_PORT}" ]; then
BIND_OPTIONS+=" --port ${MONGODB_PORT}"
# Create MongoDB data directory if it doesn't exist
if [ ! -d "$MONGO_DATA_DIR" ]; then
mkdir -p "$MONGO_DATA_DIR"
chmod 755 "$MONGO_DATA_DIR"
fi
if [ "syslog" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to syslog"
mongod --dbpath ${SNAP_COMMON} --syslog ${BIND_OPTIONS} --quiet
exit 0
fi
# Set MongoDB log file path
export MONGO_LOG_FILE="${SNAP_COMMON}/mongodb.log"
if [ "snapcommon" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to $SNAP_COMMON"
mongod --dbpath ${SNAP_COMMON} --logpath ${SNAP_COMMON}/mongodb.log --logappend ${BIND_OPTIONS} --quiet
fi
# Start MongoDB with appropriate version
echo "Starting MongoDB with detected version..."
log_version_detection "Starting MongoDB server"
if [ "devnull" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to /dev/null"
mongod --dbpath ${SNAP_COMMON} --logpath /dev/null ${BIND_OPTIONS} --quiet
fi
#echo "mongodb log destination: ${MONGO_LOG_DESTINATION}" >> "${SNAP_COMMON}/settings.log"
# Get the active version
ACTIVE_VERSION=$(cat "${SNAP_COMMON}/mongodb-active-version" 2>/dev/null || echo "7")
# Disable MongoDB telemetry and free monitoring
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'disableTelemetry();' --port ${MONGODB_PORT}
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.disableFreeMonitoring();' --port ${MONGODB_PORT}
# Snap: Disable apparmor="DENIED" at syslog
# https://github.com/wekan/wekan/issues/4855
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.adminCommand({ setParameter: 1, diagnosticDataCollectionEnabled: false});' --port ${MONGODB_PORT}
# Drop indexes on database upgrade, when starting MongoDB
#mongosh wekan --eval "db.getCollectionNames().forEach(function(col_name) { var coll = db.getCollection(col_name); coll.dropIndexes(); });" $BIND_OPTIONS
# Set MongoDB feature compatibility version
#mongosh wekan --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "4.4" });' ${BIND_OPTIONS}
# Delete incomplete uploads so that they would not prevent starting WeKan
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.getCollection("cfs.attachments.filerecord").find( { "uploadedAt": { "$exists": true }, "copies.attachments" : null,"failures.copies.attachments.doneTrying" : {"$ne" : true}});' --port ${MONGODB_PORT}
else
if [ "syslog" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to syslog"
mongod --dbpath ${SNAP_COMMON} --syslog ${MONGO_URL} --quiet
fi
if [ "snapcommon" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to ${SNAP_COMMON}"
mongod --dbpath ${SNAP_COMMON} --logpath ${SNAP_COMMON}/mongodb.log --logappend ${MONGO_URL} --quiet
fi
if [ "devnull" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to /dev/null"
mongod --dbpath ${SNAP_COMMON} --logpath /dev/null ${MONGO_URL} --quiet
fi
# Disable MongoDB telemetry and free monitoring
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'disableTelemetry();'
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.disableFreeMonitoring();'
# Snap: Disable apparmor="DENIED" at syslog
# https://github.com/wekan/wekan/issues/4855
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.adminCommand({ setParameter: 1, diagnosticDataCollectionEnabled: false});'
# Drop indexes on database upgrade, when starting MongoDB
#mongosh wekan --eval "db.getCollectionNames().forEach(function(col_name) { var coll = db.getCollection(col_name); coll.dropIndexes(); });" $BIND_OPTIONS
# Set MongoDB feature compatibility version
#/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "4.4" });'
# Delete incomplete uploads so that they would not prevent starting WeKan
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.getCollection("cfs.attachments.filerecord").find( { "uploadedAt": { "$exists": true }, "copies.attachments" : null,"failures.copies.attachments.doneTrying" : {"$ne" : true}});'
fi
case "$ACTIVE_VERSION" in
"3")
echo "Starting MongoDB 3.x server..."
log_version_detection "Starting MongoDB 3.x server"
exec /snap/${SNAP_NAME}/current/migratemongo/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
"4")
echo "Starting MongoDB 4.x server..."
log_version_detection "Starting MongoDB 4.x server"
exec /snap/${SNAP_NAME}/current/mongodb4/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
"5")
echo "Starting MongoDB 5.x server..."
log_version_detection "Starting MongoDB 5.x server"
exec /snap/${SNAP_NAME}/current/mongodb5/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
"6")
echo "Starting MongoDB 6.x server..."
log_version_detection "Starting MongoDB 6.x server"
exec /snap/${SNAP_NAME}/current/mongodb6/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
"7"|*)
echo "Starting MongoDB 7.x server..."
log_version_detection "Starting MongoDB 7.x server"
exec /snap/${SNAP_NAME}/current/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
"8")
echo "Starting MongoDB 8.x server..."
log_version_detection "Starting MongoDB 8.x server"
exec /snap/${SNAP_NAME}/current/mongodb8/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
;;
esac

179
snap-src/bin/mongodb-control-new Normal file
View file

@ -0,0 +1,179 @@
#!/bin/bash
# MongoDB Control Script
# Handles MongoDB server startup with automatic version detection and switching
# get wekan/mongo settings
source $SNAP/bin/wekan-read-settings
if [ "true" == "${DISABLE_MONGODB}" ]; then
echo "mongodb is disabled. Stop service"
snapctl stop --disable ${SNAP_NAME}.mongodb
exit 0
fi
# MongoDB Version Detection and Auto-Switching System
# Detects MongoDB server version from connection attempts and switches to correct binary
# MongoDB binary paths for different versions
MONGO3_BIN="/snap/${SNAP_NAME}/current/migratemongo/bin"
MONGO7_BIN="/snap/${SNAP_NAME}/current/bin"
MONGO3_LIB="/snap/${SNAP_NAME}/current/migratemongo/lib"
MONGO7_LIB="/snap/${SNAP_NAME}/current/usr/lib"
# Version detection log
VERSION_DETECTION_LOG="${SNAP_COMMON}/mongodb-version-detection.log"
# Log version detection events
log_version_detection() {
echo "$(date): $1" >> "$VERSION_DETECTION_LOG"
}
# Detect MongoDB server version by attempting connection
detect_mongodb_version() {
local mongo_url="${MONGO_URL:-mongodb://127.0.0.1:27017/wekan}"
local detected_version=""
log_version_detection "Starting MongoDB version detection for: $mongo_url"
# Try to connect with MongoDB 7 first (latest)
log_version_detection "Attempting connection with MongoDB 7 binary"
if timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>/dev/null | grep -q "7\."; then
detected_version="7"
log_version_detection "Detected MongoDB 7.x server"
elif timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>&1 | grep -q "protocol version\|wire protocol"; then
# Check for wire protocol errors that indicate older version
local error_output=$(timeout 10s /snap/${SNAP_NAME}/current/bin/mongosh --quiet --eval "db.runCommand({buildInfo: 1}).version" "$mongo_url" 2>&1)
if echo "$error_output" | grep -q "protocol version 0\|wire protocol version 0"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 0)"
elif echo "$error_output" | grep -q "protocol version 1\|wire protocol version 1"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 1)"
elif echo "$error_output" | grep -q "protocol version 2\|wire protocol version 2"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 2)"
elif echo "$error_output" | grep -q "protocol version 3\|wire protocol version 3"; then
detected_version="3"
log_version_detection "Detected MongoDB 3.x server (wire protocol 3)"
else
log_version_detection "Unknown wire protocol error: $error_output"
fi
else
log_version_detection "No MongoDB server running or connection failed"
fi
echo "$detected_version"
}
# Switch to appropriate MongoDB binary based on detected version
switch_mongodb_binary() {
local version="$1"
case "$version" in
"3")
log_version_detection "Switching to MongoDB 3.x binary"
export PATH="${MONGO3_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO3_LIB}:${MONGO3_LIB}/x86_64-linux-gnu:${LD_LIBRARY_PATH}"
echo "3" > "${SNAP_COMMON}/mongodb-active-version"
;;
"7"|"")
log_version_detection "Using MongoDB 7.x binary (default)"
export PATH="${MONGO7_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO7_LIB}:${LD_LIBRARY_PATH}"
echo "7" > "${SNAP_COMMON}/mongodb-active-version"
;;
*)
log_version_detection "Unknown version $version, using default MongoDB 7.x"
export PATH="${MONGO7_BIN}:${PATH}"
export LD_LIBRARY_PATH="${MONGO7_LIB}:${LD_LIBRARY_PATH}"
echo "7" > "${SNAP_COMMON}/mongodb-active-version"
;;
esac
}
# Main version detection and switching logic
setup_mongodb_version() {
# Check if we have a cached version
if [ -f "${SNAP_COMMON}/mongodb-active-version" ]; then
local cached_version=$(cat "${SNAP_COMMON}/mongodb-active-version")
log_version_detection "Using cached MongoDB version: $cached_version"
switch_mongodb_binary "$cached_version"
return
fi
# Detect version and switch
local detected_version=$(detect_mongodb_version)
if [ -n "$detected_version" ]; then
switch_mongodb_binary "$detected_version"
else
# Default to MongoDB 7 if detection fails
log_version_detection "Version detection failed, using default MongoDB 7.x"
switch_mongodb_binary "7"
fi
}
# Run version detection and setup
setup_mongodb_version
# make sure we have set minimum env variables for locale
if [ -z "${LANG}" ]; then
export LANG=en_US.UTF-8
fi
export LC_ALL=C
# If CPU does not support AVX, use Qemu that supports AVX.
# Migratemongo is at https://github.com/wekan/migratemongo
# and at directory /snap/${SNAP_NAME}/current/migratemongo/avx
# is bash scripts like mongod, mongosh check avx support and use Qemu if needed.
export PATH=/snap/${SNAP_NAME}/current/migratemongo/avx:/snap/${SNAP_NAME}/current/usr/bin:/snap/${SNAP_NAME}/current/bin:${PATH}
export LD_LIBRARY_PATH=/snap/${SNAP_NAME}/current/lib:/snap/${SNAP_NAME}/current/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH}
# If temporary settings log exists, delete it
if [ -f ${SNAP_COMMON}/settings.log ]; then
rm ${SNAP_COMMON}/settings.log
fi
# Set MongoDB log destination to snapcommon for log file detection
export MONGO_LOG_DESTINATION="snapcommon"
# Set MongoDB data directory
export MONGO_DATA_DIR="${SNAP_COMMON}/wekan"
# Create MongoDB data directory if it doesn't exist
if [ ! -d "$MONGO_DATA_DIR" ]; then
mkdir -p "$MONGO_DATA_DIR"
chmod 755 "$MONGO_DATA_DIR"
fi
# Set MongoDB log file path
export MONGO_LOG_FILE="${SNAP_COMMON}/mongodb.log"
# Start MongoDB with appropriate version
echo "Starting MongoDB with detected version..."
log_version_detection "Starting MongoDB server"
# Get the active version
ACTIVE_VERSION=$(cat "${SNAP_COMMON}/mongodb-active-version" 2>/dev/null || echo "7")
if [ "$ACTIVE_VERSION" = "3" ]; then
echo "Starting MongoDB 3.x server..."
log_version_detection "Starting MongoDB 3.x server"
exec /snap/${SNAP_NAME}/current/migratemongo/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
else
echo "Starting MongoDB 7.x server..."
log_version_detection "Starting MongoDB 7.x server"
exec /snap/${SNAP_NAME}/current/bin/mongod \
--dbpath="$MONGO_DATA_DIR" \
--logpath="$MONGO_LOG_FILE" \
--logappend \
--bind_ip=127.0.0.1 \
--port=27017 \
--fork
fi

281
snap-src/bin/mongodb-control-old Executable file
View file

@ -0,0 +1,281 @@
#!/bin/bash
# MongoDB Control Script
# Handles MongoDB server startup with automatic version detection and switching
# get wekan/mongo settings
source $SNAP/bin/wekan-read-settings
if [ "true" == "${DISABLE_MONGODB}" ]; then
echo "mongodb is disabled. Stop service"
snapctl stop --disable ${SNAP_NAME}.mongodb
exit 0
fi
# Check if migration is needed
check_migration_needed() {
if [ -f "$MIGRATION_STATUS" ]; then
local status=$(jq -r '.status' "$MIGRATION_STATUS" 2>/dev/null || echo "unknown")
if [ "$status" = "completed" ]; then
return 1
elif [ "$status" = "running" ]; then
return 0
fi
fi
# Check if we have MongoDB 3 data
if [ -d "${SNAP_COMMON}/wekan" ] && [ ! -f "${SNAP_COMMON}/mongodb-version-7" ]; then
return 0
fi
return 1
}
# Handle migration - only if MIGRATE_MONGODB=true
handle_migration() {
echo "MongoDB migration needed, starting migration process..."
# Stop Wekan (meteor) process before migration
echo "Stopping Wekan (meteor) process for migration..."
snapctl stop --disable ${SNAP_NAME}.wekan || true
snapctl stop --disable ${SNAP_NAME} || true
# Wait a moment for processes to stop
sleep 2
# Kill any remaining meteor/node processes
pkill -f "node.*main.js" || true
pkill -f "meteor" || true
sleep 1
# Start migration web interface in background
$SNAP/bin/mongodb-migration-web &
local web_pid=$!
echo "$web_pid" > "${SNAP_COMMON}/migration-web.pid"
# Run migration
if $SNAP/bin/mongodb-migrate; then
echo "MongoDB migration completed successfully"
# Kill migration web interface
if [ -f "${SNAP_COMMON}/migration-web.pid" ]; then
local web_pid=$(cat "${SNAP_COMMON}/migration-web.pid")
kill "$web_pid" 2>/dev/null || true
rm -f "${SNAP_COMMON}/migration-web.pid"
fi
# Clean up temporary Node.js server file
rm -f "${SNAP_COMMON}/migration-web-server.js"
echo "Migration completed. Wekan will be restarted automatically."
else
echo "MongoDB migration failed"
# Kill migration web interface
if [ -f "${SNAP_COMMON}/migration-web.pid" ]; then
local web_pid=$(cat "${SNAP_COMMON}/migration-web.pid")
kill "$web_pid" 2>/dev/null || true
rm -f "${SNAP_COMMON}/migration-web.pid"
fi
# Clean up temporary Node.js server file
rm -f "${SNAP_COMMON}/migration-web-server.js"
exit 1
fi
}
# Check if revert is requested
if [ -f "$REVERT_FILE" ]; then
echo "Revert requested, stopping Wekan and MongoDB, then reverting migration..."
# Stop Wekan (meteor) process before revert
echo "Stopping Wekan (meteor) process for revert..."
snapctl stop --disable ${SNAP_NAME}.wekan || true
snapctl stop --disable ${SNAP_NAME} || true
# Wait a moment for processes to stop
sleep 2
# Kill any remaining meteor/node processes
pkill -f "node.*main.js" || true
pkill -f "meteor" || true
sleep 1
# Stop MongoDB
snapctl stop --disable ${SNAP_NAME}.mongodb
# Run migration (which will handle revert)
$SNAP/bin/mongodb-migrate
exit $?
fi
# Check if migration is needed - only if MIGRATE_MONGODB=true
if [ "$MIGRATE_MONGODB" = "true" ]; then
if check_migration_needed; then
handle_migration
else
echo "MIGRATE_MONGODB=true but no migration needed"
fi
else
echo "MIGRATE_MONGODB not set to 'true' - skipping migration check"
fi
# make sure we have set minimum env variables for locale
if [ -z "${LANG}" ]; then
export LANG=en_US.UTF-8
fi
export LC_ALL=C
# If CPU does not support AVX, use Qemu that supports AVX.
# Migratemongo is at https://github.com/wekan/migratemongo
# and at directory /snap/${SNAP_NAME}/current/migratemongo/avx
# is bash scripts like mongod, mongosh check avx support and use Qemu if needed.
export PATH=/snap/${SNAP_NAME}/current/migratemongo/avx:/snap/${SNAP_NAME}/current/usr/bin:/snap/${SNAP_NAME}/current/bin:${PATH}
export LD_LIBRARY_PATH=/snap/${SNAP_NAME}/current/lib:/snap/${SNAP_NAME}/current/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH}
# If temporary settings log exists, delete it
if [ -f ${SNAP_COMMON}/settings.log ]; then
rm ${SNAP_COMMON}/settings.log
fi
#if test -f "$SNAP_COMMON/01-migrate-mongo3-to-mongo5.txt"; then
# touch "$SNAP_COMMON/01-migrate-mongo3-to-mongo5.txt"
# # Stop MongoDB
# touch "$SNAP_COMMON/02-disable-mongo.txt"
# snapctl stop --disable ${SNAP_NAME}.mongodb
# touch "$SNAP_COMMON/03-eval-stop-mongo.txt"
# mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# # Start MongoDB 4.4
# touch "$SNAP_COMMON/04-start-mongo44.txt"
# $SNAP/mongo44bin/mongod --dbpath $SNAP_COMMON --logpath $SNAP_COMMON/02_mongodb_log_while_migrate.txt --logappend --journal $MONGO_URL --quiet
# # Wait MongoDB 4.4 to start
# touch "$SNAP_COMMON/05-wait-2s-mongo44-start.txt"
# sleep 2s
# # Dump Old MongoDB 3.x database
# touch "$SNAP_COMMON/06-dump-database.txt"
# (cd $SNAP_COMMON && mongodump --port ${MONGODB_PORT})
# # Stop MongoDB 4.4
# touch "$SNAP_COMMON/07-stop-mongo44.txt"
# $SNAP/mongo44bin/mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# # Wait MongoDB 4.4 to stop
# touch "$SNAP_COMMON/08-wait-2s-mongo44-stop.txt"
# sleep 2s
# # Start MongoDB 5
# touch "$SNAP_COMMON/09-start-mongo5.txt"
# mongod --dbpath $SNAP_COMMON --logpath $SNAP_COMMON/10_mongodb_log_while_migrate.txt --logappend --journal $MONGO_URL --quiet
# # Restore database
# touch "$SNAP_COMMON/11-mongorestore-to-mongo5.txt"
# (cd $SNAP_COMMON && mongorestore --port ${MONGODB_PORT})
# # Wait 5s
# touch "$SNAP_COMMON/12-wait-5s-after-restore.txt"
# sleep 5s
# # Shutdown mongodb
# touch "$SNAP_COMMON/13-shutdown-mongodb.txt"
# mongo wekan --eval "db.getSiblingDB('admin').shutdownServer()" $BIND_OPTIONS
# touch "$SNAP_COMMON/14-wait-5s-after-mongo5-shutdown.txt"
# sleep 5s
# # Enable MongoDB 5
# touch "$SNAP_COMMON/15-enable-mongo-5.txt"
# snapctl start --enable ${SNAP_NAME}.mongodb
#fi
# When starting MongoDB, if logfile exist, delete it, because now uses syslog instead of logfile,
# because syslog usually already has log rotation.
# https://github.com/wekan/wekan-snap/issues/92
#if test -f "$SNAP_COMMON/mongodb.log"; then
# rm -f "$SNAP_COMMON/mongodb.log"
#fi
# Alternative: When starting MongoDB, and using logfile, truncate log to last 1000 lines of text.
# 1) If file exists:
#if test -f "$SNAP_COMMON/mongodb.log"; then
# # 2) Copy last 1000 lines to variable loglast1000lines.
# loglast1000lines=$(tail -1000 "$SNAP_COMMON/mongodb.log")
# # 3) Copy variable to replace original MongoDB log.
# echo "$loglast1000lines" > "$SNAP_COMMON/mongodb.log"
# # 4) Set variable to be empty.
# loglast1000lines=""
#fi
if [ -z "${MONGO_URL}" ]; then
# start mongo deamon
BIND_OPTIONS=""
if [ "nill" != "${MONGODB_BIND_UNIX_SOCKET}" ] && [ "x" != "x${MONGODB_BIND_UNIX_SOCKET}" ]; then
BIND_OPTIONS+=" --unixSocketPrefix ${MONGODB_BIND_UNIX_SOCKET}"
fi
# Newest MongoDB uses --host or --bind_ip
if [ "x" != "x${MONGODB_BIND_IP}" ]; then
BIND_OPTIONS+=" --bind_ip $MONGODB_BIND_IP"
fi
if [ "x" != "x${MONGODB_PORT}" ]; then
BIND_OPTIONS+=" --port ${MONGODB_PORT}"
fi
if [ "syslog" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to syslog"
mongod --dbpath ${SNAP_COMMON} --syslog ${BIND_OPTIONS} --quiet
exit 0
fi
if [ "snapcommon" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to $SNAP_COMMON"
mongod --dbpath ${SNAP_COMMON} --logpath ${SNAP_COMMON}/mongodb.log --logappend ${BIND_OPTIONS} --quiet
fi
if [ "devnull" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to /dev/null"
mongod --dbpath ${SNAP_COMMON} --logpath /dev/null ${BIND_OPTIONS} --quiet
fi
#echo "mongodb log destination: ${MONGO_LOG_DESTINATION}" >> "${SNAP_COMMON}/settings.log"
# Disable MongoDB telemetry and free monitoring
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'disableTelemetry();' --port ${MONGODB_PORT}
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.disableFreeMonitoring();' --port ${MONGODB_PORT}
# Snap: Disable apparmor="DENIED" at syslog
# https://github.com/wekan/wekan/issues/4855
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.adminCommand({ setParameter: 1, diagnosticDataCollectionEnabled: false});' --port ${MONGODB_PORT}
# Drop indexes on database upgrade, when starting MongoDB
#mongosh wekan --eval "db.getCollectionNames().forEach(function(col_name) { var coll = db.getCollection(col_name); coll.dropIndexes(); });" $BIND_OPTIONS
# Set MongoDB feature compatibility version
#mongosh wekan --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "4.4" });' ${BIND_OPTIONS}
# Delete incomplete uploads so that they would not prevent starting WeKan
/snap/${SNAP_NAME}/current/bin/mongosh wekan --eval 'db.getCollection("cfs.attachments.filerecord").find( { "uploadedAt": { "$exists": true }, "copies.attachments" : null,"failures.copies.attachments.doneTrying" : {"$ne" : true}});' --port ${MONGODB_PORT}
else
if [ "syslog" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to syslog"
mongod --dbpath ${SNAP_COMMON} --syslog ${MONGO_URL} --quiet
fi
if [ "snapcommon" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to ${SNAP_COMMON}"
mongod --dbpath ${SNAP_COMMON} --logpath ${SNAP_COMMON}/mongodb.log --logappend ${MONGO_URL} --quiet
fi
if [ "devnull" == "${MONGO_LOG_DESTINATION}" ]; then
echo "Sending mongodb logs to /dev/null"
mongod --dbpath ${SNAP_COMMON} --logpath /dev/null ${MONGO_URL} --quiet
fi
# Disable MongoDB telemetry and free monitoring
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'disableTelemetry();'
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.disableFreeMonitoring();'
# Snap: Disable apparmor="DENIED" at syslog
# https://github.com/wekan/wekan/issues/4855
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.adminCommand({ setParameter: 1, diagnosticDataCollectionEnabled: false});'
# Drop indexes on database upgrade, when starting MongoDB
#mongosh wekan --eval "db.getCollectionNames().forEach(function(col_name) { var coll = db.getCollection(col_name); coll.dropIndexes(); });" $BIND_OPTIONS
# Set MongoDB feature compatibility version
#/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "4.4" });'
# Delete incomplete uploads so that they would not prevent starting WeKan
/snap/${SNAP_NAME}/current/bin/mongosh ${MONGO_URL} --eval 'db.getCollection("cfs.attachments.filerecord").find( { "uploadedAt": { "$exists": true }, "copies.attachments" : null,"failures.copies.attachments.doneTrying" : {"$ne" : true}});'
fi

876
snap-src/bin/mongodb-migrate
View file

@ -1,876 +0,0 @@
#!/bin/bash
# MongoDB Migration Script from version 3 to 7
# This script handles migration with disk space checks, progress tracking, and error handling
#
# IMPORTANT: Migration is only triggered when MIGRATE_MONGODB=true environment variable is set
# IMPORTANT: All operations are contained within SNAP_COMMON directory
# IMPORTANT: No snap settings or channel changes are made during migration
# This is the only writable directory in a snap environment
set -e
# Source settings
source $SNAP/bin/wekan-read-settings
# Migration configuration
MIGRATION_LOG="${SNAP_COMMON}/mongodb-migration-log.txt"
MIGRATION_STATUS="${SNAP_COMMON}/mongodb-migration-status.json"
MIGRATION_PROGRESS="${SNAP_COMMON}/mongodb-migration-progress.html"
REVERT_FILE="${SNAP_COMMON}/revert-mongodb-migration.txt"
TEMP_DIR="${SNAP_COMMON}/mongodb-migration-temp"
BACKUP_DIR="${SNAP_COMMON}/mongodb-backup-$(date +%Y%m%d-%H%M%S)"
# MongoDB paths
MONGO3_BIN="/snap/${SNAP_NAME}/current/migratemongo/bin"
MONGO7_BIN="/snap/${SNAP_NAME}/current/bin"
MONGO3_LIB="/snap/${SNAP_NAME}/current/migratemongo/lib"
MONGO7_LIB="/snap/${SNAP_NAME}/current/usr/lib"
# Set up environment for MongoDB 3 tools
export LD_LIBRARY_PATH="${MONGO3_LIB}:${MONGO3_LIB}/x86_64-linux-gnu:${LD_LIBRARY_PATH}"
export PATH="${MONGO3_BIN}:${MONGO7_BIN}:${PATH}"
# Set MongoDB log destination to snapcommon for log file detection
export MONGO_LOG_DESTINATION="snapcommon"
# Validate that all operations are within SNAP_COMMON
validate_snap_common_path() {
local path="$1"
local description="$2"
if [[ "$path" != "${SNAP_COMMON}"* ]]; then
log_error "Path outside SNAP_COMMON detected: $path ($description)"
log_error "SNAP_COMMON: $SNAP_COMMON"
return 1
fi
return 0
}
# Validate all critical paths
validate_all_paths() {
log_message "Validating all paths are within SNAP_COMMON"
validate_snap_common_path "$MIGRATION_LOG" "Migration log" || return 1
validate_snap_common_path "$MIGRATION_STATUS" "Migration status" || return 1
validate_snap_common_path "$MIGRATION_PROGRESS" "Migration progress" || return 1
validate_snap_common_path "$REVERT_FILE" "Revert file" || return 1
validate_snap_common_path "$TEMP_DIR" "Temporary directory" || return 1
validate_snap_common_path "$BACKUP_DIR" "Backup directory" || return 1
log_success "All paths validated within SNAP_COMMON"
return 0
}
# Logging functions
log_message() {
local message="$1"
local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
echo "[$timestamp] $message" | tee -a "$MIGRATION_LOG"
}
log_error() {
local message="$1"
log_message "ERROR: $message"
}
log_success() {
local message="$1"
log_message "SUCCESS: $message"
}
log_warning() {
local message="$1"
log_message "WARNING: $message"
}
# Disk space checking functions
check_disk_space() {
local required_space_gb="$1"
local available_space_gb=$(df "$SNAP_COMMON" | awk 'NR==2 {print int($4/1024/1024)}')
if [ "$available_space_gb" -lt "$required_space_gb" ]; then
log_error "Insufficient disk space. Required: ${required_space_gb}GB, Available: ${available_space_gb}GB"
return 1
fi
log_message "Disk space check passed. Available: ${available_space_gb}GB, Required: ${required_space_gb}GB"
return 0
}
# Progress tracking functions
update_progress() {
local step="$1"
local total_steps="$2"
local description="$3"
local percentage=$((step * 100 / total_steps))
# Update JSON status file
cat > "$MIGRATION_STATUS" << EOF
{
"step": $step,
"total_steps": $total_steps,
"percentage": $percentage,
"description": "$description",
"timestamp": "$(date -Iseconds)",
"status": "running"
}
EOF
# Update HTML progress page
cat > "$MIGRATION_PROGRESS" << EOF
<!DOCTYPE html>
<html>
<head>
<title>MongoDB Migration Progress</title>
<meta http-equiv="refresh" content="5">
<style>
body { font-family: Arial, sans-serif; margin: 40px; }
.progress-bar { width: 100%; background-color: #f0f0f0; border-radius: 5px; }
.progress-fill { height: 30px; background-color: #4CAF50; border-radius: 5px; width: ${percentage}%; }
.status { margin: 20px 0; }
.error { color: red; }
.success { color: green; }
</style>
</head>
<body>
<h1>MongoDB Migration Progress</h1>
<div class="progress-bar">
<div class="progress-fill"></div>
</div>
<div class="status">
<p><strong>Progress:</strong> $step of $total_steps steps ($percentage%)</p>
<p><strong>Current Step:</strong> $description</p>
<p><strong>Last Updated:</strong> $(date)</p>
</div>
<p><em>This page will refresh automatically every 5 seconds.</em></p>
</body>
</html>
EOF
log_message "Progress: $step/$total_steps ($percentage%) - $description"
}
# Estimate completion time
estimate_completion_time() {
local start_time="$1"
local current_step="$2"
local total_steps="$3"
if [ "$current_step" -gt 0 ]; then
local elapsed=$(($(date +%s) - start_time))
local avg_time_per_step=$((elapsed / current_step))
local remaining_steps=$((total_steps - current_step))
local estimated_remaining=$((remaining_steps * avg_time_per_step))
local hours=$((estimated_remaining / 3600))
local minutes=$(((estimated_remaining % 3600) / 60))
local seconds=$((estimated_remaining % 60))
echo "${hours}h ${minutes}m ${seconds}s"
else
echo "Calculating..."
fi
}
# Create backup before migration
create_backup() {
log_message "Creating backup of MongoDB 3 database"
# Check disk space for backup (estimate 2x current database size)
local db_size=$(du -s "${SNAP_COMMON}/wekan" 2>/dev/null | awk '{print $1}' || echo "0")
local required_space=$((db_size * 2 / 1024 / 1024 + 1)) # Convert to GB and add 1GB buffer
if ! check_disk_space "$required_space"; then
log_error "Insufficient disk space for backup"
return 1
fi
# Create backup directory
mkdir -p "$BACKUP_DIR"
# Copy database files
if [ -d "${SNAP_COMMON}/wekan" ]; then
cp -r "${SNAP_COMMON}/wekan" "$BACKUP_DIR/"
log_success "Database backup created at $BACKUP_DIR"
return 0
else
log_error "No database found to backup"
return 1
fi
}
# Check if migration is needed
check_migration_needed() {
if [ -f "$MIGRATION_STATUS" ]; then
local status=$(jq -r '.status' "$MIGRATION_STATUS" 2>/dev/null || echo "unknown")
if [ "$status" = "completed" ]; then
log_message "Migration already completed"
return 1
elif [ "$status" = "running" ]; then
log_message "Migration already in progress"
return 0
fi
fi
# Check if we have MongoDB 3 data (either in wekan directory or raw database files)
if [ -d "${SNAP_COMMON}/wekan" ] && [ ! -f "${SNAP_COMMON}/mongodb-version-7" ]; then
log_message "MongoDB 3 data detected in wekan directory"
return 0
fi
# Check for MongoDB upgrade needed by examining log file
if detect_mongodb_upgrade_needed; then
log_message "MongoDB upgrade needed detected from log file"
return 0
fi
log_message "No migration needed"
return 1
}
# Display MongoDB log content for debugging
display_mongodb_log_content() {
local mongodb_log="${SNAP_COMMON}/mongodb.log"
if [ ! -f "$mongodb_log" ]; then
log_message "MongoDB log file not found: $mongodb_log"
return 1
fi
log_message "MongoDB log file content (last 50 lines):"
if [ -r "$mongodb_log" ]; then
tail -50 "$mongodb_log" | while read -r line; do
log_message "LOG: $line"
done
else
log_message "MongoDB log file not readable, trying with sudo"
sudo tail -50 "$mongodb_log" 2>/dev/null | while read -r line; do
log_message "LOG: $line"
done
fi
}
# Detect if MongoDB upgrade is needed by checking log file
detect_mongodb_upgrade_needed() {
local mongodb_log="${SNAP_COMMON}/mongodb.log"
# Check if MongoDB log file exists
if [ ! -f "$mongodb_log" ]; then
log_message "MongoDB log file not found: $mongodb_log"
return 1
fi
# Display log content for debugging
display_mongodb_log_content
# Check file permissions and try to read with appropriate method
if [ ! -r "$mongodb_log" ]; then
log_message "MongoDB log file not readable, trying with sudo"
# Try to read with sudo if not readable
if ! sudo grep -q "too recent to start up on the existing data files" "$mongodb_log" 2>/dev/null; then
log_message "No MongoDB upgrade needed detected in log file (via sudo)"
return 1
fi
else
# Check for various error messages that indicate upgrade is needed
# The exact message may vary between MongoDB versions
local upgrade_patterns=(
"This version of MongoDB is too recent to start up on the existing data files"
"too recent to start up on the existing data files"
"Try MongoDB 4.2 or earlier"
"unsupported format version"
"data files are incompatible"
"database files are incompatible"
"version too new"
"version too recent"
)
local found_upgrade_needed=false
for pattern in "${upgrade_patterns[@]}"; do
if grep -q "$pattern" "$mongodb_log" 2>/dev/null; then
log_message "Found upgrade pattern in log: '$pattern'"
found_upgrade_needed=true
break
fi
done
if [ "$found_upgrade_needed" = false ]; then
log_message "No MongoDB upgrade needed detected in log file"
return 1
fi
fi
log_message "MongoDB upgrade needed detected in log file"
return 0
}
# Log rotation function for MongoDB logs
rotate_mongodb_logs() {
local mongodb_log="${SNAP_COMMON}/mongodb.log"
local max_size_mb=100
local keep_copies=10
# Check if log file exists and is large enough to rotate
if [ ! -f "$mongodb_log" ]; then
log_message "MongoDB log file not found, skipping rotation"
return 0
fi
# Get log file size in MB
local log_size_mb=$(du -m "$mongodb_log" | cut -f1)
if [ "$log_size_mb" -lt "$max_size_mb" ]; then
log_message "MongoDB log size (${log_size_mb}MB) is below rotation threshold (${max_size_mb}MB)"
return 0
fi
log_message "Rotating MongoDB log file (size: ${log_size_mb}MB)"
# Create rotated log file with timestamp
local timestamp=$(date +%Y%m%d-%H%M%S)
local rotated_log="${mongodb_log}.${timestamp}"
# Copy current log to rotated file
if cp "$mongodb_log" "$rotated_log"; then
log_message "Created rotated log file: $rotated_log"
# Truncate original log file
if > "$mongodb_log"; then
log_message "Truncated original log file"
else
log_error "Failed to truncate original log file"
return 1
fi
# Compress rotated log file
if gzip "$rotated_log"; then
log_message "Compressed rotated log file: ${rotated_log}.gz"
else
log_warning "Failed to compress rotated log file"
fi
# Clean up old rotated logs (keep only specified number)
local old_logs=$(ls -t "${mongodb_log}".* 2>/dev/null | tail -n +$((keep_copies + 1)))
if [ -n "$old_logs" ]; then
echo "$old_logs" | xargs rm -f
log_message "Cleaned up old rotated log files"
fi
log_success "MongoDB log rotation completed successfully"
return 0
else
log_error "Failed to create rotated log file"
return 1
fi
}
# Enhanced log rotation function for migration logs
rotate_migration_logs() {
local migration_log="${SNAP_COMMON}/mongodb-migration-log.txt"
local max_size_mb=50
local keep_copies=5
# Check if migration log file exists and is large enough to rotate
if [ ! -f "$migration_log" ]; then
log_message "Migration log file not found, skipping rotation"
return 0
fi
# Get log file size in MB
local log_size_mb=$(du -m "$migration_log" | cut -f1)
if [ "$log_size_mb" -lt "$max_size_mb" ]; then
log_message "Migration log size (${log_size_mb}MB) is below rotation threshold (${max_size_mb}MB)"
return 0
fi
log_message "Rotating migration log file (size: ${log_size_mb}MB)"
# Create rotated log file with timestamp
local timestamp=$(date +%Y%m%d-%H%M%S)
local rotated_log="${migration_log}.${timestamp}"
# Copy current log to rotated file
if cp "$migration_log" "$rotated_log"; then
log_message "Created rotated migration log file: $rotated_log"
# Truncate original log file
if > "$migration_log"; then
log_message "Truncated original migration log file"
else
log_error "Failed to truncate original migration log file"
return 1
fi
# Compress rotated log file
if gzip "$rotated_log"; then
log_message "Compressed rotated migration log file: ${rotated_log}.gz"
else
log_warning "Failed to compress rotated migration log file"
fi
# Clean up old rotated logs (keep only specified number)
local old_logs=$(ls -t "${migration_log}".* 2>/dev/null | tail -n +$((keep_copies + 1)))
if [ -n "$old_logs" ]; then
echo "$old_logs" | xargs rm -f
log_message "Cleaned up old rotated migration log files"
fi
log_success "Migration log rotation completed successfully"
return 0
else
log_error "Failed to create rotated migration log file"
return 1
fi
}
# Reset MONGO_LOG_DESTINATION to devnull after successful migration - DISABLED
# Snap settings changes are permanently disabled during migration
reset_mongo_log_destination() {
log_message "MONGO_LOG_DESTINATION reset disabled - snap settings changes not allowed during migration"
# Snap settings changes permanently disabled during migration
# This ensures no snap configuration is modified during the migration process
# if snap set wekan mongo-log-destination="devnull" 2>/dev/null; then
# log_success "MONGO_LOG_DESTINATION reset to devnull successfully"
# else
# log_error "Failed to reset MONGO_LOG_DESTINATION to devnull"
# # Don't fail the migration for this setting issue
# fi
}
# Migrate raw MongoDB 3 database files
migrate_raw_database_files() {
log_message "Starting raw MongoDB 3 database files migration"
# Validate paths are within SNAP_COMMON
if ! validate_snap_common_path "${SNAP_COMMON}" "Database path"; then
log_error "Database path validation failed"
return 1
fi
# Stop any running MongoDB processes
log_message "Stopping any running MongoDB processes"
pkill -f mongod || true
sleep 3
# Start MongoDB 3 with raw files
log_message "Starting MongoDB 3 with raw database files"
local mongo3_pid
mongod --dbpath "${SNAP_COMMON}" --port "${MONGODB_PORT:-27019}" --quiet &
mongo3_pid=$!
# Wait for MongoDB 3 to start
local retry_count=0
while [ $retry_count -lt 30 ]; do
if mongosh --quiet --eval "db.adminCommand('ping')" "mongodb://localhost:${MONGODB_PORT:-27019}/admin" >/dev/null 2>&1; then
log_message "MongoDB 3 started successfully"
break
fi
sleep 1
retry_count=$((retry_count + 1))
done
if [ $retry_count -eq 30 ]; then
log_error "MongoDB 3 failed to start"
kill $mongo3_pid 2>/dev/null || true
return 1
fi
# Dump all databases from MongoDB 3
log_message "Dumping databases from MongoDB 3"
if ! mongodump --port "${MONGODB_PORT:-27019}" --out "$TEMP_DIR" --dbpath "${SNAP_COMMON}"; then
log_error "Failed to dump databases from MongoDB 3"
kill $mongo3_pid 2>/dev/null || true
return 1
fi
# Stop MongoDB 3
log_message "Stopping MongoDB 3"
kill $mongo3_pid 2>/dev/null || true
sleep 3
# Start MongoDB 7
log_message "Starting MongoDB 7"
local mongo7_pid
mongod --dbpath "${SNAP_COMMON}" --port "${MONGODB_PORT:-27019}" --quiet &
mongo7_pid=$!
# Wait for MongoDB 7 to start
retry_count=0
while [ $retry_count -lt 30 ]; do
if mongosh --quiet --eval "db.adminCommand('ping')" "mongodb://localhost:${MONGODB_PORT:-27019}/admin" >/dev/null 2>&1; then
log_message "MongoDB 7 started successfully"
break
fi
sleep 1
retry_count=$((retry_count + 1))
done
if [ $retry_count -eq 30 ]; then
log_error "MongoDB 7 failed to start"
kill $mongo7_pid 2>/dev/null || true
return 1
fi
# Restore databases to MongoDB 7
log_message "Restoring databases to MongoDB 7"
if ! mongorestore --port "${MONGODB_PORT:-27019}" --dbpath "${SNAP_COMMON}" "$TEMP_DIR"; then
log_error "Failed to restore databases to MongoDB 7"
kill $mongo7_pid 2>/dev/null || true
return 1
fi
# Stop MongoDB 7
log_message "Stopping MongoDB 7"
kill $mongo7_pid 2>/dev/null || true
sleep 3
# Clean up old MongoDB 3 files
log_message "Cleaning up old MongoDB 3 files"
find "${SNAP_COMMON}" -maxdepth 1 -name "*.0" -o -name "*.1" -o -name "*.ns" -o -name "j._*" -o -name "mongod.lock" | while read -r file; do
if [ -f "$file" ]; then
rm -f "$file"
log_message "Removed old file: $file"
fi
done
# Remove journal directory if it exists
if [ -d "${SNAP_COMMON}/journal" ]; then
rm -rf "${SNAP_COMMON}/journal"
log_message "Removed journal directory"
fi
log_success "Raw database files migration completed successfully"
# Reset MONGO_LOG_DESTINATION to devnull after successful migration
reset_mongo_log_destination
return 0
}
# Snap channel detection and switching functions
get_current_snap_channel() {
local snap_name="$1"
snap list "$snap_name" 2>/dev/null | awk 'NR==2 {print $4}' || echo "unknown"
}
is_wekan_snap() {
local snap_name="$1"
case "$snap_name" in
wekan|wekan-gantt-gpl|wekan-ondra)
return 0
;;
*)
return 1
;;
esac
}
switch_to_stable_channel() {
local snap_name="$1"
local current_channel=$(get_current_snap_channel "$snap_name")
log_message "Snap channel switching disabled - not changing $snap_name channel (currently: $current_channel)"
log_message "Snap channel changes are permanently disabled during migration"
return 0
# Snap channel switching permanently disabled during migration
# This ensures no snap channels are modified during the migration process
# if [ "$current_channel" != "stable" ] && [ "$current_channel" != "unknown" ]; then
# log_message "Switching $snap_name from $current_channel to stable channel"
# if snap refresh "$snap_name" --channel=stable; then
# log_success "Successfully switched $snap_name to stable channel"
# return 0
# else
# log_error "Failed to switch $snap_name to stable channel"
# return 1
# fi
# else
# log_message "$snap_name is already on stable channel or not installed"
# return 0
# fi
}
switch_all_wekan_snaps_to_stable() {
log_message "Checking for Wekan-related snaps to switch to stable channel"
local wekan_snaps=("wekan" "wekan-gantt-gpl" "wekan-ondra")
local switched_count=0
local failed_count=0
for snap_name in "${wekan_snaps[@]}"; do
if snap list "$snap_name" >/dev/null 2>&1; then
if switch_to_stable_channel "$snap_name"; then
switched_count=$((switched_count + 1))
else
failed_count=$((failed_count + 1))
fi
fi
done
log_message "Channel switching completed: $switched_count successful, $failed_count failed"
if [ "$failed_count" -gt 0 ]; then
return 1
else
return 0
fi
}
# Get database collections
get_collections() {
local mongo_url="$1"
local collections=$(mongosh --quiet --eval "db.getCollectionNames().join('\n')" "$mongo_url" 2>/dev/null | grep -v "^$" || echo "")
echo "$collections"
}
# Migrate a single collection
migrate_collection() {
local collection="$1"
local mongo3_url="$2"
local mongo7_url="$3"
local step="$4"
local total_steps="$5"
log_message "Migrating collection: $collection"
# Check disk space before each collection (estimate 2x collection size)
local collection_size=$(mongosh --quiet --eval "db.$collection.stats().size" "$mongo3_url" 2>/dev/null || echo "0")
local required_space=$((collection_size * 2 / 1024 / 1024 / 1024 + 1)) # Convert to GB and add 1GB buffer
if ! check_disk_space "$required_space"; then
log_error "Insufficient disk space for collection $collection"
return 1
fi
# Dump collection
local dump_file="${TEMP_DIR}/${collection}.bson"
log_message "Dumping collection $collection to $dump_file"
if ! mongodump --db wekan --collection "$collection" --out "$TEMP_DIR" --port "${MONGODB_PORT:-27019}" --dbpath "${SNAP_COMMON}"; then
log_error "Failed to dump collection $collection"
return 1
fi
# Restore collection
log_message "Restoring collection $collection to MongoDB 7"
if ! mongorestore --db wekan --collection "$collection" "$dump_file" --port "${MONGODB_PORT:-27019}" --dbpath "${SNAP_COMMON}"; then
log_error "Failed to restore collection $collection"
return 1
fi
# Update progress
update_progress "$step" "$total_steps" "Migrated collection: $collection"
# Clean up dump file
rm -f "$dump_file"
log_success "Collection $collection migrated successfully"
return 0
}
# Main migration function
perform_migration() {
local start_time=$(date +%s)
log_message "Starting MongoDB migration from version 3 to 7"
# Rotate MongoDB logs before migration if needed
log_message "Checking if MongoDB log rotation is needed"
if ! rotate_mongodb_logs; then
log_warning "MongoDB log rotation failed, continuing with migration"
fi
# Rotate migration logs before migration if needed
log_message "Checking if migration log rotation is needed"
if ! rotate_migration_logs; then
log_warning "Migration log rotation failed, continuing with migration"
fi
# Create backup before migration
log_message "Creating backup before migration"
if ! create_backup; then
log_error "Failed to create backup, aborting migration"
return 1
fi
# Create temporary directory
mkdir -p "$TEMP_DIR"
# Check if we need to migrate raw database files
if detect_mongodb_upgrade_needed; then
log_message "MongoDB upgrade needed detected, starting raw file migration"
if ! migrate_raw_database_files; then
log_error "Failed to migrate raw database files"
return 1
fi
fi
# Get MongoDB connection details
local mongo3_url="mongodb://localhost:${MONGODB_PORT:-27019}/wekan"
local mongo7_url="mongodb://localhost:${MONGODB_PORT:-27019}/wekan"
# Get collections to migrate
log_message "Getting list of collections to migrate"
local collections=$(get_collections "$mongo3_url")
if [ -z "$collections" ]; then
log_error "No collections found to migrate"
return 1
fi
local collection_count=$(echo "$collections" | wc -l)
log_message "Found $collection_count collections to migrate"
# Migrate each collection
local current_step=0
for collection in $collections; do
current_step=$((current_step + 1))
if ! migrate_collection "$collection" "$mongo3_url" "$mongo7_url" "$current_step" "$collection_count"; then
log_error "Migration failed at collection $collection"
return 1
fi
# Update completion time estimate
local estimated_time=$(estimate_completion_time "$start_time" "$current_step" "$collection_count")
log_message "Estimated completion time: $estimated_time"
done
# Mark migration as completed
cat > "$MIGRATION_STATUS" << EOF
{
"step": $collection_count,
"total_steps": $collection_count,
"percentage": 100,
"description": "Migration completed successfully",
"timestamp": "$(date -Iseconds)",
"status": "completed"
}
EOF
# Create MongoDB 7 version marker
touch "${SNAP_COMMON}/mongodb-version-7"
# Clean up temporary files
rm -rf "$TEMP_DIR"
# Switch Wekan snaps to stable channel after successful migration
log_message "Switching Wekan snaps to stable channel after successful migration"
if switch_all_wekan_snaps_to_stable; then
log_success "All Wekan snaps switched to stable channel successfully"
else
log_error "Some Wekan snaps failed to switch to stable channel"
# Don't fail the migration for channel switching issues
fi
log_success "MongoDB migration completed successfully"
# Rotate MongoDB logs after successful migration
log_message "Rotating MongoDB logs after successful migration"
if ! rotate_mongodb_logs; then
log_warning "MongoDB log rotation after migration failed"
fi
# Rotate migration logs after successful migration
log_message "Rotating migration logs after successful migration"
if ! rotate_migration_logs; then
log_warning "Migration log rotation after migration failed"
fi
# Reset MONGO_LOG_DESTINATION to devnull after successful migration
reset_mongo_log_destination
return 0
}
# Revert migration
revert_migration() {
log_message "Reverting MongoDB migration"
if [ ! -f "$REVERT_FILE" ]; then
log_error "Revert file not found: $REVERT_FILE"
return 1
fi
# Stop MongoDB 7 - DISABLED
log_message "MongoDB stopping disabled - not using snapctl stop"
# snapctl stop --disable "${SNAP_NAME}.mongodb"
# Remove MongoDB 7 version marker
rm -f "${SNAP_COMMON}/mongodb-version-7"
# Find the most recent backup directory
local latest_backup=$(ls -td "${SNAP_COMMON}/mongodb-backup-"* 2>/dev/null | head -1)
if [ -n "$latest_backup" ] && [ -d "$latest_backup" ]; then
log_message "Restoring from backup: $latest_backup"
# Stop any running MongoDB processes
pkill -f mongod || true
sleep 2
# Remove current database directory
rm -rf "${SNAP_COMMON}/wekan"
# Restore from backup
cp -r "$latest_backup"/* "${SNAP_COMMON}/"
# Clean up backup directory
rm -rf "$latest_backup"
log_success "Database restored from backup"
else
log_error "No backup found for revert"
return 1
fi
# Remove revert file
rm -f "$REVERT_FILE"
# Clear migration status
rm -f "$MIGRATION_STATUS"
# Start MongoDB 3 - DISABLED
log_message "MongoDB starting disabled - not using snapctl start"
# snapctl start --enable "${SNAP_NAME}.mongodb"
log_success "Migration reverted successfully"
return 0
}
# Main execution
main() {
log_message "MongoDB Migration Script started"
# Validate all paths are within SNAP_COMMON
if ! validate_all_paths; then
log_error "Path validation failed - aborting migration"
exit 1
fi
# Check if revert is requested
if [ -f "$REVERT_FILE" ]; then
revert_migration
exit $?
fi
# Check if migration is needed
if ! check_migration_needed; then
exit 0
fi
# Perform migration
if perform_migration; then
log_success "Migration completed successfully"
exit 0
else
log_error "Migration failed"
exit 1
fi
}
# Run main function
main "$@"

126
snap-src/bin/mongodb-migration-status
View file

@ -1,126 +0,0 @@
#!/bin/bash
# MongoDB Migration Status Script
# Provides status information about the migration process
# Source settings
source $SNAP/bin/wekan-read-settings
# Configuration
MIGRATION_STATUS="${SNAP_COMMON}/mongodb-migration-status.json"
MIGRATION_LOG="${SNAP_COMMON}/mongodb-migration-log.txt"
REVERT_FILE="${SNAP_COMMON}/revert-mongodb-migration.txt"
# Show migration status
show_status() {
if [ -f "$MIGRATION_STATUS" ]; then
echo "=== MongoDB Migration Status ==="
echo "Status: $(jq -r '.status' "$MIGRATION_STATUS" 2>/dev/null || echo "unknown")"
echo "Progress: $(jq -r '.step' "$MIGRATION_STATUS" 2>/dev/null || echo "0")/$(jq -r '.total_steps' "$MIGRATION_STATUS" 2>/dev/null || echo "0") steps"
echo "Percentage: $(jq -r '.percentage' "$MIGRATION_STATUS" 2>/dev/null || echo "0")%"
echo "Current Step: $(jq -r '.description' "$MIGRATION_STATUS" 2>/dev/null || echo "Unknown")"
echo "Last Updated: $(jq -r '.timestamp' "$MIGRATION_STATUS" 2>/dev/null || echo "Unknown")"
echo ""
if [ -f "$MIGRATION_LOG" ]; then
echo "=== Recent Migration Log ==="
tail -10 "$MIGRATION_LOG"
fi
else
echo "No migration in progress or completed."
fi
}
# Show migration log
show_log() {
if [ -f "$MIGRATION_LOG" ]; then
echo "=== Full Migration Log ==="
cat "$MIGRATION_LOG"
else
echo "No migration log found."
fi
}
# Check if migration is needed
check_needed() {
if [ -d "${SNAP_COMMON}/wekan" ] && [ ! -f "${SNAP_COMMON}/mongodb-version-7" ]; then
echo "Migration needed: MongoDB 3 data detected"
return 0
else
echo "No migration needed"
return 1
fi
}
# Start migration
start_migration() {
if check_needed; then
echo "Starting MongoDB migration..."
$SNAP/bin/mongodb-migrate
else
echo "Migration not needed or already completed"
fi
}
# Request revert
request_revert() {
echo "Requesting migration revert..."
touch "$REVERT_FILE"
echo "Revert requested. Restart Wekan to apply revert."
}
# Clear migration status
clear_status() {
echo "Clearing migration status..."
rm -f "$MIGRATION_STATUS"
rm -f "$MIGRATION_LOG"
rm -f "$REVERT_FILE"
echo "Migration status cleared."
}
# Show help
show_help() {
echo "MongoDB Migration Status Tool"
echo ""
echo "Usage: $0 [command]"
echo ""
echo "Commands:"
echo " status - Show current migration status (default)"
echo " log - Show full migration log"
echo " check - Check if migration is needed"
echo " start - Start migration if needed"
echo " revert - Request migration revert"
echo " clear - Clear migration status"
echo " help - Show this help"
echo ""
}
# Main execution
case "${1:-status}" in
"status")
show_status
;;
"log")
show_log
;;
"check")
check_needed
;;
"start")
start_migration
;;
"revert")
request_revert
;;
"clear")
clear_status
;;
"help"|"-h"|"--help")
show_help
;;
*)
echo "Unknown command: $1"
show_help
exit 1
;;
esac

251
snap-src/bin/mongodb-migration-web
View file

@ -1,251 +0,0 @@
#!/bin/bash
# MongoDB Migration Web Interface
# Serves migration progress at ROOT_URL/migration-progress using Node.js
# Source settings
source $SNAP/bin/wekan-read-settings
# Set up Node.js environment like wekan-control
export NODE_PATH=$SNAP/bin
# Configuration
MIGRATION_STATUS="${SNAP_COMMON}/mongodb-migration-status.json"
MIGRATION_LOG="${SNAP_COMMON}/mongodb-migration-log.txt"
MIGRATION_PROGRESS="${SNAP_COMMON}/mongodb-migration-progress.html"
# Use same PORT as wekan-control, but add 1 to avoid conflicts
MIGRATION_PORT=$((PORT + 1))
# Create Node.js HTTP server script
create_node_server() {
cat > "${SNAP_COMMON}/migration-web-server.js" << 'EOF'
const http = require('http');
const fs = require('fs');
const path = require('path');
const PORT = process.env.MIGRATION_PORT || 8081;
const SNAP_COMMON = process.env.SNAP_COMMON;
const ROOT_URL = process.env.ROOT_URL || 'http://127.0.0.1';
const MIGRATION_STATUS = path.join(SNAP_COMMON, 'mongodb-migration-status.json');
const MIGRATION_LOG = path.join(SNAP_COMMON, 'mongodb-migration-log.txt');
function readFileSafe(filePath) {
try {
return fs.readFileSync(filePath, 'utf8');
} catch (error) {
return null;
}
}
function getMigrationStatus() {
const statusContent = readFileSafe(MIGRATION_STATUS);
if (!statusContent) {
return null;
}
try {
return JSON.parse(statusContent);
} catch (error) {
return null;
}
}
function getMigrationLog() {
const logContent = readFileSafe(MIGRATION_LOG);
if (!logContent) {
return 'No log available';
}
const lines = logContent.split('\n');
return lines.slice(-20).join('\n');
}
function generateHTML(status) {
if (!status) {
return `<!DOCTYPE html>
<html>
<head>
<title>MongoDB Migration Progress</title>
<meta http-equiv="refresh" content="5">
<style>
body { font-family: Arial, sans-serif; margin: 40px; }
.container { max-width: 800px; margin: 0 auto; text-align: center; }
</style>
</head>
<body>
<div class="container">
<h1>MongoDB Migration</h1>
<p>No migration in progress.</p>
<p><em>This page will refresh automatically every 5 seconds.</em></p>
</div>
</body>
</html>`;
}
const { status: statusValue, step, total_steps, percentage, description, timestamp } = status;
const logContent = getMigrationLog();
return `<!DOCTYPE html>
<html>
<head>
<title>MongoDB Migration Progress</title>
<meta http-equiv="refresh" content="5">
<style>
body {
font-family: Arial, sans-serif;
margin: 40px;
background-color: #f5f5f5;
}
.container {
max-width: 800px;
margin: 0 auto;
background: white;
padding: 30px;
border-radius: 10px;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
.progress-bar {
width: 100%;
background-color: #e0e0e0;
border-radius: 10px;
overflow: hidden;
margin: 20px 0;
}
.progress-fill {
height: 40px;
background: linear-gradient(90deg, #4CAF50, #45a049);
border-radius: 10px;
width: ${percentage}%;
transition: width 0.3s ease;
display: flex;
align-items: center;
justify-content: center;
color: white;
font-weight: bold;
}
.status {
margin: 20px 0;
padding: 20px;
background-color: #f9f9f9;
border-radius: 5px;
}
.error { color: #d32f2f; }
.success { color: #388e3c; }
.warning { color: #f57c00; }
.info { color: #1976d2; }
.log-container {
margin-top: 30px;
max-height: 300px;
overflow-y: auto;
background-color: #f5f5f5;
padding: 15px;
border-radius: 5px;
font-family: monospace;
font-size: 12px;
}
.header {
text-align: center;
margin-bottom: 30px;
}
.status-indicator {
display: inline-block;
width: 12px;
height: 12px;
border-radius: 50%;
margin-right: 8px;
}
.status-running { background-color: #ff9800; }
.status-completed { background-color: #4caf50; }
.status-error { background-color: #f44336; }
.status-unknown { background-color: #9e9e9e; }
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>MongoDB Migration Progress</h1>
<p>Migrating from MongoDB 3 to MongoDB 7</p>
</div>
<div class="progress-bar">
<div class="progress-fill">${percentage}%</div>
</div>
<div class="status">
<p><span class="status-indicator status-${statusValue}"></span><strong>Status:</strong> ${statusValue}</p>
<p><strong>Progress:</strong> ${step} of ${total_steps} steps</p>
<p><strong>Current Step:</strong> ${description}</p>
<p><strong>Last Updated:</strong> ${timestamp}</p>
</div>
<div class="log-container">
<h3>Migration Log (Last 20 lines):</h3>
<pre>${logContent}</pre>
</div>
<p style="text-align: center; margin-top: 30px; color: #666;">
<em>This page will refresh automatically every 5 seconds.</em><br>
<em>Migration URL: ${ROOT_URL}/migration-progress</em>
</p>
</div>
</body>
</html>`;
}
const server = http.createServer((req, res) => {
if (req.url === '/migration-progress' || req.url === '/') {
const status = getMigrationStatus();
const html = generateHTML(status);
res.writeHead(200, {
'Content-Type': 'text/html; charset=utf-8',
'Cache-Control': 'no-cache',
'Connection': 'close'
});
res.end(html);
} else {
res.writeHead(404, { 'Content-Type': 'text/plain' });
res.end('Not Found');
}
});
server.listen(PORT, () => {
console.log(`MongoDB Migration Web Server running on port ${PORT}`);
});
// Handle graceful shutdown
process.on('SIGTERM', () => {
console.log('Received SIGTERM, shutting down gracefully');
server.close(() => {
process.exit(0);
});
});
process.on('SIGINT', () => {
console.log('Received SIGINT, shutting down gracefully');
server.close(() => {
process.exit(0);
});
});
EOF
}
# Start the Node.js web server
start_node_server() {
echo "Starting MongoDB migration web server using Node.js..."
echo "Migration server will be available at: ${ROOT_URL}/migration-progress"
echo "Migration server port: ${MIGRATION_PORT}"
# Create the Node.js server script
create_node_server
# Export environment variables for the Node.js process
export MIGRATION_PORT
export ROOT_URL
# Start the server using Node.js from SNAP/bin
$NODE_PATH/node "${SNAP_COMMON}/migration-web-server.js"
}
# Start the web server
start_node_server

138
snap-src/bin/mongodb-version-manager Executable file
View file

@ -0,0 +1,138 @@
#!/bin/bash
# MongoDB Version Manager
# Helps manage MongoDB server binaries for different versions (3-8)
# This script provides utilities for adding and managing MongoDB server versions
# Source settings
source $SNAP/bin/wekan-read-settings
# MongoDB version information
declare -A MONGO_VERSIONS=(
["3"]="3.2.22"
["4"]="4.4.28"
["5"]="5.0.28"
["6"]="6.0.15"
["7"]="7.0.25"
["8"]="8.0.4"
)
# MongoDB binary paths
declare -A MONGO_PATHS=(
["3"]="/snap/${SNAP_NAME}/current/migratemongo/bin"
["4"]="/snap/${SNAP_NAME}/current/mongodb4/bin"
["5"]="/snap/${SNAP_NAME}/current/mongodb5/bin"
["6"]="/snap/${SNAP_NAME}/current/mongodb6/bin"
["7"]="/snap/${SNAP_NAME}/current/bin"
["8"]="/snap/${SNAP_NAME}/current/mongodb8/bin"
)
# Check which MongoDB versions are available
check_available_versions() {
echo "=== Available MongoDB Server Versions ==="
for version in "${!MONGO_VERSIONS[@]}"; do
local path="${MONGO_PATHS[$version]}"
if [ -f "$path/mongod" ]; then
echo "✓ MongoDB $version.x (${MONGO_VERSIONS[$version]}) - Available at $path"
else
echo "✗ MongoDB $version.x (${MONGO_VERSIONS[$version]}) - Not available at $path"
fi
done
echo ""
}
# Check which MongoDB Node.js drivers are available
check_available_drivers() {
echo "=== Available MongoDB Node.js Drivers ==="
if [ -f "/home/wekan/repos/wekan/package.json" ]; then
for version in "${!MONGO_VERSIONS[@]}"; do
if grep -q "mongodb${version}legacy" "/home/wekan/repos/wekan/package.json"; then
echo "✓ MongoDB $version.x Node.js driver - Available (mongodb${version}legacy)"
else
echo "✗ MongoDB $version.x Node.js driver - Not available"
fi
done
else
echo "package.json not found"
fi
echo ""
}
# Show current active MongoDB version
show_active_version() {
if [ -f "${SNAP_COMMON}/mongodb-active-version" ]; then
local active_version=$(cat "${SNAP_COMMON}/mongodb-active-version")
echo "=== Current Active MongoDB Version ==="
echo "Active Version: MongoDB $active_version.x"
echo "Version File: ${SNAP_COMMON}/mongodb-active-version"
echo ""
else
echo "=== Current Active MongoDB Version ==="
echo "No active version file found (will use default MongoDB 7.x)"
echo ""
fi
}
# Show version detection log
show_detection_log() {
if [ -f "${SNAP_COMMON}/mongodb-version-detection.log" ]; then
echo "=== MongoDB Version Detection Log ==="
tail -20 "${SNAP_COMMON}/mongodb-version-detection.log"
echo ""
else
echo "=== MongoDB Version Detection Log ==="
echo "No detection log found"
echo ""
fi
}
# Force version detection
force_detection() {
echo "=== Forcing MongoDB Version Detection ==="
rm -f "${SNAP_COMMON}/mongodb-active-version"
echo "Cleared cached version. Next MongoDB restart will detect version automatically."
echo ""
}
# Show help
show_help() {
echo "MongoDB Version Manager"
echo ""
echo "Usage: $0 [command]"
echo ""
echo "Commands:"
echo " versions - Show available MongoDB server versions"
echo " drivers - Show available MongoDB Node.js drivers"
echo " active - Show current active MongoDB version"
echo " log - Show version detection log"
echo " detect - Force version detection on next restart"
echo " help - Show this help message"
echo ""
echo "Examples:"
echo " $0 versions # Check which MongoDB versions are available"
echo " $0 active # Show which version is currently active"
echo " $0 detect # Force re-detection on next restart"
echo ""
}
# Main command handling
case "${1:-help}" in
"versions")
check_available_versions
;;
"drivers")
check_available_drivers
;;
"active")
show_active_version
;;
"log")
show_detection_log
;;
"detect")
force_detection
;;
"help"|*)
show_help
;;
esac