mirror of
https://github.com/wekan/wekan.git
synced 2025-12-24 03:10:12 +01:00
Reorganized Docs. In Progress.
This commit is contained in:
Lauri Ojansivu
parent
1961e22cbd
commit
ce89ff4833
202 changed files with 0 additions and 0 deletions
784
docs/Backup/Backup.md
Normal file
784
docs/Backup/Backup.md
Normal file
|
|
@ -0,0 +1,784 @@
|
|||
[Sandstorm](Sandstorm) - [Sandstorm Backup](Export-from-Wekan-Sandstorm-grain-.zip-file)
|
||||
|
||||
# Upcoming
|
||||
|
||||
[Transferring to Minio and SQLite](https://github.com/wekan/minio-metadata)
|
||||
|
||||
# Backup Docker
|
||||
|
||||
[Also see: Upgrading Synology with Wekan quay images](https://github.com/wekan/wekan/issues/3874#issuecomment-867526249)
|
||||
|
||||
Note: Do not run `docker-compose down` without verifying your docker-compose file, it does not delete the data by default but caution is advised. Refer to https://docs.docker.com/compose/reference/down/.
|
||||
|
||||
[docker-compose.yml](https://raw.githubusercontent.com/wekan/wekan/master/docker-compose.yml)
|
||||
|
||||
This presumes your Wekan Docker is currently running with:
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
Backup to directory dump:
|
||||
```bash
|
||||
docker stop wekan-app
|
||||
docker exec wekan-db rm -rf /data/dump
|
||||
docker exec wekan-db mongodump -o /data/dump
|
||||
docker cp wekan-db:/data/dump .
|
||||
docker start wekan-app
|
||||
```
|
||||
Copy dump directory to other server or to your backup.
|
||||
|
||||
# Restore Docker
|
||||
```bash
|
||||
docker stop wekan-app
|
||||
docker exec wekan-db rm -rf /data/dump
|
||||
docker cp dump wekan-db:/data/
|
||||
docker exec wekan-db mongorestore --drop --dir=/data/dump
|
||||
docker start wekan-app
|
||||
```
|
||||
# Upgrade Docker Wekan version
|
||||
|
||||
## Newest info
|
||||
|
||||
https://github.com/wekan/wekan/discussions/5367
|
||||
|
||||
## Old info
|
||||
|
||||
Note: Do not run `docker-compose down` without verifying your docker-compose file, it does not delete the data by default but caution is advised. Refer to https://docs.docker.com/compose/reference/down/.
|
||||
```bash
|
||||
docker-compose stop
|
||||
docker rm wekan-app
|
||||
```
|
||||
a) For example, if you in docker-compose.yml use `image: wekanteam/wekan` or `image: quay.io/wekan/wekan` for latest development version
|
||||
|
||||
b) Or in docker-compose.yml change version tag, or use version tag like `image: wekanteam/wekan:v5.50` or `image: quay.io/wekan/wekan:v5.50`
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
# Backup Wekan Snap to directory dump
|
||||
```bash
|
||||
sudo snap stop wekan.wekan
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
export PATH="/snap/wekan/current/bin:$PATH"
|
||||
mongodump --port 27019
|
||||
sudo snap get wekan > snap-settings.sh
|
||||
```
|
||||
NOTE for Arch Linux: Use this path instead. [Source](https://github.com/wekan/wekan/issues/3941).
|
||||
```bash
|
||||
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH-}:/var/lib/snapd/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
```
|
||||
username is your /home/username
|
||||
```bash
|
||||
sudo chown username:username snap-settings.sh
|
||||
sudo snap start wekan.wekan
|
||||
```
|
||||
Modify snap-settings.sh so that it has commands like:
|
||||
```bash
|
||||
sudo snap set wekan root-url='http://localhost'
|
||||
```
|
||||
Set snap-settings.sh executeable:
|
||||
```bash
|
||||
chmod +x snap-settings.sh
|
||||
```
|
||||
|
||||
# Restore Wekan Snap
|
||||
```bash
|
||||
sudo snap stop wekan.wekan
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
export PATH="$PATH:/snap/wekan/current/bin"
|
||||
mongorestore --drop --port 27019
|
||||
sudo snap start wekan.wekan
|
||||
./snap-settings.sh
|
||||
```
|
||||
# Upgrade Snap manually immediately (usually it updates automatically)
|
||||
|
||||
```bash
|
||||
sudo snap refresh
|
||||
```
|
||||
|
||||
# Backup Wekan Gantt GPLv2 Snap to directory dump
|
||||
```bash
|
||||
sudo snap stop wekan-gantt-gpl.wekan
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan-gantt-gpl/current/lib/x86_64-linux-gnu
|
||||
export PATH="$PATH:/snap/wekan-gantt-gpl/current/bin"
|
||||
mongodump --port 27019
|
||||
sudo snap get wekan-gantt-gpl > snap-settings.sh
|
||||
```
|
||||
username is your /home/username
|
||||
```bash
|
||||
sudo chown username:username snap-settings.sh
|
||||
sudo snap start wekan-gantt-gpl.wekan
|
||||
```
|
||||
Modify snap-settings.sh so that it has command like:
|
||||
```bash
|
||||
sudo snap set wekan-gantt-gpl root-url='http://localhost'
|
||||
```
|
||||
Set snap-settings.sh executeable:
|
||||
```bash
|
||||
chmod +x snap-settings.sh
|
||||
```
|
||||
|
||||
# Restore Wekan Gantt GPLv2 Snap
|
||||
```bash
|
||||
sudo snap stop wekan-gantt-gpl.wekan
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan-gantt-gpl/current/lib/x86_64-linux-gnu
|
||||
export PATH="$PATH:/snap/wekan-gantt-gpl/current/bin"
|
||||
mongorestore --drop --port 27019
|
||||
sudo snap start wekan-gantt-gpl.wekan
|
||||
./snap-settings.sh
|
||||
```
|
||||
|
||||
# DBGate, Open Source MongoDB GUI
|
||||
|
||||
https://dbgate.org - https://github.com/dbgate/dbgate
|
||||
|
||||
# Using nosqlbooster closed source MongoDB GUI with Wekan Snap to edit MongoDB database
|
||||
|
||||
https://nosqlbooster.com/downloads
|
||||
|
||||
### At server where Wekan Snap is installed, MongoDB is running at localhost port 27019
|
||||
|
||||
<img src="https://wekan.github.io/nosqlbooster-basic-connection.png" width="60%" alt="Wekan logo" />
|
||||
|
||||
### You can tunnel via SSH to server, using password auth or private key auth dropdown selection
|
||||
|
||||
<img src="https://wekan.github.io/nosqlbooster-ssh-tunnel.png" width="60%" alt="Wekan logo" />
|
||||
|
||||
# Scheduled backups to local or remote server
|
||||
|
||||
For below scheduled backup scripts, no info from above of this wiki page is required. Backup scripts below have the required settings.
|
||||
|
||||
This does backup of [Wekan+RocketChat snap databases](OAuth2) and php website etc.
|
||||
|
||||
If you need to backup some remote server or cloud, you can use scp, or read [rclone docs](https://rclone.org/docs/) about how to configure saving to some other remote server or cloud.
|
||||
|
||||
The following .sh bash scripts are added as root user to your `/root/` directory.
|
||||
|
||||
Backups are created to subdirectories of `/root/backups/`.
|
||||
|
||||
Cron is used to schedule backups, for example once a day.
|
||||
|
||||
1. To add bash scripts, you change to root user with this command, and your sudo or root password.
|
||||
```bash
|
||||
sudo su
|
||||
```
|
||||
2. Use nano editor for editing cron. If you don't have it installed, type:
|
||||
```bash
|
||||
apt install nano
|
||||
```
|
||||
3. Then we set text editor to be nano. Otherwise it probably uses vi, that is harder to use.
|
||||
```bash
|
||||
export EDITOR=nano
|
||||
```
|
||||
4. Now we start editing cron scheduler.
|
||||
```bash
|
||||
crontab -e
|
||||
```
|
||||
For more info how to make cron time, see https://crontab.guru
|
||||
|
||||
In this example, we set backups every day at 04:00, then it runs backup.sh script, and saves output of the backup commands to the bottom of textfile backup.log.txt
|
||||
```bash
|
||||
# m h dom mon dow command
|
||||
0 4 * * * /root/backup.sh >> /root/backup.log.txt 2>&1
|
||||
```
|
||||
- For changing to `/root` directory, type: `cd /root`
|
||||
- for editing backup.sh file, type: `nano backup.sh`
|
||||
- For saving in nano, press Ctrl-o Enter
|
||||
- For exiting nano, press Ctrl-x Enter
|
||||
- Set every .sh file as executeable, for example: `chmod +x backup.sh`
|
||||
|
||||
This is content of `backup.sh` script. It runs all the other backup scripts.
|
||||
If you do not need to backup rocketchat or website, or do not need to use rclone,
|
||||
you don't need to add those command lines at all.
|
||||
```bash
|
||||
cd /root
|
||||
./backup-wekan.sh
|
||||
./backup-rocketchat.sh
|
||||
./backup-website.sh
|
||||
rclone move backups cloudname:backup.example.com
|
||||
```
|
||||
More about rclone:
|
||||
|
||||
/root/rclone-ls-all.sh , shows directory contests at cloud:
|
||||
```bash
|
||||
rclone lsd cloudname:
|
||||
```
|
||||
In this example, cron does run backup scripts as root.
|
||||
This is if you edit cron with command `crontab -e` as root user,
|
||||
so it edits the cron of root user.
|
||||
|
||||
If mongodump command works as normal user for you, you could instead
|
||||
run backups as normal user, by exiting root user with `exit` and
|
||||
then as normal user editing cron with `crontab -e`.
|
||||
You can also list current cron with command `crontab -l`.
|
||||
|
||||
If you like to backup Wekan snap settings with this command, then it
|
||||
only works with sudo at front, or as a root user without sudo at front.
|
||||
```bash
|
||||
sudo snap get wekan > snap-settings.txt
|
||||
```
|
||||
|
||||
This below is backup script for backing up Wekan.
|
||||
|
||||
/root/backup-wekan.sh
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
makeDump()
|
||||
{
|
||||
|
||||
# Backups will be created below this directory.
|
||||
backupdir="/root/backups/wekan"
|
||||
|
||||
# Gets the version of the snap.
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
# Gets current time to variable "now"
|
||||
now=$(date +'%Y-%m-%d_%H.%M.%S')
|
||||
|
||||
# Creates new backup directory like BACKUPDIR/BACKUPVERSIO-TIMENOW
|
||||
mkdir -p $backupdir/$version-$now
|
||||
|
||||
# Targets the dump file.
|
||||
#dump=$"/snap/wekan/$version/bin/mongodump"
|
||||
|
||||
# Changes to backup directory
|
||||
cd $backupdir/$version-$now
|
||||
|
||||
# Backup Caddy settings
|
||||
snap get wekan > snap-settings.txt
|
||||
|
||||
# Show text that database backup is in progress
|
||||
printf "\nThe database backup is in progress.\n\n"
|
||||
|
||||
# Backup to current directory, creates subdirectory called "dump"
|
||||
# with database dump files
|
||||
mongodump --port 27019
|
||||
|
||||
# Change diretory (=cd) to parent directory
|
||||
cd ..
|
||||
|
||||
# Show text "Makes the tar.gz archive file"
|
||||
printf "\nMakes the tar.gz archive file.\n"
|
||||
|
||||
# Creates tar.gz archive file. This works similarly like creating .zip file.
|
||||
tar -zcvf $version-$now.tar.gz $version-$now
|
||||
|
||||
# Delete temporary files that have already been
|
||||
# compressed to above tar.gz file
|
||||
rm -rf $version-$now
|
||||
|
||||
# Shows text "Backup done."
|
||||
printf "\nBackup done.\n"
|
||||
|
||||
# Show where backup archive file is.
|
||||
echo "Backup is archived to .tar.gz file at $backupdir/${version}-${now}.tar.gz"
|
||||
}
|
||||
|
||||
# Checks is the user is sudo/root
|
||||
if [ "$UID" -ne "0" ]
|
||||
then
|
||||
echo "This program must be launched with sudo/root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Starts
|
||||
makeDump
|
||||
```bash
|
||||
/root/backup-rocketchat.sh
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
makeDump()
|
||||
{
|
||||
|
||||
backupdir="/root/backups/rocketchat"
|
||||
|
||||
# Gets the version of the snap.
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
# Prepares.
|
||||
now=$(date +'%Y-%m-%d_%H.%M.%S')
|
||||
mkdir -p $backupdir/$version-$now
|
||||
|
||||
# Targets the dump file.
|
||||
dump=$"/snap/wekan/$version/bin/mongodump"
|
||||
|
||||
# Makes the backup.
|
||||
cd $backupdir/$version-$now
|
||||
printf "\nThe database backup is in progress.\n\n"
|
||||
$dump --port 27017
|
||||
|
||||
# Makes the tar.gz file.
|
||||
cd ..
|
||||
printf "\nMakes the tar.gz file.\n"
|
||||
tar -zcvf $version-$now.tar.gz $version-$now
|
||||
|
||||
# Cleanups
|
||||
rm -rf $version-$now
|
||||
|
||||
# End.
|
||||
printf "\nBackup done.\n"
|
||||
echo "Backup is archived to .tar.gz file at $backupdir/${version}-${now}.tar.gz"
|
||||
}
|
||||
|
||||
# Checks is the user is sudo/root
|
||||
if [ "$UID" -ne "0" ]
|
||||
then
|
||||
echo "This program must be launched with sudo/root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Starts
|
||||
makeDump
|
||||
```
|
||||
/root/backup-website.sh
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
makeDump()
|
||||
{
|
||||
|
||||
backupdir="/root/backups/example.com"
|
||||
|
||||
# Gets the version of the snap.
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
# Prepares.
|
||||
now=$(date +'%Y-%m-%d_%H.%M.%S')
|
||||
mkdir -p $backupdir/$version-$now
|
||||
|
||||
# Makes the backup.
|
||||
cd $backupdir/$version-$now
|
||||
printf "\nThe file backup is in progress.\n\n"
|
||||
|
||||
# Makes the tar.gz file.
|
||||
cd ..
|
||||
printf "\nMakes the tar.gz file.\n"
|
||||
cp -pR /var/snap/wekan/common/example.com $version-$now
|
||||
tar -zcvf $version-$now.tar.gz $version-$now
|
||||
|
||||
# Cleanups
|
||||
rm -rf $version-$now
|
||||
|
||||
# End.
|
||||
printf "\nBackup done.\n"
|
||||
echo "Backup is archived to .tar.gz file at $backupdir/${version}-${now}.tar.gz"
|
||||
}
|
||||
|
||||
# Checks is the user is sudo/root
|
||||
if [ "$UID" -ne "0" ]
|
||||
then
|
||||
echo "This program must be launched with sudo/root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Starts
|
||||
makeDump
|
||||
```
|
||||
/var/snap/wekan/common/Caddyfile
|
||||
```bash
|
||||
chat.example.com {
|
||||
proxy / localhost:3000 {
|
||||
websocket
|
||||
transparent
|
||||
}
|
||||
}
|
||||
|
||||
https://boards.example.com {
|
||||
proxy / localhost:3001 {
|
||||
websocket
|
||||
transparent
|
||||
}
|
||||
}
|
||||
|
||||
example.com {
|
||||
root /var/snap/wekan/common/example.com
|
||||
fastcgi / /var/run/php/php7.0-fpm.sock php
|
||||
}
|
||||
|
||||
matomo.example.com {
|
||||
root /var/snap/wekan/common/matomo.example.com
|
||||
fastcgi / /var/run/php/php7.0-fpm.sock php
|
||||
}
|
||||
|
||||
# Example CloudFlare free wildcard SSL Origin Certificate, there is example.com.pem at certificates directory with private key at to and cert at bottom.
|
||||
http://example.com https://example.com {
|
||||
tls {
|
||||
load /var/snap/wekan/common/certificates
|
||||
alpn http/1.1
|
||||
}
|
||||
root /var/snap/wekan/common/example.com
|
||||
browse
|
||||
}
|
||||
|
||||
static.example.com {
|
||||
root /var/snap/wekan/common/static.example.com
|
||||
}
|
||||
```
|
||||
|
||||
## Related talk about MongoDB backup
|
||||
|
||||
Related talk, search for "mongodb" this page:
|
||||
https://fosdem.org/2020/schedule/events/
|
||||
|
||||
There is this:
|
||||
|
||||
Percona Backup for MongoDB: Status and Plans
|
||||
|
||||
Open Source solution for consistent backups of multi-shard MongoDB
|
||||
|
||||
- [Slides](https://fosdem.org/2020/schedule/event/perconamongodb/attachments/slides/3768/export/events/attachments/perconamongodb/slides/3768/Percona_Backup_for_MongoDB.pdf)
|
||||
- [Video webm](https://video.fosdem.org/2020/UD2.119/perconamongodb.webm)
|
||||
- [Same Video mp4](https://video.fosdem.org/2020/UD2.119/perconamongodb.mp4)
|
||||
|
||||
## Related Sandstorm issue
|
||||
|
||||
[Creating a backup while the grain is running could cause corruption](https://github.com/sandstorm-io/sandstorm/issues/3186).
|
||||
|
||||
## Combining old and new Wekan version data
|
||||
|
||||
Note: Do mongodump/mongorestore only when Wekan is stopped: wekan.wekan (Snap) or wekan-app (Docker).
|
||||
|
||||
1. From new Wekan export all boards to Wekan JSON.
|
||||
2. Backup new Wekan with mongodump.
|
||||
3. Backup old Wekan with mongodump.
|
||||
4. Restore old Wekan data to new Wekan with mongorestore.
|
||||
5. Restore new Wekan JSON exported boards by importing them.
|
||||
|
||||
## Rescuing board that does not load
|
||||
|
||||
Wekan web UI Import/Export JSON does not have all content currently. To upgrade from old Wekan version, use mongodump/mongorestore to newest Wekan, like described below.
|
||||
|
||||
To import big JSON file, on Linux you can use xclip to copy textfile to clipboard:
|
||||
```bash
|
||||
sudo apt-get install xclip
|
||||
cat board.json | xclip -se c
|
||||
```
|
||||
Then paste to webbrowser Wekan Add Board / Import / From previous export.
|
||||
|
||||
You can [save all MongoDB database content as JSON files](Export-from-Wekan-Sandstorm-grain-.zip-file). Files are base64 encoded in JSON files.
|
||||
|
||||
Export board to Wekan JSON, and import as Wekan JSON can make some part of board to load, but you should check is some data missing.
|
||||
|
||||
With Wekan Snap, you can use [nosqlbooster GUI](https://nosqlbooster.com/downloads) to login through SSH to Wekan server localhost port 27019 and browse data.
|
||||
|
||||
You could use [daff](https://github.com/paulfitz/daff) to compare tables.
|
||||
|
||||
## Using Snap Mongo commands on your bash CLI
|
||||
|
||||
Add to your `~/.bashrc`
|
||||
```bash
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
export PATH="$PATH:/snap/wekan/current/bin"
|
||||
```
|
||||
Then you can backup:
|
||||
```bash
|
||||
mongodump --port 27019
|
||||
```
|
||||
And restore:
|
||||
```bash
|
||||
mongorestore --drop --port 27019
|
||||
```
|
||||
|
||||
## MongoDB shell on Wekan Snap
|
||||
|
||||
mongoshell.sh
|
||||
```bash
|
||||
#/bin/bash
|
||||
export LC_ALL=C
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
mongo=$"/snap/wekan/$version/bin/mongo"
|
||||
$mongo --port 27019
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
# Snap backup-restore v2
|
||||
|
||||
Originally from https://github.com/wekan/wekan-snap/issues/62#issuecomment-470622601
|
||||
|
||||
## Backup
|
||||
|
||||
wekan-backup.sh
|
||||
```bash
|
||||
#!/bin/bash
|
||||
export LC_ALL=C
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
now=$(date +"%Y%m%d-%H%M%S")
|
||||
parent_dir="/data/backups/wekan"
|
||||
backup_dir="${parent_dir}/${now}"
|
||||
log_file="${parent_dir}/backup-progress.log.${now}"
|
||||
|
||||
error () {
|
||||
printf "%s: %s\n" "$(basename "${BASH_SOURCE}")" "${1}" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
trap 'error "An unexpected error occurred."' ERR
|
||||
|
||||
take_backup () {
|
||||
mkdir -p "${backup_dir}"
|
||||
|
||||
cd "${backup_dir}"
|
||||
|
||||
/snap/wekan/$version/bin/mongodump --quiet --port 27019
|
||||
|
||||
cd ..
|
||||
|
||||
tar -zcf "${now}.tar.gz" "${now}"
|
||||
|
||||
rm -rf "${now}"
|
||||
}
|
||||
|
||||
printf "\n======================================================================="
|
||||
printf "\nWekan Backup"
|
||||
printf "\n======================================================================="
|
||||
printf "\nBackup in progress..."
|
||||
|
||||
take_backup 2> "${log_file}"
|
||||
|
||||
if [[ -s "${log_file}" ]]
|
||||
then
|
||||
printf "\nBackup failure! Check ${log_file} for more information."
|
||||
printf "\n=======================================================================\n\n"
|
||||
else
|
||||
rm "${log_file}"
|
||||
printf "...SUCCESS!\n"
|
||||
printf "Backup created at ${backup_dir}.tar.gz"
|
||||
printf "\n=======================================================================\n\n"
|
||||
fi
|
||||
```
|
||||
wekan-restore.sh
|
||||
```bash
|
||||
#!/bin/bash
|
||||
export LC_ALL=C
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
|
||||
makesRestore()
|
||||
{
|
||||
file=$1
|
||||
|
||||
ext=$"$(basename $file)"
|
||||
parentDir=$"${file:0:${#file}-${#ext}}"
|
||||
cd "${parentDir}"
|
||||
|
||||
printf "\nMakes the untar of the archive.\n"
|
||||
|
||||
tar -zxvf "${file}"
|
||||
file="${file:0:${#file}-7}"
|
||||
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
restore=$"/snap/wekan/${version}/bin/mongorestore"
|
||||
|
||||
printf "\nThe database restore is in progress.\n\n"
|
||||
|
||||
## Only if you get errors about existing indexes, use this below instead:
|
||||
## $restore --quiet --drop --noIndexRestore -d wekan --port 27019 "${file}/dump/wekan"
|
||||
|
||||
$restore --quiet --drop -d wekan --port 27019 "${file}/dump/wekan"
|
||||
|
||||
rm -rf "${file}"
|
||||
|
||||
printf "\nRestore done.\n"
|
||||
}
|
||||
|
||||
makesRestore $1
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
# Snap backup-restore v1
|
||||
|
||||
## Backup script for MongoDB Data, if running Snap MongoDB at port 27019
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
makeDump()
|
||||
{
|
||||
# Gets the version of the snap.
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
# Prepares.
|
||||
now=$(date +'%Y-%m-%d_%H.%M.%S')
|
||||
mkdir -p /var/backups/wekan/$version-$now
|
||||
|
||||
# Targets the dump file.
|
||||
dump=$"/snap/wekan/$version/bin/mongodump"
|
||||
|
||||
# Makes the backup.
|
||||
cd /var/backups/wekan/$version-$now
|
||||
printf "\nThe database backup is in progress.\n\n"
|
||||
$dump --port 27019
|
||||
|
||||
# Makes the tar.gz file.
|
||||
cd ..
|
||||
printf "\nMakes the tar.gz file.\n"
|
||||
tar -zcvf $version-$now.tar.gz $version-$now
|
||||
|
||||
# Cleanups
|
||||
rm -rf $version-$now
|
||||
|
||||
# End.
|
||||
printf "\nBackup done.\n"
|
||||
echo "Backup is archived to .tar.gz file at /var/backups/wekan/${version}-${now}.tar.gz"
|
||||
}
|
||||
|
||||
# Checks is the user is sudo/root
|
||||
if [ "$UID" -ne "0" ]
|
||||
then
|
||||
echo "This program must be launched with sudo/root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
# Starts
|
||||
makeDump
|
||||
|
||||
```
|
||||
|
||||
## Restore script for MongoDB Data, if running Snap MongoDB at port 27019 with a tar.gz archive.
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
makesRestore()
|
||||
{
|
||||
# Prepares the folder used for the backup.
|
||||
file=$1
|
||||
if [[ "$file" != *tar.gz* ]]
|
||||
then
|
||||
echo "The backup archive must be a tar.gz."
|
||||
exit -1
|
||||
fi
|
||||
|
||||
# Goes into the parent directory.
|
||||
ext=$"$(basename $file)"
|
||||
parentDir=$"${file:0:${#file}-${#ext}}"
|
||||
cd $parentDir
|
||||
|
||||
# Untar the archive.
|
||||
printf "\nMakes the untar of the archive.\n"
|
||||
tar -zxvf $file
|
||||
file="${file:0:${#file}-7}"
|
||||
|
||||
# Gets the version of the snap.
|
||||
version=$(snap list | grep wekan | awk -F ' ' '{print $3}')
|
||||
|
||||
# Targets the dump file.
|
||||
restore=$"/snap/wekan/$version/bin/mongorestore"
|
||||
|
||||
# Restores.
|
||||
printf "\nThe database restore is in progress.\n\n"
|
||||
## Only if you get errors about existing indexes, use this below instead:
|
||||
## $restore --drop --noIndexRestore wekan --port 27019 $file/dump/wekan
|
||||
$restore --drop wekan --port 27019 $file/dump/wekan
|
||||
printf "\nRestore done.\n"
|
||||
|
||||
# Cleanups
|
||||
rm -rf $file
|
||||
}
|
||||
|
||||
# Checks is the user is sudo/root.
|
||||
if [ "$UID" -ne "0" ]
|
||||
then
|
||||
echo "This program must be launched with sudo/root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
# Start.
|
||||
makesRestore $1
|
||||
|
||||
```
|
||||
|
||||
## Docker Backup and Restore
|
||||
|
||||
[Docker Backup and Restore](Export-Docker-Mongo-Data)
|
||||
|
||||
[Wekan Docker Upgrade](https://github.com/wekan/wekan-mongodb#backup-before-upgrading)
|
||||
|
||||
## Snap Backup
|
||||
|
||||
[Snap Backup and Restore](https://github.com/wekan/wekan-snap/wiki/Backup-and-restore)
|
||||
|
||||
[Wekan Snap upgrade](https://github.com/wekan/wekan-snap/wiki/Install#5-install-all-snap-updates-automatically-between-0200am-and-0400am)
|
||||
|
||||
## Sandstorm Backup
|
||||
|
||||
Download Wekan grain with arrow down download button to .zip file. You can restore it later.
|
||||
|
||||
[Export data from Wekan Sandstorm grain .zip file](Export-from-Wekan-Sandstorm-grain-.zip-file)
|
||||
|
||||
## <a name="cloudron">Cloudron
|
||||
|
||||
If those [Backup](Backup) ways are not easily found at [Cloudron](Cloudron), one way is to install [Redash](https://redash.io/) and then backup this way:
|
||||
|
||||
Redash works with this kind of queries:
|
||||
```json
|
||||
{
|
||||
"collection": "accountSettings",
|
||||
"query": {
|
||||
"type": 1
|
||||
},
|
||||
"fields": {
|
||||
"_id": 1,
|
||||
"booleanValue": 1,
|
||||
"createdAt": 1,
|
||||
"modifiedAt": 1,
|
||||
}
|
||||
}
|
||||
```
|
||||
So:
|
||||
|
||||
1) Create this kind of query:
|
||||
```json
|
||||
{
|
||||
"collection": "boards"
|
||||
}
|
||||
```
|
||||
|
||||
Later when you modify query, you can remove text like boards with double-click-with-mouse and delete-key-at-keyboard (or select characters with mouse and delete-key-at-keyboard), and then click collection/table >> button to insert name of next collection/table.
|
||||
|
||||
2) Click Save
|
||||
|
||||
3) Click Execute. This will cache query for use with REST API.
|
||||
|
||||
4) Click at right top `[...]` => `Show API key`
|
||||
|
||||
It looks like this:
|
||||
|
||||
https://redash.example.com/api/queries/1/results.json?api_key=...
|
||||
|
||||
5) Only when saving first collection/table, Save API key to text file script like this `dl.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
# Example: ./dl.sh boards
|
||||
|
||||
export APIKEY=https://redash.example.com/api/queries/1/results.json?api_key=...
|
||||
|
||||
curl -o $1.json $APIKEY
|
||||
```
|
||||
|
||||
6) Run save script like:
|
||||
```bash
|
||||
./dl.sh boards
|
||||
```
|
||||
Note: 1) Save 2) Execute => webbrowser can give this kind of timeout,
|
||||
but downloading with API script still works:
|
||||
|
||||
> wekan
|
||||
> Error running query: failed communicating with server. Please check your Internet connection and try again.
|
||||
|
||||
7) Repeat steps 1-4 and 6 for every collection/table like boards,cards, etc
|
||||
|
||||
8) Remove from downloaded .json files extra query related data, so that it is similar like [any other Wekan database backup JSON files](Export-from-Wekan-Sandstorm-grain-.zip-file)
|
||||
|
||||
9) Insert data to some other Wekan install with nosqlbooster like mentioned at page [Backup](Backup)
|
||||
|
||||
|
||||
155
docs/Backup/Rclone.md
Normal file
155
docs/Backup/Rclone.md
Normal file
|
|
@ -0,0 +1,155 @@
|
|||
# Upgrading
|
||||
|
||||
Newest WeKan uses MongoDB 6.0.3.
|
||||
|
||||
Current working way to upgrade is:
|
||||
|
||||
## Docker
|
||||
|
||||
1. Create Mongodump of your database like at [Backup page](https://github.com/wekan/wekan/wiki/Backup)
|
||||
2. Use newest WeKan docker-compose.yml from https://github.com/wekan/wekan with WRITABLE_PATH setup to Rclone/MinIO like below this page
|
||||
3. Mongorestore to MongoDB 6.0.3 database
|
||||
4. Use WeKan Admin Panel `Move all attachments to filesystem` button to move all attachments to MinIO.
|
||||
5. Create Mongodump, that this time does not have any attachments, and is much smaller.
|
||||
6. Mongorestore to MongoDB 6.0.3 database.
|
||||
7. Start WeKan.
|
||||
|
||||
## Snap
|
||||
|
||||
Similar Mongodump and Mongorestore above, with these info:
|
||||
- https://github.com/wekan/wekan-snap/wiki/Candidate-WeKan-Snap-Manual-Upgrade
|
||||
- https://github.com/wekan/wekan/issues/4780
|
||||
|
||||
# Introduction
|
||||
|
||||
With Rclone https://rclone.org , it's possible to use many cloud filesystems, like AWS S3, MinIO https://min.io , etc.
|
||||
|
||||
Newest WeKan has features to move files between MongoDB GridFS and filesystem.
|
||||
|
||||
Instead of filesystem, Rclone mounted cloud filesystem directory can be used, like MinIO.
|
||||
|
||||
## Screenshot 1: Move to filesystem button moves to cloud filesystem like MinIO. S3 button does not work yet.
|
||||
|
||||
Note: In some cases, only buttons `Move all attachments` at top are visible. In some other cases, there is more visible, like moving all attachments of board, etc, maybe when some have been already moved.
|
||||
|
||||
<img src="https://wekan.github.io/rclone/wekan-admin-panel.png" width="100%" alt="Wekan Admin Panel file move" />
|
||||
|
||||
## Screenshot 2: Files at MinIO after moving all to filesystem
|
||||
|
||||
<img src="https://wekan.github.io/rclone/minio1.png" width="100%" alt="MinIO 1" />
|
||||
|
||||
## Screenshot 3: Files at MinIO after moving all to filesystem
|
||||
|
||||
<img src="https://wekan.github.io/rclone/minio2.png" width="100%" alt="MinIO 2" />
|
||||
|
||||
## Rclone config
|
||||
|
||||
Create config for example with command:
|
||||
```
|
||||
rclone config
|
||||
```
|
||||
At S3 providers, there is MinIO at https://rclone.org/s3/#minio
|
||||
|
||||
`/root/.config/rclone/rclone.conf`
|
||||
|
||||
```
|
||||
[aws]
|
||||
type = s3
|
||||
provider = AWS
|
||||
access_key_id = ACCESS-KEY-HERE
|
||||
secret_access_key = SECRET-KEY-HERE
|
||||
region = eu-north-1
|
||||
location_constraint = eu-north-1
|
||||
acl = private
|
||||
|
||||
[minio]
|
||||
type = s3
|
||||
provider = Minio
|
||||
access_key_id = ACCESS-KEY-HERE
|
||||
secret_access_key = SECRET-KEY-HERE
|
||||
endpoint = http://192.168.0.100:9000
|
||||
acl = private
|
||||
```
|
||||
## Listing files with Rclone
|
||||
```
|
||||
rclone ls aws:
|
||||
|
||||
rclone ls minio:
|
||||
```
|
||||
## MinIO config
|
||||
|
||||
https://min.io
|
||||
|
||||
Running MinIO server binary, storing files at `/home/wekan/minio/data`
|
||||
```
|
||||
MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password ./minio server /home/wekan/minio/data --console-address ":9001"
|
||||
```
|
||||
Create bucket, access key, secret key. Then connect them with `mc` command installed from https://min.io website, here bucket wekan:
|
||||
```
|
||||
mc config host add wekan http://192.168.0.100:9000 ACCESS-KEY-HERE SECRET-KEY-HERE
|
||||
```
|
||||
## Snap
|
||||
This uses minimal cache to make it work. Note that Rclone stays at foreground running, showing related messages, it does not daemonize to background.
|
||||
```
|
||||
sudo su
|
||||
|
||||
rclone mount minio:wekan/files /var/snap/wekan/common/files --vfs-cache-mode minimal
|
||||
```
|
||||
Sometimes it shows this kind of message:
|
||||
```
|
||||
2022/12/27 10:22:13 NOTICE: S3 bucket wekan path files: Streaming uploads using chunk size 5Mi will have maximum file size of 48.828Gi
|
||||
```
|
||||
## Docker
|
||||
|
||||
Rclone, short story:
|
||||
```
|
||||
sudo su
|
||||
|
||||
rclone mount minio:wekan/files /var/lib/docker/volumes/wekan_wekan-files/_data --vfs-cache-mode minimal
|
||||
```
|
||||
Longer story:
|
||||
|
||||
`docker-compose.yml` at https://github.com/wekan/wekan
|
||||
```
|
||||
- WRITABLE_PATH=/data
|
||||
```
|
||||
Docker volumes are there:
|
||||
```
|
||||
sudo ls /var/lib/docker/volumes/
|
||||
|
||||
wekan_wekan-db-dump
|
||||
wekan_wekan-db
|
||||
wekan_wekan-files
|
||||
```
|
||||
Directories there are:
|
||||
```
|
||||
/var/lib/docker/volumes/_data/attachments
|
||||
/var/lib/docker/volumes/_data/avatars
|
||||
```
|
||||
## Bundle or Source
|
||||
|
||||
Set `WRITABLE_PATH`like in Docker example above.
|
||||
|
||||
## Development
|
||||
|
||||
In Progress direct support without Rclone, if getting it working sometime.
|
||||
|
||||
Meteor-Files currently uses official AWS-SDK that does not yet directly support custom endpoint like MinIO:
|
||||
- https://github.com/veliovgroup/Meteor-Files/issues/862
|
||||
- https://github.com/veliovgroup/Meteor-Files/blob/master/docs/aws-s3-integration.md
|
||||
- https://github.com/wekan/wekan/issues/142#issuecomment-1365249290
|
||||
|
||||
MinIO code examples:
|
||||
- https://github.com/minio/minio-js/tree/master/examples
|
||||
|
||||
Related RocketChat docs about MinIO:
|
||||
- https://docs.rocket.chat/guides/administration/admin-panel/settings/file-upload/minio
|
||||
- https://github.com/RocketChat/Rocket.Chat/tree/develop/apps/meteor/app/file-upload
|
||||
- https://github.com/RocketChat/Rocket.Chat/pulls?q=is%3Apr+minio+is%3Aclosed
|
||||
|
||||
Storage path during developing WeKan, for example:
|
||||
```
|
||||
./rebuild-wekan.sh
|
||||
|
||||
rclone mount minio:wekan/files /home/wekan/repos/wekan/.meteor/local/.build-garbage-uslyxi.uokel/programs --vfs-cache-mode minimal
|
||||
```
|
||||
165
docs/Backup/Repair-MongoDB.md
Normal file
165
docs/Backup/Repair-MongoDB.md
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
## Repair Snap WeKan
|
||||
|
||||
```
|
||||
sudo su
|
||||
|
||||
snap stop wekan
|
||||
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
|
||||
export PATH="/snap/wekan/current/bin:$PATH"
|
||||
|
||||
mongod --dbpath "/var/snap/wekan/common" --repair >> /root/repairlog.txt
|
||||
|
||||
snap start wekan
|
||||
|
||||
exit
|
||||
```
|
||||
|
||||
Logs from Snap:
|
||||
|
||||
https://github.com/wekan/wekan/issues/5073
|
||||
|
||||
## Repair Docker WeKan
|
||||
|
||||
At step 12 below is actual repair MongoDB command.
|
||||
|
||||
Some customer of [WeKan Commercial Support](https://wekan.team/commercial-support/) ordered restore and repair of WeKan MongoDB database at UCS appliance. This was needed when changing back to previous UCS 4.x major version, when in new major UCS 5.x version Docker containers were all the time crashing and restarting, or something else got broken. Here are commands that were used while repairing.
|
||||
|
||||
Similarly, MongoDB may require repair, if:
|
||||
- MongoDB does not recover from sudden power failure cleanly
|
||||
- Disk comes full, that is worst case. Recommended are daily backups, in this case sometimes repair does not work well enough.
|
||||
|
||||
1. SSH to UCS appliance as root:
|
||||
```
|
||||
ssh root@192.168.0.100
|
||||
```
|
||||
2. If backup or raw database files is at external USB harddrive, look where it is inserted:
|
||||
```
|
||||
ls /dev
|
||||
|
||||
fdisk -l /dev/sdb
|
||||
|
||||
fdisk -l /dev/sdc
|
||||
|
||||
fdisk -l /dev/sdd
|
||||
```
|
||||
2. If external USB drive has NTFS file format, and mount complains about not having NTFS support, install NTFS support to Linux:
|
||||
```
|
||||
apt install ntfs-3g
|
||||
```
|
||||
3. Mount USB drive to /mnt
|
||||
```
|
||||
mount /dev/sdd1 /mnt
|
||||
```
|
||||
4. Stop WeKan Docker containers. If copying raw database files, WeKan and MongoDB should not be running, so that MongoDB would not be even more corrupted. (If MongoDB is running, and you have mongodump backup, you can [Backup with mongorestore](Backup)
|
||||
```
|
||||
docker stop wekan-db wekan-app
|
||||
```
|
||||
5. Find WeKan database raw files. MongoDB has various database engines, like WiredTiger, how to save raw data compressed, or other formats. WiredTiger is file format to save compressed MongoDB data. Similar like MySQL has ISAM, InnoDB etc.
|
||||
```
|
||||
apt -y install mlocate
|
||||
updatedb
|
||||
locate WiredTiger.wt
|
||||
```
|
||||
It can show for example USB drive and Docker container directory:
|
||||
```
|
||||
root@ucs:/mnt/wekan/data/db# locate WiredTiger.wt
|
||||
/mnt/wekan/data/db/WiredTiger.wt
|
||||
/var/lib/univention-appcenter/apps/wekan/data/db/WiredTiger.wt
|
||||
```
|
||||
6. Change to newly installed MongoDB data directory:
|
||||
```
|
||||
cd /var/lib/univention-appcenter/apps/wekan/data/db/
|
||||
```
|
||||
7. Look at directory files owner permissions:
|
||||
```
|
||||
ls -lah
|
||||
```
|
||||
There could be like this:
|
||||
```
|
||||
-rw-r--r-- 1 tss tss 1004 Jan 25 13:09 WiredTiger.turtle
|
||||
-rw-r--r-- 1 tss tss 392K Jan 25 13:09 WiredTiger.wt
|
||||
```
|
||||
8. Move that probably empty newly installed MongoDB data elsewhere:
|
||||
```
|
||||
mkdir /root/new-empty-wekan-data
|
||||
mv * /root/new-empty-wekan-data
|
||||
```
|
||||
9. Copy MongoDB raw database files from USB harddrive to docker container directory:
|
||||
```
|
||||
root@ucs-bdc:/var/lib/univention-appcenter/apps/wekan/data/db# cp -pR /mnt/wekan/data/db/* .
|
||||
```
|
||||
10. Like looked at step 7, change file owner permissions to be correct, and change directory
|
||||
```
|
||||
chown -R tss:tss *
|
||||
cd /root
|
||||
```
|
||||
11. Find mongod command:
|
||||
```
|
||||
locate /usr/bin/mongod
|
||||
```
|
||||
12. Repair Docker WeKan with version of Docker MongoDB that WeKan uses, change mongod path to below. Repairing logged to textfile.
|
||||
```
|
||||
root@ucs:~# /var/lib/docker/overlay2/7b58483a16a2f67ee50486c00ec669940f7a95d460ee8188966fee0096e81fa2/diff/usr/bin/mongod --dbpath "/var/lib/univention-appcenter/apps/wekan/data/db" --repair >> repairlog.txt
|
||||
```
|
||||
13. Look what containers are running:
|
||||
```
|
||||
docker ps
|
||||
```
|
||||
14. Look what containers are not running:
|
||||
```
|
||||
docker ps -a
|
||||
```
|
||||
15. Start MongoDB first:
|
||||
```
|
||||
docker start wekan-db
|
||||
```
|
||||
16. Look what happends at MongoDB container:
|
||||
```
|
||||
docker logs wekan-db
|
||||
```
|
||||
17. Start WeKan next.
|
||||
```
|
||||
docker start wekan-app
|
||||
```
|
||||
18. Look what happends at MongoDB container.
|
||||
```
|
||||
docker logs wekan-app
|
||||
```
|
||||
19. Check are both wekan-db and wekan-app containers running, and not restarting:
|
||||
```
|
||||
docker ps
|
||||
```
|
||||
20. Try to login to WeKan with webbrowser.
|
||||
21. Backup WeKan database now after repair:
|
||||
```
|
||||
docker exec -it wekan-db bash
|
||||
cd /data
|
||||
rm -rf dump
|
||||
mongodump
|
||||
exit
|
||||
docker cp wekan-db:/data/dump .
|
||||
zip -r wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip dump
|
||||
```
|
||||
Now backup is at wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip file.
|
||||
|
||||
22. At your local computer terminal (not at UCS server), transfer file to your local computer with scp:
|
||||
```
|
||||
scp root@192.168.0.100:/root/wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip .
|
||||
```
|
||||
23. Copy backup to external USB harddrive, change YEAR-MONTH-DATE-TIME-HERE to current date and time:
|
||||
```
|
||||
cp /root/wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip /mnt/
|
||||
```
|
||||
24. Look what drives are mounted:
|
||||
```
|
||||
df -h
|
||||
```
|
||||
25. Sync and save unwritted data to disk, and unmount external USB harddrive safely:
|
||||
```
|
||||
sync
|
||||
umount /dev/sdd1
|
||||
```
|
||||
26. Remove external USB harddisk from server.
|
||||
27. Login to WeKan, check do all WeKan boards work. In this case, all did work.
|
||||
15
docs/Backup/Upgrade.md
Normal file
15
docs/Backup/Upgrade.md
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
|
||||
```
|
||||
sudo snap stop wekan.wekan
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
|
||||
export PATH2=$PATH
|
||||
export PATH=/snap/wekan/current/bin:$PATH
|
||||
mongodump --port 27019
|
||||
sudo snap get wekan > snap-settings.txt
|
||||
sudo snap stop wekan.mongodb
|
||||
sudo mv /var/snap/wekan/common .
|
||||
sudo mkdir /var/snap/wekan/common
|
||||
sudo snap refresh wekan --channel=latest/candidate
|
||||
```
|
||||
|
||||
To be continued...
|
||||
Loading…
Add table
Add a link
Reference in a new issue