Compare commits

...

68 Commits

Author SHA1 Message Date
3006abac3e Merge pull request #192 from jkaninda/dependabot/docker/golang-1.24.4
Some checks failed
Tests / test (push) Has been cancelled
Lint / Run on Ubuntu (push) Has been cancelled
chore(deps): bump golang from 1.24.3 to 1.24.4
2025-06-09 19:53:25 +02:00
dependabot[bot]
3a16f6929c chore(deps): bump golang from 1.24.3 to 1.24.4
Bumps golang from 1.24.3 to 1.24.4.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.24.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-09 10:17:20 +00:00
2f3ed7d0d8 Merge pull request #191 from jkaninda/dependabot/docker/alpine-3.22.0
chore(deps): bump alpine from 3.21.3 to 3.22.0
2025-06-06 20:01:27 +02:00
dependabot[bot]
a92bba05e4 chore(deps): bump alpine from 3.21.3 to 3.22.0
Bumps alpine from 3.21.3 to 3.22.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: 3.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-02 11:09:07 +00:00
7b565d54bd Merge pull request #190 from jkaninda/dependabot/docker/golang-1.24.3
Some checks failed
Lint / Run on Ubuntu (push) Successful in 18m52s
Tests / test (push) Failing after 1m14s
chore(deps): bump golang from 1.24.2 to 1.24.3
2025-05-14 23:03:58 +02:00
dependabot[bot]
d10321dac6 chore(deps): bump golang from 1.24.2 to 1.24.3
Bumps golang from 1.24.2 to 1.24.3.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.24.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-12 09:54:31 +00:00
d151c50caa Merge pull request #189 from jkaninda/nightly
Some checks failed
Tests / test (push) Has been cancelled
Lint / Run on Ubuntu (push) Has been cancelled
Nightly
2025-05-11 20:10:05 +02:00
07ecec57b3 Add why use MySQL-BKUP 2025-05-11 20:09:19 +02:00
bc4aab6ed0 Add Docker labels 2025-05-11 20:05:16 +02:00
12c17c18d6 doc: update key features (#188)
Some checks failed
Tests / test (push) Has been cancelled
Deploy Documenation site to GitHub Pages / build (push) Has been cancelled
Lint / Run on Ubuntu (push) Has been cancelled
Deploy Documenation site to GitHub Pages / deploy (push) Has been cancelled
2025-05-10 17:47:12 +02:00
80cd70e153 Merge pull request #187 from jkaninda/dependabot/docker/golang-1.24.2
Some checks failed
Lint / Run on Ubuntu (push) Successful in 18m40s
Tests / test (push) Failing after 25s
chore(deps): bump golang from 1.24.1 to 1.24.2
2025-04-08 19:29:23 +02:00
dependabot[bot]
80abdb4299 chore(deps): bump golang from 1.24.1 to 1.24.2
Bumps golang from 1.24.1 to 1.24.2.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.24.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 10:37:49 +00:00
72b21f00e6 Merge pull request #186 from jkaninda/nightly
Some checks failed
Lint / Run on Ubuntu (push) Successful in 18m44s
Tests / test (push) Failing after 18s
Nightly
2025-03-17 09:09:09 +01:00
3439c74257 Merge branch 'main' of github.com:jkaninda/mysql-bkup into nightly 2025-03-17 09:05:24 +01:00
4110ff4e64 Merge branch 'docs' into nightly 2025-03-17 09:05:16 +01:00
cb299a35bf enh: enhancement of logging 2025-03-17 09:05:04 +01:00
ba7c096bf3 Merge pull request #185 from jkaninda/docs
Some checks failed
Deploy Documenation site to GitHub Pages / build (push) Failing after 9m27s
Deploy Documenation site to GitHub Pages / deploy (push) Has been skipped
Lint / Run on Ubuntu (push) Successful in 18m44s
Tests / test (push) Failing after 20s
doc: update deployment examples
2025-03-16 11:50:32 +01:00
6ac8dcef9e doc: update deployment examples
All checks were successful
Lint / Run on Ubuntu (push) Successful in 18m39s
2025-03-16 11:49:36 +01:00
6bcc5d6bd4 Merge pull request #184 from jkaninda/docs
Some checks failed
Deploy Documenation site to GitHub Pages / build (push) Failing after 9m27s
Deploy Documenation site to GitHub Pages / deploy (push) Has been skipped
Lint / Run on Ubuntu (push) Successful in 18m36s
Tests / test (push) Failing after 18s
doc: update configuration deployment
2025-03-16 05:55:14 +01:00
fd35fabf97 doc: update configuration deployment 2025-03-16 05:45:12 +01:00
23cce10e8c Merge branch 'main' of github.com:jkaninda/mysql-bkup into nightly 2025-03-16 05:37:34 +01:00
e666466d27 fix: database name not set error when using flag -d (#183)
Some checks failed
Lint / Run on Ubuntu (push) Successful in 18m39s
Tests / test (push) Failing after 13s
* fix: database name not set error when using flag -d
2025-03-14 14:38:06 +01:00
36bca254a9 ci: Set -d flag for database name
Some checks failed
Build / docker (push) Failing after 8s
Lint / Run on Ubuntu (push) Successful in 18m37s
Tests / test (push) Failing after 1m0s
2025-03-14 14:34:25 +01:00
ad18d42145 fix: database name not set error when using flag -d 2025-03-14 14:32:38 +01:00
d9d44c2798 Merge pull request #182 from jkaninda/nightly
Some checks failed
Deploy Documenation site to GitHub Pages / build (push) Failing after 9m29s
Deploy Documenation site to GitHub Pages / deploy (push) Has been skipped
Lint / Run on Ubuntu (push) Successful in 18m40s
Tests / test (push) Failing after 2m19s
Nightly
2025-03-14 09:59:33 +01:00
300a592508 Merge branch 'main' of github.com:jkaninda/mysql-bkup into nightly
Some checks failed
Build / docker (push) Failing after 9s
Lint / Run on Ubuntu (push) Successful in 18m40s
Tests / test (push) Failing after 19s
2025-03-14 09:58:47 +01:00
be82e841e7 ci: set docker tests on main 2025-03-14 09:58:41 +01:00
a73a365ebf ci: set docker tests on main 2025-03-14 09:57:59 +01:00
75e965c0c5 Merge pull request #181 from jkaninda/nightly
doc: update reference
2025-03-14 09:55:02 +01:00
fc60ddb308 doc: update reference 2025-03-14 09:53:38 +01:00
573ef15ef3 Merge pull request #180 from jkaninda/nightly
ci: update  Github pages action
2025-03-14 09:50:49 +01:00
b1776d3689 ci: update Github pages action 2025-03-14 09:50:13 +01:00
376d47f738 Merge pull request #178 from jkaninda/nightly
feat: add backup all databases separately
2025-03-14 09:43:43 +01:00
eb6268f8ec ci: add Docker tests (#179) 2025-03-14 09:41:37 +01:00
731e2d789d ci: add go lint 2025-03-14 05:24:46 +01:00
6300a8f2dd feat: add backup all databases 2025-03-14 05:20:54 +01:00
cd827a9277 chore: comment code
Some checks failed
Build / docker (push) Failing after 16s
2025-03-13 14:44:22 +01:00
71cf3fae85 chore: improve log message 2025-03-13 14:26:32 +01:00
528282bbd4 feat: add backup all databases separately 2025-03-13 07:48:28 +01:00
002c93a796 Merge pull request #176 from jkaninda/dependabot/docker/golang-1.24.1
chore(deps): bump golang from 1.24.0 to 1.24.1
2025-03-12 16:29:14 +01:00
b6192f4c42 feat: add backup all databases
Some checks failed
Build / docker (push) Failing after 14s
2025-03-12 16:04:26 +01:00
d5061453b0 feat: add backup all databases 2025-03-12 15:50:30 +01:00
0bc7497512 fix: warning message when using MYSQL_PASSWORD env 2025-03-12 14:13:21 +01:00
489dfdf842 fix: backup error output 2025-03-12 13:27:31 +01:00
dependabot[bot]
907e70d552 chore(deps): bump golang from 1.24.0 to 1.24.1
Bumps golang from 1.24.0 to 1.24.1.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 09:25:40 +00:00
696477fe5c Merge pull request #175 from jkaninda/dependabot/go_modules/github.com/jkaninda/go-utils-0.1.1
chore(deps): bump github.com/jkaninda/go-utils from 0.0.0-20250122060806-26119182077a to 0.1.1
2025-02-27 12:33:17 +01:00
dependabot[bot]
56a8b51660 chore(deps): bump github.com/jkaninda/go-utils
Bumps [github.com/jkaninda/go-utils](https://github.com/jkaninda/go-utils) from 0.0.0-20250122060806-26119182077a to 0.1.1.
- [Release notes](https://github.com/jkaninda/go-utils/releases)
- [Commits](https://github.com/jkaninda/go-utils/commits/v0.1.1)

---
updated-dependencies:
- dependency-name: github.com/jkaninda/go-utils
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-24 10:27:41 +00:00
c76a00139c Merge pull request #172 from jkaninda/dependabot/go_modules/github.com/spf13/cobra-1.9.1
chore(deps): bump github.com/spf13/cobra from 1.8.1 to 1.9.1
2025-02-21 11:38:48 +01:00
0f43871765 Merge pull request #173 from jkaninda/dependabot/docker/golang-1.24.0
chore(deps): bump golang from 1.23.6 to 1.24.0
2025-02-21 11:38:37 +01:00
9ba6abe3f4 Merge pull request #174 from jkaninda/dependabot/docker/alpine-3.21.3
chore(deps): bump alpine from 3.21.2 to 3.21.3
2025-02-21 11:38:27 +01:00
dependabot[bot]
764583d88f chore(deps): bump alpine from 3.21.2 to 3.21.3
Bumps alpine from 3.21.2 to 3.21.3.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-17 10:11:17 +00:00
dependabot[bot]
dbf4dc596a chore(deps): bump golang from 1.23.6 to 1.24.0
Bumps golang from 1.23.6 to 1.24.0.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-17 10:11:10 +00:00
dependabot[bot]
06c89a9b78 chore(deps): bump github.com/spf13/cobra from 1.8.1 to 1.9.1
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.8.1 to 1.9.1.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.8.1...v1.9.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-17 10:01:09 +00:00
ec8bdd806c Merge pull request #171 from jkaninda/dependabot/docker/golang-1.23.6
chore(deps): bump golang from 1.23.5 to 1.23.6
2025-02-10 20:15:31 +01:00
dependabot[bot]
828b11c6dd chore(deps): bump golang from 1.23.5 to 1.23.6
Bumps golang from 1.23.5 to 1.23.6.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-10 09:34:49 +00:00
1d01e13909 Merge pull request #170 from jkaninda/nightly
chore: update helper func to check env with prefix or suffix for multi backups
2025-02-05 07:44:57 +01:00
bd65db2418 chore: update helper func to check env with prefix or suffix for multi backups
Some checks failed
Build / docker (push) Failing after 14m58s
2025-02-05 07:39:52 +01:00
75b809511e fix go lint
Some checks failed
Build / docker (push) Failing after 8s
2025-01-26 13:54:41 +01:00
fc028a2c55 feat: add multiple backup rescued mode for scheduled mode 2025-01-26 13:43:39 +01:00
7fa0c6a118 Merge pull request #169 from jkaninda/nightly
Some checks failed
Deploy Documenation site to GitHub Pages / build (push) Failing after 9m23s
Deploy Documenation site to GitHub Pages / deploy (push) Has been skipped
docs: add quick restore
2025-01-26 12:12:53 +01:00
661702a97e docs: add quick restore 2025-01-26 12:11:29 +01:00
dd5f33f17d Merge pull request #168 from jkaninda/nightly
Some checks failed
Deploy Documenation site to GitHub Pages / build (push) Failing after 9m28s
Deploy Documenation site to GitHub Pages / deploy (push) Has been skipped
Nightly
2025-01-25 09:36:19 +01:00
b7cdfebd9c chore: notification remove MAIL_USERNAME and MAIL_PASSWORD from required env
Some checks failed
Build / docker (push) Failing after 13s
2025-01-25 09:19:23 +01:00
4b93becdf2 feat: add Set default values from environment variables if not provided for multiple backup 2025-01-25 09:12:28 +01:00
748cccec58 Merge pull request #167 from jkaninda/nightly
feat: add backup duration
2025-01-22 07:23:29 +01:00
3e8bfabc44 feat: add backup duration
Some checks failed
Build / docker (push) Failing after 12s
2025-01-22 07:22:56 +01:00
777b59fd7c Merge pull request #166 from jkaninda/dependabot/docker/golang-1.23.5
chore(deps): bump golang from 1.23.4 to 1.23.5
2025-01-21 02:53:48 +01:00
dependabot[bot]
2b25f39c0a chore(deps): bump golang from 1.23.4 to 1.23.5
Bumps golang from 1.23.4 to 1.23.5.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-20 10:09:46 +00:00
33 changed files with 1215 additions and 390 deletions

View File

@@ -32,14 +32,14 @@ jobs:
working-directory: docs working-directory: docs
- name: Setup Pages - name: Setup Pages
id: pages id: pages
uses: actions/configure-pages@v2 uses: actions/configure-pages@v5
- name: Build with Jekyll - name: Build with Jekyll
working-directory: docs working-directory: docs
run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}" run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}"
env: env:
JEKYLL_ENV: production JEKYLL_ENV: production
- name: Upload artifact - name: Upload artifact
uses: actions/upload-pages-artifact@v1 uses: actions/upload-pages-artifact@v3
with: with:
path: 'docs/_site/' path: 'docs/_site/'
@@ -52,4 +52,4 @@ jobs:
steps: steps:
- name: Deploy to GitHub Pages - name: Deploy to GitHub Pages
id: deployment id: deployment
uses: actions/deploy-pages@v1 uses: actions/deploy-pages@v4

23
.github/workflows/lint.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: Lint
on:
push:
pull_request:
jobs:
lint:
name: Run on Ubuntu
runs-on: ubuntu-latest
steps:
- name: Clone the code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '~1.23'
- name: Run linter
uses: golangci/golangci-lint-action@v6
with:
version: v1.61

289
.github/workflows/tests.yml vendored Normal file
View File

@@ -0,0 +1,289 @@
name: Tests
on:
push:
branches:
- main
- nightly
pull_request:
branches:
- main
env:
IMAGE_NAME: mysql-bkup
jobs:
test:
runs-on: ubuntu-latest
services:
mysql:
image: mysql:9
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: testdb
MYSQL_USER: user
MYSQL_PASSWORD: password
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uuser -ppassword"
--health-interval=10s
--health-timeout=5s
--health-retries=5
mysql8:
image: mysql:8
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: testdb
MYSQL_USER: user
MYSQL_PASSWORD: password
ports:
- 3308:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uuser -ppassword"
--health-interval=10s
--health-timeout=5s
--health-retries=5
mysql5:
image: mysql:5
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: testdb
MYSQL_USER: user
MYSQL_PASSWORD: password
ports:
- 3305:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uuser -ppassword"
--health-interval=10s
--health-timeout=5s
--health-retries=5
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create Minio container
run: |
docker run -d --rm --name minio \
--network host \
-p 9000:9000 \
-e MINIO_ACCESS_KEY=minioadmin \
-e MINIO_SECRET_KEY=minioadmin \
-e MINIO_REGION_NAME="eu" \
minio/minio server /data
echo "Create Minio container completed"
- name: Install MinIO Client (mc)
run: |
curl -O https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
- name: Wait for MinIO to be ready
run: sleep 5
- name: Configure MinIO Client
run: |
mc alias set local http://localhost:9000 minioadmin minioadmin
mc alias list
- name: Create MinIO Bucket
run: |
mc mb local/backups
echo "Bucket backups created successfully."
# Build the Docker image
- name: Build Docker Image
run: |
docker buildx build --build-arg appVersion=test -t ${{ env.IMAGE_NAME }}:latest --load .
- name: Verify Docker images
run: |
docker images
- name: Wait for MySQL to be ready
run: |
docker run --rm --network host mysql:9 mysqladmin ping -h 127.0.0.1 -uuser -ppassword --wait
- name: Test restore
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest restore -f init.sql
echo "Database restore completed"
- name: Test restore Mysql8
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_PORT=3308 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest restore -f init.sql
echo "Test restore Mysql8 completed"
- name: Test restore Mysql5
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_PORT=3305 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest restore -f init.sql
echo "Test restore Mysql5 completed"
- name: Test backup
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest backup
echo "Database backup completed"
- name: Test backup Mysql8
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_PORT=3308 \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest backup
echo "Test backup Mysql8 completed"
- name: Test backup Mysql5
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_PORT=3305 \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest backup
echo "Test backup Mysql5 completed"
- name: Test encrypted backup
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e GPG_PASSPHRASE=password \
${{ env.IMAGE_NAME }}:latest backup -d testdb --disable-compression --custom-name encrypted-bkup
echo "Database encrypted backup completed"
- name: Test restore encrypted backup | testdb -> testdb2
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e GPG_PASSPHRASE=password \
-e DB_NAME=testdb2 \
${{ env.IMAGE_NAME }}:latest restore -f /backup/encrypted-bkup.sql.gpg
echo "Test restore encrypted backup completed"
- name: Test migrate database testdb -> testdb3
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e GPG_PASSPHRASE=password \
-e DB_NAME=testdb \
-e TARGET_DB_HOST=127.0.0.1 \
-e TARGET_DB_PORT=3306 \
-e TARGET_DB_NAME=testdb3 \
-e TARGET_DB_USERNAME=root \
-e TARGET_DB_PASSWORD=password \
${{ env.IMAGE_NAME }}:latest migrate
echo "Test migrate database testdb -> testdb3 completed"
- name: Test backup all databases
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=root \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest backup --all-databases
echo "Database backup completed"
- name: Test multiple backup
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e TESTDB2_DB_USERNAME=root \
-e TESTDB2_DB_PASSWORD=password \
-e TESTDB2_DB_HOST=127.0.0.1 \
${{ env.IMAGE_NAME }}:latest backup -c /backup/test_config.yaml
echo "Database backup completed"
- name: Test backup Minio (s3)
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
-e AWS_S3_ENDPOINT="http://127.0.0.1:9000" \
-e AWS_S3_BUCKET_NAME=backups \
-e AWS_ACCESS_KEY=minioadmin \
-e AWS_SECRET_KEY=minioadmin \
-e AWS_DISABLE_SSL="true" \
-e AWS_REGION="eu" \
-e AWS_FORCE_PATH_STYLE="true" ${{ env.IMAGE_NAME }}:latest backup -s s3 --custom-name minio-backup
echo "Test backup Minio (s3) completed"
- name: Test restore Minio (s3)
run: |
docker run --rm --name ${{ env.IMAGE_NAME }} \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
-e AWS_S3_ENDPOINT="http://127.0.0.1:9000" \
-e AWS_S3_BUCKET_NAME=backups \
-e AWS_ACCESS_KEY=minioadmin \
-e AWS_SECRET_KEY=minioadmin \
-e AWS_DISABLE_SSL="true" \
-e AWS_REGION="eu" \
-e AWS_FORCE_PATH_STYLE="true" ${{ env.IMAGE_NAME }}:latest restore -s s3 -f minio-backup.sql.gz
echo "Test backup Minio (s3) completed"
- name: Test scheduled backup
run: |
docker run -d --rm --name ${{ env.IMAGE_NAME }} \
-v ./migrations:/backup/ \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_USERNAME=user \
-e DB_PASSWORD=password \
-e DB_NAME=testdb \
${{ env.IMAGE_NAME }}:latest backup -e "@every 10s"
echo "Waiting for backup to be done..."
sleep 25
docker logs ${{ env.IMAGE_NAME }}
echo "Test scheduled backup completed"
# Cleanup: Stop and remove containers
- name: Clean up
run: |
docker stop ${{ env.IMAGE_NAME }} || true
docker rm ${{ env.IMAGE_NAME }} || true

View File

@@ -27,6 +27,7 @@ linters:
- gosimple - gosimple
- govet - govet
- ineffassign - ineffassign
# - lll
- misspell - misspell
- nakedret - nakedret
- prealloc - prealloc

View File

@@ -1,4 +1,4 @@
FROM golang:1.23.4 AS build FROM golang:1.24.4 AS build
WORKDIR /app WORKDIR /app
ARG appVersion="" ARG appVersion=""
@@ -10,7 +10,7 @@ RUN go mod download
# Build # Build
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-X 'github.com/jkaninda/mysql-bkup/utils.Version=${appVersion}'" -o /app/mysql-bkup RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-X 'github.com/jkaninda/mysql-bkup/utils.Version=${appVersion}'" -o /app/mysql-bkup
FROM alpine:3.21.2 FROM alpine:3.22.0
ENV TZ=UTC ENV TZ=UTC
ARG WORKDIR="/config" ARG WORKDIR="/config"
ARG BACKUPDIR="/backup" ARG BACKUPDIR="/backup"
@@ -18,9 +18,12 @@ ARG BACKUP_TMP_DIR="/tmp/backup"
ARG TEMPLATES_DIR="/config/templates" ARG TEMPLATES_DIR="/config/templates"
ARG appVersion="" ARG appVersion=""
ENV VERSION=${appVersion} ENV VERSION=${appVersion}
LABEL author="Jonas Kaninda" LABEL org.opencontainers.image.title="mysql-bkup"
LABEL version=${appVersion} LABEL org.opencontainers.image.description="A lightweight MySQL backup and restore tool"
LABEL github="github.com/jkaninda/mysql-bkup" LABEL org.opencontainers.image.licenses="MIT"
LABEL org.opencontainers.image.authors="Jonas Kaninda <me@jonaskaninda.com>"
LABEL org.opencontainers.image.version=${appVersion}
LABEL org.opencontainers.image.source="https://github.com/jkaninda/mysql-bkup"
RUN apk --update add --no-cache mysql-client mariadb-connector-c tzdata ca-certificates RUN apk --update add --no-cache mysql-client mariadb-connector-c tzdata ca-certificates
RUN mkdir -p $WORKDIR $BACKUPDIR $TEMPLATES_DIR $BACKUP_TMP_DIR && \ RUN mkdir -p $WORKDIR $BACKUPDIR $TEMPLATES_DIR $BACKUP_TMP_DIR && \

242
README.md
View File

@@ -3,6 +3,14 @@
**MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**. **MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**.
It supports a variety of storage options and ensures data security through GPG encryption. It supports a variety of storage options and ensures data security through GPG encryption.
MYSQL-BKUP is designed for seamless deployment on **Docker** and **Kubernetes**, simplifying MySQL backup, restoration, and migration across environments.
It is a lightweight, multi-architecture solution compatible with **Docker**, **Docker Swarm**, **Kubernetes**, and other container orchestration platforms.
[![Tests](https://github.com/jkaninda/mysql-bkup/actions/workflows/tests.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/tests.yml)
[![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml) [![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml)
[![Go Report](https://goreportcard.com/badge/github.com/jkaninda/mysql-bkup)](https://goreportcard.com/report/github.com/jkaninda/mysql-bkup) [![Go Report](https://goreportcard.com/badge/github.com/jkaninda/mysql-bkup)](https://goreportcard.com/report/github.com/jkaninda/mysql-bkup)
![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/jkaninda/mysql-bkup?style=flat-square) ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/jkaninda/mysql-bkup?style=flat-square)
@@ -11,9 +19,9 @@ It supports a variety of storage options and ensures data security through GPG e
## Features ## Features
- **Storage Options:** - **Flexible Storage Backends:**
- Local storage - Local filesystem
- AWS S3 or any S3-compatible object storage - Amazon S3 & S3-compatible storage (e.g., MinIO, Wasabi)
- FTP - FTP
- SSH-compatible storage - SSH-compatible storage
- Azure Blob storage - Azure Blob storage
@@ -33,16 +41,19 @@ It supports a variety of storage options and ensures data security through GPG e
- **Telegram** - **Telegram**
- **Email** - **Email**
## Use Cases ## 💡Use Cases
- **Automated Recurring Backups:** Schedule regular backups for MySQL databases. - **Scheduled Backups**: Automate recurring backups using Docker or Kubernetes.
- **Cross-Environment Migration:** Easily migrate your MySQL databases across different environments using supported storage options. - **Disaster Recovery:** Quickly restore backups to a clean MySQL instance.
- **Secure Backup Management:** Protect your data with GPG encryption. - **Database Migration**: Seamlessly move data across environments using the built-in `migrate` feature.
- **Secure Archiving:** Keep backups encrypted and safely stored in the cloud or remote servers.
Successfully tested on: ## ✅ Verified Platforms:
MYSQL-BKUP has been tested and runs successfully on:
- Docker - Docker
- Docker in Swarm mode - Docker Swarm
- Kubernetes - Kubernetes
- OpenShift - OpenShift
@@ -58,45 +69,75 @@ Successfully tested on:
- [PostgreSQL](https://github.com/jkaninda/pg-bkup) - [PostgreSQL](https://github.com/jkaninda/pg-bkup)
## Storage:
- Local
- AWS S3 or any S3 Alternatives for Object Storage
- SSH remote storage server
- FTP remote storage server
- Azure Blob storage
## Quickstart ## Quickstart
### Simple backup using Docker CLI ### Simple Backup Using Docker CLI
To run a one time backup, bind your local volume to `/backup` in the container and run the `backup` command: To perform a one-time backup, bind your local volume to `/backup` in the container and run the `backup` command:
```shell ```shell
docker run --rm --network your_network_name \ docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \ -e "DB_HOST=dbhost" \
-e "DB_USERNAME=username" \ -e "DB_PORT=3306" \
-e "DB_PASSWORD=password" \ -e "DB_USERNAME=username" \
jkaninda/mysql-bkup backup -d database_name -e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup -d database_name
``` ```
Alternatively, pass a `--env-file` in order to use a full config as described below. Alternatively, use an environment file (`--env-file`) for configuration:
```yaml ```shell
docker run --rm --network your_network_name \ docker run --rm --network your_network_name \
--env-file your-env-file \ --env-file your-env-file \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
jkaninda/mysql-bkup backup -d database_name jkaninda/mysql-bkup backup -d database_name
``` ```
### Simple backup in docker compose file ### Backup All Databases
To back up all databases on the server, use the `--all-databases` or `-a` flag. By default, this creates individual backup files for each database.
```shell
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup --all-databases --disable-compression
```
> **Note:** Use the `--all-in-one` or `-A` flag to combine backups into a single file.
---
### Simple Restore Using Docker CLI
To restore a database, bind your local volume to `/backup` and run the `restore` command:
```shell
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup restore -d database_name -f backup_file.sql.gz
```
---
### Backup with Docker Compose
Below is an example of a `docker-compose.yml` file for running a one-time backup:
```yaml ```yaml
services: services:
mysql-bkup: pg-bkup:
# In production, it is advised to lock your image tag to a proper # In production, pin your image tag to a specific release version instead of `latest`.
# release version instead of using `latest`. # See available releases: https://github.com/jkaninda/mysql-bkup/releases
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup image: jkaninda/mysql-bkup
container_name: mysql-bkup container_name: mysql-bkup
command: backup command: backup
@@ -109,29 +150,39 @@ services:
- DB_USERNAME=bar - DB_USERNAME=bar
- DB_PASSWORD=password - DB_PASSWORD=password
- TZ=Europe/Paris - TZ=Europe/Paris
# mysql-bkup container must be connected to the same network with your database
networks: networks:
- web - web
networks: networks:
web: web:
``` ```
### Docker recurring backup
---
### Recurring Backups with Docker
You can schedule recurring backups using the `--cron-expression` or `-e` flag:
```shell ```shell
docker run --rm --network network_name \ docker run --rm --network network_name \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
-e "DB_HOST=hostname" \ -e "DB_HOST=hostname" \
-e "DB_USERNAME=user" \ -e "DB_USERNAME=user" \
-e "DB_PASSWORD=password" \ -e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup -d dbName --cron-expression "@every 15m" #@midnight jkaninda/mysql-bkup backup -d dbName --cron-expression "@every 15m"
``` ```
See: https://jkaninda.github.io/mysql-bkup/reference/#predefined-schedules
For predefined schedules, refer to the [documentation](https://jkaninda.github.io/mysql-bkup/reference/#predefined-schedules).
---
## Deploy on Kubernetes ## Deploy on Kubernetes
For Kubernetes, you don't need to run it in scheduled mode. You can deploy it as Job or CronJob. For Kubernetes, you can deploy `mysql-bkup` as a Job or CronJob. Below are examples for both.
### Simple Kubernetes backup Job : ### Kubernetes Backup Job
This example defines a one-time backup job:
```yaml ```yaml
apiVersion: batch/v1 apiVersion: batch/v1
@@ -144,15 +195,10 @@ spec:
spec: spec:
containers: containers:
- name: mysql-bkup - name: mysql-bkup
# In production, it is advised to lock your image tag to a proper # Pin the image tag to a specific release version in production.
# release version instead of using `latest`. # See available releases: https://github.com/jkaninda/mysql-bkup/releases
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup image: jkaninda/mysql-bkup
command: command: ["backup", "-d", "dbname"]
- /bin/sh
- -c
- backup -d dbname
resources: resources:
limits: limits:
memory: "128Mi" memory: "128Mi"
@@ -170,10 +216,94 @@ spec:
volumes: volumes:
- name: backup - name: backup
hostPath: hostPath:
path: /home/toto/backup # directory location on host path: /home/toto/backup # Directory location on the host
type: Directory # this field is optional type: Directory # Optional field
restartPolicy: Never restartPolicy: Never
``` ```
### Kubernetes CronJob for Scheduled Backups
For scheduled backups, use a `CronJob`:
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: pg-bkup-cronjob
spec:
schedule: "0 2 * * *" # Runs daily at 2 AM
jobTemplate:
spec:
template:
spec:
containers:
- name: pg-bkup
image: jkaninda/mysql-bkup
command: ["backup", "-d", "dbname"]
env:
- name: DB_HOST
value: "mysql"
- name: DB_USERNAME
value: "user"
- name: DB_PASSWORD
value: "password"
volumeMounts:
- mountPath: /backup
name: backup
volumes:
- name: backup
hostPath:
path: /home/toto/backup
type: Directory
restartPolicy: OnFailure
```
---
## 🚀 Why Use MYSQL-BKUP?
**MYSQL-BKUP** isn't just another MySQL backup tool, it's a robust, production-ready solution purpose-built for modern DevOps workflows.
Heres why developers, sysadmins, and DevOps choose **MYSQL-BKUP**:
### ✅ All-in-One Backup, Restore & Migration
Whether you're backing up a single database, restoring critical data, or migrating across environments, MYSQL-BKUP handles it all with a **single, unified CLI** no scripting gymnastics required.
### 🔄 Works Everywhere You Deploy
Designed to be cloud-native:
* **Runs seamlessly on Docker, Docker Swarm, and Kubernetes**
* Supports **CronJobs** for automated scheduled backups
* Compatible with GitOps and CI/CD workflows
### ☁️ Flexible Storage Integrations
Store your backups **anywhere**:
* Local disks
* Amazon S3, MinIO, Wasabi, Azure Blob, FTP, SSH
### 🔒 Enterprise-Grade Security
* **GPG Encryption**: Protect sensitive data with optional encryption before storing backups locally or in the cloud.
* **Secure Storage** Options: Supports S3, Azure Blob, SFTP, and SSH with encrypted transfers, keeping backups safe from unauthorized access.
### 📬 Instant Notifications
Stay in the loop with real-time notifications via **Telegram** and **Email**. Know immediately when a backup succeeds—or fails.
### 🏃‍♂️ Lightweight and Fast
Written in **Go**, MYSQL-BKUP is fast, multi-arch compatible (`amd64`, `arm64`, `arm/v7`), and optimized for minimal memory and CPU usage. Ideal for both cloud and edge deployments.
### 🧪 Tested. Verified. Trusted.
Actively maintained with **automated testing**, **Docker image size optimizations**, and verified support across major container platforms.
---
## Available image registries ## Available image registries
This Docker image is published to both Docker Hub and the GitHub container registry. This Docker image is published to both Docker Hub and the GitHub container registry.

View File

@@ -44,11 +44,14 @@ var BackupCmd = &cobra.Command{
} }
func init() { func init() {
//Backup // Backup
BackupCmd.PersistentFlags().StringP("storage", "s", "local", "Define storage: local, s3, ssh, ftp, azure") BackupCmd.PersistentFlags().StringP("storage", "s", "local", "Define storage: local, s3, ssh, ftp, azure")
BackupCmd.PersistentFlags().StringP("path", "P", "", "Storage path without file name. e.g: /custom_path or ssh remote path `/home/foo/backup`") BackupCmd.PersistentFlags().StringP("path", "P", "", "Storage path without file name. e.g: /custom_path or ssh remote path `/home/foo/backup`")
BackupCmd.PersistentFlags().StringP("cron-expression", "e", "", "Backup cron expression (e.g., `0 0 * * *` or `@daily`)") BackupCmd.PersistentFlags().StringP("cron-expression", "e", "", "Backup cron expression (e.g., `0 0 * * *` or `@daily`)")
BackupCmd.PersistentFlags().StringP("config", "c", "", "Configuration file for multi database backup. (e.g: `/backup/config.yaml`)") BackupCmd.PersistentFlags().StringP("config", "c", "", "Configuration file for multi database backup. (e.g: `/backup/config.yaml`)")
BackupCmd.PersistentFlags().BoolP("disable-compression", "", false, "Disable backup compression") BackupCmd.PersistentFlags().BoolP("disable-compression", "", false, "Disable backup compression")
BackupCmd.PersistentFlags().BoolP("all-databases", "a", false, "Backup all databases")
BackupCmd.PersistentFlags().BoolP("all-in-one", "A", false, "Backup all databases in a single file")
BackupCmd.PersistentFlags().StringP("custom-name", "", "", "Custom backup name")
} }

View File

@@ -46,7 +46,7 @@ var RestoreCmd = &cobra.Command{
} }
func init() { func init() {
//Restore // Restore
RestoreCmd.PersistentFlags().StringP("file", "f", "", "File name of database") RestoreCmd.PersistentFlags().StringP("file", "f", "", "File name of database")
RestoreCmd.PersistentFlags().StringP("storage", "s", "local", "Define storage: local, s3, ssh, ftp") RestoreCmd.PersistentFlags().StringP("storage", "s", "local", "Define storage: local, s3, ssh, ftp")
RestoreCmd.PersistentFlags().StringP("path", "P", "", "AWS S3 path without file name. eg: /custom_path or ssh remote path `/home/foo/backup`") RestoreCmd.PersistentFlags().StringP("path", "P", "", "AWS S3 path without file name. eg: /custom_path or ssh remote path `/home/foo/backup`")

View File

@@ -38,7 +38,6 @@ var rootCmd = &cobra.Command{
Example: utils.MainExample, Example: utils.MainExample,
Version: appVersion, Version: appVersion,
} }
var operation = ""
// Execute adds all child commands to the root command and sets flags appropriately. // Execute adds all child commands to the root command and sets flags appropriately.
// This is called by main.main(). It only needs to happen once to the rootCmd. // This is called by main.main(). It only needs to happen once to the rootCmd.

View File

@@ -0,0 +1,61 @@
---
title: Backup all databases in the server
layout: default
parent: How Tos
nav_order: 12
---
# Backup All Databases
MySQL-Bkup supports backing up all databases on the server using the `--all-databases` (`-a`) flag. By default, this creates separate backup files for each database. If you prefer a single backup file, you can use the `--all-in-one` (`-A`) flag.
Backing up all databases is useful for creating a snapshot of the entire database server, whether for disaster recovery or migration purposes.
## Backup Modes
### Separate Backup Files (Default)
Using --all-databases without --all-in-one creates individual backup files for each database.
- Creates separate backup files for each database.
- Provides more flexibility in restoring individual databases or tables.
- Can be more manageable in cases where different databases have different retention policies.
- Might take slightly longer due to multiple file operations.
- It is the default behavior when using the `--all-databases` flag.
- It does not backup system databases (`information_schema`, `performance_schema`, `mysql`, `sys`, `innodb`,...).
**Command:**
```bash
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup --all-databases
```
### Single Backup File
Using --all-in-one (-A) creates a single backup file containing all databases.
- Creates a single backup file containing all databases.
- Easier to manage if you need to restore everything at once.
- Faster to back up and restore in bulk.
- Can be problematic if you only need to restore a specific database or table.
- It is recommended to use this option for disaster recovery purposes.
- It backups system databases as well.
```bash
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup --all-in-one
```
### When to Use Which?
- Use `--all-in-one` if you want a quick, simple backup for disaster recovery where you'll restore everything at once.
- Use `--all-databases` if you need granularity in restoring specific databases or tables without affecting others.

View File

@@ -8,51 +8,62 @@ nav_order: 11
# Multiple Backup Schedules # Multiple Backup Schedules
You can configure multiple backup schedules with different configurations by using a configuration file. This tool supports running multiple database backup schedules within the same container.
You can configure these schedules with different settings using a **configuration file**. This flexibility allows you to manage backups for multiple databases efficiently.
This file can be mounted into the container at `/config/config.yaml`, `/config/config.yml`, or specified via the `BACKUP_CONFIG_FILE` environment variable.
--- ---
## Configuration File ## Configuration File Setup
The configuration file allows you to define multiple databases and their respective backup settings. The configuration file can be mounted into the container at `/config/config.yaml`, `/config/config.yml`, or specified via the `BACKUP_CONFIG_FILE` environment variable.
Below is an example configuration file: ### Key Features:
- **Global Environment Variables**: Use these for databases that share the same configuration.
- **Database-Specific Overrides**: Override global settings for individual databases by specifying them in the configuration file or using the database name as a prefix or suffix in the variable name (e.g., `DB_HOST_DATABASENAME` or `DATABASENAME_DB_HOST`).
- **Global Cron Expression**: Define a global `cronExpression` in the configuration file to schedule backups for all databases. If omitted, backups will run immediately.
- **Configuration File Path**: Specify the configuration file path using:
- The `BACKUP_CONFIG_FILE` environment variable.
- The `--config` or `-c` flag for the backup command.
---
## Configuration File Example
Below is an example configuration file (`config.yaml`) that defines multiple databases and their respective backup settings:
```yaml ```yaml
# Optional: Define a global cron expression for scheduled backups # Optional: Define a global cron expression for scheduled backups.
# cronExpression: "@every 20m" # Example: "@every 20m" (runs every 20 minutes). If omitted, backups run immediately.
cronExpression: "" cronExpression: "" # Optional: Define a global cron expression for scheduled backups.
backupRescueMode: false # Optional: Set to true to enable rescue mode for backups.
databases: databases:
- host: mysql1 - host: mysql1 # Optional: Overrides DB_HOST or uses DB_HOST_DATABASE1.
port: 3306 port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_DATABASE1.
name: database1 name: database1 # Required: Database name.
user: database1 user: database1 # Optional: Overrides DB_USERNAME or uses DB_USERNAME_DATABASE1.
password: password password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_DATABASE1.
path: /s3-path/database1 # For SSH or FTP, define the full path (e.g., /home/toto/backup/) path: /s3-path/database1 # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
- host: mysql2 - host: mysql2 # Optional: Overrides DB_HOST or uses DB_HOST_LLAP.
port: 3306 port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_LLAP.
name: lldap name: lldap # Required: Database name.
user: lldap user: lldap # Optional: Overrides DB_USERNAME or uses DB_USERNAME_LLAP.
password: password password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_LLAP.
path: /s3-path/lldap # For SSH or FTP, define the full path (e.g., /home/toto/backup/) path: /s3-path/lldap # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
- host: mysql3 - host: mysql3 # Optional: Overrides DB_HOST or uses DB_HOST_KEYCLOAK.
port: 3306 port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_KEYCLOAK.
name: keycloak name: keycloak # Required: Database name.
user: keycloak user: keycloak # Optional: Overrides DB_USERNAME or uses DB_USERNAME_KEYCLOAK.
password: password password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_KEYCLOAK.
path: /s3-path/keycloak # For SSH or FTP, define the full path (e.g., /home/toto/backup/) path: /s3-path/keycloak # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
- host: mysql4 - host: mysql4 # Optional: Overrides DB_HOST or uses DB_HOST_JOPLIN.
port: 3306 port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_JOPLIN.
name: joplin name: joplin # Required: Database name.
user: joplin user: joplin # Optional: Overrides DB_USERNAME or uses DB_USERNAME_JOPLIN.
password: password password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_JOPLIN.
path: /s3-path/joplin # For SSH or FTP, define the full path (e.g., /home/toto/backup/) path: /s3-path/joplin # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
``` ```
--- ---
@@ -78,7 +89,7 @@ services:
environment: environment:
## Specify the path to the configuration file ## Specify the path to the configuration file
- BACKUP_CONFIG_FILE=/backup/config.yaml - BACKUP_CONFIG_FILE=/backup/config.yaml
# Ensure the pg-bkup container is connected to the same network as your database # Ensure the mysql-bkup container is connected to the same network as your database
networks: networks:
- web - web
@@ -88,9 +99,5 @@ networks:
--- ---
## Key Notes
- **Global Cron Expression**: You can define a global `cronExpression` in the configuration file to schedule backups for all databases. If omitted, backups will run immediately.
- **Database-Specific Paths**: For SSH or FTP storage, ensure the `path` field contains the full remote path (e.g., `/home/toto/backup/`).
- **Environment Variables**: Use the `BACKUP_CONFIG_FILE` environment variable to specify the path to the configuration file.
- **Security**: Avoid hardcoding sensitive information like passwords in the configuration file. Use environment variables or secrets management tools instead.

View File

@@ -2,7 +2,7 @@
title: Receive notifications title: Receive notifications
layout: default layout: default
parent: How Tos parent: How Tos
nav_order: 12 nav_order: 13
--- ---
# Receive Notifications # Receive Notifications

View File

@@ -10,6 +10,8 @@ nav_order: 1
**MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**. **MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**.
It supports a variety of storage options and ensures data security through GPG encryption. It supports a variety of storage options and ensures data security through GPG encryption.
**MYSQL-BKUP** is designed for seamless deployment on **Docker** and **Kubernetes**, simplifying MySQL backup, restoration, and migration across environments.
It is a lightweight, multi-architecture solution compatible with **Docker**, **Docker Swarm**, **Kubernetes**, and other container orchestration platforms.
--- ---
## Key Features ## Key Features
@@ -39,11 +41,21 @@ It supports a variety of storage options and ensures data security through GPG e
--- ---
## Use Cases ## 💡Use Cases
- **Automated Recurring Backups:** Schedule regular backups for MySQL databases. - **Scheduled Backups**: Automate recurring backups using Docker or Kubernetes.
- **Cross-Environment Migration:** Easily migrate MySQL databases across different environments using supported storage options. - **Disaster Recovery:** Quickly restore backups to a clean MySQL instance.
- **Secure Backup Management:** Protect your data with GPG encryption. - **Database Migration**: Seamlessly move data across environments using the built-in `migrate` feature.
- **Secure Archiving:** Keep backups encrypted and safely stored in the cloud or remote servers.
## ✅ Verified Platforms:
MYSQL-BKUP has been tested and runs successfully on:
- Docker
- Docker Swarm
- Kubernetes
- OpenShift
--- ---

View File

@@ -10,41 +10,72 @@ This guide provides quick examples for running backups using Docker CLI, Docker
--- ---
## Simple Backup Using Docker CLI ### Simple Backup Using Docker CLI
To run a one-time backup, bind your local volume to `/backup` in the container and execute the `backup` command: To perform a one-time backup, bind your local volume to `/backup` in the container and run the `backup` command:
```bash ```shell
docker run --rm --network your_network_name \ docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \ -e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \ -e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \ -e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup -d database_name jkaninda/mysql-bkup backup -d database_name
``` ```
### Using an Environment File Alternatively, use an environment file (`--env-file`) for configuration:
Alternatively, you can use an `--env-file` to pass a full configuration: ```shell
```bash
docker run --rm --network your_network_name \ docker run --rm --network your_network_name \
--env-file your-env-file \ --env-file your-env-file \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
jkaninda/mysql-bkup backup -d database_name jkaninda/mysql-bkup backup -d database_name
``` ```
### Backup All Databases
To back up all databases on the server, use the `--all-databases` or `-a` flag. By default, this creates individual backup files for each database.
```shell
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup backup --all-databases --disable-compression
```
> **Note:** Use the `--all-in-one` or `-A` flag to combine backups into a single file.
--- ---
## Simple Backup Using Docker Compose ### Simple Restore Using Docker CLI
Below is an example `docker-compose.yml` configuration for running a backup: To restore a database, bind your local volume to `/backup` and run the `restore` command:
```shell
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_PORT=3306" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup restore -d database_name -f backup_file.sql.gz
```
---
### Backup with Docker Compose
Below is an example of a `docker-compose.yml` file for running a one-time backup:
```yaml ```yaml
services: services:
mysql-bkup: pg-bkup:
# In production, lock the image tag to a specific release version. # In production, pin your image tag to a specific release version instead of `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases for available releases. # See available releases: https://github.com/jkaninda/mysql-bkup/releases
image: jkaninda/mysql-bkup image: jkaninda/mysql-bkup
container_name: mysql-bkup container_name: mysql-bkup
command: backup command: backup
@@ -57,7 +88,6 @@ services:
- DB_USERNAME=bar - DB_USERNAME=bar
- DB_PASSWORD=password - DB_PASSWORD=password
- TZ=Europe/Paris - TZ=Europe/Paris
# Ensure the mysql-bkup container is connected to the same network as your database.
networks: networks:
- web - web
@@ -67,11 +97,11 @@ networks:
--- ---
## Recurring Backup with Docker ### Recurring Backups with Docker
To schedule recurring backups, use the `--cron-expression` flag: You can schedule recurring backups using the `--cron-expression` or `-e` flag:
```bash ```shell
docker run --rm --network network_name \ docker run --rm --network network_name \
-v $PWD/backup:/backup/ \ -v $PWD/backup:/backup/ \
-e "DB_HOST=hostname" \ -e "DB_HOST=hostname" \
@@ -84,9 +114,13 @@ For predefined schedules, refer to the [documentation](https://jkaninda.github.i
--- ---
## Backup Using Kubernetes ## Deploy on Kubernetes
Below is an example Kubernetes `Job` configuration for running a backup: For Kubernetes, you can deploy `mysql-bkup` as a Job or CronJob. Below are examples for both.
### Kubernetes Backup Job
This example defines a one-time backup job:
```yaml ```yaml
apiVersion: batch/v1 apiVersion: batch/v1
@@ -99,8 +133,8 @@ spec:
spec: spec:
containers: containers:
- name: mysql-bkup - name: mysql-bkup
# In production, lock the image tag to a specific release version. # Pin the image tag to a specific release version in production.
# Check https://github.com/jkaninda/mysql-bkup/releases for available releases. # See available releases: https://github.com/jkaninda/mysql-bkup/releases
image: jkaninda/mysql-bkup image: jkaninda/mysql-bkup
command: command:
- /bin/sh - /bin/sh
@@ -114,7 +148,7 @@ spec:
- name: DB_HOST - name: DB_HOST
value: "mysql" value: "mysql"
- name: DB_USERNAME - name: DB_USERNAME
value: "postgres" value: "user"
- name: DB_PASSWORD - name: DB_PASSWORD
value: "password" value: "password"
volumeMounts: volumeMounts:
@@ -123,11 +157,51 @@ spec:
volumes: volumes:
- name: backup - name: backup
hostPath: hostPath:
path: /home/toto/backup # Directory location on the host path: /home/toto/backup # Directory location on the host
type: Directory # Optional field type: Directory # Optional field
restartPolicy: Never restartPolicy: Never
``` ```
### Kubernetes CronJob for Scheduled Backups
For scheduled backups, use a `CronJob`:
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: pg-bkup-cronjob
spec:
schedule: "0 2 * * *" # Runs daily at 2 AM
jobTemplate:
spec:
template:
spec:
containers:
- name: pg-bkup
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup -d dbname
env:
- name: DB_HOST
value: "mysql"
- name: DB_USERNAME
value: "user"
- name: DB_PASSWORD
value: "password"
volumeMounts:
- mountPath: /backup
name: backup
volumes:
- name: backup
hostPath:
path: /home/toto/backup
type: Directory
restartPolicy: OnFailure
```
--- ---
## Key Notes ## Key Notes

View File

@@ -6,28 +6,31 @@ nav_order: 3
# Configuration Reference # Configuration Reference
Backup, restore, and migration targets, schedules, and retention policies are configured using **environment variables** or **CLI flags**. MySQL backup, restore, and migration processes can be configured using **environment variables** or **CLI flags**.
---
## CLI Utility Usage ## CLI Utility Usage
| Option | Short Flag | Description | The `mysql-bkup` CLI provides commands and options to manage MySQL backups efficiently.
|-------------------------|------------|-------------------------------------------------------------------------------|
| `pg-bkup` | `bkup` | CLI utility for managing PostgreSQL backups. | | Option | Short Flag | Description |
| `backup` | | Perform a backup operation. | |-------------------------|------------|-----------------------------------------------------------------------------------------|
| `restore` | | Perform a restore operation. | | `mysql-bkup` | `bkup` | CLI tool for managing MySQL backups, restoration, and migration. |
| `migrate` | | Migrate a database from one instance to another. | | `backup` | | Executes a backup operation. |
| `--storage` | `-s` | Storage type (`local`, `s3`, `ssh`, etc.). Default: `local`. | | `restore` | | Restores a database from a backup file. |
| `--file` | `-f` | File name for restoration. | | `migrate` | | Migrates a database from one instance to another. |
| `--path` | | Path for storage (e.g., `/custom_path` for S3 or `/home/foo/backup` for SSH). | | `--storage` | `-s` | Specifies the storage type (`local`, `s3`, `ssh`, etc.). Default: `local`. |
| `--config` | `-c` | Configuration file for multi database backup. (e.g: `/backup/config.yaml`). | | `--file` | `-f` | Defines the backup file name for restoration. |
| `--dbname` | `-d` | Database name. | | `--path` | | Sets the storage path (e.g., `/custom_path` for S3 or `/home/foo/backup` for SSH). |
| `--port` | `-p` | Database port. Default: `3306`. | | `--config` | `-c` | Provides a configuration file for multi-database backups (e.g., `/backup/config.yaml`). |
| `--disable-compression` | | Disable compression for database backups. | | `--dbname` | `-d` | Specifies the database name to back up or restore. |
| `--cron-expression` | `-e` | Cron expression for scheduled backups (e.g., `0 0 * * *` or `@daily`). | | `--port` | `-p` | Defines the database port. Default: `3306`. |
| `--help` | `-h` | Display help message and exit. | | `--disable-compression` | | Disables compression for database backups. |
| `--version` | `-V` | Display version information and exit. | | `--cron-expression` | `-e` | Schedules backups using a cron expression (e.g., `0 0 * * *` or `@daily`). |
| `--all-databases` | `-a` | Backs up all databases separately (e.g., `backup --all-databases`). |
| `--all-in-one` | `-A` | Backs up all databases in a single file (e.g., `backup --all-databases --single-file`). |
| `--custom-name` | `` | Sets custom backup name for one time backup |
| `--help` | `-h` | Displays the help message and exits. |
| `--version` | `-V` | Shows version information and exits. |
--- ---
@@ -40,6 +43,8 @@ Backup, restore, and migration targets, schedules, and retention policies are co
| `DB_NAME` | Optional (if provided via `-d` flag) | Database name. | | `DB_NAME` | Optional (if provided via `-d` flag) | Database name. |
| `DB_USERNAME` | Required | Database username. | | `DB_USERNAME` | Required | Database username. |
| `DB_PASSWORD` | Required | Database password. | | `DB_PASSWORD` | Required | Database password. |
| `DB_SSL_CA` | Optional | Database client CA certificate file |
| `DB_SSL_MODE` | Optional(`0 or 1`) default: `0` | Database client Enable CA validation |
| `AWS_ACCESS_KEY` | Required for S3 storage | AWS S3 Access Key. | | `AWS_ACCESS_KEY` | Required for S3 storage | AWS S3 Access Key. |
| `AWS_SECRET_KEY` | Required for S3 storage | AWS S3 Secret Key. | | `AWS_SECRET_KEY` | Required for S3 storage | AWS S3 Secret Key. |
| `AWS_BUCKET_NAME` | Required for S3 storage | AWS S3 Bucket Name. | | `AWS_BUCKET_NAME` | Required for S3 storage | AWS S3 Bucket Name. |

5
go.mod
View File

@@ -2,14 +2,15 @@ module github.com/jkaninda/mysql-bkup
go 1.23.2 go 1.23.2
require github.com/spf13/pflag v1.0.5 // indirect require github.com/spf13/pflag v1.0.6 // indirect
require ( require (
github.com/go-mail/mail v2.3.1+incompatible github.com/go-mail/mail v2.3.1+incompatible
github.com/jkaninda/encryptor v0.0.0-20241111100652-926393c9437e github.com/jkaninda/encryptor v0.0.0-20241111100652-926393c9437e
github.com/jkaninda/go-storage v0.1.3 github.com/jkaninda/go-storage v0.1.3
github.com/jkaninda/go-utils v0.1.1
github.com/robfig/cron/v3 v3.0.1 github.com/robfig/cron/v3 v3.0.1
github.com/spf13/cobra v1.8.1 github.com/spf13/cobra v1.9.1
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )

12
go.sum
View File

@@ -22,7 +22,7 @@ github.com/bramvdbogaerde/go-scp v1.5.0 h1:a9BinAjTfQh273eh7vd3qUgmBC+bx+3TRDtkZ
github.com/bramvdbogaerde/go-scp v1.5.0/go.mod h1:on2aH5AxaFb2G0N5Vsdy6B0Ml7k9HuHSwfo1y0QzAbQ= github.com/bramvdbogaerde/go-scp v1.5.0/go.mod h1:on2aH5AxaFb2G0N5Vsdy6B0Ml7k9HuHSwfo1y0QzAbQ=
github.com/cloudflare/circl v1.3.7 h1:qlCDlTPz2n9fu58M0Nh1J/JzcFpfgkFHHX3O35r5vcU= github.com/cloudflare/circl v1.3.7 h1:qlCDlTPz2n9fu58M0Nh1J/JzcFpfgkFHHX3O35r5vcU=
github.com/cloudflare/circl v1.3.7/go.mod h1:sRTcRWXGLrKw6yIGJ+l7amYJFfAXbZG0kBSc8r4zxgA= github.com/cloudflare/circl v1.3.7/go.mod h1:sRTcRWXGLrKw6yIGJ+l7amYJFfAXbZG0kBSc8r4zxgA=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -43,6 +43,8 @@ github.com/jkaninda/encryptor v0.0.0-20241111100652-926393c9437e h1:jtFKZHt/PLGQ
github.com/jkaninda/encryptor v0.0.0-20241111100652-926393c9437e/go.mod h1:Y1EXpPWQ9PNd7y7E6ez3xgnzZc8fuDWXwX/1/dXNCE4= github.com/jkaninda/encryptor v0.0.0-20241111100652-926393c9437e/go.mod h1:Y1EXpPWQ9PNd7y7E6ez3xgnzZc8fuDWXwX/1/dXNCE4=
github.com/jkaninda/go-storage v0.1.3 h1:lEpHVgFLKSvjsi/6tAek96Y07za3vxmsXF2/+jiCMZU= github.com/jkaninda/go-storage v0.1.3 h1:lEpHVgFLKSvjsi/6tAek96Y07za3vxmsXF2/+jiCMZU=
github.com/jkaninda/go-storage v0.1.3/go.mod h1:zVRnLprBk/9AUz2+za6Y03MgoNYrqKLy3edVtjqMaps= github.com/jkaninda/go-storage v0.1.3/go.mod h1:zVRnLprBk/9AUz2+za6Y03MgoNYrqKLy3edVtjqMaps=
github.com/jkaninda/go-utils v0.1.1 h1:PMrtXR9d51YzHo85y9Z6YVL0YyBURbRTPemHVbFDqZg=
github.com/jkaninda/go-utils v0.1.1/go.mod h1:pf0/U6k4JbxlablM2G4eSTZdQ2LFshfAsCK5Q8qNfGo=
github.com/jlaffaye/ftp v0.2.0 h1:lXNvW7cBu7R/68bknOX3MrRIIqZ61zELs1P2RAiA3lg= github.com/jlaffaye/ftp v0.2.0 h1:lXNvW7cBu7R/68bknOX3MrRIIqZ61zELs1P2RAiA3lg=
github.com/jlaffaye/ftp v0.2.0/go.mod h1:is2Ds5qkhceAPy2xD6RLI6hmp/qysSoymZ+Z2uTnspI= github.com/jlaffaye/ftp v0.2.0/go.mod h1:is2Ds5qkhceAPy2xD6RLI6hmp/qysSoymZ+Z2uTnspI=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
@@ -66,10 +68,10 @@ github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzG
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8= github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4= github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM= github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y= github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=

35
migrations/init.sql Normal file
View File

@@ -0,0 +1,35 @@
-- Create the database testdb2 and testdb3
CREATE DATABASE IF NOT EXISTS testdb2;
CREATE DATABASE IF NOT EXISTS testdb3;
CREATE DATABASE IF NOT EXISTS fakedb;
USE testdb;
-- Create the 'users' table
CREATE TABLE users (
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(100) NOT NULL,
email VARCHAR(100) NOT NULL UNIQUE,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- Create the 'orders' table
CREATE TABLE orders (
id INT AUTO_INCREMENT PRIMARY KEY,
user_id INT NOT NULL,
amount DECIMAL(10,2) NOT NULL,
status ENUM('pending', 'completed', 'canceled') NOT NULL DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
-- Insert fake users
INSERT INTO users (name, email) VALUES
('Alice Smith', 'alice@example.com'),
('Bob Johnson', 'bob@example.com'),
('Charlie Brown', 'charlie@example.com');
-- Insert fake orders
INSERT INTO orders (user_id, amount, status) VALUES
(1, 100.50, 'completed'),
(2, 200.75, 'pending'),
(3, 50.00, 'canceled');

View File

@@ -0,0 +1,13 @@
#cronExpression: "@every 20s"
#backupRescueMode: false
databases:
- host: 127.0.0.1
port: 3306
name: testdb
user: user
password: password
- name: testdb2
# database credentials from environment variables
#TESTDB2_DB_USERNAME
#TESTDB2_DB_PASSWORD
#TESTDB2_DB_HOST

View File

@@ -27,6 +27,7 @@ package pkg
import ( import (
"fmt" "fmt"
"github.com/jkaninda/go-storage/pkg/azure" "github.com/jkaninda/go-storage/pkg/azure"
goutils "github.com/jkaninda/go-utils"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"os" "os"
@@ -36,10 +37,13 @@ import (
func azureBackup(db *dbConfig, config *BackupConfig) { func azureBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to Azure Blob Storage") utils.Info("Backup database to Azure Blob Storage")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database // Backup database
BackupDatabase(db, config.backupFileName, disableCompression) err := BackupDatabase(db, config.backupFileName, disableCompression, config.all, config.allInOne)
if err != nil {
recoverMode(err, "Error backing up database")
return
}
finalFileName := config.backupFileName finalFileName := config.backupFileName
if config.encryption { if config.encryption {
encryptBackup(config) encryptBackup(config)
@@ -87,6 +91,8 @@ func azureBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize))) utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize)))
utils.Info("Uploading backup archive to Azure Blob storage ... done ") utils.Info("Uploading backup archive to Azure Blob storage ... done ")
duration := goutils.FormatDuration(time.Since(startTime), 0)
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
File: finalFileName, File: finalFileName,
@@ -94,12 +100,11 @@ func azureBackup(db *dbConfig, config *BackupConfig) {
Database: db.dbName, Database: db.dbName,
Storage: config.storage, Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName), BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime, Duration: duration,
EndTime: time.Now().Format(utils.TimeFormat()),
}) })
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") utils.Info("The backup of the %s database has been completed in %s", db.dbName, duration)
} }
func azureRestore(db *dbConfig, conf *RestoreConfig) { func azureRestore(db *dbConfig, conf *RestoreConfig) {
utils.Info("Restore database from Azure Blob storage") utils.Info("Restore database from Azure Blob storage")

View File

@@ -26,16 +26,19 @@ SOFTWARE.
package pkg package pkg
import ( import (
"bytes"
"errors"
"fmt" "fmt"
"github.com/jkaninda/encryptor" "github.com/jkaninda/encryptor"
"github.com/jkaninda/go-storage/pkg/local" "github.com/jkaninda/go-storage/pkg/local"
goutils "github.com/jkaninda/go-utils"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/robfig/cron/v3" "github.com/robfig/cron/v3"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"log"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"strings"
"time" "time"
) )
@@ -48,7 +51,8 @@ func StartBackup(cmd *cobra.Command) {
if err != nil { if err != nil {
dbConf = initDbConfig(cmd) dbConf = initDbConfig(cmd)
if config.cronExpression == "" { if config.cronExpression == "" {
BackupTask(dbConf, config) config.allowCustomName = true
createBackupTask(dbConf, config)
} else { } else {
if utils.IsValidCronExpression(config.cronExpression) { if utils.IsValidCronExpression(config.cronExpression) {
scheduledMode(dbConf, config) scheduledMode(dbConf, config)
@@ -71,14 +75,18 @@ func scheduledMode(db *dbConfig, config *BackupConfig) {
// Test backup // Test backup
utils.Info("Testing backup configurations...") utils.Info("Testing backup configurations...")
testDatabaseConnection(db) err := testDatabaseConnection(db)
if err != nil {
utils.Error("Error connecting to database: %s", db.dbName)
utils.Fatal("Error: %s", err)
}
utils.Info("Testing backup configurations...done") utils.Info("Testing backup configurations...done")
utils.Info("Creating backup job...") utils.Info("Creating backup job...")
// Create a new cron instance // Create a new cron instance
c := cron.New() c := cron.New()
_, err := c.AddFunc(config.cronExpression, func() { _, err = c.AddFunc(config.cronExpression, func() {
BackupTask(db, config) createBackupTask(db, config)
utils.Info("Next backup time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat)) utils.Info("Next backup time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat))
}) })
@@ -100,27 +108,69 @@ func multiBackupTask(databases []Database, bkConfig *BackupConfig) {
if db.Path != "" { if db.Path != "" {
bkConfig.remotePath = db.Path bkConfig.remotePath = db.Path
} }
BackupTask(getDatabase(db), bkConfig) createBackupTask(getDatabase(db), bkConfig)
} }
} }
// BackupTask backups database // createBackupTask backup task
func BackupTask(db *dbConfig, config *BackupConfig) { func createBackupTask(db *dbConfig, config *BackupConfig) {
if config.all && !config.allInOne {
backupAll(db, config)
} else {
if db.dbName == "" && !config.all {
utils.Fatal("Database name is required, use DB_NAME environment variable or -d flag")
}
backupTask(db, config)
}
}
// backupAll backup all databases
func backupAll(db *dbConfig, config *BackupConfig) {
databases, err := listDatabases(*db)
if err != nil {
utils.Fatal("Error listing databases: %s", err)
}
for _, dbName := range databases {
if dbName == "information_schema" || dbName == "performance_schema" || dbName == "mysql" || dbName == "sys" || dbName == "innodb" || dbName == "Database" {
continue
}
db.dbName = dbName
config.backupFileName = fmt.Sprintf("%s_%s.sql.gz", dbName, time.Now().Format("20060102_150405"))
backupTask(db, config)
}
}
// backupTask backup task
func backupTask(db *dbConfig, config *BackupConfig) {
utils.Info("Starting backup task...") utils.Info("Starting backup task...")
startTime = time.Now()
prefix := db.dbName
if config.all && config.allInOne {
prefix = "all_databases"
}
// Generate file name // Generate file name
backupFileName := fmt.Sprintf("%s_%s.sql.gz", db.dbName, time.Now().Format("20060102_150405")) backupFileName := fmt.Sprintf("%s_%s.sql.gz", prefix, time.Now().Format("20060102_150405"))
if config.disableCompression { if config.disableCompression {
backupFileName = fmt.Sprintf("%s_%s.sql", db.dbName, time.Now().Format("20060102_150405")) backupFileName = fmt.Sprintf("%s_%s.sql", prefix, time.Now().Format("20060102_150405"))
}
if config.customName != "" && config.allowCustomName && !config.all {
backupFileName = fmt.Sprintf("%s.sql.gz", config.customName)
if config.disableCompression {
backupFileName = fmt.Sprintf("%s.sql", config.customName)
}
} }
config.backupFileName = backupFileName config.backupFileName = backupFileName
switch config.storage { s := strings.ToLower(config.storage)
switch s {
case "local": case "local":
localBackup(db, config) localBackup(db, config)
case "s3", "S3": case "s3":
s3Backup(db, config) s3Backup(db, config)
case "ssh", "SSH", "remote": case "ssh", "remote", "sftp":
sshBackup(db, config) sshBackup(db, config)
case "ftp", "FTP": case "ftp":
ftpBackup(db, config) ftpBackup(db, config)
case "azure": case "azure":
azureBackup(db, config) azureBackup(db, config)
@@ -128,6 +178,8 @@ func BackupTask(db *dbConfig, config *BackupConfig) {
localBackup(db, config) localBackup(db, config)
} }
} }
// startMultiBackup start multi backup
func startMultiBackup(bkConfig *BackupConfig, configFile string) { func startMultiBackup(bkConfig *BackupConfig, configFile string) {
utils.Info("Starting Multi backup task...") utils.Info("Starting Multi backup task...")
conf, err := readConf(configFile) conf, err := readConf(configFile)
@@ -145,6 +197,7 @@ func startMultiBackup(bkConfig *BackupConfig, configFile string) {
if bkConfig.cronExpression == "" { if bkConfig.cronExpression == "" {
multiBackupTask(conf.Databases, bkConfig) multiBackupTask(conf.Databases, bkConfig)
} else { } else {
backupRescueMode = conf.BackupRescueMode
// Check if cronExpression is valid // Check if cronExpression is valid
if utils.IsValidCronExpression(bkConfig.cronExpression) { if utils.IsValidCronExpression(bkConfig.cronExpression) {
utils.Info("Running backup in Scheduled mode") utils.Info("Running backup in Scheduled mode")
@@ -155,7 +208,11 @@ func startMultiBackup(bkConfig *BackupConfig, configFile string) {
// Test backup // Test backup
utils.Info("Testing backup configurations...") utils.Info("Testing backup configurations...")
for _, db := range conf.Databases { for _, db := range conf.Databases {
testDatabaseConnection(getDatabase(db)) err = testDatabaseConnection(getDatabase(db))
if err != nil {
recoverMode(err, fmt.Sprintf("Error connecting to database: %s", db.Name))
continue
}
} }
utils.Info("Testing backup configurations...done") utils.Info("Testing backup configurations...done")
utils.Info("Creating backup job...") utils.Info("Creating backup job...")
@@ -185,79 +242,85 @@ func startMultiBackup(bkConfig *BackupConfig, configFile string) {
} }
// BackupDatabase backup database // BackupDatabase backup database
func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool) { func BackupDatabase(db *dbConfig, backupFileName string, disableCompression, all, singleFile bool) error {
storagePath = os.Getenv("STORAGE_PATH") storagePath = os.Getenv("STORAGE_PATH")
utils.Info("Starting database backup...") utils.Info("Starting database backup...")
err := os.Setenv("MYSQL_PWD", db.dbPassword) if err := testDatabaseConnection(db); err != nil {
if err != nil { return fmt.Errorf("database connection failed: %w", err)
return
} }
testDatabaseConnection(db)
// Backup Database database
utils.Info("Backing up database...")
// Verify is compression is disabled
if disableCompression {
// Execute mysqldump
cmd := exec.Command("mysqldump",
"-h", db.dbHost,
"-P", db.dbPort,
"-u", db.dbUserName,
db.dbName,
)
output, err := cmd.Output()
if err != nil {
utils.Fatal(err.Error())
}
// save output
file, err := os.Create(filepath.Join(tmpPath, backupFileName))
if err != nil {
utils.Fatal(err.Error())
}
defer func(file *os.File) {
err := file.Close()
if err != nil {
utils.Fatal(err.Error())
}
}(file)
_, err = file.Write(output)
if err != nil {
utils.Fatal(err.Error())
}
utils.Info("Database has been backed up")
dumpArgs := []string{fmt.Sprintf("--defaults-file=%s", mysqlClientConfig)}
if all && singleFile {
utils.Info("Backing up all databases...")
dumpArgs = append(dumpArgs, "--all-databases", "--single-transaction", "--routines", "--triggers")
} else { } else {
// Execute mysqldump utils.Info("Backing up %s database...", db.dbName)
cmd := exec.Command("mysqldump", "-h", db.dbHost, "-P", db.dbPort, "-u", db.dbUserName, db.dbName) dumpArgs = append(dumpArgs, db.dbName)
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Fatal(err)
}
gzipCmd := exec.Command("gzip")
gzipCmd.Stdin = stdout
gzipCmd.Stdout, err = os.Create(filepath.Join(tmpPath, backupFileName))
err = gzipCmd.Start()
if err != nil {
return
}
if err := cmd.Run(); err != nil {
log.Fatal(err)
}
if err := gzipCmd.Wait(); err != nil {
log.Fatal(err)
}
utils.Info("Database has been backed up")
} }
backupPath := filepath.Join(tmpPath, backupFileName)
if disableCompression {
return runCommandAndSaveOutput("mysqldump", dumpArgs, backupPath)
}
return runCommandWithCompression("mysqldump", dumpArgs, backupPath)
} }
// runCommandAndSaveOutput runs a command and saves the output to a file
func runCommandAndSaveOutput(command string, args []string, outputPath string) error {
cmd := exec.Command(command, args...)
output, err := cmd.Output()
if err != nil {
return fmt.Errorf("failed to execute %s: %v, output: %s", command, err, string(output))
}
return os.WriteFile(outputPath, output, 0644)
}
// runCommandWithCompression runs a command and compresses the output
func runCommandWithCompression(command string, args []string, outputPath string) error {
cmd := exec.Command(command, args...)
stdout, err := cmd.StdoutPipe()
if err != nil {
return fmt.Errorf("failed to create stdout pipe: %w", err)
}
gzipCmd := exec.Command("gzip")
gzipCmd.Stdin = stdout
gzipFile, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("failed to create gzip file: %w", err)
}
defer func(gzipFile *os.File) {
err := gzipFile.Close()
if err != nil {
utils.Error("Error closing gzip file: %v", err)
}
}(gzipFile)
gzipCmd.Stdout = gzipFile
if err := gzipCmd.Start(); err != nil {
return fmt.Errorf("failed to start gzip: %w", err)
}
if err := cmd.Run(); err != nil {
return fmt.Errorf("failed to execute %s: %w", command, err)
}
if err := gzipCmd.Wait(); err != nil {
return fmt.Errorf("failed to wait for gzip completion: %w", err)
}
utils.Info("Database has been backed up")
return nil
}
// localBackup backup database to local storage
func localBackup(db *dbConfig, config *BackupConfig) { func localBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to local storage") utils.Info("Backup database to local storage")
startTime = time.Now().Format(utils.TimeFormat()) err := BackupDatabase(db, config.backupFileName, disableCompression, config.all, config.allInOne)
BackupDatabase(db, config.backupFileName, disableCompression) if err != nil {
recoverMode(err, "Error backing up database")
return
}
finalFileName := config.backupFileName finalFileName := config.backupFileName
if config.encryption { if config.encryption {
encryptBackup(config) encryptBackup(config)
@@ -279,6 +342,8 @@ func localBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup name is %s", finalFileName) utils.Info("Backup name is %s", finalFileName)
utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize))) utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize)))
utils.Info("Backup saved in %s", filepath.Join(storagePath, finalFileName)) utils.Info("Backup saved in %s", filepath.Join(storagePath, finalFileName))
duration := goutils.FormatDuration(time.Since(startTime), 0)
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
File: finalFileName, File: finalFileName,
@@ -286,8 +351,7 @@ func localBackup(db *dbConfig, config *BackupConfig) {
Database: db.dbName, Database: db.dbName,
Storage: config.storage, Storage: config.storage,
BackupLocation: filepath.Join(storagePath, finalFileName), BackupLocation: filepath.Join(storagePath, finalFileName),
StartTime: startTime, Duration: duration,
EndTime: time.Now().Format(utils.TimeFormat()),
}) })
// Delete old backup // Delete old backup
if config.prune { if config.prune {
@@ -299,9 +363,10 @@ func localBackup(db *dbConfig, config *BackupConfig) {
} }
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") utils.Info("The backup of the %s database has been completed in %s", db.dbName, duration)
} }
// encryptBackup encrypt backup
func encryptBackup(config *BackupConfig) { func encryptBackup(config *BackupConfig) {
backupFile, err := os.ReadFile(filepath.Join(tmpPath, config.backupFileName)) backupFile, err := os.ReadFile(filepath.Join(tmpPath, config.backupFileName))
outputFile := fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension) outputFile := fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension)
@@ -331,3 +396,43 @@ func encryptBackup(config *BackupConfig) {
} }
} }
// listDatabases list all databases
func listDatabases(db dbConfig) ([]string, error) {
databases := []string{}
// Create the mysql client config file
if err := createMysqlClientConfigFile(db); err != nil {
return databases, errors.New(err.Error())
}
utils.Info("Listing databases...")
// Step 1: List all databases
cmd := exec.Command("mariadb", fmt.Sprintf("--defaults-file=%s", mysqlClientConfig), "-e", "SHOW DATABASES;")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
return databases, fmt.Errorf("failed to list databases: %s", err)
}
// Step 2: Parse the output
for _, _db := range strings.Split(out.String(), "\n") {
if _db != "" {
databases = append(databases, _db)
}
}
return databases, nil
}
func recoverMode(err error, msg string) {
if err != nil {
if backupRescueMode {
utils.NotifyError(fmt.Sprintf("%s : %v", msg, err))
utils.Error("Error: %s", msg)
utils.Error("Backup rescue mode is enabled")
utils.Error("Backup will continue")
} else {
utils.Error("Error: %s", msg)
utils.Fatal("Error: %v", err)
return
}
}
}

View File

@@ -30,6 +30,7 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
"os" "os"
"strconv" "strconv"
"strings"
) )
type Database struct { type Database struct {
@@ -41,8 +42,9 @@ type Database struct {
Path string `yaml:"path"` Path string `yaml:"path"`
} }
type Config struct { type Config struct {
Databases []Database `yaml:"databases"` CronExpression string `yaml:"cronExpression"`
CronExpression string `yaml:"cronExpression"` BackupRescueMode bool `yaml:"backupRescueMode"`
Databases []Database `yaml:"databases"`
} }
type dbConfig struct { type dbConfig struct {
@@ -75,6 +77,10 @@ type BackupConfig struct {
publicKey string publicKey string
storage string storage string
cronExpression string cronExpression string
all bool
allInOne bool
customName string
allowCustomName bool
} }
type FTPConfig struct { type FTPConfig struct {
host string host string
@@ -113,7 +119,7 @@ func initDbConfig(cmd *cobra.Command) *dbConfig {
utils.GetEnv(cmd, "dbname", "DB_NAME") utils.GetEnv(cmd, "dbname", "DB_NAME")
dConf := dbConfig{} dConf := dbConfig{}
dConf.dbHost = os.Getenv("DB_HOST") dConf.dbHost = os.Getenv("DB_HOST")
dConf.dbPort = os.Getenv("DB_PORT") dConf.dbPort = utils.EnvWithDefault("DB_PORT", "3306")
dConf.dbName = os.Getenv("DB_NAME") dConf.dbName = os.Getenv("DB_NAME")
dConf.dbUserName = os.Getenv("DB_USERNAME") dConf.dbUserName = os.Getenv("DB_USERNAME")
dConf.dbPassword = os.Getenv("DB_PASSWORD") dConf.dbPassword = os.Getenv("DB_PASSWORD")
@@ -127,6 +133,11 @@ func initDbConfig(cmd *cobra.Command) *dbConfig {
} }
func getDatabase(database Database) *dbConfig { func getDatabase(database Database) *dbConfig {
// Set default values from environment variables if not provided
database.User = getEnvOrDefault(database.User, "DB_USERNAME", database.Name, "")
database.Password = getEnvOrDefault(database.Password, "DB_PASSWORD", database.Name, "")
database.Host = getEnvOrDefault(database.Host, "DB_HOST", database.Name, "")
database.Port = getEnvOrDefault(database.Port, "DB_PORT", database.Name, "3306")
return &dbConfig{ return &dbConfig{
dbHost: database.Host, dbHost: database.Host,
dbPort: database.Port, dbPort: database.Port,
@@ -136,6 +147,31 @@ func getDatabase(database Database) *dbConfig {
} }
} }
// Helper function to get environment variable or use a default value
func getEnvOrDefault(currentValue, envKey, suffix, defaultValue string) string {
// Return the current value if it's already set
if currentValue != "" {
return currentValue
}
// Check for suffixed or prefixed environment variables if a suffix is provided
if suffix != "" {
suffixUpper := strings.ToUpper(suffix)
envSuffix := os.Getenv(fmt.Sprintf("%s_%s", envKey, suffixUpper))
if envSuffix != "" {
return envSuffix
}
envPrefix := os.Getenv(fmt.Sprintf("%s_%s", suffixUpper, envKey))
if envPrefix != "" {
return envPrefix
}
}
// Fall back to the default value using a helper function
return utils.EnvWithDefault(envKey, defaultValue)
}
// loadSSHConfig loads the SSH configuration from environment variables // loadSSHConfig loads the SSH configuration from environment variables
func loadSSHConfig() (*SSHConfig, error) { func loadSSHConfig() (*SSHConfig, error) {
utils.GetEnvVariable("SSH_HOST", "SSH_HOST_NAME") utils.GetEnvVariable("SSH_HOST", "SSH_HOST_NAME")
@@ -224,6 +260,12 @@ func initBackupConfig(cmd *cobra.Command) *BackupConfig {
prune = true prune = true
} }
disableCompression, _ = cmd.Flags().GetBool("disable-compression") disableCompression, _ = cmd.Flags().GetBool("disable-compression")
customName, _ := cmd.Flags().GetString("custom-name")
all, _ := cmd.Flags().GetBool("all-databases")
allInOne, _ := cmd.Flags().GetBool("all-in-one")
if allInOne {
all = true
}
_, _ = cmd.Flags().GetString("mode") _, _ = cmd.Flags().GetString("mode")
passphrase := os.Getenv("GPG_PASSPHRASE") passphrase := os.Getenv("GPG_PASSPHRASE")
_ = utils.GetEnv(cmd, "path", "AWS_S3_PATH") _ = utils.GetEnv(cmd, "path", "AWS_S3_PATH")
@@ -249,6 +291,9 @@ func initBackupConfig(cmd *cobra.Command) *BackupConfig {
config.publicKey = publicKeyFile config.publicKey = publicKeyFile
config.usingKey = usingKey config.usingKey = usingKey
config.cronExpression = cronExpression config.cronExpression = cronExpression
config.all = all
config.allInOne = allInOne
config.customName = customName
return &config return &config
} }

View File

@@ -26,7 +26,9 @@ package pkg
import ( import (
"bytes" "bytes"
"errors"
"fmt" "fmt"
goutils "github.com/jkaninda/go-utils"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"os" "os"
@@ -36,7 +38,7 @@ import (
) )
func intro() { func intro() {
fmt.Println("Starting MySQL Backup...") fmt.Println("Starting MYSQL-BKUP...")
fmt.Printf("Version: %s\n", utils.Version) fmt.Printf("Version: %s\n", utils.Version)
fmt.Println("Copyright (c) 2024 Jonas Kaninda") fmt.Println("Copyright (c) 2024 Jonas Kaninda")
} }
@@ -65,27 +67,30 @@ func deleteTemp() {
} }
} }
// TestDatabaseConnection tests the database connection // TestDatabaseConnection tests the database connection
func testDatabaseConnection(db *dbConfig) { func testDatabaseConnection(db *dbConfig) error {
err := os.Setenv("MYSQL_PWD", db.dbPassword) // Create the mysql client config file
if err != nil { if err := createMysqlClientConfigFile(*db); err != nil {
return return errors.New(err.Error())
} }
utils.Info("Connecting to %s database ...", db.dbName) utils.Info("Connecting to %s database ...", db.dbName)
// Set database name for notification error // Set database name for notification error
utils.DatabaseName = db.dbName utils.DatabaseName = db.dbName
cmd := exec.Command("mariadb", "-h", db.dbHost, "-P", db.dbPort, "-u", db.dbUserName, db.dbName, "-e", "quit")
// Prepare the command to test the database connection
cmd := exec.Command("mariadb", fmt.Sprintf("--defaults-file=%s", mysqlClientConfig), db.dbName, "-e", "quit")
// Capture the output // Capture the output
var out bytes.Buffer var out bytes.Buffer
cmd.Stdout = &out cmd.Stdout = &out
cmd.Stderr = &out cmd.Stderr = &out
err = cmd.Run()
if err != nil {
utils.Fatal("Error testing database connection: %v\nOutput: %s", err, out.String())
// Run the command
if err := cmd.Run(); err != nil {
return fmt.Errorf("failed to connect to database %s: %v, output: %s", db.dbName, err, out.String())
} }
utils.Info("Successfully connected to %s database", db.dbName)
utils.Info("Successfully connected to %s database", db.dbName)
return nil
} }
// checkPubKeyFile checks gpg public key // checkPubKeyFile checks gpg public key
@@ -183,3 +188,16 @@ func RemoveLastExtension(filename string) string {
} }
return filename return filename
} }
// Create mysql client config file
func createMysqlClientConfigFile(db dbConfig) error {
caCertPath := goutils.GetStringEnvWithDefault("DB_SSL_CA", "/etc/ssl/certs/ca-certificates.crt")
sslMode := goutils.GetStringEnvWithDefault("DB_SSL_MODE", "0")
// Create the mysql client config file
mysqlClientConfigFile := filepath.Join(tmpPath, "my.cnf")
mysqlCl := fmt.Sprintf("[client]\nhost=%s\nport=%s\nuser=%s\npassword=%s\nssl-ca=%s\nssl=%s\n", db.dbHost, db.dbPort, db.dbUserName, db.dbPassword, caCertPath, sslMode)
if err := os.WriteFile(mysqlClientConfigFile, []byte(mysqlCl), 0644); err != nil {
return fmt.Errorf("failed to create mysql client config file: %v", err)
}
return nil
}

View File

@@ -51,7 +51,10 @@ func StartMigration(cmd *cobra.Command) {
conf := &RestoreConfig{} conf := &RestoreConfig{}
conf.file = backupFileName conf.file = backupFileName
// Backup source Database // Backup source Database
BackupDatabase(dbConf, backupFileName, true) err := BackupDatabase(dbConf, backupFileName, true, false, false)
if err != nil {
utils.Fatal("Error backing up database: %s", err)
}
// Restore source database into target database // Restore source database into target database
utils.Info("Restoring [%s] database into [%s] database...", dbConf.dbName, targetDbConf.targetDbName) utils.Info("Restoring [%s] database into [%s] database...", dbConf.dbName, targetDbConf.targetDbName)
RestoreDatabase(&newDbConfig, conf) RestoreDatabase(&newDbConfig, conf)

View File

@@ -28,6 +28,7 @@ import (
"fmt" "fmt"
"github.com/jkaninda/go-storage/pkg/ftp" "github.com/jkaninda/go-storage/pkg/ftp"
"github.com/jkaninda/go-storage/pkg/ssh" "github.com/jkaninda/go-storage/pkg/ssh"
goutils "github.com/jkaninda/go-utils"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"os" "os"
@@ -37,9 +38,12 @@ import (
func sshBackup(db *dbConfig, config *BackupConfig) { func sshBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to Remote server") utils.Info("Backup database to Remote server")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database // Backup database
BackupDatabase(db, config.backupFileName, disableCompression) err := BackupDatabase(db, config.backupFileName, disableCompression, config.all, config.allInOne)
if err != nil {
recoverMode(err, "Error backing up database")
return
}
finalFileName := config.backupFileName finalFileName := config.backupFileName
if config.encryption { if config.encryption {
encryptBackup(config) encryptBackup(config)
@@ -91,6 +95,8 @@ func sshBackup(db *dbConfig, config *BackupConfig) {
} }
utils.Info("Uploading backup archive to remote storage ... done ") utils.Info("Uploading backup archive to remote storage ... done ")
duration := goutils.FormatDuration(time.Since(startTime), 0)
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
File: finalFileName, File: finalFileName,
@@ -98,12 +104,11 @@ func sshBackup(db *dbConfig, config *BackupConfig) {
Database: db.dbName, Database: db.dbName,
Storage: config.storage, Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName), BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime, Duration: duration,
EndTime: time.Now().Format(utils.TimeFormat()),
}) })
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") utils.Info("The backup of the %s database has been completed in %s", db.dbName, duration)
} }
func remoteRestore(db *dbConfig, conf *RestoreConfig) { func remoteRestore(db *dbConfig, conf *RestoreConfig) {
@@ -153,10 +158,13 @@ func ftpRestore(db *dbConfig, conf *RestoreConfig) {
} }
func ftpBackup(db *dbConfig, config *BackupConfig) { func ftpBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to the remote FTP server") utils.Info("Backup database to the remote FTP server")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database // Backup database
BackupDatabase(db, config.backupFileName, disableCompression) err := BackupDatabase(db, config.backupFileName, disableCompression, config.all, config.allInOne)
if err != nil {
recoverMode(err, "Error backing up database")
return
}
finalFileName := config.backupFileName finalFileName := config.backupFileName
if config.encryption { if config.encryption {
encryptBackup(config) encryptBackup(config)
@@ -203,6 +211,7 @@ func ftpBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup name is %s", finalFileName) utils.Info("Backup name is %s", finalFileName)
utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize))) utils.Info("Backup size: %s", utils.ConvertBytes(uint64(backupSize)))
utils.Info("Uploading backup archive to the remote FTP server ... done ") utils.Info("Uploading backup archive to the remote FTP server ... done ")
duration := goutils.FormatDuration(time.Since(startTime), 0)
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
@@ -211,10 +220,9 @@ func ftpBackup(db *dbConfig, config *BackupConfig) {
Database: db.dbName, Database: db.dbName,
Storage: config.storage, Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName), BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime, Duration: duration,
EndTime: time.Now().Format(utils.TimeFormat()),
}) })
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") utils.Info("The backup of the %s database has been completed in %s", db.dbName, duration)
} }

View File

@@ -25,6 +25,7 @@ SOFTWARE.
package pkg package pkg
import ( import (
"fmt"
"github.com/jkaninda/encryptor" "github.com/jkaninda/encryptor"
"github.com/jkaninda/go-storage/pkg/local" "github.com/jkaninda/go-storage/pkg/local"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
@@ -56,11 +57,17 @@ func StartRestore(cmd *cobra.Command) {
} }
func localRestore(dbConf *dbConfig, restoreConf *RestoreConfig) { func localRestore(dbConf *dbConfig, restoreConf *RestoreConfig) {
utils.Info("Restore database from local") utils.Info("Restore database from local")
basePath := filepath.Dir(restoreConf.file)
fileName := filepath.Base(restoreConf.file)
restoreConf.file = fileName
if basePath == "" || basePath == "." {
basePath = storagePath
}
localStorage := local.NewStorage(local.Config{ localStorage := local.NewStorage(local.Config{
RemotePath: storagePath, RemotePath: basePath,
LocalPath: tmpPath, LocalPath: tmpPath,
}) })
err := localStorage.CopyFrom(restoreConf.file) err := localStorage.CopyFrom(fileName)
if err != nil { if err != nil {
utils.Fatal("Error copying backup file: %s", err) utils.Fatal("Error copying backup file: %s", err)
} }
@@ -68,88 +75,79 @@ func localRestore(dbConf *dbConfig, restoreConf *RestoreConfig) {
} }
// RestoreDatabase restore database // RestoreDatabase restores the database from a backup file
func RestoreDatabase(db *dbConfig, conf *RestoreConfig) { func RestoreDatabase(db *dbConfig, conf *RestoreConfig) {
if conf.file == "" { if conf.file == "" {
utils.Fatal("Error, file required") utils.Fatal("Error, file required")
} }
extension := filepath.Ext(filepath.Join(tmpPath, conf.file))
rFile, err := os.ReadFile(filepath.Join(tmpPath, conf.file)) filePath := filepath.Join(tmpPath, conf.file)
outputFile := RemoveLastExtension(filepath.Join(tmpPath, conf.file)) rFile, err := os.ReadFile(filePath)
if err != nil { if err != nil {
utils.Fatal("Error reading backup file: %s ", err) utils.Fatal("Error reading backup file: %v", err)
} }
extension := filepath.Ext(filePath)
outputFile := RemoveLastExtension(filePath)
if extension == ".gpg" { if extension == ".gpg" {
decryptBackup(conf, rFile, outputFile)
if conf.usingKey {
utils.Info("Decrypting backup using private key...")
utils.Warn("Backup decryption using a private key is not fully supported")
prKey, err := os.ReadFile(conf.privateKey)
if err != nil {
utils.Fatal("Error reading public key: %s ", err)
}
err = encryptor.DecryptWithPrivateKey(rFile, outputFile, prKey, conf.passphrase)
if err != nil {
utils.Fatal("error during decrypting backup %v", err)
}
utils.Info("Decrypting backup using private key...done")
} else {
if conf.passphrase == "" {
utils.Error("Error, passphrase or private key required")
utils.Fatal("Your file seems to be a GPG file.\nYou need to provide GPG keys. GPG_PASSPHRASE or GPG_PRIVATE_KEY environment variable is required.")
} else {
utils.Info("Decrypting backup using passphrase...")
// decryptWithGPG file
err := encryptor.Decrypt(rFile, outputFile, conf.passphrase)
if err != nil {
utils.Fatal("Error decrypting file %s %v", file, err)
}
utils.Info("Decrypting backup using passphrase...done")
// Update file name
conf.file = RemoveLastExtension(file)
}
}
} }
if utils.FileExists(filepath.Join(tmpPath, conf.file)) { restorationFile := filepath.Join(tmpPath, conf.file)
err := os.Setenv("MYSQL_PWD", db.dbPassword) if !utils.FileExists(restorationFile) {
utils.Fatal("File not found: %s", restorationFile)
}
if err := testDatabaseConnection(db); err != nil {
utils.Fatal("Error connecting to the database: %v", err)
}
utils.Info("Restoring database...")
restoreDatabaseFile(db, restorationFile)
}
func decryptBackup(conf *RestoreConfig, rFile []byte, outputFile string) {
if conf.usingKey {
utils.Info("Decrypting backup using private key...")
prKey, err := os.ReadFile(conf.privateKey)
if err != nil { if err != nil {
return utils.Fatal("Error reading private key: %v", err)
} }
testDatabaseConnection(db) if err := encryptor.DecryptWithPrivateKey(rFile, outputFile, prKey, conf.passphrase); err != nil {
utils.Info("Restoring database...") utils.Fatal("Error decrypting backup: %v", err)
extension := filepath.Ext(filepath.Join(tmpPath, conf.file))
// Restore from compressed file / .sql.gz
if extension == ".gz" {
str := "zcat " + filepath.Join(tmpPath, conf.file) + " | mariadb -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName
_, err := exec.Command("sh", "-c", str).Output()
if err != nil {
utils.Fatal("Error, in restoring the database %v", err)
}
utils.Info("Restoring database... done")
utils.Info("Database has been restored")
// Delete temp
deleteTemp()
} else if extension == ".sql" {
// Restore from sql file
str := "cat " + filepath.Join(tmpPath, conf.file) + " | mariadb -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName
_, err := exec.Command("sh", "-c", str).Output()
if err != nil {
utils.Fatal("Error in restoring the database %v", err)
}
utils.Info("Restoring database... done")
utils.Info("Database has been restored")
// Delete temp
deleteTemp()
} else {
utils.Fatal("Unknown file extension %s", extension)
} }
} else { } else {
utils.Fatal("File not found in %s", filepath.Join(tmpPath, conf.file)) if conf.passphrase == "" {
utils.Fatal("Passphrase or private key required for GPG file.")
}
utils.Info("Decrypting backup using passphrase...")
if err := encryptor.Decrypt(rFile, outputFile, conf.passphrase); err != nil {
utils.Fatal("Error decrypting file: %v", err)
}
conf.file = RemoveLastExtension(conf.file)
} }
} }
func restoreDatabaseFile(db *dbConfig, restorationFile string) {
extension := filepath.Ext(restorationFile)
var cmdStr string
switch extension {
case ".gz":
cmdStr = fmt.Sprintf("zcat %s | mariadb --defaults-file=%s %s", restorationFile, mysqlClientConfig, db.dbName)
case ".sql":
cmdStr = fmt.Sprintf("cat %s | mariadb --defaults-file=%s %s", restorationFile, mysqlClientConfig, db.dbName)
default:
utils.Fatal("Unknown file extension: %s", extension)
}
cmd := exec.Command("sh", "-c", cmdStr)
output, err := cmd.CombinedOutput()
if err != nil {
utils.Fatal("Error restoring database: %v\nOutput: %s", err, string(output))
}
utils.Info("Database has been restored successfully.")
deleteTemp()
}

View File

@@ -27,6 +27,7 @@ package pkg
import ( import (
"fmt" "fmt"
"github.com/jkaninda/go-storage/pkg/s3" "github.com/jkaninda/go-storage/pkg/s3"
goutils "github.com/jkaninda/go-utils"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"os" "os"
@@ -37,9 +38,12 @@ import (
func s3Backup(db *dbConfig, config *BackupConfig) { func s3Backup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to s3 storage") utils.Info("Backup database to s3 storage")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database // Backup database
BackupDatabase(db, config.backupFileName, disableCompression) err := BackupDatabase(db, config.backupFileName, disableCompression, config.all, config.allInOne)
if err != nil {
recoverMode(err, "Error backing up database")
return
}
finalFileName := config.backupFileName finalFileName := config.backupFileName
if config.encryption { if config.encryption {
encryptBackup(config) encryptBackup(config)
@@ -91,6 +95,7 @@ func s3Backup(db *dbConfig, config *BackupConfig) {
} }
utils.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName)) utils.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
utils.Info("Uploading backup archive to remote storage S3 ... done ") utils.Info("Uploading backup archive to remote storage S3 ... done ")
duration := goutils.FormatDuration(time.Since(startTime), 0)
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
File: finalFileName, File: finalFileName,
@@ -98,12 +103,11 @@ func s3Backup(db *dbConfig, config *BackupConfig) {
Database: db.dbName, Database: db.dbName,
Storage: config.storage, Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName), BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime, Duration: duration,
EndTime: time.Now().Format(utils.TimeFormat()),
}) })
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") utils.Info("The backup of the %s database has been completed in %s", db.dbName, duration)
} }
func s3Restore(db *dbConfig, conf *RestoreConfig) { func s3Restore(db *dbConfig, conf *RestoreConfig) {

View File

@@ -24,6 +24,11 @@ SOFTWARE.
package pkg package pkg
import (
"path/filepath"
"time"
)
const tmpPath = "/tmp/backup" const tmpPath = "/tmp/backup"
const gpgHome = "/config/gnupg" const gpgHome = "/config/gnupg"
const gpgExtension = "gpg" const gpgExtension = "gpg"
@@ -39,7 +44,9 @@ var (
encryption = false encryption = false
usingKey = false usingKey = false
backupSize int64 = 0 backupSize int64 = 0
startTime string startTime = time.Now()
backupRescueMode = false
mysqlClientConfig = filepath.Join(tmpPath, "my.cnf")
) )
// dbHVars Required environment variables for database // dbHVars Required environment variables for database
@@ -47,7 +54,6 @@ var dbHVars = []string{
"DB_HOST", "DB_HOST",
"DB_PASSWORD", "DB_PASSWORD",
"DB_USERNAME", "DB_USERNAME",
"DB_NAME",
} }
var tdbRVars = []string{ var tdbRVars = []string{
"TARGET_DB_HOST", "TARGET_DB_HOST",
@@ -59,13 +65,6 @@ var tdbRVars = []string{
var dbConf *dbConfig var dbConf *dbConfig
var targetDbConf *targetDbConfig var targetDbConf *targetDbConfig
// sshVars Required environment variables for SSH remote server storage
var sshVars = []string{
"SSH_USER",
"SSH_HOST_NAME",
"SSH_PORT",
"REMOTE_PATH",
}
var ftpVars = []string{ var ftpVars = []string{
"FTP_HOST_NAME", "FTP_HOST_NAME",
"FTP_USER", "FTP_USER",

View File

@@ -60,10 +60,10 @@
<p>We recommend investigating the issue as soon as possible to prevent potential data loss or service disruptions.</p> <p>We recommend investigating the issue as soon as possible to prevent potential data loss or service disruptions.</p>
<p>For more information, visit the <a href="https://jkaninda.github.io/pg-bkup">pg-bkup documentation</a>.</p> <p>For more information, visit the <a href="https://jkaninda.github.io/mysql-bkup">mysql-bkup documentation</a>.</p>
<footer> <footer>
&copy; 2024 <a href="https://github.com/jkaninda/pg-bkup">pg-bkup</a> | Automated Backup System &copy; 2024 <a href="https://github.com/jkaninda/mysql-bkup">mysql-bkup</a> | Automated Backup System
</footer> </footer>
</body> </body>
</html> </html>

View File

@@ -52,8 +52,7 @@
<h3>Backup Details:</h3> <h3>Backup Details:</h3>
<ul> <ul>
<li><strong>Database Name:</strong> {{.Database}}</li> <li><strong>Database Name:</strong> {{.Database}}</li>
<li><strong>Backup Start Time:</strong> {{.StartTime}}</li> <li><strong>Backup Duration:</strong> {{.Duration}}</li>
<li><strong>Backup End Time:</strong> {{.EndTime}}</li>
<li><strong>Backup Storage:</strong> {{.Storage}}</li> <li><strong>Backup Storage:</strong> {{.Storage}}</li>
<li><strong>Backup Location:</strong> {{.BackupLocation}}</li> <li><strong>Backup Location:</strong> {{.BackupLocation}}</li>
<li><strong>Backup Size:</strong> {{.BackupSize}}</li> <li><strong>Backup Size:</strong> {{.BackupSize}}</li>

View File

@@ -6,8 +6,7 @@ Please find the details below:
Backup Details: Backup Details:
- Database Name: {{.Database}} - Database Name: {{.Database}}
- Backup Start Time: {{.StartTime}} - Backup Duration: {{.Duration}}
- Backup EndTime: {{.EndTime}}
- Backup Storage: {{.Storage}} - Backup Storage: {{.Storage}}
- Backup Location: {{.BackupLocation}} - Backup Location: {{.BackupLocation}}
- Backup Size: {{.BackupSize}} - Backup Size: {{.BackupSize}}

View File

@@ -39,8 +39,7 @@ type NotificationData struct {
File string File string
BackupSize string BackupSize string
Database string Database string
StartTime string Duration string
EndTime string
Storage string Storage string
BackupLocation string BackupLocation string
BackupReference string BackupReference string
@@ -84,3 +83,13 @@ func backupReference() string {
const templatePath = "/config/templates" const templatePath = "/config/templates"
var DatabaseName = "" var DatabaseName = ""
var vars = []string{
"TG_TOKEN",
"TG_CHAT_ID",
}
var mailVars = []string{
"MAIL_HOST",
"MAIL_PORT",
"MAIL_FROM",
"MAIL_TO",
}

View File

@@ -107,19 +107,6 @@ func sendMessage(msg string) error {
} }
func NotifySuccess(notificationData *NotificationData) { func NotifySuccess(notificationData *NotificationData) {
notificationData.BackupReference = backupReference() notificationData.BackupReference = backupReference()
var vars = []string{
"TG_TOKEN",
"TG_CHAT_ID",
}
var mailVars = []string{
"MAIL_HOST",
"MAIL_PORT",
"MAIL_USERNAME",
"MAIL_PASSWORD",
"MAIL_FROM",
"MAIL_TO",
}
// Email notification // Email notification
err := CheckEnvVars(mailVars) err := CheckEnvVars(mailVars)
if err == nil { if err == nil {
@@ -147,18 +134,6 @@ func NotifySuccess(notificationData *NotificationData) {
} }
} }
func NotifyError(error string) { func NotifyError(error string) {
var vars = []string{
"TG_TOKEN",
"TG_CHAT_ID",
}
var mailVars = []string{
"MAIL_HOST",
"MAIL_PORT",
"MAIL_USERNAME",
"MAIL_PASSWORD",
"MAIL_FROM",
"MAIL_TO",
}
// Email notification // Email notification
err := CheckEnvVars(mailVars) err := CheckEnvVars(mailVars)