Skip to content

[Bug] Backup fails with Windows NFS storage due to colons in filename #1255

@ralite

Description

@ralite

Documentation

Describe the bug

When using the Backup CRD to create database backups with a Windows NFS server as storage, the backup job fails with an "Input/output error" when attempting to write to the backup file. This occurs because the default filename format includes colons (:) in the timestamp format (%Y-%m-%dT%H:%M:%SZ), which are not allowed in Windows filesystems.

In the job container, we can observe:

root@backup-manual-job-fixed-9z2fb:/# cat /backup/0-backup-target.txt
/backup/backup.2025-04-18T12:06:38Z.sql
root@backup-manual-job-fixed-9z2fb:/# $(cat /backup/0-backup-target.txt)
bash: /backup/backup.2025-04-18T12:06:38Z.sql: Input/output error

From the logs:

mariadb 💾 Exporting env
mariadb 💾 Writing target file: /backup/0-backup-target.txt
mariadb 💾 Taking backup: /backup/backup.2025-04-18T12:31:05Z.sql
mariadb bash: $(cat '/backup/0-backup-target.txt'): Input/output error

The underlying issue is that Windows filesystems do not allow colons in filenames, which makes the backup feature incompatible with Windows-based NFS servers.

Expected behaviour

The backup job should use a filename format that is compatible with all common filesystems, especially Windows filesystems which are frequently used for NFS servers in enterprise environments.

One solution would be to use a different date format that doesn't include colons, such as %Y-%m-%dT%H-%M-%SZ (using hyphens instead of colons for hours, minutes, seconds).

Steps to reproduce the bug

  1. Set up a Windows-based NFS server as storage for backups
  2. Create a MariaDB instance using the operator
  3. Create a Backup resource pointing to this MariaDB instance with storage mounted from the Windows NFS server
  4. Observe the backup job failing with I/O errors when trying to write to a file with colons in the name

Debug information

  • From the failing job pod, we see the command:
set -euo pipefail;echo 💾 Exporting env;export BACKUP_FILE=/backup/backup.$(date -u +'%Y-%m-%dT%H:%M:%SZ').sql;echo 💾 Writing target file: /backup/0-backup-target.txt;printf "${BACKUP_FILE}" > /backup/0-backup-target.txt;echo 💾 Taking backup: $(cat '/backup/0-backup-target.txt');mariadb-dump --user=${MARIADB_OPERATOR_USER} --password=${MARIADB_OPERATOR_PASSWORD} --host=myapp-db.test-jerome.svc.cluster.local --port=3306 --single-transaction --events --routines --all-databases --ssl --ssl-ca /etc/pki/ca.crt --ssl-cert /etc/pki/client.crt --ssl-key /etc/pki/client.key --ssl-verify-server-cert --verbose > $(cat '/backup/0-backup-target.txt')

When the backup job tries to write to this file, it fails with an I/O error because the filename contains colons which are invalid in Windows filesystems.

Environment details:

  • Kubernetes version: 1.31.4
  • Kubernetes distribution: RKE
  • MariaDB Operator version: 0.38.0
  • MariaDB Server version: [Version number]
  • Install method: Helm
  • Storage: Windows-based NFS server

Additional context

The issue is critical for environments using Windows-based NFS servers for storage, which is common in many enterprise setups. The current implementation makes the backup feature unusable in these environments.

The issue appears to be in the backup script's use of the date format. A simple change to the date format in the script would fix this issue:

Change:

export BACKUP_FILE=/backup/backup.$(date -u +'%Y-%m-%dT%H:%M:%SZ').sql

To:

export BACKUP_FILE=/backup/backup.$(date -u +'%Y-%m-%dT%H-%M-%SZ').sql

This would avoid the problematic colons in filenames while still maintaining ISO 8601 readability.

Proposed solution

Either:

  1. Change the default date format to use hyphens instead of colons, making it compatible with all common filesystems
  2. Add an option in the Backup CRD to configure the date format used in backup filenames, with a safe default that doesn't include characters problematic for filesystems (like colons)

This issue affects cross-platform compatibility and should be addressed to ensure the operator works in diverse storage environments.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy