Debian Tutorials

Debian Tutorials


Step by step tutorials showing you how to install and configure various applications and services on Debian based Linux distros.

April 2018
M T W T F S S
« Mar    
 1
2345678
9101112131415
16171819202122
23242526272829
30  

Categories


Automated backups to FTP server using rsync and curlftpfs

Ástþór IPÁstþór IP

Using curlftpfs and FUSE, a ftp site is mapped to the local file system and rsync will backup files to the ftp server automatically.

1. Install required packages

apt-get install curlftpfs rsync

2. Create directory to mount the ftp site

mkdir /mnt/ftpbackup

3. Add the curlftpfs mount into fstab to make it mount everytime the system is started (pico /etc/fstab)

curlftpfs#{username}:{password}@{host} /mnt/ftpbackup fuse rw,allow_other,uid={userid} 0 0

{username} = FTP username
{password} = FTP password
{host} = FTP host/ip
{userid} = ID of a local user (ex. 1001)

4. Mount the ftp site

mount /mnt/ftpbackup

5. Backup using rsync

rsync -avz --no-owner --no-group /var/www /mnt/ftpbackup

All files in the /var/www folder will be synced to the remote machine into a folder named backup

6. Automate the backup using cron (crontab -e)

0 3 * * * rsync -az --no-owner --no-group /var/www /mnt/ftpbackup >> /dev/null 2>&1

Your /var/www will be synced to the remote machine at 3am every day

Comments 13
  • fatih genç
    Posted on

    fatih genç fatih genç

    Author

    I am getting an error like
    rsync: mkstemp “/mnt/ftp_backup/backup/Documents/myfile.eqU6Y0” failed: Operation not supported (95)


  • Daniel
    Posted on

    Daniel Daniel

    Author

    Does not seem to be possible with curlftpfs 0.9.2, since buffered writing
    has been turned off. A lot of tools fail now (rsync, tiobench, iozone, etc.) The project seems to be abandoned 🙁


  • Rem
    Posted on

    Rem Rem

    Author

    Is it necessary to close the connection after the server has stored the data?
    With “mount / mnt / ftpbackup”?

    Thx


  • Rem
    Posted on

    Rem Rem

    Author

    Sorry, with “umount / mnt / ftpbackup” ??


  • Damian
    Posted on

    Damian Damian

    Author

    No matter what I tried, this is unable to work on files over 1gb in size.


  • Deter
    Posted on

    Deter Deter

    Author

    Great, all my data are in safe now .)


  • Olivier
    Posted on

    Olivier Olivier

    Author

    Just a heads up for those old comments from fatih genç and Daniel: I’ve had the same problem with mkstemp failing for the reason “Operation not supported”, using curlftpfs 0.9.2…

    I’ve worked around it by using this:
    mkdir /tmp/temp-files
    rsync -avz -T “/tmp/temp-files/” –no-perms –no-owner –no-group –delete “source” “destination”

    The -T option instructs rsync to create temporary files at a specified place. Just put them somewhere not mounted with curlftpfs and it’ll work 🙂


  • Ahmed
    Posted on

    Ahmed Ahmed

    Author

    HI Olivier,

    Thanks for your idea, but when I do it this way, the rsync resend all the files already present on the ftp server.
    Do you have another workaround for that?


  • Blaf
    Posted on

    Blaf Blaf

    Author

    Hi Ahmed,

    I’ve used the –size-only option to tell rsync not to check for modification time but just for differing size. It’s not optimal though, if you change a file but not its size, it won’t be uploaded.

    Cheers
    Blaf


  • jim
    Posted on

    jim jim

    Author

    Hi,
    I tried the suggestion for temporary files from Oliver, and got error 5 instead, and the file was still written. I am still trying to figure out from curlftpfs what errors were being caused.

    rsync -avz -T “/var/tmp/rsync” –no-perms –no-owner –no-group ./cloud /export/ftp/htdocs

    got the error 5. As I said it is just “Input/Output” error from rsync, so I need to tap into the curlftpfs process non demonized and see what it is doing and work backward. There was little in googling suggesting anything to try for this error though a number of people had the error.

    The netsol website servers were the target of this, and they have been badly hacked via ftp, and may have defences at their end, rightly so, which are causing the problem. However we have a huge amount of storage paid for there and this work is related, so it would be nice to be able to use the space there.


  • jorge huerga
    Posted on

    jorge huerga jorge huerga

    Author

    Olivier you are great! (and post owner too)


  • Hitesh Riziya
    Posted on

    Hitesh Riziya Hitesh Riziya

    Author

    I ended up using this command. Thank you, Oliver, Blaf and post owner 🙂

    [email protected]:~$ sudo rsync -avz -T “/tmp/temp-files/” –size-only –no-perms –no-owner –no-group /home/uweenggs/Documents/SVN/ /mnt/ftpbackup


  • Hue Holleran
    Posted on

    Hue Holleran Hue Holleran

    Author

    Hopefully useful for others even though it’s ~1 year later – the –inplace option worked for me instead of -T /tmp/. and avoids superfluously making a copy of the file in the tmp directory.

    Thanks all for your help and I used Hitesh’s command posted above and used –inplace, viz:

    rsync -av –inplace –size-only –no-perms –no-owner –no-group /src /tgt