Automated backups to FTP server using rsync and curlftpfs

Using curlftpfs and FUSE, a ftp site is mapped to the local file system and rsync will backup files to the ftp server automatically.

1. Install required packages

apt-get install curlftpfs rsync

2. Create directory to mount the ftp site

mkdir /mnt/ftpbackup

3. Add the curlftpfs mount into fstab to make it mount everytime the system is started (pico /etc/fstab)

curlftpfs#{username}:{password}@{host} /mnt/ftpbackup fuse rw,allow_other,uid={userid} 0 0

{username} = FTP username
{password} = FTP password
{host} = FTP host/ip
{userid} = ID of a local user (ex. 1001)

4. Mount the ftp site

mount /mnt/ftpbackup

5. Backup using rsync

rsync -avz --no-owner --no-group /var/www /mnt/ftpbackup

All files in the /var/www folder will be synced to the remote machine into a folder named backup

6. Automate the backup using cron (crontab -e)

0 3 * * * rsync -az --no-owner --no-group /var/www /mnt/ftpbackup >> /dev/null 2>&1

Your /var/www will be synced to the remote machine at 3am every day

11 comments

  1. I am getting an error like
    rsync: mkstemp “/mnt/ftp_backup/backup/Documents/myfile.eqU6Y0″ failed: Operation not supported (95)

  2. Does not seem to be possible with curlftpfs 0.9.2, since buffered writing
    has been turned off. A lot of tools fail now (rsync, tiobench, iozone, etc.) The project seems to be abandoned :-(

  3. Is it necessary to close the connection after the server has stored the data?
    With “mount / mnt / ftpbackup”?

    Thx

  4. Just a heads up for those old comments from fatih genç and Daniel: I’ve had the same problem with mkstemp failing for the reason “Operation not supported”, using curlftpfs 0.9.2…

    I’ve worked around it by using this:
    mkdir /tmp/temp-files
    rsync -avz -T “/tmp/temp-files/” –no-perms –no-owner –no-group –delete “source” “destination”

    The -T option instructs rsync to create temporary files at a specified place. Just put them somewhere not mounted with curlftpfs and it’ll work :)

  5. HI Olivier,

    Thanks for your idea, but when I do it this way, the rsync resend all the files already present on the ftp server.
    Do you have another workaround for that?

  6. Hi Ahmed,

    I’ve used the –size-only option to tell rsync not to check for modification time but just for differing size. It’s not optimal though, if you change a file but not its size, it won’t be uploaded.

    Cheers
    Blaf

  7. Hi,
    I tried the suggestion for temporary files from Oliver, and got error 5 instead, and the file was still written. I am still trying to figure out from curlftpfs what errors were being caused.

    rsync -avz -T “/var/tmp/rsync” –no-perms –no-owner –no-group ./cloud /export/ftp/htdocs

    got the error 5. As I said it is just “Input/Output” error from rsync, so I need to tap into the curlftpfs process non demonized and see what it is doing and work backward. There was little in googling suggesting anything to try for this error though a number of people had the error.

    The netsol website servers were the target of this, and they have been badly hacked via ftp, and may have defences at their end, rightly so, which are causing the problem. However we have a huge amount of storage paid for there and this work is related, so it would be nice to be able to use the space there.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>