1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[Announce] Rsync for NAS200

Discussion in 'Cisco/Linksys Network Storage Devices' started by dsc68, May 1, 2008.

  1. dsc68

    dsc68 Addicted to LI Member

    I have rsync running on a NAS200 under Jac0's alternative firmware.

    Installing
    To install rsync:

    1. Install Jac0's alternative firmware and set up ssh as per the instructions
    2. Download rsync.tar and copy it to your NAS200
    3. Log into your NAS200 and untar the file in the root directory
    4. Reboot your NAS200

    Using
    There are two ways of rsync'ing:

    1. rsync protocol (unencrypted)

    rsync -av --stats files admin@192.168.1.1::disk1

    You will need to provide an rsync module password (see below)

    2. rsync over ssh (encrypted but 5 times slower)

    rsync -av --stats -e "ssh -l root" --rsync-path=/harddisk/volume_1/conf/bin/rsync files admin@192.168.1.1::disk1

    You will need to provide both the root password and an rsync module password (see below)

    where files are the files you wish to rsync and 192.168.1.1 is the IP address of your NAS200

    Note: It doesn't seem possible to set up a user to have both ssh and web/ftp access.

    Configuration
    There are two configuration files:

    /harddisk/volume_1/conf/rsyncd.config
    Configuration information for rsyncd.
    One rsync module is set up: disk1

    /harddisk/volume_1/conf/rsyncd.secrets
    Username and password for the modules.
    Default is admin:admin

    Note
    To be accessible from the Linksys web interface, files need to be group readable. You may need to use the either "--chmod=Dg+x,g+r" on the client side or add "incoming chmod = Dg+x,g+r" in the rsyncd.conf file for the module.
     
  2. jac_goudsmit

    jac_goudsmit Super Moderator Staff Member Member

    Great stuff, dsc68!

    Thanks!

    ===Jac
     
  3. subledub

    subledub Addicted to LI Member

    Hi,
    the idea is very good... but the link to the tar is not good anymore :(
    Please dsc68, could you solve that ?

    I 'll be pleased to try your stuff
    thx
     
  4. dsc68

    dsc68 Addicted to LI Member

    Thanks,

    I've fixed the links.
     
  5. subledub

    subledub Addicted to LI Member

    thanks for the update :)
    i can't make it start, sorry .. so let's try to fix it and help the other nuts like me :confused:

    _ my root directory is / cf : Use vi to edit /etc/passwd to change the login directory from "/root" to e.g. "/"
    _ i untar file in windows an copy with winscp to /harddisk/volume_1/conf/ and /harddisk/volume_1/data/
    _ i reboot .. but rsync is not runing ...
    _ try to lunch the daemon by hand, but i don't find the good command and/or path

    I didn't modify the conf file, because it seems to be ok

    So, what did i forget to do ?

    any help will be welcome, because i want to choose between RAID1 and rsync, to have a replication of my dear data, and i would prefer using jac0 + rsync rather than R75 RAID1

    in my dream i would have prefer R75 with RAID1 with rsync with ssh and sftp
    but i am not able to make by myself .. so, waiting for the sun :)
    thx all of us
     
  6. dsc68

    dsc68 Addicted to LI Member

    Did you reboot the NAS200 after the install? The rsync daemon is started by inetd when a connection is made to the rsync port. The rsync service is added to the inetd.conf file during boot up.

    My NAS200 is currently running as my remote backup so I can't play with it at the moment. I have a second NAS200 which I will be using to develop under R75 but I am currently awaiting a new drive for it.
     
  7. fubarbloke

    fubarbloke Addicted to LI Member

    I'm having what looks like the same problem as subledub. Tried several different ways of copying and/or untarring the tarball, and rsync isn't running. Haven't tried to figure out how to launch the daemon and/or rsync manually, but that's next on the list.

    rc.rsyncd is writing to inetd.conf -- that much I can see. And the port seems to be properly assigned in /etc/services.

    One thing I did notice when I was looking at the rsyncd manual, is that according to the manual the inetd.config line should read something like:

    rsync stream tcp nowait root /usr/bin/rsync rsyncd --daemon

    The rc.rsyncd file (and consequently inetd.conf) doesn't have "rsyncd" before "--daemon". Don't know if that's important or not. I tried modifying rc.rsyncd (and inetd.conf directly, FWIW), but there seemed to be no effect.
     
  8. fubarbloke

    fubarbloke Addicted to LI Member

    I got rsync to launch just fine. I just used the following command at the $ prompt:

    Code:
    /harddisk/volume_1/conf/bin/rsync
    That pulled up the help screen, which you can go to directly at:

    Code:
    /harddisk/volume_1/conf/bin/rsync --help
    Then I tried the same thing with the daemon:

    Code:
    /harddisk/volume_1/conf/bin/rsync --daemon --help
    And it worked fine. A list of daemon options came up. Then I went right to the daemon itself:

    Code:
    /harddisk/volume_1/conf/bin/rsync --daemon
    All that came back on the screen was the $ prompt. Good news in a way -- at least it knew what I was talking about. Then I checked the rsyncd.log file in my root directory (/harddrive/volume_1/data), and saw this:

    Code:
    2008/12/09 18:13:18 [3161] rsyncd version 3.0.2 starting, listening on port 873
    2008/12/09 18:13:18 [3161] socket(10,1,6) failed: Address family not supported by protocol
    2008/12/09 18:13:18 [3161] bind() failed: Address already in use (address-family 2)
    2008/12/09 18:13:18 [3161] unable to bind any inbound sockets on port 873
    2008/12/09 18:13:18 [3161] rsync error: error in socket IO (code 10) at socket.c(535) [receiver=3.0.2]
    
    I double checked -- it's generated as a result of running /harddisk/volume_1/conf/bin/rsync --daemon

    The next task on my list is to figure out what it all means...
     
  9. fubarbloke

    fubarbloke Addicted to LI Member

    Thought about it a little, and figured that the likeliest explanation for the "in use" error is that the daemon is already running, and all I did was try to invoke it again. It's a working hypothesis, anyway.

    So I figured I'd stay with what worked, which is running rsync directly from the binary. So I tested the following command:

    Code:
    /harddisk/volume_1/conf/bin/rsync -av --stats --log-file=/harddisk/volume_1/data/rsync.log /harddisk/volume_1/data/ /harddisk/volume_2/data/
    Worked fine via the command line -- it mirrored drive 1 to drive 2. So I put it into rc.rsyncd as a cron job:

    Code:
    echo "0 20 * * * /harddisk/volume_1/conf/bin/rsync -av --stats --log-file=/harddisk/volume_1/data/rsync.log /harddisk/volume_1/data/ /harddisk/volume_2/data/ &>/devnull" >>/etc/crontab
    I verified that it was written (after a reboot of course) to /etc/crontab. And waited for 8 pm to roll around. No joy. So I tried running it again from the command line, and it still worked fine. So I must be doing something wrong with the cron job.

    That's a problem for tomorrow, though.
     
  10. fubarbloke

    fubarbloke Addicted to LI Member

    Found the problems. First, no user is specified in the cron job. Second, there's a typo in the dev/null instruction.

    Another problem might be that I'm trying to pack too much into a crontab statement. So, I took the rsync instruction and put it into a script named mirror.sh:

    Code:
    /harddisk/volume_1/conf/bin/rsync -av --stats --log-file=/harddisk/volume_1/data/rsync.log /harddisk/volume_1/data/ /harddisk/volume_2/data/
    I created a bin directory in my root user directory, and put mirror.sh into it. Then I set the permissions:

    Code:
    chmod a+rx mirror.sh
    Finally, I modified the rc.rsyncd file:

    Code:
    ln -s /harddisk/volume_1/conf/rsyncd.conf /etc/rsyncd.conf
    echo "rsync stream tcp nowait root /harddisk/volume_1/conf/bin/rsync --daemon" >>/etc/inetd.conf
    echo "0 20 * * * root /harddisk/volume_1/data/bin/mirror.sh &>/dev/null" >>/etc/crontab
    
    I rebooted with /etc/rc.d/rc.reboot, and waited.

    Seemed to work fine. The script ran (I tested with different times in the cron job, so I wouldn't have to wait until 8 tonight :)). Everything looks good. The log file shows no errors, it seems to show that the right stuff did happen, and rsync terminated.

    I think I have a simple mirroring solution. I'll let it run for a few days and see what happens.
     
  11. HrRenger

    HrRenger Addicted to LI Member

    I need some expert advice as it seems that I am stuck.

    I have installed jac2b package on my NAS200 and also put the rsync package provided by dsc68 (BTW - both great SW, many thanks for this).

    When I am rebooting the NAS, for whatever reason the rsync daemon does not seem to start. In the logfile I am getting this entry:

    2009/03/31 22:59:16 [1027] rsyncd version 3.0.2 starting, listening on port 873
    2009/03/31 22:59:16 [1027] socket(10,1,6) failed: Address family not supported by protocol
    2009/03/31 22:59:16 [1027] bind() failed: Address already in use (address-family 2)
    2009/03/31 22:59:16 [1027] unable to bind any inbound sockets on port 873
    2009/03/31 22:59:16 [1027] rsync error: error in socket IO (code 10) at socket.c(535) [receiver=3.0.2]

    If I run "/harddisk/volume_3/conf/bin/rsync --daemon" the same log entry is generated and rsync is not running in daemon mode (not listed in the the running processes).

    When I am running rsync from a command line to duplicate directories/files on the NAS200 from one place to another, this works great. So at least something is working.

    Does anybody have a clue what I am missing here? Seems to be a similar issue as brought up in this threat already (but I couldn't find if and how it was resolved).

    My rsyncd.conf file looks (/harddisk/volume_3/conf/) like this:

    log file = /harddisk/volume_3/data/rsyncd.log
    pid file = /harddisk/volume_3/data/rsyncd.pid
    lock file = /harddisk/volume_3/rsync.lock

    [backup_disk]
    path = /harddisk/volume_3/data/
    uid = root
    gid = root
    read only = no
    list = yes
    auth users = backupuser
    incoming chmod = Dg+x,g+r
    secrets file = /harddisk/volume_3/conf/rsyncd.secrets

    There is also a link /etc/rsyncd.conf -> /harddisk/volume_3/conf/rsyncd.conf. The rsyncd.secrets file is in the same directory and has the entry "root:rootpwd".

    Does anybody have an idea what I got wrong in my setup? Do the different files (rsyncd.conf, rsyncd.secrets) need to have specific permissions? Also I assume that the rootpwd is entered in plain text (as when using the passwd commend to reset the root password. In the passwd file it is contained encrypted).

    Finally, what I would like to achieve is that the NAS200 is running rsync (daemon) and I can initiate an rsync backup from another NAS to the NAS200.

    Many thanks in advance for any suggestion or ideas on how to resolve the above issues.
     
  12. jackito

    jackito LI Guru Member

    I checked the /etc/services file in the NAS200 and the rsync ports are missing. Add this lines to /etc/services:

    Code:
    rsync           873/tcp                         # rsync
    rsync           873/udp                         # rsync
    
    Then restart rsync and check if it is ok.
     
  13. HrRenger

    HrRenger Addicted to LI Member

    I checked the /etc/services file. The rsync entries as above are there.
     
  14. jackito

    jackito LI Guru Member

    Sorry it was my mistake. The entries are also in my /etc/services file.
    Also I reproduce the scenario that you proposed exactly as you describe it. But for me rsync is starting as daemon without problems at least from the shell (no startup script).

    This is a really silly idea but did you checked that you don´t have any other rsync (or something else) process running using that port?
    It is really easy to find out. Just do:

    Code:
    # netstat -na
    
    And in the output of netstat look at all the rows that end with LISTENING searching for something like 0.0.0.0:873 or something like some_ip_address:873.
    If you find something then there is your problem.
     
  15. HrRenger

    HrRenger Addicted to LI Member

    I have now made some progress. The most part of the problem described above was related to starting the rsync daemon directly while it was already started by inet. Once I figured this out, the rest was pretty straightforward.

    The only open point is the setting of a password to be stored in the rsyncd.secrets. Currently I am using admin as rsync user, however I have not managed to put the right credentials admin:admin_pwd into the secrets file. The admin password seems to be encrypted, as entering the plain pwd into the secrets file does not work. Anybody has a clue which format the pwd needs to have and where I can find this on the system. Current work around is to run rsync without pwd by wiping this out in the rsyncd.conf file.
     
  16. jackito

    jackito LI Guru Member

    Check the permissions of the rsyncd.secrets file. They should be 600 or rw only for root. Also the passwords inside the file should be in plain text.
     
  17. pmcneil

    pmcneil Addicted to LI Member

    you need to add the following line to rsyncd.conf because the rsyncd.secrets file is owned by root, not the admin user.

    strict modes = false
     
  18. pmcneil

    pmcneil Addicted to LI Member

    also beware fancy passwords with punctuation may not work :)
     
  19. pwest

    pwest Addicted to LI Member

    All:
    I just installed and configured this rsync package under Jac3, and rsync works with and without ssh. But, I have been unable to get things working with keypair authentication. Here's what I've done:

    From my PC (cygwin) I execute:
    $ rsync -av --stats -e "ssh -l root" --rsync-path=/harddisk/volume_1/conf/bin/rsync files admin@192.168.1.1::disk1

    I am prompted first for root's password, and then for admin's password. If I correctly offer these, the rsync proceeds.

    In my quest for passwordless backups, I created a new user, sshuser, (with gid=admin) with authorized keys so that I can establish passwordless ssh logins. That is, from cygwin
    $ ssh sshuser@192.168.1.1
    logs me in to a BusyBox sessions with no password needed. Also, I removed the line
    auth users = admin
    from rsyncd.conf so that rsync doesn't need passwords either.

    Thus, from cygwin, the command:
    $ rsync -av --stats . 192.168.1.1::disk1
    works with no logins. But, if I combine the two, viz
    $ rsync -av --stats -e "ssh -l sshuser" --rsync-path=/harddisk/volume_1/conf/bin/rsync files admin@192.168.1.1::disk1

    the operation does not prompt for passwords, but fails with error:
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/io.c(632) [sender=3.0.4]


    Thanks for any suggestions,
    Phil
     
  20. pwest

    pwest Addicted to LI Member

    After fiddling with things for a couple of days, all is working fine.
    To get it to work, I had to be careful with userid's, privs, and explicit paths.

    One other important thing to know is that when using ssh, rsyncd is not called, and therefore the settings in rsyncd.conf are meaningless--thus, the '::disk1' notation does not work.

    -Phil
     
  21. Jeangouytch

    Jeangouytch Addicted to LI Member

    hi

    i'm trying to rsync folder from the nas sata drive to a USB hardrive plugged in the NAS.
    It seems that all directory with uppercase name are automatically converted to lower case on the USB drive, which confuse rsync.

    Any idea of what's happening ?
    thanks
     
  22. jac_goudsmit

    jac_goudsmit Super Moderator Staff Member Member

    I bet you're running into the problem that the FAT filesystem (used on USB drives by default) is case-insensitive.

    One way around this is to format your USB drive in ext2 (but then it won't get automatically mounted or shared) or to create a loop-mount filesystem on the USB drive and and mount it, then rsyncing to there. Don't forget to unmount the loop filesystem before you unplug it, though!

    ===Jac
     
  23. matt276

    matt276 Addicted to LI Member

    This is the method i used to have rsyncd working :

    - Download rsync-3.0.2-nas200-2dsc.tar
    - uncompress it.
    - connect with root user to the nas with winscp
    - copy uncompressed tree to the root folder (/) on the nas
    - reboot the nas with /etc/rc.d/rc.reboot
    - change perms :
    chmod 777 /harddisk/volumes_1/conf/bin/rsyncd
    chmod 600 /harddisk/volumes_1/conf/rsyncd.secrets

    and it works fine (unencrypted)

    Thanks to dsc68.

    Matt
     
  24. Jeangouytch

    Jeangouytch Addicted to LI Member

    hi

    I read somewhere that fat32 drive can be mounted with an option "shortname=mixed" in order to prevent all path being converted to lowercase.

    Any chance we can change the way usb drives are mounted by default on the nas ?

    thank you

    G.
     
  25. mdsilva

    mdsilva Addicted to LI Member

    Hi everyone, I simply can't get this rsync stuff to work. I've installed it fine on the NAS200 (i think!) and I've installed cygwin+rsync and i get this error:

    Code:
    bash-3.2$ rsync -av --stats /cygdrive/f/Downloads admin@192.168.10.11::disk2
    rsync: read error: Connection reset by peer (104)
    rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6]
    
    Here is my /etc/rsyncd.conf (mostly the same as the examples)
    Code:
    log file = /harddisk/volume_1/data/rsyncd.log
    pid file = /var/run/rsyncd.pid
    lock file = /var/lock/rsync.lock
    [disk2]
    path = /harddisk/volume_2/data/public/backup
    uid = admin
    gid = administrators
    read only = no
    list = yes
    auth users = admin
    incoming chmod = Dg+x,g+r
    secrets file = /harddisk/volume_1/conf/rsyncd.secrets
    
    Any one know what i'm doing wrong?:confused:
     
  26. pwest

    pwest Addicted to LI Member

    A lot of my problems were caused by file ownership and privs. Make sure that all the files and directories at the Disk2 level and below are world readable and writable and give it another try.

    -Phil
     
  27. mdsilva

    mdsilva Addicted to LI Member

    Thanks for your help, i got it to work now :)

    Any ideas why its so slow? I run the command below, and get about 1.75MB/s In win7 when i drag and drop files to the NAS200 network share i usually get 4MB/s anyway to tweak my command i run or anything? :confused:

    Code:
    rsync -av --stats --progress /cygdrive/f/Downloads admin@192.168.10.11::disk2
    
     
  28. mdsilva

    mdsilva Addicted to LI Member

    Anyone?
     

Share This Page