How to Sync/replicate files on share between 2 Servers?

Thanks Dalight

As my servers are not in same forest, i cannot implement DFS, that leaves me to work with third party softwares.

Well i have zeroed on microsoft Robocopy, its a dos based application, come with window 2000/2003 resource kit. one can find it on 2000/2003 installation disk,

Very shortly i will be posting a dos batch script to replicate 2 way shared folder. We can startup this script on boot, by placeing it in startup schedule.

And the best part is ... it can generate logs based on date and time and files replicated & time taken to replicate.

Thanks people for your support.

Glad that you've found a solution, jayveshne. Also if you're scripting for Windows 2003/XP you may want to check out Microsoft's new scripting language, Windows Powershell (formerly called Monad).

Hello Friends,

As promised i have uploading scripts to sync/replicate between shares so that they become identical,

As i said earlier, i will be using robocopy. The current version of robocopy is "Robocopy XP010",

robocopy is very powerful command line utility, with many command line parameters . I suggest you all to go through its help. you may get the help by

[code:1]Robocopy.bat /? > C:\tmp\help-robo.txt[/code:1]

Robocopy maintains NTFS permissions on files that will be copied to remote share if required.

Let me explain major options that can be run as parameter along with robocopy..

The /l switch tells Robocopy to only list the files, folders, and permissions it would have copied. In other words, Robocopy doesn't copy any files. I recommend that you use this switch to test any copy operation before actual implementation, no matter how small the copy operation is.

The /e switch tells Robocopy to copy all folders, even empty folders.

The /np switch turns off Robocopy's progress indicator. By default, Robocopy displays a progress indicator, but you don't need this indicator when you use Robocopy in a script.

The /purge switch tells Robocopy to remove any files from the destination folder that have been deleted from the source folder since Robocopy last ran. You use this switch only if you plan to use multiple copy operations to complete a migration.

The /secfix switch migrates NTFS security permissions. The /secfix switch is a subset of the /sec switch. If you plan to complete the migration in one copy operation, you use the /sec switch. If you plan to use multiple copy operations, you use the /secfix switch because it fixes any permissions that have changed in the source folder since Robocopy last ran.

The /r:1 switch tells Robocopy how many times to retry the copy operation if it fails (in this case, one retry), and the /w:1 switch specifies how long (in seconds) to wait before each retry (in this case, 1 second). I highly recommend that you set these switches to avoid the defaults Robocopy uses—1 million attempted retries every 30 seconds can bring your copy operation to a grinding halt for 347 days!

Step by step procedure to setup replication...

1) Create a template for jobs, templates are plain text file with all pre-defined command line options. This is helpful when you have replicate many folders with different options, so each replication will have its own template. Save this templates as extension of .RJC
Example : F-drivetemplate.rjc
[code:1]
:: Source Directory :
::
/SD:F: \ :: Source Directory.

::
:: Destination Directory :
::
/DD:\\10.0.0.10\F-Drive :: Destination Directory.

::
:: Include These Files :
::
/IF :: Include Files matching these names
:: *.* :: Include all names (currently - Command Line may override)

::
:: Exclude These Directories :
::
/ :: eXclude Directories matching these names
:: :: eXclude no names (currently - Command Line may override)

::
:: Exclude These Files :
::
/XF :: eXclude Files matching these names
:: :: eXclude no names (currently - Command Line may override)
/XO :: eXclude Older files.
::
:: Copy options :
::
/MON:1 :: MONitor source; run again when more than n changes seen.
/COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
/Z :: copy files in restartable mode.
::
:: Retry Options :
::
/R:1000000 :: number of Retries on failed copies: default 1 million.
/W:30 :: Wait time between retries: default is 30 seconds.
::
:: Logging Options :
::
:: /V :: produce Verbose output, showing skipped files.
:: /LOG+:C:\robocopy\intalink.log :: output status to LOG file (append to existing log).

/maxage:7 :: Maximum file age is 7 days old.
:: /xn :: exclude newer files
/rh:0200-2359 :: set run hours between 2am and 12pm
[/code:1]


2) Create a replication calling batch file
Example : replicate.bat

[code:1]
ECHO OFF

ENVTIME > TIME.BAT
CALL TIME.BAT
ECHO.
ECHO %HOUR%:%MIN% %DOW% %MON% - %DOM% - %YR%
DEL TIME.BAT
robocopy /job: F-drivetemplate.rjc /log+:logs\%dow%%dom%%year%.log
[/code:1]

3) Remember to place robocopy.exe in %windir%, or any path folder so that it can be invoked from any location.

You need to customize the switches in F-drivetemplate.rjc to suit your environment. For example, if you plan to copy thousands of files and hundreds of directories, you can temporarily modify the script to run the Robocopy command with the /create switch. This switch creates the destination directory structure with no files and minimizes the fragmentation of the directories on the destination disk. (For more information about the /create switch, see robocopy help) If you decide you want the functionality of both the /e and /purge switches, an alternative is to use the /mir switch. Specifying the /mir switch is equivalent to specifying the /e and /purge switches. . .


To conclude…
Place replication.bat on “schedule task” on startup, and this job schedule should have appropriate credential to run, so as soon as your system is up & running , script is running at background. Its not a service/daemon

It uses standard windows file & printer sharing service to replicate, i.e port 135/137
Some times they are blocked over wan link because most worms use these ports to spread.

It doesn’t use any compression to replicate. (bottleneck to WAN links)

Every time it replicates, it gets the list of file from remote directory tree to compare, if some database was used to store this list , it would be better and faster.

Well it can generate logs.. better for analysis.

If any one has queries I will be happy to reply..
[/code]

In the first place , why do you want to maintain two drives. Since the two drives most be identical, then it is possible to maintain one share drive(F) with different share permission using NTFS formatted drives .

Therefore , no need to use any third party replication software and compression could also be supported. Sounds much more economical !

I remain
SOSE