Files Synchronization between servers

This sections will contains code regarding different PHP & MySQL features and contains help about php/mysql issues.

Files Synchronization between servers

Postby Web Guru on July 21st, 2008, 4:20 pm

If you want to synchronize the files between two ftp servers in php, then certainly this is the right post for you. In this post i will provide you the code to synchronize the files/folders between two servers in an efficient manner.

The code consists of two files, one will be placed on the source server and the other will be placed on the host server. I will call the file as "mirror-agent.php" which will be placed on the remote ftp server. The file which will be placed on the host server, i will call that as "ftp-mirror.php". Below is the code of these files.

mirror-agent.php
Code: Select all
<?php
   // disable errors ( files can be unreadable by PHP due to permissions )
   error_reporting(0);

   // disable time limit ( you never know how big this site is )
   set_time_limit(0);


   // convert all files to hex
   function recur_dir($dir)
   {
      $dirlist = opendir($dir);

      while ($file = readdir ($dirlist))
      {
         if ($file != '.' && $file != '..' && $file != '.DS_Store' && $file != 'mirror-agent.php')
         {
            $newpath = $dir.'/'.$file;
            $permissions = substr(decoct(fileperms($newpath)),-4);

            if (is_dir($newpath))
            {
               echo 'dir|'.$newpath.'|'.$permissions."\n";
               recur_dir($newpath);
            }

            else
            {
               echo 'file|'.$newpath.'|'.$permissions.'|'.bin2hex(file_get_contents($newpath))."\n";
            }
         }
      }

      closedir($dirlist);
      return $mod_array;
   }

   echo "mirror-agent starting...\n";
   recur_dir('.');
?>


ftp-mirror.php
Code: Select all
<?php
/* --------------------- configuration ---------------------- */
    // set this to the URL of the mirror-agent.php on the server you want to mirror
    $mirror_agent = 'http://www.server.com/mirror-agent.php';

    /* --------------------- functions -------------------------- */
    // convert hex to binary
    function hex2asc($temp)
    {
       for ($i = 0; $i < strlen($temp); $i += 2)
         $data .= chr(hexdec(substr($temp,$i,2)));

       return $data;
    }

    // write file to disk
    function file_write($filename, $filecontent, $mode='wb')
    {
       if($fp = fopen($filename,$mode))
       {
          fwrite($fp, $filecontent);
          fclose($fp);
          return true;
       }

       else
       {
          return false;
       }
    }

    /* --------------------- the code --------------------------- */
    // disable errors ( just in case you try this script before you read the usage above )
    //error_reporting(E_ALL);
    @ini_set('display_errors', 0);

    // disable time limit ( you never know how big this site is )
    set_time_limit(0);

    // get the hex data from mirror
    //$cont = file_get_contents($mirror_agent);          // uncomment if you don't want to use curl

    $sHandle = curl_init($mirror_agent);

    curl_setopt($sHandle, CURLOPT_HEADER, FALSE);
    curl_setopt($sHandle, CURLOPT_RETURNTRANSFER, TRUE);

    $sResponse = curl_exec($sHandle);

    curl_close ($sHandle);

     $files = explode("\n", $sResponse);

    // check if it's really the mirror_agent and if we have write permission
    if(trim($files[0]) == 'mirror-agent starting...&& is_writeable(dirname(__file__)))
    {
       $count = count($files);

       // yes - start copying
       for($i = 1; $i < $count; $i++)
       {
          if(trim($files[$i]) != '')
          {
             $parts = explode('|',trim($files[$i]));
             $permissions = intval($parts[2], 8);

             if($parts[0] == 'dir')
             {
                // it's a directory - make it
                mkdir($parts[1],$permissions);
                $done[] = $permissions.'|'.$parts[1];
             }

             elseif($parts[0] == 'file')
             {
                // it's a file - write it down
                file_write($parts[1], hex2asc($parts[3]));
                chmod($parts[1],$permissions);
                $done[] = $permissions.'|'.$parts[1];
             }
          }
       }

       // output status
       echo '
    <pre>
    following files were copied:';

       for($i = 0; $i < count($done); $i++)
       {
          echo '
    '.$done[$i];
       }

       echo '
    </pre>';
    }

    else
    {
       // error - it's not the mirror_agent, or you didn't chmod this directory
       echo '
    <pre>
    ERROR:
    '.$mirror_agent.'
    does not respond like a mirror-agent should or
    the directory '.dirname(__file__).' is not writeable.
    </pre>';
    }
?>


Just make one change in the "ftp-mirror.php" file that set the path of the "mirror-agent.php" in the top of the file. Then place the files on the server and execute this script, hopefully you will not face any error.

Any comments or questions are welcome :)
User avatar
Web Guru
 
Posts: 60
Joined: March 24th, 2008, 7:59 am
Location: Lahore, Pakistan

Server to Server File Transfer

Postby Shahzad Butt on August 6th, 2008, 9:28 am

If you need to transfer a small amount of data from one server to other server from a browser, then the following webpage will help you to do your task. If the data amount is larger then you have to pay for that.

Tool:
http://directransfer.net/start.php
Regards
Shahzad Butt
http://www.GeoSourceCode.com
Shahzad Butt
 
Posts: 19
Joined: July 9th, 2008, 11:16 am
Location: Pakistan

Server to Server Transfer using FTP Protocol

Postby Web Guru on August 8th, 2008, 10:14 am

Sometimes you need to transfer files from one server to another server. This script will help you in performing this task with ease and saves your effort and time.

This script connects to the remote server using an ftp connection, then read all files and folders from there and stores it on the local server where it is running.

ftp-transfer.php
Code: Select all
<?
   @ini_set("max_execution_time", 0);
   @ini_set("display_errors", 1);

   // Remote Server Login Info
   $sFtpServer   = "ftp.yourserver.com";
   $sFtpUsername = "ftp_username";
   $sFtpPassword = "ftp_password";
   $sFtpDir      = "/";    // server directory path from where you ant to copy files/folders

   // Directory on Local
   $sLocalDir = $_SERVER['DOCUMENT_ROOT'];

   // set up basic connection
   $hFtpCon = @ftp_connect($sFtpServer) or die("Unable to connect to the FTP Server.");

   // setting connection time-out
   @ftp_set_option($hFtpCon, FTP_TIMEOUT_SEC, 3600);

   // login with username and password
   if (@ftp_login($hFtpCon, $sFtpUsername, $sFtpPassword) == FALSE)
      die("Invalid FTP Username or Password.");

   // get list of files & folders in the specified ftp directory
   $sList = @ftp_nlist($hFtpCon, $sFtpDir);


   print ("<b>- - Start Time :</b> ".gmdate("Y-m-d H:i:s")."<br /><br />");

   ftpTransfer($sFtpDir, $sLocalDir);

   print ("<br /><b>- - End Time :</b> ".gmdate("Y-m-d H:i:s")."<br />");

   // close the connection
   @ftp_close($hFtpCon);


   // Recursive function to download all the files & folders
   function ftpTransfer($sFtpDir, $sLocalDir)
   {
      global $hFtpCon;

      if ($hFtpCon != false)
      {
         // get List of the files & folders current directory
         $sList = ftp_nlist($hFtpCon, $sFtpDir);

         foreach ($sList as $sFile)
         {
            $sFile = str_replace(($sFtpDir."/"), "", $sFile);

            if ($sFile == "." || $sFile == "..")
               continue;

            $sFtpFile   = "$sFtpDir/$sFile";
            $sLocalFile = "$sLocalDir/$sFile";

            $fFileSize  = @ftp_size($hFtpCon, $sFtpFile);

            // if Directory
            if ($fFileSize == -1)
            {
               print "<b>".$sFile."</b><br />";

               //Make Folder
               @mkdir($sLocalFile, 0777);

               ftpTransfer($sFtpFile, $sLocalFile);
            }

            elseif ($fFileSize > 0)
            {
               print "<i>$sFile</i> ";

               if (@file_exists($sLocalFile))
               {
                  if ($fFileSize == @filesize($sLocalFile))
                     print "- skipped <br />";

                  else
                  {
                     @ftp_get($hFtpCon, $sLocalFile, $sFtpFile, FTP_BINARY);

                     print "- updated <br />";
                  }
               }

               else
               {
                  @ftp_get($hFtpCon, $sLocalFile, $sFtpFile, FTP_BINARY);

                  print "<br />";
               }
            }
         }
      }
   }
?>


This is a very efficient script and can be set as a cron job as per your requirements.
User avatar
Web Guru
 
Posts: 60
Joined: March 24th, 2008, 7:59 am
Location: Lahore, Pakistan

Server to Server Files Transfer usinh PHP Script

Postby Web Guru on August 20th, 2008, 9:59 am

Here is the updated script to nicley print the files/folders list which are being copy to the local server from a remote server.

Code: Select all
<?
   @ini_set("max_execution_time", 0);
   @ini_set("display_errors", 1);

   // Remote Server Login Info
   $sFtpServer   = "";
   $sFtpUsername = "";
   $sFtpPassword = "";
   $sFtpDir      = "";
   $sPrefix      = "";

   // Directory on Local
   $sLocalDir = $_SERVER['DOCUMENT_ROOT'];

   // set up basic connection
   $hFtpCon = @ftp_connect($sFtpServer) or die("Unable to connect to the FTP Server.");

   // setting connection time-out
   @ftp_set_option($hFtpCon, FTP_TIMEOUT_SEC, 3600);

   // login with username and password
   if (@ftp_login($hFtpCon, $sFtpUsername, $sFtpPassword) == FALSE)
      die("Invalid FTP Username or Password.");

   // get list of files & folders in the specified ftp directory
   $sList = @ftp_nlist($hFtpCon, $sFtpDir);


   print ("<b>- - Start Time :</b> ".gmdate("Y-m-d H:i:s")."<br /><br />");

   ftpTransfer($sFtpDir, $sLocalDir);

   print ("<br /><b>- - End Time :</b> ".gmdate("Y-m-d H:i:s")."<br />");

   // close the connection
   @ftp_close($hFtpCon);



   // Recursive function to download all the files & folders
   function ftpTransfer($sFtpDir, $sLocalDir)
   {
      global $hFtpCon;
      global $sPrefix;

      if ($hFtpCon != false)
      {
         // get List of the files & folders current directory
         $sList = ftp_nlist($hFtpCon, $sFtpDir);

         foreach ($sList as $sFile)
         {
            $sFile = str_replace(($sFtpDir."/"), "", $sFile);

            if ($sFile == "." || $sFile == "..")
               continue;

            $sFtpFile   = "$sFtpDir/$sFile";
            $sLocalFile = "$sLocalDir/$sFile";

            $fFileSize  = @ftp_size($hFtpCon, $sFtpFile);

            // if Directory
            if ($fFileSize == -1)
            {
               print '<span style="color:#0000ff;">'.$sPrefix.$sFile."</span><br />";

               $sPrefix .= "-- ";

               //Make Folder
               @mkdir($sLocalFile, 0777);

               ftpTransfer($sFtpFile, $sLocalFile);

               $sPrefix = substr($sPrefix, 0, -3);
            }

            elseif ($fFileSize > 0)
            {
               print "<i>$sPrefix.$sFile</i>";

               if (@file_exists($sLocalFile))
               {
                  if ($fFileSize == @filesize($sLocalFile))
                     print "- <span style='color:#00ff00;'>skipped</span>";

                  else
                  {
                     @ftp_get($hFtpCon, $sLocalFile, $sFtpFile, FTP_BINARY);

                     print "- <span style='color:#0000ff;'>updated</span>";
                  }
               }

               else
                  @ftp_get($hFtpCon, $sLocalFile, $sFtpFile, FTP_BINARY);

               print "<br />";
            }
         }
      }
   }
?>

User avatar
Web Guru
 
Posts: 60
Joined: March 24th, 2008, 7:59 am
Location: Lahore, Pakistan


Return to PHP / MySQL / XML

Who is online

Users browsing this forum: No registered users and 0 guests