Now you can deploy source changes to Jetendo sites to remote servers fast via ssh/rsync integration

Thu, Oct 31, 2013 at 12:35AM

FTP is awful for team work.  You aren't sure if files are all file you might have changed are uploaded.  If you try to use a sync tool on a large project with FTP, it will take a long time to complete.   Also, a file in transfer is essentially incomplete/corrupt if it is viewed by the public while it is uploading.  This causes temporary errors every time you upload something.    If you have 2 or more people sharing the same FTP account, you have no way to know if you are working on latest version unless you rely on other tools and make sure that everyone is doing that consistently like dreamweaver check in/out.  Using FTP for team work when it comes to deployment just leads to overwriting other people's work and making mistakes that waste time and money.

Git is good for source code versioning and it seems fast enough for deployment, but it's not safe for deploying because if files on both sides are changed, it will result in a merge conflict.  A merge conflict adds info to your source code which will cause instant downtime / errors until you manually edit the affected files and re-run the git commands.  It's actually pretty hard to use git when resolving a conflict for a non-developer, which will make it really hard for a team to share deployment access with designers.     If you use git checkout instead, it can be used for deployment, but a git checkout requires downloading the entire project from scratch again.  Imagine if you had to FTP up the entire project just because you make a single CSS file change.  That would be really inefficient and slow.

Rsync to the rescue

For my deployment process, I have chosen Rsync and some PHP shell scripting to automate everything.  Rsync is able to figure out exactly what files have changed and only update those.  It also compresses data sent over the network, making it extremely fast.  It also is able to delete files from the production server without deleting the local copy.   It's really complex to use rsync correctly and to make it secure.  Here is the command I had to use to do it correctly:

    rsync -rlptDd  --delay-updates --delete -e "ssh -i /path/to/private-key" /opt/jetendo/vhosts/domain_com/public_html/

I highly doubt anyone would want to type that with the right paths EVERY time you make a change to a web site.   So this is why you would work to build automated deployment scripts for your team to work more efficiently. 

Getting the right permissions is tricky

The permissions on the local server might not be correct for the production server after rsync finishes.  For example, my test server doesn't have any Linux users for each project, it just uses a samba share with flexible permissions that help me work quickly.  But the production server has individual users for each site and stricted chmod 770 and 660 permissions.   If those permissions are preserved, there will be security or software problems later.   I had to write code to fix the permissions automatically after the rsync command completes.

Integration with Jetendo Server Manager

In addition to rsync being the best way to deploy, I now made it the easiest.  The Jetendo server manager has been upgraded to support adding multiple deployment servers.   You can configure each site to link it with 1 or more deployment servers.  Each site can deploy to different servers.  So you can have many different installations of Jetendo and move source code between them far easier now.    Once a site is configured for deployment in the browser, you can click a single button to deploy to all the servers at once in a single step.

Achieving the best security with rsync & ssh connections

To make it secure, I use an rsa key (secure certificate) instead of a password login for rsync's ssh connection to the other servers.  Each server can have a separate host name, private key and username.  Additionally, I secured the connection to the server so only rsync can be executed, and it can only run from trusted IPs.  Someone couldn't use this feature to break in and run other commands or do port forwarding.    You specify the private key path in the Jetendo server manager so the system is able to automatically do the ssh commands without prompting for a password.   Additionally, this key information is only stored on developer computers, not the live server.    

Jetendo is built on top of the Railo CFML Engine which is configured to run as a less privileged user to enhance system security.  When I execute tasks that require root privileges, I do this by setting up a queued event for a php cron job that runs every few seconds looking for more work to do.  When it finds nothing, it just sleeps and eventually re-executes again.  There is actually a 1 to 3 second delay from when you click on deploy to when it actually executes the cron job queued task.  This important extra step to handling processes that need root privileges is what allows me to have a very strict security sandbox configuration with Railo.   For example, I can disable Java and cfexecute in my Railo sandbox and greatly reduce the amount of paths in the filesystem it has access to.   I hope to replace the PHP cron jobs with Railo CLI someday when that becomes available.  But right now, you can't do command lines as root user with CFML unless you ran a whole separate installation of the server, which I'd rather not do.

Jetendo CMS will eventually be distributed as an free open source project.  Visit for more information.

Bookmark & Share