WebDAV HTTP, or SVN RA?

The answer to this question isn’t as simple as deciding which protocol is faster. Even WebDAV advocates will usually admit that SVN RA is faster for most operations. In fact, support for a new HTTPv2.0 protocol that removes much of the overhead of the current WebDAV HTTP protocol is planned for Subversion 1.7 out early next year. However, the vast majority of development organizations don’t see the performance differences between the current WebDAV HTTP protocol and SVN RA as noticeable enough to users over a LAN in most cases to overcome the significant advantages WebDAV HTTP offers over SVN RA. The user experience over a WAN may be a different story, but I’ll come to that later.

The key advantages provided by WebDAV HTTP over SVN RA include:

• Web browser access to Subversion repositories through Apache.


• The ability to leverage the authentication and authorization options offered by Apache including its native LDAP integration.

• Standardized network encryption (SSL) and certificate handling.


• Complete logging capabilities that don’t exist using SVN RA with svnserve.


• Use of a standard port (80) that makes it easy to go through corporate firewalls.

Subversion started out using WebDAV HTTP with Apache. SVN RA with SSH came along later as a way to make it easier for CVS shops that were using CVS’ pServer protocol with SSH to migrate to Subversion without having to redo their security infrastructure. If you aren’t moving from CVS to Subversion, and have to set up SSH user accounts on the server from scratch, there can be significant effort involved. In addition, SSH introduces performance overhead of its own in comparison to SSL.

In a globally distributed development organization with remote users accessing a central Subversion repository over a WAN connection, the performance differences between WebDAV HTTP and SVN RA are more noticeable to users, but either way WAN latency will have a big impact on remote developer productivity. HTTPv2.0 is designed to close this gap by removing as many network round trips as possible over a LAN or a WAN between clients and the Apache Web server front-ending Subversion. Network round trips that the current WebDAV HTTP protocol generates due to CHECKOUT requests that happen before each PUT, and unnecessary PROPFIND requests will go away. The ability to pipeline requests and process them in parallel whenever possible are some of the other major changes coming. In addition, much of the handshaking that takes place using the current WebDAV HTTP protocol to establish connections and authenticate users, before they can even access the repository will be gone.

Yet without a distributed solution for Subversion, the same problems of WAN latency, degraded central server performance as the number of remote users grows, excessive network bandwidth usage due to unnecessary read operations, and availability will still be significant issues that will negatively impact developer productivity. I’ve covered these points in detail in two recent posts:

1. Can Globally distributed Development Really be Supported with a Central Subversion Server?
   


2. The Centralized vs, Distributed Debate Continues.
   

Subversion MultiSite, which will support the new HTTPv2.0 protocol when it becomes available with Subversion 1.7 already addresses these same issues over a WAN that HTTPv2.0 will when it comes out, in addition to overcoming the downsides of a central Subversion server implementation that HTTPv2.0 won't address.

Subversion MultiSite minimizes WAN round trips and network bandwidth usage in three ways:

1. Subversion MultiSite maintains persistent physical connections between repository replicas at each site. This means that the overhead of opening and closing a connection with each request over the WAN (i.e., the 3-way TCP handshake) is eliminated already. Subversion MultiSite users don’t have to wait for HTTP v2.0 to address this over the WAN.

2. Subversion MultiSite sends an entire commit, which could include hundreds of HTTP PUTs (one for each file in a commit) over a single connection, rather than opening and closing a connection for each PUT, as WebDAV HTTP would do on its own today.

3. Subversion MultiSite doesn't generate any WAN traffic unless it is absolutely necessary. Developers checkout from their local Subversion repository replica, so there are no remote reads. Only commits and other writes generate WAN traffic.

The bottom line is that if WAN latency is driving you toward SVN RA, in spite of the major benefits offered by WebDAV, Subversion MultiSite makes giving up those benefits unnecessary and allows an IT organization to overcome the other significant challenges presented by globally distributed development.

The Centralized vs. Distributed Debate Continues

After my previous post, “Can Globally Distributed Development Really be Supported with a Central Subversion Server? ” CollabNet’s Jack Repenning recently responded with a post of his own: “Optimizing Globally Distributed Source Management”.

Needless to say, everyone at WANdisco has tremendous respect for Subversion and what the Subversion community has achieved. We’ve built a significant portion of our business around Subversion because of that respect. My post had everything to do with implementation architecture and nothing to do with the quality of Subversion. In fact, it could have been taken and applied to a debate over whether or not to centralize or distribute a database, or any other application.

And while Jack and I still agree on the list of tradeoffs in the centralized versus distributed debate, including his additional item of enterprise workflow, we still see different outcomes:


WAN costs, in terms of bandwidth and latency in large corporations are not merely a problem of, “…configured “Quality of Service” policies colliding with evolving use patterns which can be reconfigured…” WANdisco’s Fortune Global 1000 customers moved away from relying on a central Subversion server, even though they did control their networks and had ample resources to obtain extra bandwidth. Some of them are even network service providers. The bottom line is that no matter how good the network is, there’s still the unavoidable speed of light problem, which gets compounded by the number of WAN round trips required to complete a transaction. Then there’s always the risk of timeouts with each network hop as data gets transmitted over a WAN. There’s no forward recovery after a timeout, so transactions have to be resubmitted and data can be lost. Finally, if remote users lose their connection to the central server, they have no repository access at all.

Server performance can be addressed up to a point by adding more memory, CPU or disk. Some of WANdisco’s customers, including those that actually manufacture servers, did just that, or asked their hosted service provider do it for them as a stopgap. But no increase in the capacity of a central server can get away from the fact that it’s a single point of failure that’s destined to become a performance bottleneck again. Distributing Subversion repositories with the right solution gives IT organizations the ability to balance workload across their development sites and locate data close to users, which by default leads to better performance, scalability and availability (the next item on the list) in a way that no central server implementation can.

Availability is ultimately a function of the fail-over and disaster recovery features of an implementation, and since there must be something to restore from, the backup solution. Most large data centers rely on disk mirroring, coupled with backups taken at regular intervals in case the mirrors fail, to provide the data to restore from. There are three challenges with disk mirroring solutions: (1) they’re designed primarily to protect against disk failure, not total server failure, so the application still has to be brought up on another server, (2) some manual intervention is required so the risk of human error leading to data loss and extended downtime is real and (3) they only work over distances covered by a metro-LAN at best, not a WAN.

Continuous hot backup and automated disaster recovery are included with Subversion MultiSite, so there’s no need to rely on disk mirroring solutions and incremental backups, or manual procedures of any kind. Servers can even be taken offline for routine maintenance without interrupting user access, making full 24 by 7 operation a reality in a global environment. As soon as a server comes back online it catches up automatically with the other servers.

Enterprise work flow in this context refers to the ability to map and selectively replicate groups of Subversion repositories to only those sites where the development teams that use those repositories are located. As projects, teams and locations shift and change the replication map needs to change with them.

While this can get complicated without the right tools, large enterprises have implemented Subversion MultiSite partly on the basis of its ability to handle selective replication. They didn’t see it as a problem when they moved away from a central server.
One of the key features provided by Subversion MultiSite is the ability to administer all sites from a single location. Selective replication is just one facet of this.

Can Globally Distributed Development Really be Supported with a Central Subversion Server?

Based on the feedback we’ve received from our customers and prospects, the answer to this question is a resounding no, at least not in a distributed development organization of any size, with a number of users at remote sites.

There are essentially three major obstacles that distributed development organizations will run into with a central server approach:

1. WAN Latency.

WAN latency becomes a problem not only because of increased network traffic as the number of remote users grows, but also due to the fact that every remote request entails a WAN penalty. Even though Subversion clients only send changes to the central server when modifications to existing source code files are committed, when a new source code file is committed, or an existing file is checked out, the entire file is sent over the WAN.

2. Degradation of the Central Server’s Performance.

This results not only from the extra load generated by increasing numbers of remote users, but also from read transactions that would otherwise be unnecessary if those users had access to a local copy of the Subversion repository. A frequent pattern that arises with a central Subversion server configuration is that multiple developers at the same remote locations repeatedly perform checkouts, updates and other read operations against the same files.

These repeated, unnecessary reads use up central server memory and processor capacity, as well as network bandwidth.

3. Availability.

The ultimate weakness with a central server approach is its single point of failure architecture, and the impact this has on repository availability. When the network connection is lost, remote users have no repository access at all. Even a transient WAN connection failure between a remote Subversion client and the central server can slow down developers at remote sites if it takes place during a large commit, since the entire commit will have to be resubmitted.

Obviously, if the server hosting the repository is down for any reason, all users will be impacted, not just those at remote locations, unless a backup is available. Even if a backup server is available, the time involved in bringing it into service can be significant, and, there’s always the risk of data loss and extended periods of downtime due to human error during the recovery
process.

The arguments typically used in favor of a central server approach are that everyone works from a single consistent copy of the repository, maintenance and administration are only required at one location, and overall control can be implemented more effectively from both a project management and data security perspective. The bottom line for advocates of supporting globally distributed development with a central Subversion server is that these perceived benefits greatly outweigh any gains that might be achieved by distributing Subversion repositories.

Let’s take each of these arguments and examine their validity in light of the available alternatives.


Single Consistent Copy of the Subversion Repository.

The first argument that advocates of a central server approach make is that it’s the only way to maintain a single consistent copy of the repository.

There are a number of master-slave solutions available that provide a partial response to this argument. The most commonly used is svnsync, introduced with Subversion 1.4. While svnsync and the other solutions do offer the advantage of allowing remote site developers to access a local read-only slave mirror of a master Subversion repository, the slaves are only as current as the last instance of replication from the master. The lag time between each instance of replication often leaves developers at remote sites checking out stale versions of source code files. This in turn leads to update conflicts when remote developers perform their commits against the master. This then requires them to perform updates over the WAN against the master to get the latest revision and resolve any conflicts before reattempting their commit. This can negate some of the expected improvements in network performance and developer productivity, because read operations still have to be performed over the WAN. Finally, the master repository represents a single point of failure for write operations.

In contrast to a master-slave approach, Subversion MultiSite relies on a peer-to-peer architecture with no single point of failure. All of the repository replicas are readable and writeable for the entire code base, and consistency across the repositories is guaranteed. In addition, WANdisco’s active-active replication capability allows developers at all locations to work at LAN speed over a WAN for both read and write operations, while at the same time keeping all of the repository replicas continually in sync. In effect Subversion MultiSite delivers one-copy equivalence across a system of distributed Subversion repositories, and provides the same user experience that would be achieved if all of the developers worked at one location over a LAN against a single repository, instead of thousands of miles apart.

Maintenance and Administration.

The second major argument in favor of a central repository is that maintenance and administration only have to be performed at one location.

While this may sound like a major benefit at first glance, for remote developers there can be a significant negative impact if they are separated from the central server site by large time zone differences, and either the network connection or the server goes down during their location’s normal working hours. From the remote site’s perspective, it can take until the next business day to restore access.

In addition, in a typical Subversion implementation, not only Subversion, but Apache and the operating system have to be maintained as well. It is true that if Subversion repositories are distributed, maintenance, particularly in the area of applying patches and performing upgrades becomes more of a challenge. If it’s handled inconsistently, problem resolution can become incredibly complex, resulting in data loss and extended periods of downtime.

WANdisco addresses these challenges by making it possible to monitor and administers servers at all sites from a single location. In addition, Subversion MultiSite is now available as a virtual software appliance, with access to an update server. Patches and upgrades can be applied automatically for all of the components of the implementation, including Subversion, Apache, and the operating system at each location, eliminating the risks inherent in performing these tasks manually.

Backup and Recovery

Since backup and recovery is such an important aspect of Subversion repository maintenance and administration, I’d like to discuss it briefly here. In an upcoming post, I’ll cover this topic in more detail.

With a central server approach, backup and recovery solutions typically either rely on disk mirroring, or svnadmin scripts used to copy the repository to a standby backup server. In any event, even if a backup server is available, the lag time involved in bringing it into service can be significant. In addition, there’s always the risk of data loss and extended downtime resulting from human error during the failover and recovery process.

Although master-slave solutions like svnsync can be used for backup and recovery, if the master goes down, the slaves are likely to be missing data that may be unrecoverable depending upon the nature of the master server failure. The extent of data loss will depend upon the lag time and size of the changes to the master since the last instance of replication.

The other issue to be aware of is what actually gets replicated to the mirror slave repositories by the tool you’re using. For example, with svnsync, only the versioned repository data gets synchronized. Repository configuration files, user-specified repository path locks, and other items that might live in the physical repository directory but not inside the repository's virtual versioned filesystem are not replicated.

With Subversion MultiSite, continuous hot backup is achieved by default as a byproduct of active-active replication and all repository data is replicated. After an outage, recovery from any other site’s server is automatic.

Overall Control.

The third major argument used by advocates of a central Subversion server approach is that development projects can be managed more tightly. What happens in practice is often just the opposite.

Many of our customers report that prior to implementing Subversion MultiSite, remote developers often held back large commits until the end of the day or end of the week, using WAN latency as an excuse. This made it harder to monitor what everyone was doing on a day-to-day basis, and meant that it took longer to find out that developers didn’t understand the specs they were given. As a result, code was delivered that had to be rewritten and project deadlines were frequently missed.

In terms of maintaining control from a data security perspective, the goal is to achieve consistent enforcement of security policy across all development sites. When Subversion MultiSite is implemented with Subversion Access Control, the security policy configuration is automatically replicated to all sites when it’s initially set up, as are any future changes. This guarantees that access control is enforced consistently at every location. Subversion Access Control also provides audit capabilities that track every user access to the repository and alert administrators whenever access violations occur.

Data security as it relates to the contents of source code repositories has become a greater concern in recent years as IT organizations began to outsource development work to countries where enforcement of intellectual property rights is relatively weak. In addition Sarbanes-Oxley and other regulations have begun to reach into the IT organization, imposing requirements of their own. In an upcoming post, I’ll cover this topic in more detail and describe
what’s really required to secure the intellectual property stored in souce code repositories in a globally distributed environment.

Why Deliver Subversion MultiSite as a Software Appliance?

Why Deliver Subversion MultiSite as a Software Appliance?


The answer to this question is that software appliances reduce the cost and complexity of installation, deployment and maintenance to an extent that no other delivery model can. For this reason our customers have readily embraced it. Indeed, many of them have already implemented other enterprise applications as software appliances.

To understand why the software appliance approach makes such a significant difference, I’ll start by looking at the challenges posed by the other major implementation alternatives: traditional behind the firewall, and hosted software as a service (SaaS). I’ll then explain how the software appliance approach overcomes them, without sacrificing functionality or performance.


Traditional Behind the Firewall Implementations

When IT organizations implement applications, they’re faced with a number of challenges. First of all, there’s the overhead of performing routine maintenance, including manually applying patches and performing upgrades, maintaining system availability and dealing with backup and recovery. Third party components around the enterprise application add to this complexity. For example, in the case of operating systems there are literally hundreds of Linux distributions or distros as they’re called, as well as various flavors of UNIX. When the application is part of a software stack consisting of multiple components such as web servers, or databases and your staff has varying degrees of skill with these components, the potential for human error goes up dramatically. If patches are applied incorrectly to any of the components, problem resolution can become incredibly complex and lead to extended periods of downtime.

In the case of ISVs, their own development cycles can become longer, since they face many of the same challenges as their customers. In addition, it’s unrealistic to expect software vendors to QA every new release of their products on every platform, particularly when the solution is part of a stack dependent on other components. Software vendors are effectively forced to choose specific platforms to make their products available on, in order to avoid being faced with an unmanageable QA and support burden. The downside for the software vendor is that this has the effect of limiting the size of their potential market. In addition, the support burden can still be high, even when a customer implements on a supported platform, if any of the components are installed and configured incorrectly, or a patch gets applied incorrectly. If these issues crop up during an evaluation, the result can be a longer sales cycle at best, or a lost opportunity at worst.

Software as a Service (SaaS)

As an alternative to in-house behind the firewall implementations, many IT organizations have turned to hosted software as a service (SaaS) solutions. SaaS offers the benefit of immediate deployment. It also leaves the maintenance and support burden primarily in the hands of the solution provider.

However, many IT organizations don’t feel that the SaaS model is the best fit for some of their enterprise applications. This may include applications that require modification because of unique business processes, or applications that require integration with in-house legacy systems.

This had led some vendors to introduce a hybrid model that combines the hosted SaaS approach with application delivery as a software appliance. WANdisco will soon be announcing a partnership with Amazon to leverage their EC2 service to offer this hybrid model for our enterprise enabled Subversion solutions, including Subversion MultiSite, Subversion High Availability, and Subversion Clustering. This will give our customers a full range of deployment options.


Software Appliance


If the goal is to achieve the zero latency deployment and cost savings of SaaS, while at the same time maintaining the control, security and flexibility of behind the firewall implementations, the software appliance is the way to go.
A software appliance is a software application combined with just enough operating system (JeOS), a stripped down version of Linux designed to meet just the needs of the specific application, so that it can run optimally on industry standard hardware, or in a virtual machine. The result is that environmental dependencies are virtually eliminated. In the case of the Subversion MultiSite Software Appliance in particular, this has the effect of reducing installation and deployment across multiple locations to a matter of minutes. In addition, maintenance is made even simpler by the fact that access to an update server is provided so that updates are applied automatically to all sites where the appliance has been deployed, virtually eliminating any room for human error.


The one area of concern some critics point to in the case of the software appliance delivery model is performance. With the Subversion MultiSite Software Appliance there are no tradeoffs in performance or scalability. In fact, performance is often superior and scalability is much easier to achieve.

In the case of performance, with a virtual software appliance all of the components, including the operating system, are configured for the specific application and virtual machine environment. rPath’s JeOS, (Just Enough Operating System) takes the existing general purpose Linux OS, strips it down, and tunes it specifically to meet the requirements of the Subversion MultiSite Software Appliance and the supported VM that it’s installed on. This optimized OS, with its smaller footprint consumes fewer resources, and delivers superior performance than deployment under a traditional fat OS designed to run on multiple pieces of hardware and support a broad range of applications. The rPath team that built and now provides support for JeOS is uniquely qualified in this area, as it consists of Linux experts from the original brain trust that developed Red Hat Linux, Red Hat Enterprise Linux and founded the Fedora project.

In terms of the other components, the Subversion MultiSite Software Appliance includes an Apache Web server that has been preconfigured for optimal performance with Subversion MultiSite only. In many IT organizations that implement the standard Subversion MultiSite solution, the Apache Web server is often configured to support a number of applications, not just Subversion.

Recent developments are also making the overhead imposed by the extra layer of software introduced by the hypervisor a non-issue. For example, VMware’s vServices that provide direct access to the hardware infrastructure via device drivers. The applications receive services from JeOS, and JeOS communicates directly with the device drivers. Hardware manufacturers have also been implementing virtualization specific technologies to enhance performance for several years now. These improvements have taken virtualization to the point where any overhead it introduces may be completely submerged by limitations in network, storage or memory subsystems. In fact, some new virtualization solutions are even able to overcome memory and processor constraints built into many applications, delivering superior performance to what would be possible with a physical server implementation.

In the area of scalability, when additional system resources are needed to support increased user and transaction loads, traditional implementations can be more difficult to scale because the OS is closely tied in with the hardware. Virtual software appliances like the Subversion MultiSite Software Appliance can easily be moved to faster machines, even on the fly, using tools such as VMware’s vMotion and similar offerings from other vendors.

Subversion MultiSite Appliance

View more presentations from WANdisco.

Multi-site Development and the Write-Thru Proxy

With Subversion 1.5, along with other notable new features such as built-in merge tracking, the WebDAV write-thru proxy will be introduced to simplify use of svnsync for Subversion deployments based around Apache 2.2.x.

Prior to Subversion 1.5, users had to manually redirect their client to the master server whenever they executed a commit or other write transaction using the “svn switch -- relocate” command. The WebDAV write-thru proxy will now detect when a commit or other write command has been issued by a client connected to a slave repository. It will then automatically redirect the client to the master server. This should make life somewhat easier for end users, and help prevent unintended writes against slave repositories from leading to split-brain scenarios that can be difficult to recover from.

However, the WebDAV write-thru proxy leaves svnsync’s master-slave architecture unchanged. While svnsync does offer the advantage of local reads, eliminating WAN traffic that would otherwise take place between a remote client and a central Subversion server, writes only happen on the master. Thus, the master repository can become a single point of failure for write transactions. In addition, the lag time between each instance of master repository replication can result in users at remote sites checking out stale copies of source code files from their local slave. This in turn can lead to update conflicts when changes are committed against the master. If the replication process fails due to network outages or server crashes, there are no built-in recovery capabilities.

In contrast, WANdisco’s Subversion MultiSite turns every Subversion repository into a peer of every other, and every repository is readable as well as writeable for the entire code base. Replication is triggered automatically when a write operation is done at any location, and transactional consistency is guaranteed across all of the repositories. Self-healing capabilities are provided to automate the recovery process after a network outage or server crash, and prevent any data loss.

Although both svnsync and Subversion MultiSite support the WebDAV HTTP protocol, Subversion MultiSite only uses this protocol over a LAN. WANdisco’s own optimized protocol is used over a WAN on top of TCP/IP. The result is that commits consisting of hundreds of files are sent in a single pass during replication with Subversion MultiSite, rather than one-by-one as a series of HTTP PUTS, as is the case with svnsync. This enables Subversion MultiSite to deliver a significant performance boost over a wide area network.

With svnsync, user information, including access privileges must be maintained consistently across all of the servers, and there are no built-in features to support this. When WANdisco’s Subversion Access Control solution is implemented with Subversion MultiSite, the security configuration is replicated automatically when it’s initially set up, as are any changes, insuring consistency across all of the servers.


To learn more check out: Subversion MultiSite.

Get the WAN Out of the Way

Virtually every approach to globally distributed multi-site development
using Subversion leaves the WAN in the way of developer productivity. In this post, I’ll explain why this is the case, and how it can be dealt with.

The WAN performance that developers experience results from a combination of two factors: (1) the number of WAN round trip times required to complete a write operation between a Subversion client and a master server, and (2) the available throughput on the network.
I’ll focus on write operations over the WAN such as commits, since there are master-slave solutions like svnsync that allow developers to do checkouts, updates and other read operations locally, without generating WAN traffic. However, read operations over the WAN may still be required with master- slave solutions like svnsync. To understand why this is the case, see my earlier post, “Keeping Multiple Subversion Repositories in Sync.”

Let’s examine the first factor impacting WAN performance, the number of WAN round trips. With each Subversion commit using the SVN RA protocol (Subversion without Apache) up to six WAN round trips will take place between a remote developer’s Subversion client and the master Subversion server. These WAN round trips are required to open the connection, authenticate the user, and write the commit to the master server. This introduces some minimal latency, typically on the order of 2 to 3 seconds over a WAN. If Subversion is implemented with Apache using the WebDAV HTTP protocol,
4 WAN round trips will be incurred for each file in the commit, since each file will get transferred with its own separate HTTP put. With a large number of files in a commit, several minutes of wait time can be incurred over a WAN.

However, the real impact on developer productivity comes from the second factor, the amount of time required to transmit data at WAN-speed, based on the available throughput on the network. For example, consider a commit consisting of 500MB of data sent over a WAN from India to the US. Given that the typical E-1 line used between the US and India operates at approximately 2 megabits per second, it should take 2000 seconds, or a little over 33 minutes to transfer 500 MB. This assumes an absolute best-case scenario in which there’s no competition with other network traffic at the same time, and everything goes smoothly without any connection loss or communication error between the client and the remote master server.

What if a remote site developer’s commits could be processed at LAN-speed, instead of WAN-speed? Given that most LANs operate at one gigabit per second, it should take about four seconds to transfer 500MB of data between the Subversion client and the server over a LAN. Instead of waiting over 33 minutes under the best of circumstances, remote site developers would see their commits complete in four seconds! Developers would check in their changes more frequently, rather than waiting until the end of the day, or end of the week as they would have done in the past, due to the pain of poor network performance.

In addition, what if the distributed Subversion servers were kept in sync in real-time over the WAN? The 33 minutes saved would be just the tip of the iceberg. If developers across all sites had access to the latest source code without having to wait for a master server to be copied to their local read-only servers, then update conflicts and other problems could be fixed as soon as they were found. It would also be possible to achieve real-time collaboration between distributed development teams instead of having them work in silos. As a result, less time would be spent on QA and rework, and a significant amount of time and cost would be squeezed out of the development cycle.

WANdisco, with its unique active-active replication capabilities, allows all of this to be accomplished. WANdisco delivers LAN-speed performance for both read and write operations, while keeping distributed Subversion repositories in sync in real-time. WANdisco gets the WAN out of the way, so that all of the productivity improvements and cost-savings that IT organizations are seeking from globally distributed development can be achieved.

Keeping Multiple Subversion Repositories in Sync

With Subversion 1.4, svnsync was introduced for this purpose. The key problem with using svnsync for multiple Subversion repositories distributed over the WAN is its reliance on a master-slave architecture. While svnsync does provide the advantage of having local read-only repositories at each of the remote development sites, only the master repository is writeable. The master repository is then replicated to the read only slaves. However, the replication process can place a significant load on the network and servers. Because of this, replication tends to happen on an infrequent basis, leaving the read-only slave repositories that remote sites do their checkouts from out of sync with the master much of the time. As a result, commit failures due to update conflicts on the master repository can become a problem. In order to avoid commit failures, developers at the slave repository sites have to do updates over the WAN against the master Subversion repository before doing their commits. This can negate most of the expected network performance and developer productivity benefits of using svnsync in a distributed development environment.

Other solutions such as svk do allow multiple repositories to be readable as well as writeable, but there are no guarantees of consistency across the repositories. A commit can succeed on a developer’s local repository where there are no conflicts, and fail when it’s copied to other sites’ repositories due to update conflicts. This can make administration extremely difficult.

WANdisco solves these problems by turning distributed Subversion repositories into peers. All of the repositories are writeable, and consistency across the repositories is guaranteed. WANdisco’s active-active replication capabilities allow developers to work at LAN speed over the WAN for both read and write operations, while keeping all of the repositories in sync, in effect in real-time. WANdisco also provides self-healing capabilities that automate disaster recovery after a network outage or server failure.