Saturday, May 25, 2013

My experience of using Replication


I have used replication between 8 SharePoint server farms for almost a year now. I am usually a very nice and patient person, but now the patience is running out and I feel frustrated and need to write about this so maybe it can help others who are considering replication. And I am being brutally honest (sorry to all you nice people at Metalogix), but I have had it and I want to share my experience!
I used to have a nice job as the only SharePoint employee (a new role for me who always worked as a consultant) in the company, maintaining and developing our intranet in a small server farm. The company I work for is in the seismic industry and we have a fleet where the vessels have a limited amount of bandwidth. They use satellites for bandwidth, and currently they only have a 1 Mb connection. Which is a big challenge for SharePoint, as it is not supported over WAN. Our fleet users of course found our SharePoint intranet very slow (connecting to our server in Dubai), which is understandable on such a low bandwidth. So to solve this, I started to evaluate some replication products, since Microsoft does not have a product or a solution for this. And hey Microsoft, I know the cloud is the new thing ánd that is cool when you have a lot of Mb's, but what about your users that has limited low bandwidths and MUST have data available if they become offline (which happens quite often at sea)?
Anyhow, I chose Metalogix Replicator because they have been in the business for quite a while, they have similar clients and also their product is built into the CA so we did not need another client to monitor the replication. It seemed to fulfill all our needs also, and with that I mean that we have a very “out of the box” intranet so it should not be complicated to replicate the functionality.
Could have been a success story for us. But the truth is, almost a year later, I barely have any time over to do my regular job because of all the replication issues and we still are not replicating all functionality! My job has become a nightmare, where I have to fix things that the replication has messed up or monitor the package queues to see what has run into “error” or “conflicts”. It was a long time ago I could sit down and have a look at new development projects on the intranet. So why is it like this? Well first the obvious reason, and which is not due to the product (it would be the same with any replication product), is that I now have 8 instead of 1 SharePoint servers (and SQLs) to maintain. I have 8 instead of 1 intranet sites to monitor and all the work that comes with that. But that was expected and OK. I expected that to take maybe 40-50% of my time which should have been fine.
What I did not expect was the constant (I mean almost 100%) need of my time to monitor what the replication has done since I left it. That is, fearing that permissions has been removed or not replicated at all or replicated incorrectly so that sensitive information became visible (luckily I managed to restore permissions FAST before too many users discovered it), site collection features that got deactivated and ruined data from third party products we had installed (and that are critical for the safety on board) which led to a complete RESTORE of a site which took almost 2 days to get back (and unavailable to users of course), all the countless hours of troubleshooting errors and phone meetings with technical support! And also the lack of product support in replicating InfoPath forms. Which is still not solved, although stated to be solved in the last version. The product does not deliver what it promises!
Our intranet is very “out of the box” although it does not look like it, I can do magic without code J So it does not contain any custom code, I have branded it using our own css file, added some jquery stuff, and InfoPath or modified xsl. And also, we have used proper third party web parts that we have bought and used no “homemade” stuff. So I would expect that our functions should be easy and able to replicate. But I was wrong.
I have provided Metalogix with detailed descriptions of the errors both in replication of permissions, workflows and InfoPath replication. Their product manager have interviewed me to hear my opinion on the UI and I had a lot of improvement suggestions after having using it for half a year. But despite that, our problems are still there and I have been very patient but I can feel it running out now and it is getting really annoying instead. It is taking TOO much of my daily work and the users are annoyed that they STILL need to surf into our local office server to be able to fill in their Reimbursement forms etc (not to mention the confusion as they have to surf into ANOTHER url to do some stuff). I had hoped all this could be available on board when the replication was installed, which is almost a year ago now. I do understand that our third party web parts cannot be replicated, that is fine but COME ON! – forms using Data Connections which is by design in InfoPath? Lists with customized forms are not replicated in full (the ifs files are missing)? Permissions with only SP groups and AD groups in them? Workflow that sends repeated mails from the same form to users so they are SPAMMED and mail me in desperation to stop the replication, because it loops the packages and constantly “updates” the list items (when they are in fact NOT updated, just sent back and forth between the servers) but makes the workflow engine think they are updated items? List/document items with versioning enabled generates so many packages that I need to disable the replicator and it takes forever to clear the queues?

What I wonder is: where are all the other replication customers and their demands/problems? They must have the same issues as us? I seriously doubt that we are alone in this. And I can’t help but feel that I am contributing to the development of their product and usually I love to contribute with that to a company that listens, but this feels like BASIC product functionality that is already promised to be there. And it has cost us a LOT of troubleshooting and valuable time and dissatisfied users.
And not only the time at work… all my sleepless nights when I worry about this shit, afraid that users can see what they shouldn’t and the weekends I have had to spend solving things in panic or with a restore, instead of relaxing. All the hours where I have had to add back lost functionality, over and over!

Yes, there are good sides of Replicator also of course. It works really well for simple lists, pages, images and documents, (unless you use versioning, which can lead into endless loops where the package queues are so huge that you have to disable replication to be able to delete the hundreds or thousands packages that a FEW changes made in a list!), and it transfers the packages fast and at its smallest amount of data. Also I like the simple UI, and that it is integrated into CA.
And the people working at Metalogix are great and tries to help in all way, and I feel that they really take our problems seriously. But that does not help my frustration unfortunately, and they don’t seem to test real life scenarios since they obviously not had tested an InfoPath form that has Data Connections, and that is often the reason why you use IP in the first place, otherwise you could just use the simple lists.

So to conclude – using Replicator for simple, not customized, things works absolutely fine and it is indeed instant! But if you have an intranet with InfoPath forms using DC's, unique permissions, workflows, versioning etc then you might run into the same problems as we have and I would not recommend replication until all these issues are solved. We are using version 6 which is the latest version.
I hope with this article that I might find someone out there who is also using Replication (any products). Because when you google stuff about replication, you rarely find anything useful. It would be great to get some feedback and stories from others about this!