Status Report on the Switch from cPanel to Site Tools

It has been а little over a year since we launched our new client interfaces – the revamped Client Area and the in-house developed Site Tools that is replacing cPanel. In August 2019 we started onboarding all new clients to the new interfaces and shortly after we began working on the migration of our existing clients. As of now, all our clients are using the new Client Area and we have also successfully converted more than 9000 servers from cPanel to Site Tools. 

Looking back on the past 12 months, evaluating the complexity of the transfer process and the non-transfer related challenges 2020 has placed in front of us, I believe that we have managed to keep a healthy migration pace. This was achieved thanks to the tremendous amount of work of all teams involved in the transfer and despite the occasional slowdowns coming from various circumstances.

Still, we have a lot more accounts and servers to convert and many of you may be wondering why it is taking time and when is your site’s turn coming. That is why we decided to do a follow-up and tell you the story of what we have done behind the scenes during the past year and what’s the forecast for the upcoming months. 

The innate complexity of the migration

It is not surprising that the migration of over a million live sites from cPanel to Site Tools is a tremendously complex process. I might say it is comparable in terms of efforts and resources needed for the creation of the new interfaces and systems themselves. It is quite a challenge to move operational sites from one platform with a certain structure to a new platform with a completely different one, without affecting the availability and functioning of these sites. 

On the surface, it seems that there is just one simple difference between the two frameworks —  no addon sites under one hood. What this actually means is that we have to be able to untangle any subordinate sites from the main cPanel account and recreate them as separate stand-alone accounts in our new platform. And to illustrate how complex this process may be for different website setups I will list a few examples below:

Multiple addon domains using one and the same database

Normally, this is not a reasonable thing to happen, as each website, even addon ones, should be using their own database. However, this was technically possible with cPanel and there are actual sites set up this way by our customers. When such a site needs to be migrated, our migration script has to detect the case and create a separate database for every site and copy the data. After that, the script automatically reconfigures each website to use the respective database.

Applications with absolute paths in their configuration

Another thing that the migration script should fix is multiple applications set to use absolute paths in their configuration. The system has to detect those and once they become independent sites with different system users, to automatically reconfigure them to use the new system paths.

The Addon/Parked/Subdomain infinite setup options 

The infinite number of ways users can set up and sometimes mess up their document root paths for the different applications when using the addon, parked, and subdomain functionalities in cPanel is the biggest challenge for our transfer process. We have managed to list more than 30 different ways of unorthodox document root path cases. One of the most common examples is when more than one addon domains are configured to use the same folder.

This is a common “hack” of the cPanel system people use to park a second domain to an addon site — an option that is officially not allowed in cPanel. So in such setups, the system has to automatically decide which site is each domain going to and in what role (primary or parked). In some cases, the setup is so complex that the domains cannot be configured automatically and the migration should be attended manually.

Serious Development Assignment 

Automating all possible processes

We started our first migrations in September last year very cautiously. The migrated sites were manually reviewed and each of the issues described above, plus many more that appeared, have been addressed with new iterations of the migration script. Needless to say, the manual checks of the first migrations took a lot of time and was not something that could be sustainable in the long run, so we have added several additional automations to the migration script. 

We now do automated pre-checks of all accounts to be migrated. If there is an indication of a possible problem, we fix it before the migration has started. After that, the account is actually migrated from cPanel to Site Tools. Once the migration is over, we run another automatic check for post-migration issues. If any issue is detected the account is marked for manual review.

Additionally, we have developed an automated system for communicating the progress of this process with the customer whоse accounts are being migrated. With all these systems now being in use, we are able to convert about 900 cPanel accounts per day with a very low fail rate. 

Switching to the New Customer Area first

In the beginning, we planned to migrate each customer simultaneously to the new Client Area and the new Site Tools. However, we soon realized that it will be much better to untangle the new Client Area from Site Tools and move all clients to the new Client Area first. There were several reasons for this decision: 

• First, the Client Area migration was in itself less risky, as it did not directly affect the hosted websites functionality. 
• Second, we figured out that providing the Client Area first will give all our customers some time to get used to the new interfaces. We do appreciate the effort needed to learn a new interface, so by getting used to our new UX logic with the Client Area, we wanted to make the transition to the more functionality-dense Site Tools smoother for our users. 
• And third, daily maintenance of two Client Area interfaces was taking away precious time from our technical teams. Time that could have been spent on perfecting the migration scripts.

The decision to untangle the transfer to both interfaces required temporary focus change, as we now had to invest some development time into accommodating cPanel accounts in the new Client Area interface, which was not initially planned. However, in the long run, we believe that this decision brought the end date of the whole migration closer. We are very proud to have all our clients successfully using the new Client Area since May 2020. 

The challenges of 2020

As we all know this year has been extremely unpredictable and for perfectionists like our managerial team, planning has become a nightmare with too many variables and dynamic factors that inevitably slowed down the migrations during the year. 

Transfer to Google Cloud Platform

One of the things that made us reorganize our initial plans was really positive — the finalization of our contract with Google Cloud. Moving our service to a cloud-based platform was the other big project we had been working on in the past years. We knew that moving to Google Cloud would provide a lot of immediate benefits to all our customers and completing this migration first would enable us to dedicate all resources to the much more complex switch to Site Tools.

Therefore, once we finalized the contract with Google, the migration to Google Cloud was prioritized in the queue of our DevOps and SysAdmin teams. They worked fast and did a great job – we managed to migrate our whole platform to Google Cloud in less than 4 months. And we are talking about more than 4 PB of data! Though a big part of our resources was invested in the Google migration for several months, we still managed to do considerable work on polishing the Site Tools migration scripts in the meantime.  

The COVID-19 effect

Of course, we cannot forget the COVID-19 effect. During April and May, with quarantines affecting many of our geographical markets, we have seen an unprecedented uprisal in support inquiries. It was only natural, as the online presence has suddenly become much more important for a great number of people and businesses, leading to a higher level of activity by all website owners.

We were truly swamped and for the first time since we started operations, we had to literally throw all our resources into customer support. Despite hiring new people, the volume of work was so big that every department had to focus and contribute in some way to the provisioning of the service and optimization of how that service was delivered. Again, that was putting the focus away from the migrations and even though we kept a core team working on them, the managerial and operation attention was elsewhere and migrations were going slower during this period.

That being said, I still believe we managed to address both of the big 2020 events quite well and managed to remain reasonably on track with our third main endeavor — the transfer to the new interfaces. 

Current status of the migration

At the moment Cloud accounts are being migrated with the highest priority. We aim to complete the migration of the majority of the clouds by mid-December. There is a small percentage of Cloud accounts that are not included in this plan. These are accounts where existing WHM functionality was used to create custom cPanel plans. This means that custom resource limits were set by the cloud owner to the separate cPanel accounts on the cloud. As preserving these settings is a serious additional complexity to be addressed by the migration script, we will have to postpone such accounts’ migration. (In case you own such a cloud account, and you believe you do not need to keep your custom cPanel account settings, you may contact us through the Other Technical Issues category in your Help Desk and request your cloud inclusion in the current migration schedule.)

In the meantime, we are also working hard on the shared server migrations. Due to the larger volume of data on them and the different setup specifics, their migration rate is much lower at the moment. However, now that we are close to finishing the cloud account migration, we have focused more resources on optimizing the shared servers migration process and expect to see a larger number of shared accounts migrated in the next several weeks. 

On behalf of the whole SiteGround team, I would like to thank you all for the patience this year. We are fully aware we have created expectations for a change and it may be taking longer than you have anticipated, but we really do our best to make this super complex switch happen as soon as possible and at the same time to be a safe and easy experience for all our customers.