Monitoring Your Magento Web Site

When was the last time you think your web site was down? When was the last time you knew it was down? How many 404 errors do you get in a regular day? How long does your site take to load from halfway across the country? How long does it take for a transaction to process half way around the world?

These are all questions that you should be asking yourself (and be able to answer) when your Magento web site is your main source of corporate revenue.

Groove was recently asked to evaluate a retailer’s site that had begun to show a very erratic conversion rate issue. At first glance, the eTailer had many of the best practices in place. Practices that should have enhanced the conversation process—not lowered them.

We quickly felt that these issues had to be server or network related. After setting up the proper tracking, we quickly determined that over 20% of the site’s visitors were getting a database time out error. Our developers easily fixed this error in just a few hours once found. This simple error was costing the retailer at least $50,000 a month in lost revenue. Now, that’s a leaky bucket if I’ve ever seen one.

Creating a site-monitoring plan for your Magento site does not have to be complex or expensive exercise, but it does need to be consistent and cover both external (availability, load times) and internal (server, database) factors.

Here are the 5 main elements we look at with Magento systems:

1. Site uptime & load time by location
2. Transaction monitoring
3. Payment gateway & 3rd party resource availability
4. Internal system & database errors
5. 404 Page Errors

External Monitoring

Externally monitoring Magento is important as it allows you to view your site in the same way that the rest of the world sees it. This typically requires a 3rd party tool with a monthly subscription. Back in the old days, a simple ping of the home page every 10 minutes was enough. These days with more complicated software and an abundance of 3rd party tools being utilized you’ll get detailed information to help you determine where and how your site falls short.

Site Uptime/Load time by Location – This is the basis of all site monitoring. Is your site available for the world to see, and what is your load time across various locations/regions? For most sites, an availability check every five minutes will suffice. If there is a failure, it should check once per minute until it becomes available again. You will also want to check load times by specific locations. For example, you may want to see how quickly your site loads from a major city in each U.S. time zone. If you have a wider reach you may want to check by the international cities where you have large pockets of customers. We recommend at least four locations outside of your region.

Transaction Monitoring – This is the ability for a visitor to perform a complete transaction from each of the locations in your list. This is done through an automated process at various intervals. This type of monitoring is significantly more expensive than uptime reporting and can be done at longer intervals based on your traffic/order volume. Once every 15 minutes to once per hour are acceptable time frames for most sites. Keep in mind that you will need to create a method to clear these automated orders out of your Magento database, as they must take the same path as your customers to be valid.

Payment Gateway / 3rd Party Resource Monitoring – Last summer Authorize.net had an outage that left many major eCommerce sites without the ability to process transactions for over 10 hours. This type of full outage rarely happens, but you will want to protect yourself. With many sites using 3rd party API services, you will also want to keep track of those too. These services like real-time shipping rates, site search and sale tax calculation can bring Magento to its knees if not found quickly.

For one project, we created a simple PHP file that would connect to each of the respective 3rd party services each time the page loads and return a status of ‘good’ if it could connect to their APIs and ‘fail’ if it could not. Just for fun we called it ‘The Matrix’ and added the scrolling background from the movie. You can password protect this page and build further features and health stats into it over time. This page is pulled by the monitoring service every five minutes and alerts the team if any text on the page matched the word ‘fail’.

Most modern day monitoring services will alert you via text message or email if you have an outage. WebSitePulse is a great service that can handle both your uptime and transaction monitoring needs. It tends to be a more ‘do it yourself’ service that is reflected in the lower pricing. AlertSite takes that one step further with more in-depth reporting and features. It even integrates with your Google Analytics account to overlay your visitor reports over its performance reports.

Internal Monitoring

Just as important as external monitoring, keeping track of your Magento software and server health is the other half of the equation. Until recently, you had to build your own ‘Matrix’ pages or dig into some command line interface. Magento Extension Developer, AITOC recently released its Magento WatchDog product into the marketplace as a great addition.

Now your developer will probably try to convince you that all of the same information will be available from an SSH command line. For most site owners though, anything behind the black and white command line is out of sight and out of mind. For a $119 investment, it exposes many great reports into your Admin that will allow a non-technical user to see what is affecting their customers and can be notified via email or text message based upon alert criteria.

Internal System/Database Errors – No software package or server configuration is ever bug-free. Errors such as Mysql connection errors, Magento Logic errors or Upgrade/Install problems can happen for a variety of reasons. Being able to quickly diagnose and fix these errors is important. We have seen ongoing problems that can add 2-3 seconds load time Magento that once found are minor to fix.

404 Errors (Page Not Found) – 404 Errors can be caused by links that are old and outdated such as a popular product or category page from previous shopping cart software. Unscrupulous people that are phishing through your site can also cause them. All have an impact on your server performance, which affect Magento. When viewing these 404 errors, you can group these links by URL, IP or referrer. This allows you to easily create 301 redirects (Page Moved) to the new pages and block IP’s or referrers that are up to no good.

Once installed, Magento WatchDog creates an external control panel that allows you to diagnose and fix errors when Magento Admin or your database is unreachable. Of course any mission critical errors should be fixed immediately, but your site administration and development teams can address minor issues weekly.

Documenting your Plan

Now its time to document execute your plan in five easy steps. Think of it as going to the dentist or getting your oil changed. A little maintenance can go a long way in keeping Magento healthy. Just follow Groove’s 5 Steps to Monitoring Success:

1. Identify 3rd party monitoring service
2. Select monitoring locations
3. Setup site availability and transaction monitoring
4. Install Magento WatchDog
5. Review both internal and external reports weekly!

Links:
https://www.websitepulse.com/
https://smartbear.com/product/alertsite/web-monitoring/