Google Analytics is just wrong!

The other day, I was looking through TubeBattle’s syslogs…

I don’t get much time, but try to do a quick scan of it on a weekly basis just to see who is trying to hack that server (you’d be amazed at the HTTP calls people try to make!). Anyway, as I was looking through it, it occurred to me that I was getting way more hits than Google Analytics was reporting!

My syslogs were coming back with the following statistics:

  • 5,927 unique visitors for the last 14 days of January (Google was reporting 5,556)
  • 16,684 page views for the last 14 days of January (Google was reporting 15,704)
  • 8,027 unique visitors for the first 9 days of February (Google was reporting 4,215)
  • 18,684 page views for the first 9 days of February (Google was reporting 9,452)

A quick calculation shows that Google was under-reporting by:

  • 6.3% unique visitor count in January
  • 47.5% unique visitor count in February
  • 5.9% page view count in January
  • 49.5% page view count in February

In summary, it looked like Google was under-reporting more than more as TubeBattle got busier.

So off I went to investigate the cause (using Google Search, ironically), and this is summary of what I discovered:

  • Google Analytics requires the browser to make a javascript call to a Google server, to register the visit.
    1. It is susceptible to being “lost” (if the Google Analytics servers are busy or off-line, it will miss the count).
    2. If the browser does not support Javascript (such as: old browsers or proxy servers that don’t support Javascript, users who turn off Javascript support, etc) the call is never made and so that visit is never registered.
  • The syslog is the most comprehensive logging system, recording every visit.
    1. The problem is that it records visits by bots (e.g. if Google Search is scanning and indexing your site, it would still be counted as a visitor).
    2. The good news is that the bot, regardless of how many times it visits or pages that it scans, is still only counted as one unique visitor.
    3. The bad news is that Google probably has a bunch of these bots. So even though one bot counts as one unique visitor, ultimately all the bots from all the search engines out there could probably throw off stats by a few hundred (or possibly even more).
    4. But then again, if the majority of your site’s stat count is from search engines, you’ve got bigger problems to worry about :P

So all up, it looks like Google Analytics under-reports and syslogs report everything. Whereas the number that people really care about is somewhere in between (but closer to syslogs’ answer).

With this finding, I’m removing Google Analytics from all my sites and going back to the syslog analysis. Right now, I’m using awstats and webalizer (those were pre-loaded on my server). If anyone has a better syslog based analysis tool, please shoot me an email or leave a comment!

TubeBattle’s ongoing development

So it’s been 3 weeks since TubeBattle went live. The stats have been impressive so far.

  • Total # of visitors since opening: 9,267 (3,089 per week or 441 per day)
  • Total # of repeat visitors (visited more than 1 time): 845 (9.1%)
  • Total # of videos served up: 10,560
  • Total # of votes cast: 644

Here’s a snapshot of my Google Analytics Geomap, showing where all the users are coming from:
Tubebattle users' from around the world

As part of the ongoing development of the site, my developer has completed most of the visible changes. The site now displays some “activity counters”, such as # of users online currently, # of videos in each battle, # of votes that have been cast (in total as well as for each battle). I’ve noticed a huge up-tick in votes being cast, since the counters went in. I’m also overjoyed by the fact that the “# of users online” has always been in the 3-15 range every time I check the site, so it looks like people are really enjoying the website!

GotoMyPC - Satan or Saviour?

With a long career in IT Management, GotoMyPC has alwasy been the arch-enemy. Any application that allows an external system to bypass corporate firewalls inevitably gets the “shoot first, figure it out later” treatment.

With me being overseas, and still needing to access my PC in Vegas I needed a solution. GotoMyPC (which is on the top of the “shoot first” list of every IT organization I’ve ever worked at) was my first and only choice:

  • It obviously had enough marketshare: otherwise business poeple wouldn’t be begging their IT for it
  • It obviously worked well: otherwise IT wouldn’t be so fearful of it

Rather than giving an over-simplified “dummies” review or a overly-complicated “techies” review, I’ll just summarize my findings after a few days of use:

  • It does everything their marketing blurb suggests
  • The installation is quick and simple. Although I wish they wouldn’t use Java (I just hate the fact that the Java run time environment keeps popping up, seemingly needing to upgrade every week, and tries to subliminally make me buy Sun stuff).
  • It uses a Citrix client (for the techies, nothing else needs to be said after that endorsement)

As long as I have to access my machine remotely, GotoMyPC is a simple and cheap solution that’s technically sound.

Oh, and a word to my IT breathren. Yes, GotoMyPC still needs to be at the top of that arch-enemy list ;)

Domain transfers from Godaddy.com

Today, I found a cool domain name to register (for a future project). I decided to use namecheap.com for the registration. $7.99 (with coupon code) was pretty darn cheap for a .com registration, with free WhoisGuard protection.

While I was there, I finally bit the bullet and decided to transfer some of my current domains from Godaddy.com to namecheap.com. Turns out, it isn’t as easy as you might expect! Getting namecheap to initiate the transfer process was simple enough, but getting Godaddy to release the domains was a lot more obscure (can’t say I blame them, they’re losing a customer after all!). So here’s what I learnt…

  • When initiating a domian transfer, the new registrar requires a EPP (Extensible Provisioning Protocol) code from the old registrar. Basically, the new registrar will queue the transfer until you (the owner) enters the EPP code.
  • Finding the EPP code from Godaddy.com was rather obscure! But basically you need to login to Godaddy, open up the domain you want transferred, and on the bottom of the left colum will be a link that says “Authorization Code: Send by Email”. Clicking on the link will result in an email being sent to the domain owner (the Technical Contact I presume) with the EPP code.
  • Enter that EPP code to the new registrar, and then the domain will get cut over.
  • If you have Godaddy’s DomainsByProxy, make sure you cancel it. Otherwise you’ll give Godaddy another excuse to abort your domain transfer.

Once you know this “trick” it’s simple enough.
But just a quick google shows that there are many webmasters out there wondering why Godaddy doesn’t release domains and what they can do about it. Now I know, and hopefully so do you :)

James posted a really useful hint in the comments below. Basically, if you go to Godaddy’s Domain Management page, there’s a menu option for “Pending Transfers” on the left side. If you go in there, you can manually approve any pending transfers. That’ll reduce the transfer time from days to hours. Thanks for the great tip James!

CelebFace “under new management”

On New Years day, I had the opportunity to buy CelebFace, a celebrity gossip site.

Although it is my most recent acquisition, it’s the first one to “go live. I’ve got a few other projects on the boil that is imminently going public, but they’re about 1-2 weeks out.

I’ve got big plans for CelebFace, and have already engaged some people to work on it. For prosperity’s sake here are a few snapshots of the the original format (as I bought it)…

CelebFace Home (Original)
CelebFace Article (Original) 

I won’t give away the new design yet, but here’s a teaser with the new logo design…

CelebFace Logo (New)