Site too large to officially use Google Analytics?

Question

We just got this email from the Google Analytics team:

We love that you love our product and use it as much as you do. We have observed however, that a website you are tracking with Google Analytics is sending over 1 million hits per day to Google Analytics servers. This is well above the "5 million pageviews per month per account" limit specified in the Google Analytics Terms of Service.

Processing this amount of data multiple times a day takes up valuable resources that enable us to continue to develop the product for all Google Analytics users.

As such, starting August 23rd, 2010, the metrics in your reports will be updated once a day, as opposed to multiple times during the course of the day. You will continue to receive all the reports and features in Google Analytics as usual. The only change will be that data for a given day will appear the following day.

We trust you understand the reasons for this change.

I totally respect this decision, and I think it's very generous to not kick us out.

But how do we do this the right way -- what's the official, blessed Google way to use Google Analytics if you're a "whale" website with lots of hits per day?

Or, are there other analytics services that would be more appropriate for very large websites?

Answer 1

Stackoverflow.com does blow well past that limit doesn't it! Fortunately, per paragraph 2 of the terms of service...

FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 5 million pageviews per month per account, and if You have an active Adwords campaign in good standing, the Service is provided without charge to You without a pageview limitation.

... as long as you have an "active Adwords campaign in good standing", then the service is provided "without charge to You without a pageview limitation".

EDIT:

As of October 2011, the terms of service have been updated such that the page view limit was raised from 5 million to 10 million page views per month HOWEVER the AdWords exemption has been removed:

FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 10 million pageviews per month per account.

Consistent with this answer here it is not exactly clear what the intent is for sites that exceed the limit and if/how the limit is really enforced.

For reference, the current full terms of service are documented here:

http://www.google.com/analytics/tos.html

Answer 2

I think that for being free, getting data updated once a day is just fine. I don't even know why with such large numbers you need more then that. Did you really lose anything?

If you really want alternatives, I suggest you try http://piwik.org/. I have no idea if it can take the load, but since it is up to you to install and configure, I guess all is possible (only questions is, how much work?)

Other hosted option is http://chartbeat.com/ which since you actually pay for the service, I don't think they will have a problem accepting your money.

Update: based on comments, http://www.getclicky.com/ is not an option.

Answer 3

Or just check results once a day. Is your site/service dependent on reports multiple times a day?

Answer 4

If you aren't using AdWords, it may be worth contacting a Google representative to see if they're willing to make a special arrangement for you. If they're not, I'd be shocked.

Answer 5

Google Analytics is actually based on Urchin, which you can install and host yourself on your server (cross-platform). The interface looks similar to GA but it may not be as polished. It's also very expensive - nearly $3,000 for a license!

If Google is still allowing you to track every hit then there is probably no reason to switch - do you really need to check data up to the minute?

Answer 6

We run Piwik for a client with about 3 million page views per month. The mailing list has faster response times then Google customer support! With a bit of hacking it should be possible to run piwik on multiple machines.

The other solution is to only run Google Analytics as a sampling tool. For example you only log 1 out of every 10 visitors to the site. This is what some large sites do with some of the paid for Analytical tools to save on license fee's.

Answer 7

The limit documented in the Google Analytics Terms of Service has been raised to 10 million pageviews per month, with no "active Adwords account" exemption.

1. FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 10 million pageviews per month per account.

It's not clear how actively this will be enforced.

For sites looking to exceed this cap, Google Analytics is now offering Google Analytics Premium, an enterprise-focused paid version of Google Analytics.

Answer 8

perhaps you should take a look at the high end analytic tools like Omniture, Webtrends, Coremetrics, Webtrekk, who are specialised in dealing with high traffic pages and allow much more customised tracking and evaluation. They cost quite some money though, but if you are that big, it's definitely an investment to take into account. Most big sites use one of these vendors and sometimes google analytics as a control.

Disclaimer: I am actually working for Webtrekk.

Answer 9

You could also use Google Analytics's own sampling functionality, which will sample traffic by user cookie and yield consistent results in the reports-- you just need to scale up the numbers you see in Analytics by dividing by your sampling rate.

http://code.google.com/apis/analytics/docs/concepts/gaConceptsSampling.html

Answer 10

That's fascinating stuff. My website has 10 million page views a month and I have never heard from Google. However, I've noticed a few months ago when I go to Google Analytics in the morning the numbers from the previous day will sometimes be a lot less then they should be. If I go back later in the day, then the data will be updated and I can stop freaking out at my loss of 10K visitors.

I've have recently started tracking events with Google Analytics on a much larger scale (about 60K events a day) and I wonder if all that data processing effects the speed with which my site stats are updated.

Answer 11

Google Analytics supports sampling data collection as standard now.

With this kind of volumes, a real WA solution should be considered. The Urchin7 license (really, don't go to Urchin6, I speak from experience) may be $10k, but it's a one time fee. With many products

Answer 12

I have always heard good things about Mint: http://www.haveamint.com/

It not a hosted service so you can throw it on your own servers.

Answer 13

Suggestion:

Only use analytics for conversion-related pages and metrics. For example, tracking signup conversion %, actual users referred from adwords hits, non-members, and referrals:from:other:sites. (the last can help PageRank, I have no clue why...)

Perhaps:

What you are really looking at is a two-part solution. Rather than entirely replacing google analytics, you reduce the volume by the method above and also either use another 3rd-party analytics tool or create a server-sided one.

It really is a matter of what analytic data is important to you, and do you really need Google to help you with that?

Answer 14

I've heard good things about StatsCounter, even when specifically compared to Google Analytics. Peter-Paul Koch from Quirksmode.org likes it because it's supposedly got very good browser detection accuracy

Answer 15

What about Open Web Analytics? It's free and looks googlish. Might be worth a try. It says here that it does have mouse tracking and heatclick maps. However you have to host it but it has no Data/Logging Limits.