Rokkit Marketing Google Analytics Bot Blog Image 1

That Bot Skewing Your Data – How to Identify and Filter Your Bots in Google Analytics

by Ruth O’Keefe and Justin McGarvie

It’s that time – time to review your website data to get a glimpse of your overall web traffic. You log in to your Google Analytic account and see, wow, a huge spike in traffic! Hurray… or is it?

If you’ve reviewed your metrics and see, no, your huge spike wasn’t due to a great social media post or a new ad you placed then it could just be a bot messing with the quality of your analytic data.

There are good bots and bad bots but all bots skew your data. Basically they do everything from inflate visits to increase bounce rates which all effect your sampling and understanding of your website traffic, not to mention conversion metrics.

So let’s walk through how you can determine if it’s time to hide a certain bot from your Google Analytic reports so you can get back to business.

Identifying Your Bot in Google Analytic

There are several different ways to identify your bot depending on what kind of bot it is, whether it’s malicious (spam bot) or a good ol’ crawling bot. Here are the three areas we tend to search first…

Step 1: Location (Audience > Geo > Location)

Google Analytics - geographically region appearing as (not set)

In our example you see that not only was the geographically region appearing as “(not set)” but 100% were new sessions with a high bounce rate and short session duration. These 3 items alone indicate bot: session length, bounce rate and new vs returning sessions or views. This is still not a completed confirmation the surge was a bot but definitely worth more digging!

 

Step 2: Traffic Source (Acquisition > Overview)

Google Analytics - High amount of Direct Traffic with high Bounce Rate

Red flag #2! Look at how high the direct traffic was! In the context of this example, it is very unlikely that so many sessions resulted from someone typing the url directly into their browser. Now our eyebrows are definitely raised so we check out one more place before declaring this a bot situation and taking action.

Step 3: Networks (Audience > Technology > Network)

Google Analytics - google inc. - bot service provider

Ding! Ding! Ding! Take a look at the number 1 provider and percentage of sessions, for a little Service Provider named “google inc.” The Service Provider section is the names of the Internet service providers (ISPs) used by visitors to your site and this tells us google is our bot. Common service providers that are typically bots are: amazon.com inc., google inc., microsoft corp, amazonaws.com but there are many more.

Now that we’ve identified our bot – let’s remove it from our reports to keep it from skewing our data moving forward.

Filtering Your Account for Your Newly Found Bot

 

Step 1: Go To Your Admin settings.

Google Analytics - How To Filter Bots

Step 2: Select Filters under View > All Web Site Data

Google Analytics - How To Filter Bots

Step 3: Select +New Filter

Rokkit-Marketing_Google-Analytics_Bot-Blog_7

Step 4: Name your Filter and select Custom

Google Analytics - How To Filter Bots

Step 5: Select Exclude and under Filter Field select ISP Organization. Here you enter in your filter, in our example you would put: ^(google inc.)$

Looking to filter more than one? Here are some other common filters you could include: ^(microsoft corp(oration)?|inktomi corporation|yahoo! inc.|google inc.|stumbleupon inc.)$|gomez

Google Analytics - How To Filter Bots

Step 5: Save your new Filter

Google Analytics - How To Filter Bots

Congratulations you just entered your first filter to remove the bots from your view. It is important to note that this does not exclude the bots from visiting your site but does allow for a more accurate portrayal of your true website traffic.

Other Important Reminders:

Figuring out what is really a bot can be difficult once you’ve excluded all the big players. But remember the key indicators and it could help up front. High bounce rate, short session duration and high amount of new visitors!

Make sure to check the box in your view settings under Bot Filtering to Exclude all hits from known bots and spiders.

To do this:

Step 1: Go back to your Admin Settings and under VIEW: All Web Site Data select View Settings

Google Analytics - Bot Filtering

Step 2: Scroll down until you see Bot Filtering. Check the box labeled “Exclude all hits from known bots and spiders”

Google Analytics - Bot Filtering

Step 3: Remember to Save

Google Analytics - Bot Filtering

There are many ways to identify bots crawling through your website. Tell us about ways you uncovered a bot and how you handled it in the Comments section below.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *