by Ruth O’Keefe and Justin McGarvie
It’s that time – time to review your website data to get a glimpse of your overall web traffic. You log in to your Google Analytic account and see, wow, a huge spike in traffic! Hurray… or is it?
If you’ve reviewed your metrics and see, no, your huge spike wasn’t due to a great social media post or a new ad you placed then it could just be a bot messing with the quality of your analytic data.
There are good bots and bad bots but all bots skew your data. Basically they do everything from inflate visits to increase bounce rates which all effect your sampling and understanding of your website traffic, not to mention conversion metrics.
So let’s walk through how you can determine if it’s time to hide a certain bot from your Google Analytic reports so you can get back to business.
Identifying Your Bot in Google Analytic
There are several different ways to identify your bot depending on what kind of bot it is, whether it’s malicious (spam bot) or a good ol’ crawling bot. Here are the three areas we tend to search first…
Step 1: Location (Audience > Geo > Location)
In our example you see that not only was the geographically region appearing as “(not set)” but 100% were new sessions with a high bounce rate and short session duration. These 3 items alone indicate bot: session length, bounce rate and new vs returning sessions or views. This is still not a completed confirmation the surge was a bot but definitely worth more digging!
Step 2: Traffic Source (Acquisition > Overview)
Red flag #2! Look at how high the direct traffic was! In the context of this example, it is very unlikely that so many sessions resulted from someone typing the url directly into their browser. Now our eyebrows are definitely raised so we check out one more place before declaring this a bot situation and taking action.
Step 3: Networks (Audience > Technology > Network)
Ding! Ding! Ding! Take a look at the number 1 provider and percentage of sessions, for a little Service Provider named “google inc.” The Service Provider section is the names of the Internet service providers (ISPs) used by visitors to your site and this tells us google is our bot. Common service providers that are typically bots are: amazon.com inc., google inc., microsoft corp, amazonaws.com but there are many more.
Now that we’ve identified our bot – let’s remove it from our reports to keep it from skewing our data moving forward.
Filtering Your Account for Your Newly Found Bot
Step 1: Go To Your Admin settings.
Step 2: Select Filters under View > All Web Site Data
Step 3: Select +New Filter
Step 4: Name your Filter and select Custom
Step 5: Select Exclude and under Filter Field select ISP Organization. Here you enter in your filter, in our example you would put: ^(google inc.)$
Looking to filter more than one? Here are some other common filters you could include: ^(microsoft corp(oration)?|inktomi corporation|yahoo! inc.|google inc.|stumbleupon inc.)$|gomez
Step 5: Save your new Filter
Congratulations you just entered your first filter to remove the bots from your view. It is important to note that this does not exclude the bots from visiting your site but does allow for a more accurate portrayal of your true website traffic.
Other Important Reminders:
Figuring out what is really a bot can be difficult once you’ve excluded all the big players. But remember the key indicators and it could help up front. High bounce rate, short session duration and high amount of new visitors!
Make sure to check the box in your view settings under Bot Filtering to Exclude all hits from known bots and spiders.
To do this:
Step 1: Go back to your Admin Settings and under VIEW: All Web Site Data select View Settings
Step 2: Scroll down until you see Bot Filtering. Check the box labeled “Exclude all hits from known bots and spiders”
Step 3: Remember to Save
There are many ways to identify bots crawling through your website. Tell us about ways you uncovered a bot and how you handled it in the Comments section below.