What is bot traffic?

Bot traffic is the name given to non-human traffic and traffic created by various spiders and programs.

This traffic can cause misleading results in your web page data and Google Analytics reports. It could also have an impact on overall business performance.

More importantly, it’s vital for your business to be proactive in detecting bot traffic, and then to filter out bot activity. Doing this will help you make effective data driven decisions.

Detecting suspicious activity in GA

Do you review your web traffic and analytics results on a daily or weekly basis? Examining your analytics results regularly and knowing your web traffic and traffic sources can help you identify bot and spam traffic. You should also set up alerts that notify you automatically when a threshold is reached.

The screenshot below shows an increase in traffic. There are also two other spikes that aren’t in the normal flow of traffic distribution.

Was there a special campaign or an activity that would lead to an increase in web traffic in March? If not, and the traffic is not internally sourced, it can be concluded that this traffic may be generated by bots.

Before starting this research, it is very important to check whether the standard Google Analytics bot traffic and spider filter is selected in the Analytics settings or not. This option filters many bots that Google can identify as a bot or spider. To view this option, go to Admin and then View Settings in Google Analytics. Make sure that the Bot Filtering box is ticked in this section.

Three steps to identify bot traffic in GA

Step 1: In the Google Analytics Master View section, select Acquisition from the left side column and select All Traffic and then Channels. Click Referral in the Default Channel Grouping list. In this section, you can see various referral sourced visits and referral information.

Do you know all the listed sources? Examine them in detail by clicking the Show Rows option at the bottom of the page. Then browse the traffic distribution by choosing a wide date range.

In the referral information on this screenshot, we can see the visits from various sources. The traffic from a source called 'getbotraffic4free.club' is an unknown source and looks suspicious.

Step 2: Examine the traffic sources in the source list you think is suspicious. Use different metrics such as Bounce Rate, Average Visit Duration, Sessions and Users.

If a 100% bounce rate has a visit duration close to 00:00:00, this traffic may be bot traffic.

Step 3: If there’s referrer spam that disguises itself, use the Secondary Dimension option to search for different options. For example, Hostname, Network Domain, City, Campaign.

Any website without your website's hostname can be defined as bot traffic.

An example of some bot traffic sources is:

• /getbotraffic4free.club/

• /Ashburn City bot traffic/

• /bot.traffic.icu/

• /bot.traffic.live/

Four options to filter bot traffic from GA

To filter bot traffic from your GA, consider four options:

1. Use the Referral Exclusion list

2. Use GA filters

3. Create segments

4. Use cloud service offerings

1. Use the Referral Exclusion list

A popular way to exclude bot traffic referral sources from analytics results is the Referral Exclusion List. This is in the Admin section of GA.

You can filter any suspicious traffic or traffic coming from a bot traffic source from our results by using this feature.

However, this filtering doesn’t impact the previous results and website data. This feature filters referral from your results, starting from the date of application.

It’s also important to test the Referral Exclusion List under Testing View, or a view created for the purpose of preserving your current collection methods.

2. Use GA filters

Filter bot traffic by using Google Analytics' existing filtering feature. This provides more advanced options and filtering capabilities compared to the Referral Exclusion List as you can create more detailed filters.

You can also create filters based on City, Network Domain, Hostname or Campaign Source by using the Filters section in the Admin section. Since some bot traffic doesn’t have referral information, you can use this section to create detailed filters.

The screenshot below is a bot filtering example. It filters 'traffic' and 'bot' keywords sources from analytics results. This was tested under Test View before applying it to the Master View.

3. Create segments

Bot or spam traffic in your GA results may cause problems when creating reports or making decisions about your business.

Get accurate results by creating Google Analytics Segments. This allows you to separate bot traffic from your previous data (data before the filter is applied).

In the example below, we created a segment to filter bot traffic originating from Ashburn from the web page results.

You too can create a segment like this one and use it when reporting your data retrospectively.

4. Use cloud service offerings

As another level of protection, you can use a bot protection service by different cloud services providers such as Azure, AWS Cloudflare or Barracuda.

For instance, Microsoft’s Azure Web Application Firewall can help you to filter potential bot traffic and spiders. Bots and spiders can be blocked by the firewall, and you can also specify and block the specific bot traffic sources. Thus, they can't use up the resources and services of your website and impact your Analytics results. Also, some services such as Amazon Bot Management feature can provide you Bot Analytics and give insights related to your website’s bot traffic activity.

Our Digital Experience team works with many customers to get the best from Google Analytics by creating insightful reports and dashboards that help influence decisions.

Working with Codehouse

Our experts are helping customers tackle their data challenges. To find out more about how we can enable your business to get the very best from your website data, get in touch. We’re here to help your current digital marketing challenges.

Images sourced from Google Analytics