Organic Traffic Study

Abstract

The objective of this study is to determine whether content optimization solely driven by RelevancyRank recommendations impacts organic search traffic from Google. The study used 3 randomly applied treatments of RelevancyRank modified content, random keyword modified content, and unmodified content. RelevancyRank modified content saw an organic traffic lift of 17.5%, random keyword modified content saw a 10% drop and unmodified content saw drop in traffic of 15%. This indicates that RelevancyRank optimized content can create increases in organic search traffic that cannot be explained solely by content freshness measurements or random fluctuations.

Methodology

1. Randomly select 200 pages from a single website: We use a single website to prevent domain authority from influencing the effectiveness of content modifications.
2. Identify top keyword referring traffic to each of 200 selected pages
3. Randomly apply treatments to each of the 200 pages
   1. Mod1: Insert 50 recommended words from RelevancyRank Keyword Recommendations for Keyword and Page Content via Paid API
   2. Mod2: Insert 50 random words from dictionary to page (This helps us control for content freshness bumps)
   3. Mod3: Make no modification to the content
4. Record 3 weeks worth of organic google traffic for each keyword prior to transformation
5. Record 3 weeks worth of organic google traffic beginning 1 month after the launch of transformation
6. Record shift in organic Google traffic for each page
7. Record average shift for each transformation method

Results

There were 3 separate ways we could look at this data.

1. Determine the mean percentage increase for each page in each cohort,
2. determine the combined mean percentage increase across all pages in each cohort,
3. and determine the percentage likelihood that a page would increase in organic traffic at all in each cohort.

We look at each of these measurements for all 3 cohorts below.

Unmodified Cohort

The average traffic shift for each page in the unmodified cohort was 105.65%. This means that on each page, on average, saw a 5.65% lift. However, we know this was skewed by looking at the following 2 measurements. The average traffic change when all pages were combined was 75.84%. This means that many of the more popular pages actually lost traffic during this time. Finally, only 1/3 of the pages in the unmodified cohort saw any positive movement at all. If you look at the graph above, you can see quite clearly that the unmodified cohort was skewed by a single outlier that jumped from 1 visit to 5 visits.

Random Keyword Insertion Cohort

Within the random keyword insertion cohort, the average shift in traffic was 119.16%. This too was skewed by a very large outlier at 800% traffic increase. We can use the other two tests to confirm this. The total traffic shift when including all of the visits to the pages within this cohort was 84.61%, indicating a loss in traffic over the time period. Only 40.84% of the randomly modified pages saw a jump in traffic, the majority remaining unchanged or losing traffic.

RelevancyRank Cohort

The cohort modified by RelevancyRank recommendations saw an average shift in traffic of 131.48%. This traffic shift corresponded with both the second and third metrics. When all traffic was combined, the RelevancyRank modified content still saw a 116.17% traffic shift, the only cohort to record a positive number and which was more than 30% greater than random. Perhaps most importantly, 63.49% of the RelevancyRank modified content had positive growth, while only a minority remained the same or lost traffic.

Conclusions

Blind insertion of RelevancyRank recommended keywords to content can increase organic Google traffic, and that improvement cannot be described as caused by freshness ranking factors.

Reservations

It is absolutely necessary that we make clear what conclusions cannot be drawn from this study…

1. We cannot conclude that topic modeling or content relevancy is used in Google’s algorithm.
2. We cannot determine the exact mechanism by which RelevancyRank recommended keywords increase Google traffic.
3. We can, however, provide evidence that insertion of RelevancyRank recommended keywords do increase Google traffic.

About the Graphs

The graphs above show the raw results of traffic fluctuations following transformations for each keyword. There was variation in each cohort size because they were randomly applied.