How To Choose The Right Keywords for Organic Google Ranking

Many keyword tools (Ahrefs included) are trying to solve it for you by showing some sort of “keyword difficulty” or “keyword competitiveness” metric – but can you rely on their judgement?

Well, the goal of the article that goes below is to give you a definitive answer to this question.

No one really knows how Google ranks pages

Basically, the entire SEO industry is nothing else but hundreds of thousands of people trying to figure out via trial and error how Google ranks pages.

In a nutshell, all we know today is that Google uses over 200 different ranking factors and 3 most important of them are Links, Content and RankBrain (not necessarily in this particular order).

We also know that Google is experimenting a lot with machine learning and artificial intelligence algorithms, which should completely revolutionize “search” in the next few years.

So where am I going with this?

If you want to determine keyword difficulty with 100% accuracy – you need to be using exactly the same algorithms that Google uses to rank pages.

So does any third-party tool have access to Google’s ranking algorithms?

Nope.

Maybe it can develop information processing algorithms that would boast the same level of sophistication as Google?

Very unlikely.

That is why no keyword difficulty checker is perfect and each tool can only give you their best estimate.

But even an estimate is better than nothing, right? And besides, certain tools are much more accurate than others (wink).

Important: a lot of people who are new to SEO mistakenly rely on the “Competition” metric that they see in Google Keyword Tool. Please be advised that this metric has nothing to do with ranking difficulty and only shows how many advertisers are bidding to show their ads in the search results for a given keyword.

How to determine the ranking difficulty of a keyword

The only way to learn how difficult it would be to rank on top of Google for a specific keyword is by carefully analysing the pages that already rank there.

Ideally, you’d want to vet these pages for all 200+ aforementioned ranking factors. But since no one (except Google) really knows how much each individual factor contributes to the resulting ranking of a page, it only makes sense to focus on the biggest ones: links and content.

Links

Let’s use one of the keywords that we’re aiming to rank for with Ahrefs Blog as an example: “anchor text”

The quickest way to see the amount of backlinks of the top10 ranking pages for this keyword is to put it into Ahrefs’ Keywords Explorer tool and scroll down to the “SERP overview” report:

The “Domains” column shows how many unique websites link to a given page. And it’s not that hard to see a general pattern: the more sites link to a page, the higher it ranks in Google.

In fact, we’ve studied the correlation between the number of referring domains to a page and it’s position in Google across 2 million keywords and it turned out to be pretty solid:

One other interesting takeaway from the above graph is that the number of referring domains to a page has a better correlation with Google rankings than just a raw number of backlinks. So, as a general rule, it is suggested to get a link from 10 different websites rather than 10 links from a single website.

But apart from the sheer quantity, there’s also a quality factor in place: a small number of high quality links may trump a larger number of lower quality ones.

For that we have a metric called URL Rating (or “UR”).

You can see from the graph above that UR correlates with Google ranking much better than the raw number of linking domains. That’s because Ahrefs’ URL Rating takes into account the quality of backlinks (to a certain extent) and was specifically designed to reflect the ability of a page to rank well in Google (read more about Ahrefs’ metrics here).

An yet, even with UR (which is the highest correlated metric in the SEO industry) we’re only scratching the surface of how Google would process backlink factors.

There’s just too much to consider:

 

• Where the link is located on the page?
• Is that link likely to be noticed and/or send traffic?
• What is the anchor text of that link?
• What is the surrounding text of that link?
• How many other backlinks are on the page?
• At what pace the page is aquiring new backlinks?
• & so on.

 

Authority of a domain

A lot of SEOs believe that the so-called “domain authority” (or “domain rating”) has a big influence on a page’s ability to rank.

But at the same time many SEO professionals are convinced that such a thing as “domain authority” does not exist.

So who’s right and who’s wrong?

Well, here at Ahrefs we’ve studied the correlation of domain-level backlink factors across 2 million keyword searches and plotted them alongside with the page-level factors:

As you can tell from our data above, domain-level factors have significantly smaller correlation with rankings than page-level factors. And yet that correlation is still quite solid.

Does this mean that Domain Rating helps you rank higher?

I’m afraid we can’t confirm that just based on this correlation. Correlation ≠ causation.

But what our data suggests is that you should be able outrank high-DR websites if you have more links coming to your page.

And this wraps up my very brief overview of how to approach keyword difficulty from a backlinks standpoint.

Usually SEOs won’t go too deep in reviewing a given SERP: they will just look at the number of linking domains and UR/DR of the top-ranking pages and settle with that information. But for some important keywords you may want to go as far as reviewing the actual backlinks, where they come from and what would it take to replicate them.

Content

It is true that you can easily outrank pages with vastly more backlinks if they’re lacking relevance to the search query.

Here’s a keyword that perfectly illustrates what I mean: “chocolate lab”

Looks like the pages with only 6-20 referring domains are outranking the pages with 900-1000 referring domains.

How is that possible?

Well, if you open that Wikipedia page with over a thousand referring domains, you’ll see that “chocolate labrador” is only a small sub-section of a very big article:

While the articles that rank above that Wikipedia page are entirely dedicated to this specific breed:

This is a perfect illustration of how relevant content can overweight even the strongest backlink profile.

But don’t get too excited about it just yet.

What we see in this example is called “lack of relevant content” – the top-ranking results are targeting a broader search query (labrador retriever), rather than a very specific one that people are searching for (chocolate lab).

That is a massive opportunity for relevant content to shine, and that is how those two articles got to the top without a lot of backlinks.

But you don’t see this kind of thing very often.

Usually what you get in the SERP is “slightly imperfect content” (at best) – the top-ranking results are 100% relevant to a search query, but they could do a slightly better job of giving visitors what they’re looking for.

This kind of SERP won’t give you the same level of competitive advantage as “lack of relevant content”, where you can rank without backlinks.

So how do you know if the search results for your keyword are lacking relevant content and you can beat them without links?

And how do you make your own page 100% relevant to a given keyword in the eyes of Google?

Let me try to address these two things.

“Conventional On Page SEO” vs Topical relevance

Imagine you put your target keyword in Google and see that the top-ranking pages don’t use that keyword in their Title/URL/Headline.

This is an indication that you can easily outrank them if you use that keyword in Title/URL/Headline of your own page, right?

Wrong!

The best practices of on page SEO in 2017 are not as straightforward as they were back in 2010.

Back then Google didn’t have fancy things like Hummingbird and RankBrain, so it needed some very strong clues to understand what your page is about. This is when putting your exact match keyword in Title/URL/Headline of your page gave a strong competitive edge over the pages that weren’t doing that.

But this trick doesn’t work anymore. Today Google is smart enough to understand what your page is about even when a target keyword is never mentioned on the page.

In fact, by studying 2 Million keyword searches we have discovered that almost 75% of pages that rank in Google top 10 don’t have even a single mention of an exact match keyword in their content.

Check out the SERP for the keyword “guest writing” to see what I’m talking about:

Clearly, Google can understand that things like “guest writing”, “guest blogging” and “guest posting” are closely related. So if you perfectly optimize your page for the keyword “guest writing” in accordance with these oldshcool on page SEO best practices – that won’t give you any competitive edge at all.

How to make your page relevant?

Or should I re-phrase that question to “how to make your page MORE relevant than the pages that currently rank in top10, so that Google would rank your page higher even with less backlinks?”

Well, I’m afraid that there’s no easy and straightforward way to do it.

In order to make your page perfectly relevant to Google, you first need to understand how Google interprets search queries and matches them to topics and entities that it extracts from web pages.

Sounds complicated? That’s because it is.

You can try studying things like latent semantic indexing (LSI), latent Dirichlet allocation (LDA) and some other topic modeling algorithms, but most people won’t obviously go that deep.

And why should they in the first place?

Since Google is getting so smart to the point where it almost “reads” the pages of your website, why should you even bother about adjusting your pages to meet some complex criteria of its algorithms and not just “write for humans”?

Well, the important word here is “almost”. Despite its impressive complexity, Google is still a machine and if you understand how it works and can adjust your pages accordingly – you’ll be one step ahead of everyone else.

We’re going to properly cover the topic of “new on page SEO” in one of our upcoming articles, so now I’ll leave you with 2 quick tips:

1. Use Ahrefs’ Site Explorer tool to analyse top ranking pages for your target keyword and see what other keywords they also rank for. This will give you some clues as to what topics Google thinks they are relevant to.
2. Open top10 pages that rank for your target keyword and use one very sophisticated tool to extract topics from them – your brain. If you Google around and study all pages related to your topic, you will naturally build a good thesaurus of words and topics that will help Google identify your own page as perfectly relevant.

User Intent

In most cases “relevance” and “user intent” go hand in hand. But sometimes a perfectly relevant search result might not give the user what he was looking for.

In this case Google will always favour “user intent” over “relevance”.

Sounds confusing? I have a great example for you.

If you search for “online survey” from United States, you get search results that look like this:

9 search results are suggesting you tools to create online surveys and 1 search result is offering you to get paid for “online survey jobs”.

But let’s search for exactly the same keyword from United Kingdom:

This time only 5 of the search results are offering you tools for online surveys, while the other 5 are offering you “online survey jobs”.

This example shows that people might be looking for different things when they search for a general keyword that may have multiple meanings.

And Google has somehow identified that most people in US are looking for an online survey tool, while a lot of people in UK are also interested in making some money by participating in online surveys.

But how does Google know what people are looking for?

It has not been officially confirmed, but the rumours are they might be looking at thinks like:

• how long people stay on the page after clicking on it search results (otherwise known as dwell time);
• whether people will click on any other search resutls or just settle with the one they picked first;
• whether people will get what they were looking for from their first search or if they will keep refining it and clicking on more search results.

And these kinds of things can sometimes overweight the topical relevancy of a page.

I mean if more people in UK start clicking on search results related to “online survey jobs”, Google will see that and drop a bunch of “online survey tools” results from the front page. Even though they were perfectly relevant to the keyword “online survey” and had tons of backlinks.

SERP history

A good way to learn if Google is happy with the search results or if it is still figuring out what’s best for users is to look at the SERP history.

Here’s SERP position history that Ahrefs’ Keywords Explorer tool shows for the keyword “search engine optimization”:

The same 5 pages are ranking at the top for quite a while, with only small shifts in their positions.

And here’s SERP ranking history for the keyword “twitter marketing”:

The pages were jumping on and off top10 for almost a year. And only recently Google seems to have figured what kind of search results satisfy people the most.

In other words, SERP position history can be somewhat indicative of Google’s own level of satisfaction with their search results and thus reflect your chances of squeezing your own page there.

And that was the last thing that you might want to look at when assessing the difficulty of a certain keyword.

Now let’s discuss the Keyword Difficulty metric that we have in Ahrefs and how it can make things easier for you.

Ahrefs’ Keyword Difficulty

Once you put your keyword into Ahrefs’ Keywords Explorer tool, the first metric you’ll see will be the Keyword Difficulty score:

We measure KD on a scale from 0 to 100, with the latter being the hardest to rank for.

But that doesn’t really help you to understand what KD 15 or KD 65 means, right?

I figured it would be best to explain Ahrefs’ Keyword Difficulty score and how to use it in a Q&A format:

1. What does Ahrefs’ KD score mean?

Our keyword difficulty metric gives you an estimate of how hard it would be to rank in top10 search results for a given keyword.

Attention: I didn’t say “rank #1 for a given keyword”, I said “rank in top10 for a given keyword”.

This little nuance causes a lot of confusion: people see an insanely strong page ranking #1 and decide that our KD score is not accurate. But if you look at pages #2-10, they may be a lot weaker and thus easier to outrank.

Why top 10?

Here at Ahrefs we believe that Google heavily relies on backlinks for identifying which pages deserve to rank in top 10.

But once your page reaches the front page of Google, all sorts of other important factors kick in:

• which search result attracts the most clicks?
• which page shows the best behavioural factors?
• do people keep looking for more results by refining their search query?

And lets not forget about those sophisticated topic modeling algorithms that I’ve mentioned earlier.

In other words, creating a Keyword Difficulty score that would accurately predict the #1 is as easy as building our own Google here at Ahrefs.

That’s why for now we’ve settled with predicting the chances of ranking in top10, which we do quite accurately.

2. How do you calculate KD?

We look at how many backlinks the top10 ranking pages for a given keyword have.

We’re not taking into account things like: Domain Rating, age of the website, usage of keyword in Title/URL/H1, etc (I’ll explain why below).

We also don’t differentiate between dofollow and nofollow links, because the SEO community still haven’t decided if nofollow backlinks help you rank or not.

3. Why don’t you take into account any “on page SEO” factors?

First of all, let’s define the “on page SEO factors” that we’re talking about.

Most likely you’re referring to things like:

• Using exact match keyword in Title/URL/Headline;
• Keyword density of your target keyword in content;
• Length of content;
• Number of social shares;
• Outgoing links to authority sites;
• Page load time;
• etc.

Well, numerous studies (ours included) have confirmed that these kinds of things have a very minuscule correlation with rankings, compared to backlink factors.

So even if we were to include them in our KD calculation, they would change the resulting KD score by no more than +/- 3 points. Which is not significant enough to bother spending CPU resources on it.

But if we’re talking about advanced on page SEO concepts that I’ve mentioned above: topic modeling, TF-IDF, entity salience, etc. – we’re obviously working in this direction, but we’re not ready yet to apply what we have to our KD formula.

4. How accurate is your Keyword Difficulty score?

Given that Ahrefs boasts the world’s best database of live backlinks, our Keyword Difficulty score represents the most accurate picture of how competitive a SERP is backlink-wise.

The accuracy of our metric was confirmed by a third party test of existing keyword difficulty tools, where Ahrefs won the first prize:

But even though Ahrefs is more accurate than any other tool, we don’t recommend you to blindly rely your SEO decisions on our KD score alone.

It can be a great “first filter” to weed out the keywords where you need way too many backlinks to rank, but then you’ll have to look at the actual SERP and vet the top ranking pages manually in a way I explained above.

5. How do I read your KD scale?

Because Ahrefs’ Keyword Difficulty metric is tied to backlinks and nothing else, the scale is pretty straightforward.

Each KD score translates into an average number of referring domains that top10 ranking results have:

KD 0 = 0 Ref. Domains
KD 10 = 10 Ref. Domains
KD 20 = 22 Ref. Domains
KD 30 = 36 Ref. Domains
KD 40 = 56 Ref. Domains
KD 50 = 84 Ref. Domains
KD 60 = 129 Ref. Domains
KD 70 = 202 Ref. Domains
KD 80 = 353 Ref. Domains
KD 90 = 756 Ref. Domains

From the above graph you can see that our KD scale is exponential, so there’s a vastly bigger difference in referring domains between KD 80 and KD 90 than between KD 20 and KD 30.

6. What range of KD can be considered “easy”?

This question is tricky, because what seems to be “easy” to someone, might turn out “insanely hard” to someone else.

And yet from the conversations with many of our customers we figured that a universally accepted KD scale should look like this:

Even for us at Ahrefs getting ~40 websites to link to our page is pretty hard thing to do.

7. What KD range can I safely target with my website?

You can easily identify your “safe” KD range by looking at the keywords that you currently rank for.

Put your website into Ahrefs’ Site Explorer tool, go to “Organic Keywords” report and apply “Position” and “Volume” filters to find the best keywords that you rank in top5:

On the screenshot above I filtered all keywords with Volume over 500 searches per month where ahrefs.com ranks in Positions 1-5.

That resulted in 26 keywords (many of which are duplicated because they rank in SERP features).

Then I exported that report and copy/pasted these 26 keywords into Keywords Explorer.

It automatically removed the duplicates and showed me aggregated data for 16 keywords that were left:

As you can see from the “Difficulty distribution” graph, most of the keywords in my list fall into the KD 40-60 bucket.

This means that we can safely target any keywords with KD up to 60.

Replicate the same steps for your own website and you’ll see what is the maximum possible KD that you can target.

8. Why a lot of KDs in my report are gray?

Here’s a screenshot of what it looks like:

We have 2 types of Keyword Difficulty in Keywords Explorer:

• Cached – we regularly process SERPs for millions of keywords and calculate KD score based on the pages that we see ranking there at this time. Some keywords are updated every day, others might take up to a month to renew. We show cached KD in gray color to illustrate that it’s not up to date;
• Fresh – once you click the “Get metrics” button next to any keyword, we’ll get the fresh SERP and calculate KD based on the pages that rank there at this very moment. The resulting number will be assigned a color in accordance to our scale above.

Cached KD might not be very accurate, because SERPs are subject to change, but it can be extremely handy when you need to filter a huge list with a couple thousand keyword ideas to just a few ones that don’t require many backlinks to rank for:

Time to find some easy keywords to rank for!

So now you’re armed with all the knowledge you need to accurately gauge the ranking difficulty of a keyword.

Ahrefs KD score is a good starting point for your research, but please make sure you always check the SERP manually before making a decision to put your time and money into some keyword.

Read the full article on the Ahrefs website