SEO guide
Chapter 2

SEO Keyword Research: A step-by-step guide with tips from the world's top SEO experts

I went into this wanting to create a step-by-step guide for effective keyword research. In researching this piece, I read 89 great, great guides and expert articles about keyword research post RankBrain. I’ve included tips and advice from the best influencers in SEO and only selected methods that lead to quick results.

In this 2d chapter of our efficient SEO guide, you will find the basics, as well as advanced techniques recommended by some of the world’s leading SEO experts, including:

Eric Enge

Eric Enge SEO expert, founder and CEO of Stone Temple Consulting

Michael King SEO

Michael King Founder of IpullRank

Larry Kim

Larry Kim Founder of Wordstream

Brian Dean

Brian Dean Founder of Backlinko

Dr Peter J. Meyers

Dr. Peter J. Meyers Marketing Scientist at Moz

Tim Soulo

Tim Soulo Head of Marketing at Ahrefs

Marie Haynes

Marie Haynes SEO expert, specialist on Google penalties

Matt Janaway

Matt Janaway CEO of Marketing Labs

Rand Fishkin

Rand Fishkin Founder of Moz

edward sturm seo specialist

Edward Sturm, SEO consultant

Casie Gilette

Casie Gilette Senior Director of Digital Marketing at KoMarketing

Brian Lang SEO blogger for small business

The experts featured here represent a total of 1.2 million followers on Twitter!

Alright, let’s get started!

Always start with numbers

In this SMX SlideShare, Casie Gilette provides the statistics you should have in mind before even starting your keyword research:

  • 50% of search queries are 4 words or longer
  • 27% of search queries are questions
  • Average voice search query is 5-6 words

 

Keyword research has changed a lot over the years. Here are the recommended steps, post RankBrain.

Keyword Research Basics

The notion of keywords

In this guide, when I use the term “keyword,” it can mean a single word or several. For example, “camp” might be a keyword but “cheap summer camp” can also be a keyword.

Before starting your keyword research, it’s important to structure your keyword list correctly. A keyword list isn’t just a long list of all the keywords related to your activity. It has to be organized so that you can start working with it right away. If your goal is to get quick results, then you have to organize your keyword research very efficiently. Your first keyword list should be structured something like this:

SEO how to organize your keyword research

Let’s explore each column of this keyword list.

 

Topics

“Topic” can mean different things, depending on the type of website you’re optimizing.

E-commerce:

  • a category of product, e.g. “ultraportable laptops”
  • a subcategory of product, e.g. “ultraportable laptops with ssd”
  • a specific product that a replica datejust many people are searching for, e.g. “Dell XPS 13”
  • a blog post, e.g. best ultraportables
  • a question, e.g. “what ultraportable should I buy”

 

Blogging:

  • a theme, e.g. “Spring cocktails ideas”
  • a problem, e.g. “broken iPhone”

 

Services:

  • a specific usage of your service. For example, the same place could rank for “wedding venue”, “seminar venue”, “meeting space,” and “conference room.”
  • a subsection of the services you offer: “horseback ridding”, “riding lessons”, “horse riding club”

 

Local:

  • an activity in a specific place (general topic + location), e.g. “meeting space new york city”
  • different continue reading activities in different locations (specific topics + location), e.g. “horse riding club new york,” “horse riding club chicago,” “horseback ridding new york”, “horseback ridding chicago,” etc.

Search volume: use it to rank your keywords, not to predict future traffic

Most keyword tools usually provide a metric called search volume, which comes from Google AdWords’ API. However, search volume is not always accurate, which Russ Jones explains very well in this excellent article. But right now, we’re more interested in knowing how to prioritize different keywords than predicting precisely how many visitors each query might yield.

In any case, when you target a specific keyword, only 10 to 20% of your traffic to that page/topic will be a direct result of that targeted keyword; the rest of your traffic will come from different combinations of this keyword and long-tail expression containing the keyword or its synonyms.

Secondary keywords

For each topic, you will want to:

  • Find a list of derivative requests users might type into Google.
  • Rank those keywords by search volume
  • The keyword with the highest volume will be the primary keyword for this topic
  • The remaining keywords will be secondary keywords.

I will delve into this hierarchy a little bit deeper later on.

Landing page

For each topic, you will need to create a specific landing page. In this column, enter the URL of the page you created to target that specific topic.

Step 1: Brainstorm

SEO influencers recommend brainstorming first (see this infographic from Search Engine Land). The goal here is to come up with your first keyword ideas that will develop and evolve in the following steps using keyword research tools.

I’d like to add a personal piece of advice here. I’ve created five companies, and each time, I was surprised at the way my clients defined what I do. So I have one universal rule: entrepreneurs don’t know what words clients will use to find the businesses they run. This is why the main task of any entrepreneur is to meet their clients and listen to them. SEO should always start with asking clients how they would define your product/activity and carefully noting the words they use.

At Moz, they used Google Custom Audience Surveys to ask customers which keyword they were using.

Eric Enge’s advice is to use your internal site search for inspiration. If you can get a list of keywords and phrases that your existing customers are already using to find what they need on  your website, you’ll have an easier time creating your first keyword list.

Having finished the first step, you should now have a list of 10-100 keywords on your spreadsheet.

Step 2: Add keywords that people already use to find existing pages of your website

Bruce Clay’s advice is to start with what you already have. This is good advice because it’s always going to be faster to optimize and improve existing results then to create new goals. To do so, start by studying the insights from your Google search console:

Keyword research Google Search Console
  • Set up your search console
  • Log into your search console
  • Go to Search Traffic > Search Analytics
  • Note all the keywords that interest you on this list.

Some influencers also suggest listing your existing landing pages and finding the keywords that are specific to those landing pages. I personally don’t think that’s a good idea because the goal of keyword research is to define a new ideal structure for your website, not just polishing up the status quo.

Marie Haynes says that she likes using the Google search console to find keywords that bring traffic even when the website isn’t currently ranking in the top 3. It means that those keywords have high potential if you focus on them.

You should now have a complete list of new keyword ideas.

Step 3: Brainstorm with competitors’ keywords

Select some competitors that rank well for the first keywords you found. To find these, just type those keywords in a private browser tab (Incognito on Chrome). Make sure you’ve logged out of all your Google accounts and cleared your cookies and cache.

Then, you can use a competition-tracking tool to find the keywords your competitor uses.

Below are a few competition-tracking tools you may want to check out:

Mazen:

Mazen provides a list of keywords that the competitor currently ranks for, as well as a list of keywords that the competitor should target based on the semantics of their website (homepage, landing pages, etc.).

Mazen SEO keyword research competitors

SEMrush:

SEMrush is more about estimating a competitor’s existing traffic (which is sometimes accurate) and providing a list of keywords that bring this traffic. For each keyword, you get the search volume, the landing page that is currently ranking, and the number of Google results for each keyword (which helps to estimate the difficulty). SEMrush will also help you discover other competitors by displaying your competitors’ competitors.

SEMRush competitive analysis screenshot

SpyFu:

With SpyFu, you can find an exact list of your competitors’ keywords, as well as each keyword’s ranking and pertinent data about each keyword. SpyFu also provide an estimated number of visits (clicks) that each keyword yields for the competitor.

Spyfu keyword research competitors

At this point, you should have more keywords in your list, as well as relevant data for each keyword (difficulty, cost per click, and search volume), which can be obtained using dedicated tools.

Step 4: Find new keyword ideas with dedicated tools

You can literally find hundreds of different keyword suggestion tools online. I even found this list of 117 keyword tools compiled by Brian Lang.

Here are the features you should look for in these tools:

Category grouping:

In Mazen and SEMrush (under “Keyword Magic Tool”), the keywords suggested are pre-grouped into categories. For example, if you type “web agency”, you will get categories like design, marketing, SEO, digital marketing, web development, etc.

SEMrush keyword magic tool

Here’s a step-by-step guide to grouping keywords in Google AdWords:

  • Log into your Google AdWords account
  • Click on “Tools” > “Keyword Planner”,
  • Click on “Search for new keywords using a phrase, website, or category”
  • Enter your keyword idea
  • Select the “Ad group ideas” tab
Adwords group keywords by category

Artificial intelligence keyword suggestions:

When you enter a keyword to get suggestions into most keyword research tools, you’ll mainly get variations of the same keyword (see below):

SERPs.com:

Serps.com keyword suggestion

Ubersuggest:

Ubersuggest keyword research

Keywordtool.io:

Keywordtool.io SEO

In these examples, each keyword suggested contains at least 1 of the 2 words that compose the original keyword idea “web agency” and often both. It’s more about completing those keywords than finding new ideas.

That’s why you should use tools that not only provide variations, but AI recommendations as well.

Mazen produces suggestions that don’t have any common words with the original. For example, if you type “web agency,” you might get suggestions like “web design company,”  “digital marketing company,” “marketing website design,” “ecommerce website designer,” etc.

But the really interesting feature is that it will also generate keywords for the entire business scope of a web agency: web design, digital agency, web marketing, SEO services, small business design, affordable web design, e-commerce, etc.

Mazen SEO artificial intelligence keyword suggestion

SEMrush also provides strong results. In the screenshots below, you can see that the tool has found some new specific topics like wordpress agency, web development, web marketing, e-commerce, etc.

SEMrush keyword research 1
SEMRush keyowrd research 2

Step 5: Add questions

Google’s RankBrain algorithm, along with the rise of vocal search, has turned Google into a lean mean answering machine. Structuring your blog according to questions that your clients/users/readers are asking is a very good way to find new content ideas and get more qualified traffic.

Matt Janaway’s advice, and I totally agree with him, is to use AnswerThePublic to get ideas for the questions you’d be answering. BuzzSumo is another tool that allows you to find questions people might ask. BuzzSumo is very helpful because it shows you topics that have trended and received social shares or backlinks.

Here’s an example from AnswerThePublic:

Answerthepublic questions keyword research SEO

Here, we can see that there is an opportunity with questions like “what do summer camps cost?” or “what are some good summer camps?” We would then write some great content that answers those questions and includes comparisons, infographics and/or videos.

Here’s an example using BuzzSumo :

Buzzsumo for SEO keyword research

This tells us that “adult summer camp” is a great idea for content because it generates a lot of social sharing.

Step 6: Group keywords into primary and secondary

The idea of primary and secondary keywords is probably the most important notion in keyword research. A primary keyword is a general topic, whereas a secondary keyword is a variation of a primary keyword. It can also be a completely different phrase with a similar meaning.

According to a study on RankBrain by Ahrefs, it is now possible to rank for hundreds or thousands of keywords with only one page (i.e. one topic) if those keywords all target exactly the same topic. That means you don’t have to use every single keyword in your content because Google now understands that close semantic keywords have the same meaning. Some aspects of keywords that Google might treat the same include:

  • Different word order: “apartment rentals New York” and “New York rental apartments”
  • Singular/Plural: “rent a flat in new york” and “rent flats in new york”
  • Abbreviations: “rent a flat in New York” and “rent a flat NYC”
  • Synonyms: “cheap flat NYC” and “affordable flat NYC”
Close semantic keywords SEO

In a fantastic article published on Moz’s blog, Dr. Peter J. Meyers explains how RankBrain is changing how keyword research is performed. What are the consequences for this new keyword research paradigm?

  • You don’t need to do cram keywords into your text (i.e. keyword-stuffing): Google understands the meaning of your content, so you should only use variations/synonyms, but don’t obsess over it.
  • You need to create content that helps Google understand your purpose: this means longer content, but also semantically relevant content. You can read more about that here.
  • Your page should rank not only for one main version of your key phrase but also for dozens of variations. The advice here is the same that SEO specialists had for Google’s Panda algorithm. Yulia Khansvyarova, head of SEMrush Marketing, says “Stop creating pages or content tailored to only one keyword or keyword phrase“. That is to say, target general topics that will rank for a wider range of keywords. I would add that, with RankBrain, it’s essential to track your ranking for a large number of variations of your main phrase.

Take a look at what I mean below:

Rankbrain Google Keywords

In this example, the first keyword is “apartments for rent nyc” and it has the highest search volume. If we only considered the search volume for this topic, we would be missing a lot because there are tons of variations of this general topic. The global search volume of the entire topic here is 61,180 (and probably much more, since only a few examples of keywords are displayed here), which is a lot more than the search volume of the primary keyword (33,000). This means that you need to add up the search volume of each variant in order to obtain the real search volume for this topic (or a closer estimate).

 

The second reason these secondary keywords are so important is that you need to track your ranking not only for the primary, but also for each secondary. If you only rank for a small segment of the keywords, it probably means that your page is not perfectly relevant and that you should consider improving your content. If you rank for the primary but also for 70 to 80% of the secondary keywords, then your final traffic for the page will be far more significant.

When researching keywords, you should really create a hierarchy within your list:

How to organize keywords SEO

There are two keyword tools that allow you to easily group your keywords into primary and secondary buckets:

Mazen provides different variations for each keyword. You don’t need to group the keywords yourself because Mazen’s algorithm does it for you. Keywords are then sorted into primary and secondary groups.

The advantage here is that Mazen automatically detects the different topics you should target, and the different keywords (variations) that you should use for each.

Take a look at this example:

Mazen keyword suggest grouping

Moz also has a feature that allows you to group keywords. To use it, go to keyword explorer, and, in the “Group Keywords” dropdown menu, select “yes, with high lexical similarity”:

Moz group keywords feature
Moz grouped keywords suggestion

Step 7: Organize keywords

Keyword organization is probably the most time consuming task you’ll encounter in SEO. But it’s important to do it properly, because if you don’t:

  • You won’t be able to prioritize your work
  • You won’t be able to track your rankings and make decisions based on them
  • You won’t be able to optimize your content properly and evaluate the results of those improvements

The minimum information you need

Each SEO specialist has their own way of organizing keywords and adding information about them. But all the big SEO minds agree that you should, at the very least, include the following:

Data for keyword research

Let’s explore what each data point means and how to find it.

Search volume

We already explained search volume and why you need it at the beginning of this guide.

Difficulty

Keyword difficulty is important because it allows you to prioritize your work. An SEO campaign is very time consuming. It’s not really worth it to spend days trying to rank on a very difficult keyword if you can easily bring new customers to your site with easier keywords. You’ll especially want to prioritize easier queries when your website is young and you don’t have a lot of backlinks to your domain.

There are three main ways to calculate keyword difficulty

The first method is to rely on Google AdWords difficulty. This method is not very relevant for SEO because AdWords only calculates difficulty based on the number of companies that buy that keyword on Google AdWords. But interestingly enough, the most difficult AdWords keywords are sometimes quite easy to target via organic SEO.

The second method of evaluating keyword difficult is based on the total number of internet pages that have been built to specifically target a keyword. In this case, you would measure the number of pages that have made SEO efforts to target that keyword. To find that number, use and combine any of the following three indicators:

  • Number of search results for this keyword
  • Number of search results that have the keyword in the title tag
  • Number of search results that have the keyword in the URL

This method can help you figure out how many competitors are targeting the keyword, but not how serious any of those competitors might be. For example, it could end up being more difficult to rank in the top 10 for a keyword targeted by 15 competitors who have a lot of backlinks to their pages, than to rank in the top 10 against 1,500 competitors that don’t have many backlinks.

Tim Soulo, head of marketing at Ahrefs, says that, although this method worked in 2010, it isn’t as effective anymore. He explains that with Hummingbird and RankBrain, the exact keyword you use in the title tag isn’t that important since Google can understand the general meaning of your content even if the exact term isn’t found on a page. Tim Soulo explains it like this:

Check out the SERP for the keyword ‘guest writing’ to see what I’m talking about:

Clearly, Google understands that things like ‘guest writing,’ ‘guest blogging,’ and ‘guest posting’ are closely related.

Google guest writing SERP

The last (and I think best) method of measuring keyword difficulty is to tally the backlinks and page authority of the top 10 Google search results for that keyword. This method is used by Ahrefs and SEMrush, and I strongly recommend leaning on these metrics. The more backlinks and authority pages in the top 10 have, the more difficult it will be for you to rank on that keyword.

Landing page

To fill out this column, you’ll need to answer two questions:

  • Is there already a page ranking for the primary keyword?

If yes: you can optimize the content to rank better

If not, you should ask a new question:

  • If there are no pages ranking for the primary keyword, is there a page on your website dedicated to that very topic?

If there isn’t, now your know you need to create a new page to target that topic.

If your website is huge, this process can take a very long time. And if it’s not your website, it can be difficult to answer the second question. The good news is that Mazen has an algorithm integrated in its software that automatically finds the best landing page for each primary keyword… which can save you a lot of time!

Ranking

To prioritize your work, you’ll need to know which keywords are already ranking, and at which position. According to LinksManagement, keyword traffic distribution looks like this:

Traffic Google Ranking

This means that a keyword that ranks 5th or 6th would get twice as much traffic if it was ranked 3rd, and three times as much if it was ranked 2nd. If you make improvements on the pages ranked in the “golden triangle” (the top 3), you can get some quick wins that will help you save a lot of time early in your campaign.

Save 90% of the time spent on organizing keywords

To get any SEO campaign off the ground, you will need to gather information from various sources, including:

  • Keyword research tools for keyword lists and volume/competition research
  • Keyword tracking tools for rankings
  • Google analytics for traffic
  • SEO tools for backlinks

After that, you’ll also need to organize your keywords into groups of primary/secondary keywords and topic categories.

These operations are very time consuming. That’s why you should use a comprehensive tool like Mazen to organize your keywords, pages, and data without having to export a single Excel file. In Mazen, you can group and categorize keywords with a single interface.

Mazen keyword suggest grouping

The software automatically finds the pages that rank (or should rank) for each primary keyword.

Mazen SEO keyword to content mapping

Other columns you might want in your spreadsheet

SEO is a craft: we each have our own techniques and priorities. Here are some of the data columns most of our experts use:

Conversion rate

If you can get this information (specifically for e-commerce), then you’ll be able to prioritize the job on keywords/pages that are converting the best. This helps a lot, because it’s probably not worth the effort to target keywords that don’t convert. But you should also think about other types of “conversions” and goals… For example, a keyword that brings in a lot of traffic but few conversions can still be useful if it attracts backlinks.

Type of snippets available on the SERP

Depending on the keyword, there might be different snippets available in the search results. Larry Kim says that click-through rate (CTR) on the SERP is one of the most important ranking factors Google uses with RankBrain. He adds that the best way to generate traffic is to have a great CTR. It’s very useful to know how the SERP is structured for each keyword:

  • Is there an answer box ? (“Position 0”)
  • Is there a video or image snippet?
  • Is there a map, Google shopping, or customer review snippet ?

Knowing what the SERP looks like will help you create the best rich snippet for your keyword’s landing page, as well as use the right media for that page.

Commercial intent

Some SEO influencers will advise you to add a column where you can rate a keyword’s business potential. This is sometimes called “user intent.” This can be useful in prioritizing your work. For example, let’s say you have a hotel booking website; a customer searching for “what is the best destination in California?” is not as likely to convert as a customer searching for “affordable hotel San Francisco”.

Brian Dean says that commercial intent is one of the more important elements of keyword research. He explains that one of the first websites he created generated 60,000 unique visitors per month, but only $400 per month in revenue. The reason was that, when he researched his keywords, he only focused on search volume and not on at all on commercial intent.

To my mind, this method should also be complemented with “backlink potential.” Some keywords might lead you to create very interesting and high value content that is more likely to interest other bloggers/websites than generate revenue. This content can become good link baiting material.

 

Keyword categories

As you probably know, internal linking is one of the most important elements of on-page optimization. But it’s really important to plan this out thoroughly. If you want to use internal linking effectively, check out this piece I recently wrote about the subject.

The general idea is that you need to group your pages into semantic clusters. This means that your primary keywords (topics) should have a logical hierarchy. When it comes to on-page optimization, it will help you figure out which pages to link to one another.

Advanced techniques: The best of what’s in the crazy minds of the greatest SEO influencers

Branded keywords of your competitor

Why not target your competitors’ brand keywords? For example, if you were competing with Evernote, you could target a keyword like: “how to use Evernote”. To find such keywords, you can use Google’s related searches.

Target competitors keywords SEO

Persona-driven keyword research

Creating your SEO keyword list is a process not unlike market research. Michael King explains how iPullRank totally re-built its keyword research around business personas. The step-by-step guide for this method is:

  • Create personas for different types of prospective Googlers
  • Create surveys for your customers to fill out. King explains: “We develop questions that allow users to self-identify or even disqualify themselves as our target personas.” They then flesh out these user profiles by asking respondents about their search behavior in order to figure out what keywords they might use and where they are in the decision-making process.
  • Extrapolate the data. iPullRank takes the data they collected from the survey and applies that logic across all of the keywords in their list