How to run an N-Gram analysis even when Google Ads hides the data

If you are familiar with Google Ads, you probably know that Google likes to change quite often how the platform works and which data they provide through it.

In particular, last year, Google released an update that had a huge impact on all the search advertisers. Since then, they are basically hiding part of the search queries that triggered a search campaign on Google.

As you can see in the screenshot below, they added at the end of the search terms report an extra row called “Other search terms” where they provide costs, clicks, conversions,… but not the text of the search queries.

This is a big issue for many advertisers because this “hidden group” can go up to 60-70% of all the search queries data and this information is very useful when you are optimizing a campaign and especially when you want to run an N-gram analysis.

In this post, I want to share an idea I had a couple of days ago and that can help you to still run an N-gram analysis on 100% of your search queries data.

Search queries report in Google Analytics

If you connected your Google Ads Account to Google Analytics, you should be able to see part of your Google Ads data in Google Analytics.

Especially, if you look at the search queries report you’ll notice that there are some search queries that generated sessions, conversions but that don’t have clicks and costs data.

If we want to run an N-gram analysis we have to use the Google Analytics data because it’s the only report where we still have all the search queries. The problem is that we need to know how many clicks and costs the hidden queries generated, otherwise, we would take into account all the conversions but not the costs generated by those queries.

How to estimate clicks and costs for the hidden search terms

If we look at the data we have, we know that for the known queries is not a problem we have everything we need.

But for the unknown queries, we only know how many sessions and conversions they generated. Clicks and Costs on a search term level are missing.

The only way I see to attribute costs and clicks to the hidden search queries is to take costs and clicks from the “Other search terms” row (either for a specific Ad Group or Keyword) and use the % of sessions that a specific search term generated as a “weight”.

I have chosen to use the sessions because it’s the closest metric to clicks.

In the example, I took as an example the hidden search terms for the keyword “google merchandise”. In total, the queries generated 32 sessions, 10,213 clicks and 253,049.46€ costs.

The search term “google merchandise home” generated 15,63% of the 32 sessions and that’s why it gets 1,596 clicks and 39,538.98€ costs.

Conclusion

I’m currently working on the R code that will automate the whole process and maybe I’ll share it later on.

I know that especially nowadays Google CPCs on search term level can vary a lot but we have to work with the data we have and I don’t see a way to attribute a CPC that is not an average but please let me know your opinion in the comments below.