Last update: 03.05.2021
At the beginning of the year, I decided to buy a subscription plan on Datacamp because I wanted to learn R – a statistical programming language who can help you to run advanced analysis but also automatize some processes.
I’m still a beginner – but at least, I started to use it for keyword research and also pulling data from APIs.
And today, I want to talk about the Google Search Console API – that you can easily use with R. It’s a very huge topic, that’s why in this post I’ll only show you the basic commands and how to setup the connection with the API.
Learn how to install R & R Studio
1. Open R Studio and install the package
Open to R studio and write the first lines of code. First thing, you have to install two R packages called searchConsoleR .
|
1 2 3 |
#Install the two package install.packages("searchConsoleR") |
2. Load the package and enter your credentials
You have to load the package using library(). and launch the OAuth 2.0 process using scr_auth().
|
1 2 3 4 5 |
#Load the package and OAuth2 authentication library("searchConsoleR") scr_auth() |
3. Authorization code
The code will open automatically your browser and you should see the Google login page. You have to login with your Google Account (the one that owns the GSC account) in order to connect Google Search Console API to R.
At the end of the process you should see a success message. Now you can go back to R Studio and learn the first commands!
4. Basic commands
I highly recommend to read the official documentation of the searchConsoleR package because there you can find all the commands and functions available (but don’t worry! In the next post I’ll show you how to manipulate the data as I did for Data Studio) – for example, below you can see the search_analytics() function – that you can use to query the Search Analytics report.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#search_analytics() command website <- "https://www.xxxxxxxxxxxxxx.com" start <- Sys.Date() - 120 #Start of analysis period (Today-120) end <- Sys.Date() - 3 #End of analysis period (Today-3) download_dimensions <- c('query', 'page') #Dimensions to dowload: data, query, page, device, country type <- c('web') #Search type: 'web', 'video', 'image' my_dataset <- search_analytics(siteURL = website ,startDate = start ,endDate = end ,dimensions = download_dimensions ,searchType = type ,rowLimit = 1000 #,dimensionFilterExp = dimensions_filtres ,walk_data = "byBatch") View(my_dataset) #Download the data frame as a .csv file write.csv(my_dataset,"prova.csv") |
With list_websites() you can load the list of all the properties connected to your Google Search Console Account.
|
1 2 3 4 5 6 |
#Load into a dataframe all the properties connected to your GSC account my_websites <- list_websites() my_websites #You can see my_websites df in a spreasheet-style view View(my_websites) |
Super useful article thanks!
Thanks Marcin!
what about colab/jupyter? seems ok on RStudio only
Hi Jon,
I used an R Package, that’s why it only works in RStudio and not Colab (where you have to use Python)
Very informative article! Thank you!
You’re welcome!
Thank you very much for the article! I did all the steps but in the last one, when rstudio open de cloud browser it did not returs any autorization code… may you help me with this?
Thanks you a lot!
Hi Soraya,
Do you get any error message? First of all, I would re-install the searchConsoleR package and restart RStudio.
Hi ruben , i wanted to apply filter on query (dimension) containing the name of my brand , could you please help with the syntax
Hi Karan, first of all, sorry for the late reply. It’s quite easy and straight forward, I would create the “Brand / Non-Brand” column and apply the filter on that dimension instead of using the query dimension.
Hey Ruben,
I have some trouble setting up this on the Google Cloud Platform. When it asks me “add credentials to your project” it brings up a different menu and I have to select “app engine admin api” and only then I can select “Oauth” and then it asks me to choose the User Type “internal” or “external” and an “app domain” is that what I should do?
When I click on create “oauth client ID” it takes me to a “configure consent screen” any ideas?
Hi Chris, are you trying to set it up on GCP?
Hey Ruben,
Yes I am. I can send you a screenshot, basically I get an entirely different screen after step 3 picture 1. I can send you a screenshot
I think you have to setup googleAnalyticsR in a non-interactive environment. You can find here tutorial: https://datarunsdeep.com.au/blog/how-troubleshoot-non-interactive-authentication-issues-using-googleanalyticsr
I found the original fix. Its just under a different option now. as a meme once said “Im gonna be honest… im kinda slow”
Thanks a million for the help
Now it just gives me the error when I run R studio ” Error 401: invalid_client
The OAuth client was not found. “
Hi Chris,
I updated the code! Now it should work 🙂
Hi there,
I keep getting this error even on a single day.
Error in x[[1]] : subscript out of bounds
Any ideas as to why?
Unfortunately no. I would try to copy paste again the code and rerun the whole process.