Authenticate oauth2 gsc api python

This tutorial is to learn how to authenticate to the Google Search Console API with OAuth2 and Python.

I will show the steps to:

  • Connect to the developer console,
  • Get Your API Keys and authenticate to the API OAuth2
  • Make your first API call.

Follow this tutorial if you want to learn how to use Google Search Console API with Python.


Subscribe to my Newsletter


To learn more, view my complete guide on Python for SEO.

Make Your First Google Search Console API call with Python

There are many things that you can do with the Search Console API like getting search traffic and extracting indexed pages.

In this post, we will make a very basic API call: get validated properties.

The steps to use Google Search Console API in Python are:

  1. Get Your Google Search Console API Key

  2. Import Libraries

  3. Authenticate to the API

  4. Make Your API Call

Step 1: Get Your Google Search Console API Key

You will need an API key to be able to connect to the Google Search Console API.

How to Get Google Search Console API Keys

What is an API Key?

An API key is like your username and password to access the API.

Follow this guide to help you get the detailed steps to get your Google Search Console API key.

Otherwise, here are the simplified steps to get your Google Search Console API keys.

  1. Go to Google’s developers console, and sign-in;
  2. Go to “Dashboard” and click “Enable APIs and Services” ;
  3. Search for “Google Search Console API” and enable the API;
  4. Go to the “credential” tab, click on “create credential” and select “OAuth Client ID”;
  5. Click on “configure the consent screen ” and give a name to your product;
  6. Choose “Other” as the application type and click create;
  7. Copy the client ID and client Secret or go-on and save to download the JSON file.

Download your API key first, and then you’ll be ready to connect to the Search Console API with Python.

Step 2: Import Libraries

To run the OAuth 2.0 authentication you will need to install and import all those libraries.

Alternatively, you can clone the Github Repository that I made and run:

$ pip install -r requirements.txt
import argparse
import httplib2
import requests

from collections import defaultdict
from dateutil import relativedelta
from googleapiclient.discovery import build
from oauth2client import client
from oauth2client import file
from oauth2client import tools

Step 3. Authenticate to the API

Now, we will log in to the API using OAuth. Two options are possible. Using a JSON file (recommended), or using the API client_id and client_secret.

Choose your preferred solution.

Connect Using a JSON API Key

The authorize_creds() function will check the credentials file and define a .dat file to save authorised credentials so you don’t have to go through the login process each time.

def authorize_creds(creds,authorizedcreds='authorizedcreds.dat'):
    '''
    Authorize credentials using OAuth2.
    '''
    print('Authorizing Creds')
    # Variable parameter that controls the set of resources that the access token permits.
    SCOPES = ['https://www.googleapis.com/auth/webmasters.readonly'] 

    # Path to client_secrets.json file
    CLIENT_SECRETS_PATH = creds

    # Create a parser to be able to open browser for Authorization
    parser = argparse.ArgumentParser(
        formatter_class=argparse.RawDescriptionHelpFormatter,
        parents=[tools.argparser])
    flags = parser.parse_args([])

    # Creates an authorization flow from a clientsecrets file.
    # Will raise InvalidClientSecretsError for unknown types of Flows.
    flow = client.flow_from_clientsecrets(
        CLIENT_SECRETS_PATH, scope = SCOPES,
        message = tools.message_if_missing(CLIENT_SECRETS_PATH))

    # Prepare credentials and authorize HTTP
    # If they exist, get them from the storage object
    # credentials will get written back to the 'authorizedcreds.dat' file.
    storage = file.Storage(authorizedcreds)
    credentials = storage.get()

    # If authenticated credentials don't exist, open Browser to authenticate
    if credentials is None or credentials.invalid:
        credentials = tools.run_flow(flow, storage, flags)      # Add the valid creds to a variable

    # Take the credentials and authorize them using httplib2   
    http = httplib2.Http()                                      # Creates an HTTP client object to make the http request
    http = credentials.authorize(http=http)                     # Sign each request from the HTTP client with the OAuth 2.0 access token
    webmasters_service = build('searchconsole', 'v1', http=http)   # Construct a Resource to interact with the API using the Authorized HTTP Client.

    print('Auth Successful')
    return webmasters_service

Run OAuth 2.0

To run the code, simply add your credentials and run authorize_creds(). Once this is done you will be able to run Google Search Console API using the webmasters_service variable.

if __name__ == '__main__':
    creds = 'client_secrets.json'
    webmasters_service = authorize_creds(creds) 

The if name equals main line checks whether you are running the module or importing it. If you are importing it, authorize_creds() will not run.

Step 4: Make Your API Call

Now the glory!

Let’s see what websites we have validated in Google Search Console using the API.

site_list = webmasters_service.sites().list().execute()

verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry']
                       if s['permissionLevel'] != 'siteUnverifiedUser'
                          and s['siteUrl'][:4] == 'http']

for site_url in verified_sites_urls:
  print( site_url)

Full Code

Here is the full code to let you make your first API call with Python.

oauth.py

#!/usr/bin/env python
# oauth.py
# Authorize credentials using OAuth2.

# @author:    Jean-Christophe Chouinard. 
# @role:      Sr. SEO Specialist at SEEK.com.au
# @website:   jcchouinard.com
# @LinkedIn:  linkedin.com/in/jeanchristophechouinard/ 
# @Twitter:   twitter.com/@ChouinardJC

import argparse
import httplib2
import requests

from collections import defaultdict
from dateutil import relativedelta
from googleapiclient.discovery import build
from oauth2client import client
from oauth2client import file
from oauth2client import tools

def authorize_creds(creds,authorizedcreds='authorizedcreds.dat'):
    '''
    Authorize credentials using OAuth2.
    '''
    print('Authorizing Creds')
    # Variable parameter that controls the set of resources that the access token permits.
    SCOPES = ['https://www.googleapis.com/auth/webmasters'] 

    # Path to client_secrets.json file
    CLIENT_SECRETS_PATH = creds

    # Create a parser to be able to open browser for Authorization
    parser = argparse.ArgumentParser(
        formatter_class=argparse.RawDescriptionHelpFormatter,
        parents=[tools.argparser])
    flags = parser.parse_args([])

    # Creates an authorization flow from a clientsecrets file.
    # Will raise InvalidClientSecretsError for unknown types of Flows.
    flow = client.flow_from_clientsecrets(
        CLIENT_SECRETS_PATH, scope = SCOPES,
        message = tools.message_if_missing(CLIENT_SECRETS_PATH))

    # Prepare credentials and authorize HTTP
    # If they exist, get them from the storage object
    # credentials will get written back to the 'authorizedcreds.dat' file.
    storage = file.Storage(authorizedcreds)
    credentials = storage.get()

    # If authenticated credentials don't exist, open Browser to authenticate
    if credentials is None or credentials.invalid:
        credentials = tools.run_flow(flow, storage, flags)      # Add the valid creds to a variable
    # Take the credentials and authorize them using httplib2   
    http = httplib2.Http()                                      # Creates an HTTP client object to make the http request
    http = credentials.authorize(http=http)                     # Sign each request from the HTTP client with the OAuth 2.0 access token
    webmasters_service = build('searchconsole', 'v1', http=http)   # Construct a Resource to interact with the API using the Authorized HTTP Client.
    print('Auth Successful')
    return webmasters_service

# Create Function to execute your API Request
def execute_request(service, property_uri, request):
    return service.searchanalytics().query(siteUrl=property_uri, body=request).execute() 


if __name__ == '__main__':
    creds = 'client_secrets.json'
    webmasters_service = authorize_creds(creds) 

get_properties.py

#!/usr/bin/env python
from oauth import authorize_creds, execute_request

def get_property_list(webmasters_service):
    '''
    Get a list of validated properties from GSC
    '''
    site_list = webmasters_service.sites().list().execute()

    # Filter for verified websites
    verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry']
                        if s['permissionLevel'] != 'siteUnverifiedUser'
                            and s['siteUrl'][:4] == 'http']
    return verified_sites_urls

if __name__ == '__main__':
    creds = 'client_secrets.json'
    webmasters_service = authorize_creds(creds) 
    verified_sites_urls = get_property_list(webmasters_service)

Alternative OAuth2 connection: Client_Id and Client_Secrets

You might want to connect to the API simply by adding your client secrets and client id instead of using the JSON credential file.

Here is the code below:

Make sure that you don’t forget to add your own client_id and client_secrets. You can view the notebook.

parser = argparse.ArgumentParser(
    formatter_class=argparse.RawDescriptionHelpFormatter,
    parents=[tools.argparser])
flags = parser.parse_args([])

flow = client.OAuth2WebServerFlow(client_id='XXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com',
                           client_secret='XXXXXXXXXXXXXXXXXXXXX',
                           scope=SCOPES)

storage = file.Storage('searchconsole.dat')

credentials = storage.get()
if credentials is None or credentials.invalid:
  credentials = tools.run_flow(flow, storage, flags)

# Create an httplib2.Http object and authorize it with our credentials
http = credentials.authorize(http=httplib2.Http())

Search Console API Connection Errors

Error 403: access_denied in Search Console API

The Error 403: access_denied error when using Google Search Console API means that the user that validated the connection to the API does not have access to either the Google Search Console property or the Google Search Console API.

The error message goes something like this:

Access blocked: [PROJECT NAME] has not completed the Google verification process


[PROJECT NAME]  has not completed the Google verification process. The app is currently being tested and can only be accessed by developer-approved testers. If you think you should have access, contact the developer.
If you are a developer of [PROJECT NAME], see error details.
Error 403: access_denied
Access blocked Error 403: access_denied in Google search console

The Access blocked 403 error can happen if the email that you used is not allowed:

  1. access level for the property: go to Settings > Users and permissions in Google Search Console and verify that you have accessed
  2. access to the API: go to Google cloud console > OAuth Consent Screen > Test Users and add your email to the allows user list. This often happens when the app has just been created and pending being tested by Google.
adding users to API oauth consent screen

OSError: [Errno 48] Address already in use

The OSError: [Errno 48] Address already in use error in Google Search Console API means that the Port that you try to call is already taken by another running process.

This often occur when the connection to the OAuth Consent screen is still open and you try to reopen a new connection to the API. Fix this by shutting down your Jupyter Notebook in Python or killing the process like that.

$ ps -fA | grep python
$ kill PROCESS_NAME

ValueError: Client secrets must be for a web or installed app

The ValueError: Client secrets must be for a web or installed app error in Google Search Console generally means that you are using the JSON generated under the wrong type of credentials (e.g. JSON created under Service accounts and not under OAuth 2.0 client ID)s.

Error 400: invalid_request

The Error 400: invalid_request error in Google Search Console API generally means that you have a malformed URL in your API requests. Verify the URL in your browser to check if you are fetching the right API endpoint, and using all the required parameter inside your request. Check how to use OAuth2 for more information. This happened to me when I was using the Google Analytics Data API credentials file instead of the Google Search Console JSON file. Also happens when the redirect URI is wrong or the Out-of-Band (OOB) flow migration hasn’t been done by the API wrapper.

Click on the “Error details” link to verify the part of the URL that is malformed.

Conclusion

We are done. You have successfully connected to Google Search Console, calling the API With Python. Now we will learn how to Get All Your Search traffic With Google Search Console API.

4.7/5 - (7 votes)