Google URL Inspection API with Python

In this is tutorial, we will learn how to use Google Search Console’s URL Inspection API with Python.

Navigation Show

What is the URL Inspection API?

The URL Inspection API is an Application Programming Interface that allows site owners to view the indexed, or indexable status of a given URL.

Simply put, the API provides a way to interact with the URL Inspection Tool without the use of Google Search Console‘s user interface.

With the URL Inspection API, you can, among other things, see if a page is:

Indexed
Crawled
Indexable
Blocked by robots.txt
Mobile Friendly
And more

Requirements

For this tutorial, you will need a few things:

Have Python installed
Full or Owner permission level to Google Search Console

The google-auth library and the version 1.12.10 of the google-api-python-client installed (the latest version does not work).

$ pip install --upgrade google-auth
$ pip install google-api-python-client==1.12.10

Python Code to Interact with URL Inspection API

If you don’t need detailed steps to create your service account credentials, here is the code that you need to use the API. Otherwise, keep reading for detailed steps.

Huge thanks and credit to Tobias Willmann, Jan-Philipp Igla, Alex Romero and joejoinerr for helping me figure out the possible issues with the API (see the Thread).

# Credit @tobias_willmann
from google.oauth2 import service_account
from googleapiclient.discovery import build

creds = "credentials.json"
 
scopes = [
    'https://www.googleapis.com/auth/webmasters',
    'https://www.googleapis.com/auth/webmasters.readonly'
    ]

credentials = service_account.Credentials.from_service_account_file(creds, scopes=scopes)
service = build('searchconsole','v1',credentials=credentials)

request = {
  'inspectionUrl': 'https://example.com/a-page',
  'siteUrl': 'sc-domain:example.com'
}

response = service.urlInspection().index().inspect(body=request).execute()
inspectionResult = response['inspectionResult']
print(inspectionResult)

Structure of the API Response

The response returned by the URL Inspection API is in the form of a JSON file that contains 4 major components:

inspectionResultLink: Link to the user intervace
indexStatusResult: Data containing all the information about the indexation status
mobileUsabilityResult: Information about the mobile-friendliness of the URL
richResultsResult: When structured data used for rich results is found on page, the information about it will be returned here.

The example below was provided in the URL Inspection API’s Official documentation.

  {
  "inspectionResult": {
    "inspectionResultLink": "https://search.google.com/search-console/inspect?resource_id=https://developers.google.com/search/&amp;id=odaUL5Dqq3q8n0EicQzawg&amp;utm_medium=link&amp;utm_source=api",
    "indexStatusResult": {
      "verdict": "PASS",
      "coverageState": "Indexed, not submitted in sitemap",
      "robotsTxtState": "ALLOWED",
      "indexingState": "INDEXING_ALLOWED",
      "lastCrawlTime": "2022-01-31T08:39:51Z",
      "pageFetchState": "SUCCESSFUL",
      "googleCanonical": "https://developers.google.com/search/help/site-appearance-faq",
      "userCanonical": "https://developers.google.com/search/help/site-appearance-faq",
      "referringUrls": [
        "https://developers.google.com/search/updates",
        "https://developers.google.com/search/help/crawling-index-faq"
      ],
      "crawledAs": "MOBILE"
    },
    "mobileUsabilityResult": {
      "verdict": "PASS"
    },
    "richResultsResult": {
      "verdict": "PASS",
      "detectedItems": [
        {
          "richResultType": "Breadcrumbs",
          "items": [
            {
              "name": "Unnamed item"
            }
          ]
        },
        {
          "richResultType": "FAQ",
          "items": [
            {
              "name": "Unnamed item"
            }
          ]
        }
      ]
    }
  }
}

Check if you Have Access Levels to Run the API

Go to the Google URL Inspection API playground.

Add this request body.

{
  "inspectionUrl": "https://example.com/a-page",
  "siteUrl": "https://example.com/"
}

Or, if you have given access to the domain-level property.

{
  "inspectionUrl": "https://example.com/a-page",
  "siteUrl": "sc-domain:example.com"
}

If the status code is 200, then you have everything that you need to proceed.

Create your API project

Go to Google developer console and create a new project.

Give it a name and click “CREATE”.

Create a service account

The service account will act as the email that will interact with the URL Inspection Tool API.

Make sure that you are in the project that you just created.

Note that the steps blow are simlar from one API project to the other, so the name and description given may be related to another project

Go to create the API credentials.

Click on “Manage service accounts”.

Click on “CREATE SERVICE ACCOUNT”.

Then, create the service account with the information that you want.

Select the “Owner” role.

You can skip the next step and create the service account.

Create API Keys

First, store the email ending in “gserviceaccount.com” for later use. For the newly created service account, you will need to create the API keys that will act as the username and password for your application.

Click on “ADD KEY” > “Create new key”.

Select the JSON format for your private key, and then click on “CREATE”.

Save the file somewhere safe.

Enable Google Search Console API

Now, the API needs to be enabled in your project for you to use it.

Go to the Google Search Console API library and enable it in your project.

Make sure that you are in the project that you created earlier and click on “ENABLE”.

Give Owner Status to the Service Account

For the URL Inspection Tool API to work properly, you need to give ownership to your service account email in Google Search Console.

Google to Google Search Console > Settings > Users and permissions

Add the service account email as an owner.

URL Inspection API Quotas

Google’s URL Inspection API imposes quotas on the number of possible requests. Here are the usage limits:

2,000 queries per day
600 queries per minute

Debugging

Many errors can happen when using the API, and I had some issues myself.

urlInspection() method not available

See this thread that I raised to debug the issue.

Mainly, when I was using the build() function to create the service, there was not urlInspection() method.

Jan-Philipp Igla found that the latest version of google-api-python-client didn’t work.

I fixed this by installing the right version in Terminal:

$ pip install google-api-python-client==1.12.10

HttpError 403

When fetching the API, I had this message saying that “You do not own this site, or the inspected URL is not part of this property”.

HttpError: &lt;HttpError 403 when requesting https://searchconsole.googleapis.com/v1/urlInspection/index:inspect?alt=json returned "You do not own this site, or the inspected URL is not part of this property.". Details: "You do not own this site, or the inspected URL is not part of this property.">

This issue is either because:

you haven’t given owner status in Google Search Console,
the provide URL is not an exact match to the property validated in GSC

You can fix this by modifying the siteURL in the request to be an exact match to the validated property, or use the domain level sc-domain:example.com.

Interesting Work in the Community

Google’s URL Inspection API with NodeJS – Jose Luiz Hernando Sanz
Calling URL Inspection API using a simple request – joejoinerr
URL Inspection API in Colab – Tobias Willmann
Google’s URL Inspection Tool in Sheets – Mike Richardson

Conclusion

Google’s URL Inspection API is a long-awaited step forward by Google’s team to provide SEOs with more information on their indexation status. Hope you enjoyed this introductory tutorial to help you use the URL inspection API with Python.

5/5 - (1 vote)

Jean-Christophe Chouinard

SEO Strategist at Tripadvisor, ex- Seek (Melbourne, Australia). Specialized in technical SEO. Writer in Python, Information Retrieval, SEO and machine learning. Guest author at SearchEngineJournal, SearchEngineLand and OnCrawl.