In this is tutorial, we will learn how to use Google Search Console’s URL Inspection API with Python.
What is the URL Inspection API?
The URL Inspection API is an Application Programming Interface that allows site owners to view the indexed, or indexable status of a given URL.
Simply put, the API provides a way to interact with the URL Inspection Tool without the use of Google Search Console‘s user interface.
With the URL Inspection API, you can, among other things, see if a page is:
- Indexed
- Crawled
- Indexable
- Blocked by robots.txt
- Mobile Friendly
- And more
Requirements
For this tutorial, you will need a few things:
- Have Python installed
- Full or Owner permission level to Google Search Console
The google-auth library and the version 1.12.10 of the google-api-python-client installed (the latest version does not work).
$ pip install --upgrade google-auth
$ pip install google-api-python-client==1.12.10
Python Code to Interact with URL Inspection API
If you don’t need detailed steps to create your service account credentials, here is the code that you need to use the API. Otherwise, keep reading for detailed steps.
Huge thanks and credit to Tobias Willmann, Jan-Philipp Igla, Alex Romero and joejoinerr for helping me figure out the possible issues with the API (see the Thread).
# Credit @tobias_willmann
from google.oauth2 import service_account
from googleapiclient.discovery import build
creds = "credentials.json"
scopes = [
'https://www.googleapis.com/auth/webmasters',
'https://www.googleapis.com/auth/webmasters.readonly'
]
credentials = service_account.Credentials.from_service_account_file(creds, scopes=scopes)
service = build('searchconsole','v1',credentials=credentials)
request = {
'inspectionUrl': 'https://example.com/a-page',
'siteUrl': 'sc-domain:example.com'
}
response = service.urlInspection().index().inspect(body=request).execute()
inspectionResult = response['inspectionResult']
print(inspectionResult)
Structure of the API Response
The response returned by the URL Inspection API is in the form of a JSON file that contains 4 major components:
inspectionResultLink: Link to the user intervaceindexStatusResult: Data containing all the information about the indexation statusmobileUsabilityResult: Information about the mobile-friendliness of the URLrichResultsResult: When structured data used for rich results is found on page, the information about it will be returned here.
The example below was provided in the URL Inspection API’s Official documentation.
{
"inspectionResult": {
"inspectionResultLink": "https://search.google.com/search-console/inspect?resource_id=https://developers.google.com/search/&id=odaUL5Dqq3q8n0EicQzawg&utm_medium=link&utm_source=api",
"indexStatusResult": {
"verdict": "PASS",
"coverageState": "Indexed, not submitted in sitemap",
"robotsTxtState": "ALLOWED",
"indexingState": "INDEXING_ALLOWED",
"lastCrawlTime": "2022-01-31T08:39:51Z",
"pageFetchState": "SUCCESSFUL",
"googleCanonical": "https://developers.google.com/search/help/site-appearance-faq",
"userCanonical": "https://developers.google.com/search/help/site-appearance-faq",
"referringUrls": [
"https://developers.google.com/search/updates",
"https://developers.google.com/search/help/crawling-index-faq"
],
"crawledAs": "MOBILE"
},
"mobileUsabilityResult": {
"verdict": "PASS"
},
"richResultsResult": {
"verdict": "PASS",
"detectedItems": [
{
"richResultType": "Breadcrumbs",
"items": [
{
"name": "Unnamed item"
}
]
},
{
"richResultType": "FAQ",
"items": [
{
"name": "Unnamed item"
}
]
}
]
}
}
}
Check if you Have Access Levels to Run the API
Go to the Google URL Inspection API playground.
Add this request body.
{
"inspectionUrl": "https://example.com/a-page",
"siteUrl": "https://example.com/"
}
Or, if you have given access to the domain-level property.
{
"inspectionUrl": "https://example.com/a-page",
"siteUrl": "sc-domain:example.com"
}
If the status code is 200, then you have everything that you need to proceed.
Create your API project
Go to Google developer console and create a new project.
Give it a name and click “CREATE”.
Create a service account
The service account will act as the email that will interact with the URL Inspection Tool API.
Make sure that you are in the project that you just created.
Note that the steps blow are simlar from one API project to the other, so the name and description given may be related to another project
Go to create the API credentials.
Click on “Manage service accounts”.
Click on “CREATE SERVICE ACCOUNT”.
Then, create the service account with the information that you want.
Select the “Owner” role.
You can skip the next step and create the service account.
Create API Keys
First, store the email ending in “gserviceaccount.com” for later use. For the newly created service account, you will need to create the API keys that will act as the username and password for your application.
Click on “ADD KEY” > “Create new key”.
Select the JSON format for your private key, and then click on “CREATE”.
Save the file somewhere safe.
Enable Google Search Console API
Now, the API needs to be enabled in your project for you to use it.
Go to the Google Search Console API library and enable it in your project.
Make sure that you are in the project that you created earlier and click on “ENABLE”.
Give Owner Status to the Service Account
For the URL Inspection Tool API to work properly, you need to give ownership to your service account email in Google Search Console.
Google to Google Search Console > Settings > Users and permissions
Add the service account email as an owner.
URL Inspection API Quotas
Google’s URL Inspection API imposes quotas on the number of possible requests. Here are the usage limits:
- 2,000 queries per day
- 600 queries per minute
Debugging
Many errors can happen when using the API, and I had some issues myself.
urlInspection() method not available
See this thread that I raised to debug the issue.
Mainly, when I was using the build() function to create the service, there was not urlInspection() method.
Jan-Philipp Igla found that the latest version of google-api-python-client didn’t work.
I fixed this by installing the right version in Terminal:
$ pip install google-api-python-client==1.12.10
HttpError 403
When fetching the API, I had this message saying that “You do not own this site, or the inspected URL is not part of this property”.
HttpError: <HttpError 403 when requesting https://searchconsole.googleapis.com/v1/urlInspection/index:inspect?alt=json returned "You do not own this site, or the inspected URL is not part of this property.". Details: "You do not own this site, or the inspected URL is not part of this property.">
This issue is either because:
- you haven’t given owner status in Google Search Console,
- the provide URL is not an exact match to the property validated in GSC
You can fix this by modifying the siteURL in the request to be an exact match to the validated property, or use the domain level sc-domain:example.com.
Interesting Work in the Community
- Google’s URL Inspection API with NodeJS – Jose Luiz Hernando Sanz
- Calling URL Inspection API using a simple request – joejoinerr
- URL Inspection API in Colab – Tobias Willmann
- Google’s URL Inspection Tool in Sheets – Mike Richardson
Conclusion
Google’s URL Inspection API is a long-awaited step forward by Google’s team to provide SEOs with more information on their indexation status. Hope you enjoyed this introductory tutorial to help you use the URL inspection API with Python.
SEO Strategist at Tripadvisor, ex- Seek (Melbourne, Australia). Specialized in technical SEO. Writer in Python, Information Retrieval, SEO and machine learning. Guest author at SearchEngineJournal, SearchEngineLand and OnCrawl.
