How to Scrape Google without Getting Blocked (for FREE) Example

In this tutorial, I will show you how to scrape Google Search Results using Apify.

Here is an example where I scraped 5 keywords for 0.02$.

Get FREE Apify Credits

Why Use Apify For Scraping Google?

Apify allows you to scrape Google without:

Subscribe to my Newsletter

  • the fear of getting your IP blocked AND
  • the need to how to code AND
  • the need to handle proxies

AND it is super easy to learn AND relatively cheap.

Is Apify Free?

Apify is not free, but does provide free credits for you to get started.

Overview of Apify Google Web Scraper

Apify has a built-in functionality for scraping Google. It is very easy to use and scrapes things like:

  • Top Search Results Paid and Organic
  • People Also Ask
  • Related Queries
  • Emphasized keywords
  • Sitelinks
Example Scraped Google Search Result in Apify
Example Scraped Google Search Result in Apify

Customize Google Scraping Settings

With Apify‘s Google web scraper, you can also customized web scraping settings such as the country, the language or whether you want to scrape Desktop or Mobile search results.

Apify VS SEM Rush VS Ahrefs

While Apify may not be as complete as SEM Rush or Ahrefs, it has the upside of being way cheaper for people looking for small scale web scraping.

Apify also provide many web scraping tools that its competitor doesn’t such as proxies, scrapers for sites other than Google (e.g. Amazon, Twitter, TikTok, etc.)

How to Scrape Google with Apify

It takes around 5 minutes to create your Apify account, log-in, learn and to run your first Google web scraper.

Time needed: 5 minutes

  1. Sign-Up to Apify

    sign up to apify

  2. Log-in to Apify

    Automatically, you get 5$ free credits and are ready to scrape Google.
    log-in to apify

  3. Open the Apify Store

    On the left hand panel, click on the “store” buttonapify store

  4. Find the “Google Search Results Scraper”

  5. Add the Keywords that You want to Scrape Search Results

    Make a list of keywords that you want to scrape. 6 keywords will cost you around 0.02$
    keywords to scrape in Google

  6. Set-up the Number of Results to Scrape

    The “max page per search term or URL” and “Results per Google Page” allows you to define the parameters to add in the Google URL that will tell how many search results to display. If you set this to 1 and 100 respectively, you will get the top 100 search results in a single HTTP request.apify google scraper number of search results

  7. Decide to Scrape Mobile or Desktop

    You can toggle the “Mobile results” button to decide whether you want to scrape Google mobile or desktop. I usually do both.

  8. Decide the Location and Language From Where to Scrape

    Google search results are not the same in the USA as they are in Italy. You can define the Geo and the language in the “Additional filters” section.apify additional filters

  9. Store an Example of the Search Results

    If you want to come back and debug your SEO in the future, you probably will want to store a version of the HTML so that you can look at it in the future.

    You can do so by going in “Advanced options (for developers)” and toggle “Save HTML to Dataset” and “Save HTML to key-value store”.

  10. Enhance Web Scraping Settings (optional)

    apify settings advanced

  11. Run your Google Web Scraper

    Once ready to scrape, you can click on “Save & Start”run apify google scraper

Apify Scraped Search Results Dashboard

After Scraping Google Search results, you will have a dashboard showing you the scraped data from the SERP.

In the Overview tab, you get a table with the Search results where each search term is a row and each column tells you:

  • Total number of results
  • Related queries
  • Paid results
  • Paid products
  • Organic Results
  • People Also Ask.

You can expand the amount of available data by clicking on “all fields”.

This snapshot will give your the HTML and the snapshot URL, as well as few other fields.

You can choose to have the data as a table or as a JSON to your linking.

By clicking, each value of the table, you can see the underlying JSON scraped data.

If you go in the storage tab, you’ll find you SERP snapshots

As well as options to download and export the scraped Google search results data.

How to Save a Version of Google SERP with Apify

You can scrape Google and save a version of the HTML so that you can view the page in your browser in the future by using Apify Google Scraper and select “Save HTML to Dataset” and “Save HTML to key-value store” in the “Advanced options (for developers)” section.

How to Schedule Scraping Google Search Results with Apify

After you have run a Google scraper with Apify, you can schedule its execution so that you can have historical comparison for the search results.

To schedule a Google scraper with Apify, go in Actors > Runs and click on “Schedule your actor”.

In the pop-up you can set-up the scraper to run up to the minute (depending on your budget).

It is a good idea to create different scraper for different categories of keywords or site sections. This way you can diagnose whether certain site sections go better of worse over time.

Be mindful of your budget when setting this up.

What Other Google Products Can you Scrape with Apify

You can scrape many Google products using Apify such as:

  • Google Maps
  • Google Trends
  • Google Shopping
  • Google Play Reviews


This is the end of the introduction on scraping Google using Apify. You can also read how to scrape Google with Screaming Frog.

Enjoyed This Post?