I made this easy tutorial to help non-developers install Selenium Webdriver using Python. Selenium Webdriver is an amazing tool for SEO automation.
If you haven’t installed Python yet, make sure that you install Python using Anaconda.
Let’s get started by installing Selenium.
Install Selenium With Anaconda
If you have installed Python with Anaconda, go to Anaconda Prompt from your navigation.
Type in the following command.
conda install -c conda-forge selenium
Now, a line will come up asking you to proceed. Press “Y” and click enter.
Install Selenium With Pip
If you need to install Selenium with Pip, use this in your command line.
pip install Selenium
Now that we have Selenium installed, we need to install a webdriver.
Install Selenium Webdriver on Chrome
Selenium needs a webdriver to work with your browser.
Step 1: To install Selenium Webdriver, just go to Seleniumhq.org and go to “Third Party Browser Drivers” section.
Step 2: Click on “Google Chrome Driver” . If you use Mozilla, just install Mozilla GeckoDriver (or any other browser that you use).
Step 3: Click on the latest stable release.
Step 4: Download the correct version for your operating system. I am on Windows, so I’ll select chromedriver_win32.zip.
Step 5 Unzip the folder
Step 6: Go back to Anaconda Prompt and type
Step 7: Import Selenium Webdriver
In your command-line, or IDE, write this command.
from selenium import webdriver
If everything was properly installed, you should get something like this.
If you get an error message, it might be that you have forgot to type “Y” when asked to “proceed”.
Go ahread and try to reinstall Selenium in conda and read the instructions properly.
If still not working, head over to Stackoverflow to find a solution.
Next, we’ll set-up our browser.
Make Your First Automated Task With Selenium
To make your first automated task with Selenium, you will need to find the executable path of your chrome browser.
To find it just
chrome://version in your browser.
Open And Close Your Browser
To open and close your browser using selenium, you will be using the
webdriver.Chrome() and the
import time from selenium import webdriver driver=webdriver.Chrome(executable_path=r"C:\Users\j-c.chouinard\Downloads\chromedriver_win32\chromedriver.exe") #Path to Chrome Driver time.sleep(5) # Wait for 5 seconds, so that you can see something. driver.quit()
A browser will open with some kind of mention that Chrome is being controlled by automated test software and then it will close.
Note that the path is optional, but it is preferable to add it to make sure that the script works.
Open a Website With Selenium
Opening a website in your browser is simple using the
from selenium import webdriver driver=webdriver.Chrome(executable_path=r"C:\Users\j-c.chouinard\Downloads\chromedriver_win32\chromedriver.exe") #Path to Chrome Driver driver.get('https://www.jcchouinard.com/')
How to Open a Link
There are many ways that you can open links automatically with Selenium. The process is this one:
- Find an element and store it in a variable
- Get the
- Click the link
There are multiple ways that you can find an element.
#1. Find an element by link text and store it in a variable driver.get('https://moz.com/') linkText=driver.find_element_by_link_text("Blog") #Access the "text" attribute. linkText.text ##Out: 'Blog'
Let’s get the
href attribute using
driver.get('https://moz.com/') linkText=driver.find_element_by_link_text("Blog") linkText.get_attribute("href") ##Out: 'https://moz.com/blog'
Click the link using
#Open the webpage driver.get('https://moz.com/') #Add link element to a variable linkText=driver.find_element_by_link_text("Blog") #Wait 1 Second and click the link time.sleep(1) linkText.click() #Wait 1 second and close the browser time.sleep(1) driver.quit()
Congratulations! You are now ready to learn SEO reading Moz’s blog.
How To Use a Search Box
To perform a search using a search box using Selenium, you will need to get the Id of the input and of the click button.
To do this, right-click on the search box and click inspect element.
Note the ID of the Input element.
Repeat the process to find the ID of the button.
#Get to your page driver.get('https://www.jobillico.com/') #Add the ID of the search Box searchBox=driver.find_element_by_id("search-engine-keyword-job") #Add the search text searchBox.send_keys("seo") #Get Button ID submit=driver.find_element_by_id("btn-search-engine") #Click The Button submit.click()
You could also use Keys to submit the form. This will simulate a user that presses the ENTER key.
from selenium.webdriver.common.keys import Keys driver.get('https://www.jobillico.com/') searchBox=driver.find_element_by_id("search-engine-keyword-job") searchBox.send_keys("seo") searchBox.send_keys(Keys.ENTER)
These are some of the many ways that could work with any form. The next example uses another way using the
Search For Something On Google Using Selenium
You can easily search for anything on Google using Python and Selenium. By typing this bit of code, it will:
- Open a Web Browser
- Go to Google.com
- Search for a Query String
- Close the Browser
All of this will be done at a certain rate because of the
import time from selenium import webdriver from selenium.webdriver.chrome.service import Service driver=webdriver.Chrome(executable_path=r"C:\Users\j-c.chouinard\Downloads\chromedriver_win32\chromedriver.exe") #Path to Chrome Driver driver.get('http://www.google.com/'); time.sleep(5) #Search and submit search_box = driver.find_element_by_name('q') search_box.send_keys('jcchouinard') search_box.submit() #Wait for the user to see something and then close time.sleep(5) driver.quit()
Open The First Result In Google With Python
The first result in Google is of great interest to SEOs, so you might want to go and open the first link in Google.
To do this, the link name is not a good selector since it always changes.
We will need some Xpath to perform this task.
Even if you know nothing about Xpath, you can reuse some xpath functions for Google that I have created for you.
import time from selenium import webdriver from selenium.webdriver.chrome.service import Service #Open Web Driver driver=webdriver.Chrome(executable_path=r"C:\Users\j-c.chouinard\Downloads\chromedriver_win32\chromedriver.exe") #Go To Google .com driver.get('https://www.google.com/'); time.sleep(1) #Search For a Query search_box = driver.find_element_by_name('q') search_box.send_keys('jcchouinard') search_box.submit() #Select First Result Using Xpath firstResult=driver.find_element_by_xpath('(//div[contains(@class,"r")]/a[contains(@ping,"/url")])') #Click On The Link firstResult.click()
Enjoyed this post? Tell me how you automate your SEO using Selenium webdriver and Python.
Sr SEO Specialist at Seek (Melbourne, Australia). Specialized in technical SEO. In a quest to programmatic SEO for large organizations through the use of Python, R and machine learning.