Python Script Automation Using CRON on Mac

SEO automation with Python is a very popular topic right now. The easiest way you automate SEO with Python is by using crontab (cron) on Mac or Task Scheduler on Windows. In this guide, you will learn how to use crontab run your Python scripts automatically.

What Can You Automate?

You can extract data from Google Search Console and Google Analytics on a specific schedule, send yourself a weekly email report, log all your keywords every day or automate your keywords research, launch a crawl every week, save robots.txt file, and a lot more.

Before we can cover all that, we need to learn how to use Crontab.

What are Cron and Crontab?

Cron is the tool that let users run script, commands or software automatically on a specified schedule. Crontab is the file that lists the jobs that cron will be executing.

Cron doesn’t execute while the computer is asleep.

Cron is perfect to run simple task automation that can run during the day while you are working.

To automate task seamlessly whether your computer is open or not. Cron is not the right solution for you. You would need to run your scripts on servers like AWS for this kind of automation.

Overview of the Project

In this post, we will automate a very simple script to run every minute.

from datetime import datetime
import os

def write_file(filename,data):
    if os.path.isfile(filename):
        with open(filename, 'a') as f:          
            f.write('\n' + data)   
    else:
        with open(filename, 'w') as f:                   
            f.write(data)

def print_time():   
    now = datetime.now()
    current_time = now.strftime("%H:%M:%S")
    data = "Current Time = " + current_time
    return data

write_file('test.txt' , print_time())

This Python script will create a test.txt and append the file with the time when the cron job executed.

What You Will Need?

This tutorial assumes that you already have installed:

• MacOS (if not, see the Windows Task Scheduler tutorial linked in the introduction)
• Python
• Code editor (like VScode)

And that you have at least basic knowledge of:

• Python
• Command-line

If not, start from the beginning of my Python for SEO tutorial.

How to Schedule a Task With Cron

Cron uses schedule expressions to know when you want to execute a task.

It has 5 parameters (* * * * *) to let you choose when you want to run your script:

• the minute,
• the hour,
• the day of the month,
• the month,
• the day of the week.

CRON Schedule Expressions

The easiest way to create crontab schedule expressions is to use crontab.guru.

Also, here is a table with all the most common expressions that you will encounter.

How to Schedule a Script Using Cron?

To schedule a python script execution using cron, we will:

1. Create the Python Script
2. Create a crontab file
3. Create a Python Script Sheduler
4. Write the crontab

Create the Python Script

Using the script above, create a .py Python script and save it to a location that you have permission (see Troubleshooting to understand why). In my case, I created an “Automation” folder under the /User/user.name path.

Create Your First Cron Job in Crontab

We will create and run a crontab file using the command line.

Open Terminal.

There are three (3) available commands to work with the crontab file: crontab -l (to display), crontab -e (to create) and crontab -r (to delete).

List the current crontab

Check if you already have a crontab created.

crontab -l

-l means that you want to open the crontab file in the “list” mode.

If you have never created a crontab before, you will get this message: crontab: no crontab for user.name.

Create a new Crontab

Now, we’ll add a new job to crontab.

crontab -e

-e means that you want to open the crontab file in the “edit” mode.

This will open the vi editor. Don’t be scared, I will walk you through it.

Vi Editor Commands

First, let’s look at the commands you can use in the vi editor.

Create a Python Script Scheduler

We will now add the Python script scheduler command to in the crontab using the VI Editor.

How to Schedule the Python Script

The command we will use to execute the python script goes like this.

• * * * * * is the crontab schedule expressions to schedule the script to run every minute;
• cd is the command-line argument to tell crontab where the executable file is;
• /Users/user.name/Automation is where my python script is located;
• /usr/bin/python is where python is installed. You can find where you have installed it using the whereis python or the which python commands in Terminal.
• test_cron.py is the name that I have given to the script share above.

Run the Script Automation

In the vi editor.

Switch to Insert mode by pressing i to edit the file.

Add the command to execute the Python script.

* * * * * /usr/bin/python /path/to/file/<FILENAME>.py /path/to/file/<FILENAME>.log

Press esc to exit the editing mode.

Type :wq to save and quit the file. w means to write and q means quit.

After writing the crontab, you will get this message crontab: installing new crontab which tells you created the crontab.

If all went well, you should now have the test.txt file in your folder with the time being added every minute (if not, see Trouble Shooting).

Recap

Wow, that was a lot.

Put simple, here is what you do:

1. Create your Python Script;
2. Open Terminal;
3. Write crontab -e to create crontab;
4. Press i to launch edit mode;
5. Write the schedule command * * * * * /usr/bin/python /path/to/file/<FILENAME>.py;
6. Press esc to exit edit mode;
7. Write :wq to write your crontab
8. To delete the running job:
   1. To delete the entire crontab: Run crontab -r
   2. To delete a single cron job: Go to crontab -e, press i, press dd and press :wq to write the file.

Additional Tutorial

Troubleshooting

Whenever you get an error with a cron job, you will have a mail in terminal saying “You have mail in /var/mail/user.name“. You can read the mail using the mail command. Use the number next to the mail to check the error message.

Look at the message that you get to learn about the error.

To delete a message use d, or to delete all use delete *. To quit the mail, use q.

Operation not permitted

You might encounter the “[Errno 1] Operation not permitted” in MacOS when you try to run a cron job.

can't open file '<FILENAME>.py': [Errno 1] Operation not permitted

Here is how to Fix “Operation not permitted” Error in Terminal for Mac OS.

You will need to give full disk access to the terminal and to CRON like in this solution.

1.  In the  Apple menu, choose System Preferences > Privacy and Security
2. Go to Full Disk Access on the left-hand side of the panel
3. Click the lock to make changes
4. Click on the Plus (+) sign
5. Go to Applications > Utilities > Terminal.app
6. Click on the (+) sign again;
7. Press Command + Shift + G and move to /usr/sbin/cron
8. Restart the Computer

Permission denied

Cron needs access to the folder location that you work in. A lot of companies will restrict access to locations of your desktop or will have your Documents folder in an internal server to keep a backup of your desktop.

IOError: [Errno 13] Permission denied: 'test.txt'

Here is how to fix the “Permission denied” error.

Just move your script to the user path /User/user.name to see if you still get the error running the script from that location.

Kill a Currently Running Cron task

Let’s say you have created a Python script and ran it daily using CRON, but unfortunately, an error in your code created an infinite loop. How can you stop it without shutting down your computer?

You can start by finding any python script currently running. In Terminal type:

$ ps aux | grep python

Now, you will get a few lines that have this structure:

user.name  12345   0.0  0.0  1727387347    90 s004  S+    2:25pm   0:00.00 grep python
user.name  23456   0.0  0.0  111111111111111    294 s005  S+    2:27pm   0:00.00 grep python

The first number you see on each line is your process ID. Above I have two PID: 12345 and 23456.

If I want to kill the first process, I would use this line in Terminal:

$ kill -9 12345

Conclusion

That’s it! You have learned how to run a cron job on mac to automate your SEO with Python.

Jean-Christophe Chouinard

Sr SEO Specialist at Seek (Melbourne, Australia). Specialized in technical SEO. In a quest to programmatic SEO for large organizations through the use of Python, R and machine learning.