Python File Handling: Read, Write and Delete Files in Python

File handling in Python programming represents the action of interacting with files using Python built-in functions or modules. The most common function in Python to read and write to files is the open() function.

The Python open() built-in function returns a file object on which methods can be used to interact with the underlying resource.

What is File Handling in Python

File handling refers to storing the output or input of a Python program in a data file (e.g. binary or text file) for future reference.

What is a File in Python

In Python, a file is a data structure used for storing and organizing information. It can be handled in two modes: text or binary. A file can be in either a text-based format or a binary format, and each line within a file is typically terminated with a special character.

Types of Files in Python

There are two main types of files in Python:

1. Text files
2. Binary files

Text Files

Text files in Python are files that store textual data (e.g. text file) encoded using character encodings like ASCII, UTF-8, etc. When working with text files in Python, you can read and write text data using string operations and functions.

Binary Files

Binary files are files that store non-textual data (e.g. images, audio, video, etc.) stored in a binary format (0s and 1s) rather than as text. To read or write binary files in Python, use binary file handling methods or libraries, such as the struct module or specialized libraries for specific file formats.

File Operations in Python

File operations in Python generally follow the following sequence:

1. Opening a file using the open() function
2. Performing read or write operations using the built-in methods
3. Closing the file.

This allows to read or write data from files as needed, providing flexibility in handling various types of data storage and retrieval tasks.

Open a file in Python With the Open() Function

To open a file in Python, use the built-in open() function and provide the location of the file and the mode as parameters of the function.

# open() function
f = open('filename.txt') # open file
f.close() # close file

The open function has two positional arguments and multiple keyword arguments.

open(file, mode='r', buffering=- 1, encoding=None, errors=None, newline=None, closefd=True, opener=None)

The first argument is a string telling the location of the file to be opened. The second argument is a string that specifies the mode. The mode tells how the file will be used when opened (read, write, append, etc.).

The keyword parameter is a string telling the encoding of the data read from the file ('b' for bytes or default 'utf-8').

The mode and the other keyword arguments are optional in the open() function.

Modes of the open function

Below are listed the modes that can be used in the open() function in Python:

Opening a File in Python

To open a file in Python, use the open() function with the location of the file as a string parameter. Don’t forget to close the connection using the close() method on the file object.

# Open file
filename = 'my_file.txt'
f = open(filename)

# Show the open mode
print(f.mode)

r

Closing a File in Python

To close a Python file, use the close() method on the Python file object.

# Needs to be closed or it stays open
f.close()

Why it is Important to Close a File in Python

When you open a file in Python, the operating system assigns resources (such as file handles) to manage the connection between your program and the file.

If you don’t close the file, these resources may not be released, leading to resource leaks and potential issues with other programs or processes trying to access the same file.

For example, by opening 3 files in write mode, we can see the handles of the connection:

files = [open(f"files/file_{n}.txt", mode="w") for n in range(3)]

However, these connections stay open as long as you don’t close them. Too many and your system may crash.

# The OS Can manage millions of files
for f in files:
    print(f.closed)

False
False
False

Lets Close the files.

# Closing the files
for f in files:
    f.close()
    print(f.closed)

With Open – Opening Python Files with a Context Manager

Context managers are used to give and release resources at need before and after an operation that needs it.

The with statement allows you to open and close a connection once the operation is done.

It is always a good practice to use a context manager when handling files in Python.

# Open with context manager
with open('files/my_file.txt', 'r') as f:
    print(f.name)

print(f.closed)

files/my_file.txt
True

Read a Text File in Python using read()

Use the read() method on the file object to read the file in Python.

# read(): Read the contents of the file
with open('files/my_file.txt', 'r') as f:
    print(f.read())

Read File Up to a Nth Position with read(n)

To read the file up to a position, add an integer as the argument of the read method.

# read(): Read the contents of the file to a Nth position
with open('files/my_file.txt', 'r') as f:
    print(f.read(10))

Loop Through Each Line of a File in Python

To read a file line-by-line, you can use a for loop to print each line one after the other.

# Loop line-by-line
with open('files/my_file.txt', 'r') as f:
    for line in f:
        print(line)

Read a Single Line of a File with readline()

The readline() method reads the file until newline or EOF and returns an empty string if EOF is hit immediately.

file.readline(size=-1)

The only optional parameter is the size, which represents the number of bytes from to return from the line. Its default value is -1, which means the whole line.

# readline(): Reads a single line from the file
with open('files/my_file.txt', 'r') as f:
    line = f.readline()

line

Read Python File Lines to a List – readlines() VS read().split()

To read each line of a Python file and add them to a list, use the readlines() method or use the read() method to read the file and then apply the split(‘\n’) method on the output.

Using readlines()

The readlines() method allows you to read each line of a Python file and add them to a list. In Python, readlines() will return the result as a list of strings where the newline delimiter is still present on each item of the list.

file.readlines(hint=-1)

Hint can be specified to control the number of lines read. no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

# readlines(): Returns a list of lines from the file
with open('files/my_list.txt', 'r') as f:
    ls = f.readlines()
ls

['Montreal\n',
 'Tokyo\n',
 'Istanbul\n',
 'New-York\n',
 'London\n',
 'Melbourne\n',
 'Marrakesh']

Using read().split()

The split() method can be used on the read() object to split the Python using the \n character and read the Python file into a list.

file.read().split('\n')

# Read lines to list
with open('files/my_list.txt', 'r') as f:
    content = f.read()
    ls = content.split('\n')
ls

['Montreal',
 'Tokyo',
 'Istanbul',
 'New-York',
 'London',
 'Melbourne',
 'Marrakesh']

Write to a Python File using write()

The write() method is used to write string to a file object and returns the number of characters written. To use the write() method on a Python file, the file object needs to be opened using one of the following modes on the open() function: ‘w’, ‘a’ or ‘+’.

file.write(text)

Write to a File in Write Mode (‘w’)

# write(): Writes a string to the file
with open('files/my_file.txt', 'w') as f:
    f.write('Hello world')

Append to a File in Append Mode (‘a’)

# Append to a file
with open('files/my_file.txt', 'a') as f:
    f.write('\nhello world')

Write to a File in Read/Write Mode (‘r+’)

# write(): Writes a string to the file
with open('files/my_file.txt', 'r+') as f:
    f.write('Hello world')

Write Multiple Lines to a Python File using writelines()

The writelines() method is used on the Python file object to write a list of lines to a stream.

file.writelines(lines)

# Write List to a file
my_list = [
    'Line 1\n',
    'Line 2\n',
    'Line 3\n'
    ]

with open('files/writelines.txt', 'w') as f:
    f.writelines(my_list)

Note that the line separators (\n) are not added, so it is good practice to provide each line with line separator at the end.

How to Delete a File in Python

To delete a file in Python, use the remove() method from the os module.

import os
os.remove("filename.txt")

How to Handle a JSON File in Python

To open, read and write to a JSON file in Python, use the open function on the JSON file and use the json module to handle the operations.

Write to JSON file

# Write to JSON file
import json

data = {
    'name': 'Jean-Christophe', 
    'website': 'jcchouinard.com'
    }

with open('files/my_file.json', 'w') as f:
    json.dump(data, f, indent=4)

Read JSON file

# Read JSON file
with open('files/my_file.json', 'r') as f:
    data = json.load(f)

data

Check the State of the Opened File – Readable, Writable

The readable() and writeable() methods return a boolean whether or not the object was opened for reading or writing

# Check the state of the file
with open('files/my_file.txt', 'r') as f:
    print(f.readable())
    print(f.writable())

True
False

Offset The Position Where to Start Reading a File with seek()

The seek() method is used on the Python file object to offset the stream (file) position in Bytes. It is use to tell where to start the operation from.

seek(cookie, whence=0)

# seek(): Changes the file position to a specified location

# Read a file line by line
with open('files/my_file.txt', 'r') as f:
    print('No offset:', f.readline())

# Read a file line by line
with open('files/my_file.txt', 'r') as f:
    f.seek(6)
    print('Offset:', f.readline())

No offset: 1. Hello World

Offset: lo World

The offset is interpreted relative to the position indicated by whence. Values for whence are:

• 0 : start of stream (the default); offset should be zero or positive
• 1 : current stream position; offset may be negative
• 2 : end of stream; offset is usually negative

Check the Position of the Stream with tell()

The tell() method is used on the Python file object to return the current position of the stream. The example below shows how the tell() method will return the same position that we gave with the seek() method.

# tell(): Returns the current file position
# Read a file line by line
with open('files/my_file.txt', 'r') as f:
    f.seek(6)
    print(f.tell())

6

Specify How Much of the File Should be Removed with truncate()

The truncate() method is used to truncate a file and specify how much of the file should be removed in bytes.

truncate(pos=None)

# Print the current contents of the file
with open("example.txt", "r") as file:
    print("Before truncate:")
    print(file.read())

# Truncate the file to a specified position
with open("example.txt", "r+") as file:
    file.truncate(14)

# Print the modified contents of the file
with open("example.txt", "r") as file:
    print("\nAfter truncate:")
    print(file.read())

Before truncate:
Hello, World!
This is an example file.

After truncate:
Hello, World!

Check if File Supports Random Access seekable()

The seekable() method returns whether the file object supports random access. If False, seek(), tell() and truncate() will raise OSError.

# Read a file line by line
with open('files/my_file.txt', 'r') as f:
    print(f.seekable())

True

Show File ID with fileno()

The fileno() method returns underlying file descriptor if one exists. The file descriptor is a non-negative integer ID that identifies an opened file in the operating system.

# Read a file line by line
with open('files/my_file.txt', 'r') as f:
    print(f.fileno())

72

What is the flush() Method in Python

The file flush() methods flushes write buffers when applicable.

To understand what this means, know that there are two levels of buffering involved in file writing:

• Internal buffers
• Operating system buffers

The internal buffers are used by the programming language to optimize write operations, while the operating system buffers may hold the data before it is written to disk. To ensure data is written to disk, you can use the flush method to write data from program buffers to operating system buffers, and the fsync method to synchronize operating system buffers with the disk.

import os
# Open a file in write mode
file = open("example.txt", "w")

# Write data to the file
file.write("Hello, World!")

# The data is now in the program buffer
# Use flush to write the data from the program buffer to the operating system buffer
file.flush()

# At this point, the data is in the operating system buffer but may not be on disk yet

# Use fsync to synchronize the operating system buffer with the disk
os.fsync(file.fileno()) # Sync file descriptor

# Now, the data is guaranteed to be written to disk

# Close the file
file.close()

What is the detach() Method in Python

The detach() method separates the underlying buffer from the TextIOBase and returns it. Once detached, the TextIO is in an unusable state.

# Open a file in write mode
file = open("example.txt", "w")

# Write some content to the file
file.write("Hello, World!\nThis is an example file.")

# Detach the file object from the file descriptor
file_descriptor = file.detach()

# Verify that the file object is detached
print(file)
print(file_descriptor)

# Close the file descriptor separately (if needed)
file_descriptor.close()

# Don't need to close file, it has been detached. Throws ValueError
file.close()

<_io.TextIOWrapper mode='w' encoding='UTF-8'>
<_io.BufferedWriter name='example.txt'>

What is the isatty() Method in Python

The isatty() method returns whether the file object is an ‘interactive’ stream and return False if it can’t be determined

File Handling Methods

Here is a table with the methods that can be used on the file object returned by the open() function.

Ending Note on File Handling in Python

We have now covered everything that we need to know to handle files in Python such as reading, writing and deleting Python files.

We have also seen the Python file handling methods of the stream object returned by the Python open() function.

Jean-Christophe Chouinard

SEO Strategist at Tripadvisor, ex- Seek (Melbourne, Australia). Specialized in technical SEO. Writer in Python, Information Retrieval, SEO and machine learning. Guest author at SearchEngineJournal, SearchEngineLand and OnCrawl.