Mock external dependencies in Python unittest
External dependencies are a major source of pain in unit testing. They can be slow, unreliable, and difficult to control. Mocking these external services is the single most common and valuable use of the unittest.mock library [1]. This article will provide practical, hands-on examples for mocking three of the most frequent external dependencies: HTTP requests, database connections, and file system operations.
Mocking HTTP Requests
Your application likely communicates with external APIs. Making real network calls during testing is a bad practice. It makes tests slow, can incur costs, and relies on the external API being available and returning predictable data. We can use patch to replace the requests.get or requests.post functions with a mock object.
Example: Mocking an API call
Imagine a function that fetches a list of users from a remote API.
api_client.py
import requests
def get_users():
"""Fetches user data from a remote API."""
try:
response = requests.get('https://api.example.com/users')
response.raise_for_status() # Raise an exception for HTTP errors
return response.json()
except requests.exceptions.HTTPError as e:
print(f"API Error: {e}")
return None
To test this function, we will mock the requests.get call.
test_api_client.py
import unittest
from unittest.mock import patch, MagicMock
from api_client import get_users
import requests
class TestAPIClient(unittest.TestCase):
# Use @patch to replace requests.get in the module where it's used.
@patch('api_client.requests.get')
def test_get_users_success(self, mock_get):
# Configure the mock response object
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = [{'id': 1, 'name': 'Alice'}]
mock_response.raise_for_status.return_value = None
# Make the mock object returned by `requests.get` be our mock_response
mock_get.return_value = mock_response
# Call the function under test
users = get_users()
# Assert the mock was called once with the correct URL
mock_get.assert_called_once_with('https://api.example.com/users')
# Assert the function returned the expected data
self.assertEqual(len(users), 1)
self.assertEqual(users[0]['name'], 'Alice')
@patch('api_client.requests.get')
def test_get_users_api_error(self, mock_get):
# Configure the mock to raise a real exception
mock_get.side_effect = requests.exceptions.HTTPError('404 Not Found')
users = get_users()
# Assert that the function correctly handled the error and returned None
self.assertIsNone(users)
The @patch decorator is crucial here. The string 'api_client.requests.get' tells Python to find the requests module as it is imported inside the api_client module and replace its get function with a mock [5].
Mocking Database Connections
Testing database-dependent code can be slow and require a live database. We can mock the database connection to simulate queries and results without ever touching a real database.
Example: Mocking a database client
Consider a function that retrieves a user from a database using a simple client.
db_client.py
import sqlite3
class DatabaseClient:
def __init__(self, db_path):
self.conn = sqlite3.connect(db_path)
self.cursor = self.conn.cursor()
def get_user_by_id(self, user_id):
self.cursor.execute("SELECT name FROM users WHERE id=?", (user_id,))
result = self.cursor.fetchone()
return result[0] if result else None
To test this, we will mock the database connection and cursor objects. This is a great use case for MagicMock as it can simulate the chained calls of connect().cursor().fetchone() [2, 6].
test_db_client.py
import unittest
from unittest.mock import patch, MagicMock
from db_client import DatabaseClient
class TestDatabaseClient(unittest.TestCase):
# Patch the `sqlite3.connect` function in the module where it's used
@patch('db_client.sqlite3.connect')
def test_get_user_by_id_exists(self, mock_connect):
# Configure the chain of mocked objects
mock_cursor = MagicMock()
mock_connect.return_value.cursor.return_value = mock_cursor
# Set the return value of fetchone() to simulate a database row
mock_cursor.fetchone.return_value = ('Alice',)
db_client = DatabaseClient('test_db.sqlite')
user_name = db_client.get_user_by_id(1)
# Assert that the query was executed correctly
mock_cursor.execute.assert_called_once_with("SELECT name FROM users WHERE id=?", (1,))
# Assert the function returned the expected result
self.assertEqual(user_name, 'Alice')
@patch('db_client.sqlite3.connect')
def test_get_user_by_id_not_found(self, mock_connect):
mock_cursor = MagicMock()
mock_connect.return_value.cursor.return_value = mock_cursor
# Simulate a database returning no results
mock_cursor.fetchone.return_value = None
db_client = DatabaseClient('test_db.sqlite')
user_name = db_client.get_user_by_id(999)
# Assert the function handled the "not found" case
self.assertIsNone(user_name)
By patching sqlite3.connect, we intercept the first call in the chain, allowing us to control the behavior of all subsequent calls and return values without a real connection.
Mocking File System Operations
File I/O can be slow and introduce state, as a test might read from or write to a real file, affecting subsequent tests. unittest.mock provides a special helper function, mock_open, specifically for this purpose [7].
Example: Mocking file reads and writes
Consider a function that reads a configuration file and another that writes to a log file.
file_operations.py
def read_config(file_path):
"""Reads a configuration from a file."""
with open(file_path, 'r') as f:
return f.read()
def write_log(message, file_path='app.log'):
"""Appends a message to a log file."""
with open(file_path, 'a') as f:
f.write(message + '\n')
We can test these functions using mock_open.
test_file_operations.py
import unittest
from unittest.mock import patch, mock_open, call
from file_operations import read_config, write_log
class TestFileOperations(unittest.TestCase):
@patch('builtins.open', new_callable=mock_open, read_data='key: value')
def test_read_config(self, mock_file):
config = read_config('config.txt')
# The mock_open helper is configured to return 'key: value'
self.assertEqual(config, 'key: value')
# We can still assert how it was used
mock_file.assert_called_once_with('config.txt', 'r')
@patch('builtins.open', new_callable=mock_open)
def test_write_log(self, mock_file):
write_log('Test message')
# Assert the file was opened in append mode
mock_file.assert_called_once_with('app.log', 'a')
# The `mock_file.return_value` is the mock file handle
# Assert that the write method on the file handle was called
mock_file.return_value.write.assert_called_once_with('Test message\n')
The new_callable=mock_open argument to patch is key. It replaces the built-in open function with a special mock object that understands file-like operations such as read(), write(), and context management (__enter__, __exit__).
This article has demonstrated the power of mocking for isolating your code from external services. In the next article, we will explore more advanced topics, including mocking entire classes and the key differences between mocks, stubs, and fakes.
Sources
- Real Python - The Python Mocking Library
- Test-Driven Development with Python and Django
- Python Mock vs MagicMock
- Python unittest.mock documentation
- Where to patch in Python
- Mocking Database Connections in Python
- unittest.mock.mock_open helper
This video provides a great visual explanation of mocking HTTP requests in Python: Intro to Python Mocks | Python tutorial