Decorators
Decorators allow programmers to modify the functionality of a function by wrapping it in another function.
Key concepts
- Closures: Functions that capture the enclosing environment.
- Higher-Order Functions: Functions that can take other functions as arguments or return them.
- Reassigning Symbols: Assigning a function to an existing name.
Why Use Decorators?
def greet(name):
return f'Hello! {name}'
def add(x, y):
return x + y
To log function calls, you can create a logging decorator.
def log(fn):
def inner(*args, **kwargs):
print(f'Calling {fn}...')
result = fn(*args, **kwargs)
return result
return inner
add_logged = log(add)
greet_logged = log(greet)
add_logged(3, 55)
Output
Calling <function add at 0x0000011EE58EAA60>...
58
Reassign the function name to avoid changing all calls.
add = log(add)
greet = log(greet)
add(3, 55)
Output
Calling <function add at 0x0000011EE58EAA60>...
58
The log function is called the decorator, which takes the original function as an argument and returns a modified version of it.
Decorator Syntax
Instead of manually assigning the decorator.
def add(x, y):
return x + y
add = log(add)
Use the @ syntax.
@ log
def add(x, y):
return x + y
Coding
Advanced Logging with logging Module
import logging
from time import perf_counter
# Configure our logger
logging.basicConfig(
format='%(asctime)s %(levelname)s: %(message)s',
level=logging.DEBUG
)
logger = logging.getLogger('Custom Log')
def log(func):
def inner(*args, **kwargs):
start = perf_counter()
result = func(*args, **kwargs)
end = perf_counter()
logger.debug(f'called={func.__name__}, elapsed={end-start}')
return result
return inner
@log
def add(a, b, c):
return a + b + c
@log
def greet(name):
return f'Hello {name}!'
@log
def join(data, *, item_sep=',', line_sep='\n'):
return line_sep.join([item_sep.join(str(item) for item in row) for row in data])
add(10, 20, 30)
join([range(10) for _ in range(10)])
Output
# 2024-05-15 00:33:14,536 DEBUG: called=add, elapsed=1.1000010999850929e-06
# 60
# 2024-05-15 00:34:25,534 DEBUG: called=join, elapsed=2.600000152597204e-05
# '0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9\n0,1,2,3,4,5,6,7,8,9'
LRU Cache
Understanding Caching
Caching stores recent or frequently used data in memory for faster access.
Caching is useful when:
- The function is deterministic (calls with the same set of arguments return the same result).
- Re-calculating the function is costly.
Basic Caching Concept
def add(a, b, c):
return a + b + c
cache = {}
def func(a, b, c):
key = (a, b, c)
if key in cache:
return cache[key]
print('key is not in cache')
result = add(a, b, c)
cache[key] = result
return result
cache
# {}
func(1, 2, 3) # First call, adds to cache
# key is not in cache
# 6
cache
# {(1, 2, 3): 6}
func(1, 2, 3) # Second call, retrieves from cache
# 6
LRU Cache
An LRU (Least Recently Used) cache evicts the least recently accessed items first when full.
Implementing LRU Cache in Python
from funtools import lru_cache
Syntax
@lru_cache(maxsize=128)
maxsize(Optional): It sets the size of your cache. By default, it’s set to128. Ifmaxsizeis set toNone, the LRU feature will be disabled, and the cache can grow without any limitations.
Restriction
The arguments passed to the function must be hashable values since the arguments will be stored as keys.
@lru_cache
def my_func(x):
print('calling my_func')
return x
Calling with hashable arguments
my_func((10, 20))
# calling my_func
# (10, 20)
Calling with unhashable arguments
my_func([10, 20])
# ---------------------------------------------------------------------------
# TypeError Traceback (most recent call last)
# ...
# TypeError: unhashable type: 'list'
Coding
# Create an lru_cache decorator with cache size 2
from functools import lru_cache
@lru_cache(maxsize=2)
def add(a, b):
print('add called')
return a + b
add(2, 3)
# add called
# 5
add.cache_info()
# CacheInfo(hits=0, misses=1, maxsize=2, currsize=1)
add(5, 6)
# add called
# 11
add.cache_info()
# CacheInfo(hits=0, misses=2, maxsize=2, currsize=2)
add(2, 3)
# 5
add.cache_info()
# CacheInfo(hits=1, misses=2, maxsize=2, currsize=2)
add(4, 5)
# add called
# 9
add.cache_info()
# CacheInfo(hits=1, misses=3, maxsize=2, currsize=2)
add(5, 6)
# add called
# 11
add.cache_info()
# CacheInfo(hits=1, misses=4, maxsize=2, currsize=2)
Calculating Fibonacci Numbers
Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, ...
def fib(n):
print(f'fib({n}) called...')
if n <= 1:
return n
return fib(n-1) + fib(n-2)
# if we call fib(0), it just returns 0.
fib(0)
# fib(0) called...
# 0
# if we call fib(1), it just returns 1.
fib(1)
# fib(1) called...
# 1
# If we call fib(2) it will call fib(0) and fib(1) which return 0 and 1.
fib(2)
# fib(2) called...
# fib(1) called...
# fib(0) called...
# 1
# If we call fib(3) it will call fib(2) and fib(1) -> fib(1) will just return 1, but fib(2) calls fib(0) and fib(1).
fib(3)
# fib(3) called...
# fib(2) called...
# fib(1) called...
# fib(0) called...
# fib(1) called...
# 2
# Apply that LRU cache
@lru_cache
def fib(n):
print(f'fib({n}) called...')
if n <= 1:
return n
return fib(n-1) + fib(n-2)
fib(2)
# fib(2) called...
# fib(1) called...
# fib(0) called...
# 1
fib(3)
# fib(3) called...
# 2