Python Fundamentals Tutorial: Functional Programming

Functions are first-class objects in Python, meaning they have attributes and can be referenced and assigned to variables.

>>> def i_am_an_object(myarg):
...     '''I am a really nice function.
...        Please be my friend.'''
...     return myarg
...
>>> i_am_an_object(1)
1

>>> an_object_by_any_other_name = i_am_an_object
>>> an_object_by_any_other_name(2)
2

>>> i_am_an_object
<function i_am_an_object at 0x100432aa0>

>>> an_object_by_any_other_name
<function i_am_an_object at 0x100432aa0>

>>> i_am_an_object.__doc__
'I am a really nice function.\n       Please be my friend.'

Python also supports higher-order functions, meaning that functions can accept other functions as arguments and return functions to the caller.

>>> i_am_an_object(i_am_an_object)
<function i_am_an_object at 0x100519848>

In order to define non-default sorting in Python, both the sorted() function and the list’s .sort() method accept a key argument. The value passed to this argument needs to be a function object that returns the sorting key for any item in the list or iterable.

For example, given a list of tuples, Python will sort by default on the first value in each tuple. In order to sort on a different element from each tuple, a function can be passed that returns that element.

>>> def second_element(t):
...     return t[1]
...
>>> zepp = [('Guitar', 'Jimmy'), ('Vocals', 'Robert'), ('Bass', 'John Paul'), ('Drums', 'John')]

>>> sorted(zepp)
[('Bass', 'John Paul'), ('Drums', 'John'), ('Guitar', 'Jimmy'), ('Vocals', 'Robert')]

>>> sorted(zepp, key=second_element)
[('Guitar', 'Jimmy'), ('Drums', 'John'), ('Bass', 'John Paul'), ('Vocals', 'Robert')]

Anonymous functions in Python are created using the lambda statement. This approach is most commonly used when passing a simple function as an argument to another function. The syntax is shown in the next example and consists of the lambda keyword followed by a list of arguments, a colon, and the expression to evaluate and return.

>>> def call_func(f, *args):
...     return f(*args)
...
>>> call_func(lambda x, y: x + y, 4, 5)
9

Whenever you find yourself writing a simple anonymous function, check for a builtin first. The operator module is a good place to start (see the section below). For example, the lambda function above could have been replaced with the operator.add builtin.

>>> import operator
>>> def call_func(f, *args):
...     return f(*args)
...
>>> call_func(operator.add, 4, 5)
9

Functions can be defined within the scope of another function. If this type of function definition is used, the inner function is only in scope inside the outer function, so it is most often useful when the inner function is being returned (moving it to the outer scope) or when it is being passed into another function.

Notice that in the below example, a new instance of the function inner() is created on each call to outer(). That is because it is defined during the execution of outer(). The creation of the second instance has no impact on the first.

>>> def outer():
...     def inner(a):
...         return a
...     return inner
...
>>> f = outer()
>>> f
<function inner at 0x1004340c8>
>>> f(10)
10

>>> f2 = outer()
>>> f2
<function inner at 0x1004341b8>
>>> f2(11)
11

>>> f(12)
12

A nested function has access to the environment in which it was defined. Remember from above that the definition occurs during the execution of the outer function. Therefore, it is possible to return an inner function that remembers the state of the outer function, even after the outer function has completed execution. This model is referred to as a closure.

>>> def outer2(a):
...     def inner2(b):
...         return a + b
...     return inner2
...
>>> add1 = outer2(1)
>>> add1
<function inner2 at 0x100519c80>
>>> add1(4)
5
>>> add1(5)
6
>>> add2 = outer2(2)
>>> add2
<function inner2 at 0x100519cf8>
>>> add2(4)
6
>>> add2(5)
7

A common pattern that occurs while attempting to use closures, and leads to confusion, is attempting to encapsulate an internal variable using an immutable type. When it is re-assigned in the inner scope, it is interpreted as a new variable and fails because it hasn’t been defined.

>>> def outer():
...     count = 0
...     def inner():
...         count += 1
...         return count
...     return inner
...

>>> counter = outer()

>>> counter()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in inner
UnboundLocalError: local variable 'count' referenced before assignment

The standard workaround for this issue is to use a mutable datatype like a list and manage state within that object.

>>> def better_outer():
...     count = [0]
...     def inner():
...         count[0] += 1
...         return count[0]
...     return inner
...
>>> counter = better_outer()
>>> counter()
1
>>> counter()
2
>>> counter()
3

There are many builtin functions in Python that accept functions as arguments. An example is the filter() function that was used previously. However, there are some basic actions that use operators instead of functions (like + or the subscript [] or dot . operators).

The operator module provides function versions of these operators.

>>> import operator
>>> operator.add(1, 2)
3

Using closures, it’s possible to create functions dynamically that can, for example, know which item to get from a list.

>>> get_second = operator.itemgetter(1)
>>> get_second(['a', 'b', 'c', 'd'])
'b'

The itemgetter() function will return a tuple if it is given more than one index.

>>> get_02 = operator.itemgetter(0, 2)
>>> get_02(['a', 'b', 'c', 'd'])
('a', 'c')

A typical use for the itemgetter() function is as the key argument to a list sort.

>>> import operator
>>> zepp = [('Guitar', 'Jimmy'), ('Vocals', 'Robert'), ('Bass', 'John Paul'), ('Drums', 'John')]

>>> sorted(zepp)
[('Bass', 'John Paul'), ('Drums', 'John'), ('Guitar', 'Jimmy'), ('Vocals', 'Robert')]

>>> sorted(zepp, key=operator.itemgetter(1))
[('Guitar', 'Jimmy'), ('Drums', 'John'), ('Bass', 'John Paul'), ('Vocals', 'Robert')]

functional-1-reduce.py. 

'''
>>> series1 = (0, 1, 2, 3, 4, 5)
>>> series2 = (2, 4, 8, 16, 32)
>>> power_reducer = add_powers(2)
>>> power_reducer(series1)
55
>>> power_reducer(series2)
1364
>>>
>>> power_reducer = add_powers(3)
>>> power_reducer(series1)
225
>>> power_reducer(series2)
37448

Hint: use the builtin reduce() function
'''
if __name__ == '__main__':
    import doctest
    doctest.testmod()

A powerful and common, albeit slightly complex, paradigm in Python is the use of function decorators. The usual role of decorators is to reduce repetition in code by consolidating common elements of setup and teardown in functions.

Understanding decorators requires some foundation, so the first building block is a straightforward function that does nothing out of the ordinary.

The function simple() accepts a variable list of arguments and prints them to stdout.

>>> def simple(*args):
...     for arg in args:
...         print arg,
...
>>> simple('a', 'b', 'c')
a b c

The second building block is a closure. Remember that a nested, or inner, function in Python has access to the environment in which it was defined. If that environment happens to include another function, then the closure will have access to that function.

>>> def logit(func):
...     def wrapper(*args, **kwargs):
...         print 'function %s called with args %s' % (func, args)
...         func(*args, **kwargs)
...     return wrapper
...

Now, calling logit() and passing the function object simple as the argument returns us a new fuction with a reference to simple. The returned function is easily assigned to a new variable, and then called.

>>> logged_simple = logit(simple)

>>> logged_simple('a', 'b', 'c')
function <function simple at 0x100414de8> called with args ('a', 'b', 'c')
a b c

At this point, logged_simple is, in effect, a modified version of the original function that had the name simple. More often than not, the goal is to require the use of the modified version, so it makes sense to remove the ability to call the original function. To accomplish this task, assign the newly returned function to the original variable simple. Remember, function names are just variables pointing to function objects.

This technique looks like the following:

>>> simple = logit(simple)

Now, calling simple() is actually calling the inner wrapper function from logit(), which has a reference to the function that was originally referenced by simple. In other words, the only way that the original function object can be called at this point is through the wrapper.

>>> simple('a', 'b', 'c')
function <function simple at 0x100414de8> called with args ('a', 'b', 'c')
a b c

Logging is a very common example of a task that can be accomplished in a decorator, but the options are truly unlimited. If there is any common functionality that happens (or can happen) at the beginning or end of multiple function bodies, then a decorator is the model to reduce duplication of effort.

In the following example, the arguments being passed to the function are manipulated by the decorator.

This time, there are two functions, with similar but different tasks. Each takes a list of programming languages defined by the caller and returns an ordered list with claims about the languages.

functional-1-langs.py. 

import itertools

CLAIM = '{0} is the #{1} {2} language'

def best(type, *args):
    langs = []
    for i, arg in enumerate(args):
        langs.append(CLAIM.format(arg, i+1, type))
    return langs

def best_functional(*args):
    return best('functional', *args)

def best_oo(*args):
    return best('OO', *args)

for claim in itertools.chain(best_functional('Haskell', 'Erlang'), [''],
        best_oo('Objective-C', 'Java')):
    print claim

$ python functional-1-langs.py
Haskell is the #1 functional language
Erlang is the #2 functional language

Objective-C is the #1 OO language
Java is the #2 OO language

Recognizing that Python is universally the best language in both arenas, a decorator can be defined to prevent any caller from making a mistake.

functional-2-langs-decorated.py. 

import itertools

CLAIM = '{0} is the #{1} {2} language'

def python_rules(func):
    def wrapper(*args, **kwargs):
        new_args = ['Python']
        new_args.extend(args)
        return func(*new_args)
    return wrapper

def best(type, *args):
    langs = []
    for i, arg in enumerate(args):
        langs.append(CLAIM.format(arg, i+1, type))
    return langs

def best_functional(*args):
    return best('functional', *args)
best_functional = python_rules(best_functional)

def best_oo(*args):
    return best('OO', *args)
best_oo = python_rules(best_oo)

for claim in itertools.chain(best_functional('Haskell', 'Erlang'), [''],
        best_oo('Objective-C', 'Java')):
    print claim

$ python functional-2-langs-decorated.py
Python is the #1 functional language
Haskell is the #2 functional language
Erlang is the #3 functional language

Python is the #1 OO language
Objective-C is the #2 OO language
Java is the #3 OO language

The decorator paradigm is so common in Python that there is a special syntax using the @ symbol that allows the name of the decorating function to be placed immediately above the function definition line. This syntax, while it stands out, tends to lead to some confusion because it is less explicit than the previous format. So it is important to remember that the @ symbol notation does exactly the same thing under the covers as re-assigning the function variable name after the function definition.

functional-3-langs-atsyntax.py. 

import itertools

CLAIM = '{0} is the #{1} {2} language'

def python_rules(func):
    def wrapper(*args, **kwargs):
        new_args = ['Python']
        new_args.extend(args)
        return func(*new_args)
    return wrapper

def best(type, *args):
    langs = []
    for i, arg in enumerate(args):
        langs.append(CLAIM.format(arg, i+1, type))
    return langs

@python_rules
def best_functional(*args):
    return best('functional', *args)

@python_rules
def best_oo(*args):
    return best('OO', *args)

for claim in itertools.chain(best_functional('Haskell', 'Erlang'), [''],
        best_oo('Objective-C', 'Java')):
    print claim

$ python functional-3-langs-atsyntax.py
Python is the #1 functional language
Haskell is the #2 functional language
Erlang is the #3 functional language

Python is the #1 OO language
Objective-C is the #2 OO language
Java is the #3 OO language

functional-2-decorate.py. 

'''
>>> data = '{"username": "oscar", "password": "trashcan", "account": 1234, "amount": 12.03}'
>>> deposit(data)
'OK'
>>> data = '{"username": "oscar", "password": "trash", "account": 1234, "amount": 14.98}'
>>> deposit(data)
'Invalid Password'
>>> data = '{"username": "oscar", "password": "trashcan", "account": 1234, "amount": 4.12}'
>>> withdraw(data)
'OK'
>>> data = '{"username": "oscar", "password": "trashcan", "account": 1235, "amount": 2.54}'
>>> withdraw(data)
'Invalid Account'
>>> data = '{"username": "oscar", "password": "trashcan", "account": 1234}'
>>> balance(data)
'7.91'
>>> data = '{"username": "oscar", "password": "trashcan"}'
>>> balance(data)
'No Account Number Provided'

Hint: that's json data
'''

def deposit(account, amount=0.00):
    pass

def withdraw(account, amount=0.00):
    pass

def balance(account):
    pass

if __name__ == '__main__':
    import doctest
    doctest.testmod()