List Comprehension in Python

     Comprehensions in Python provide us with a short and concise way to construct new sequences.

Why We use List Comprehension?:

• The code is more concise.
• The code is generally more readable.
• The code, in most cases, will run faster.
• Efficient Memory Management

What is Python List Comprehension?

Suppose you want to take the letters in the word ‘anxiety’, and want to put them in a list. 

Using a for loop, you would:

>>> mylist=[]

>>> for i in 'anxiety':

mylist.append(i)

>>> mylist

Output

[‘a’, ‘n’, ‘x’, ‘i’, ‘e’, ‘t’, ‘y’]

But with a Python list comprehension, you can do this in one line:

>>> [i for i in 'anxiety']

Output

[‘a’, ‘n’, ‘x’, ‘i’, ‘e’, ‘t’, ‘y’]

List Comprehensions Syntax

A list comprehension creates a new list by applying an expression to each element of an iterable. The most basic

form is:

[ <expression> for <element> in <iterable> ]

There's also an optional 'if' condition:

[ <expression> for <element> in <iterable> if <condition> ]

Each <element> in the <iterable> is plugged in to the <expression> if the (optional) <condition> evaluates to true. All results are returned at once in the new list. 

To create a list of squared integers:

squares = [x * x  for x in (1, 2, 3, 4)]

# squares: [1, 4, 9, 16]

The for expression sets x to each value in turn from (1, 2, 3, 4). The result of the expression x * x is appended to an internal list. The internal list is assigned to the variable squares when completed.

Besides a speed increase (as explained here), a list comprehension is roughly equivalent to the following for-loop:

squares = []

for x in (1, 2, 3, 4):

squares.append(x * x)

# squares: [1, 4, 9, 16]

The expression applied to each element can be as complex as needed:

# Get a list of uppercase characters from a string

[s.upper() for s in "Hello World"]

# ['H', 'E', 'L', 'L', 'O', ' ', 'W', 'O', 'R', 'L', 'D']

# Strip off any commas from the end of strings in a list

[w.strip(',') for w in ['these,', 'words,,', 'mostly', 'have,commas,']]

# ['these', 'words', 'mostly', 'have,commas']

# Organize letters in words more reasonably - in an alphabetical order

sentence = "Beautiful is better than ugly"

["".join(sorted(word, key = lambda x: x.lower())) for word in sentence.split()]

# ['aBefiltuu', 'is', 'beertt', 'ahnt', 'gluy']

else:

else can be used in List comprehension constructs, but be careful regarding the syntax. The if/else clauses should be used before for loop, not after:

# create a list of characters in apple, replacing non vowels with '*'

# Ex - 'apple' --> ['a', '*', '*', '*' ,'e']

[x for x in 'apple' if x in 'aeiou' else '*']

#SyntaxError: invalid syntax

# When using if/else together use them before the loop

[x if x in 'aeiou' else '*' for x in 'apple']

#['a', '*', '*', '*', 'e']

Double Iteration

Order of double iteration [... for x in ... for y in ...] is either natural or counter-intuitive. The rule of

thumb is to follow an equivalent for loop:

def foo(i):

    return i, i + 0.5

for i in range(3):

    for x in foo(i):

    yield str(x)

This becomes:

[str(x)

for i in range(3)

for x in foo(i)

]

This can be compressed into one line as [str(x) for i in range(3) for x in foo(i)]

In-place Mutation and Other Side Effects

Before using list comprehension, understand the difference between functions called for their side effects (mutating, or in-place functions) which usually return None, and functions that return an interesting value.

Many functions (especially pure functions) simply take an object and return some object. An in-place function modifies the existing object, which is called a side effect. Other examples include input and output operations such as printing.

list.sort() sorts a list in-place (meaning that it modifies the original list) and returns the value None. Therefore, it

won't work as expected in a list comprehension:

[x.sort() for x in [[2, 1], [4, 3], [0, 1]]]

# [None, None, None]

Instead, sorted() returns a sorted list rather than sorting in-place:

[sorted(x) for x in [[2, 1], [4, 3], [0, 1]]]

# [[1, 2], [3, 4], [0, 1]]

Using comprehensions for side-effects is possible, such as I/O or in-place functions. Yet a for loop is usually more

readable. While this works in Python 3:

[print(x) for x in (1, 2, 3)]

Instead use:

for x in (1, 2, 3):

print(x)

Whitespace in list comprehensions

More complicated list comprehensions can reach an undesired length, or become less readable. Although less  common in examples, it is possible to break a list comprehension into multiple lines like so:

[

x for x

in 'foo'

if x not in 'bar'

]

Conditional List Comprehensions

Given a list comprehension you can append one or more if conditions to filter values.

[<expression> for <element> in <iterable> if <condition>]

For each <element> in <iterable>; if <condition> evaluates to True, add <expression> (usually a function of

<element>) to the returned list.

For example, this can be used to extract only even numbers from a sequence of integers:

[x for x in range(10) if x % 2 == 0]

# Out: [0, 2, 4, 6, 8]

The above code is equivalent to:

even_numbers = []

for x in range(10):

    if x % 2 == 0:

    even_numbers.append(x)

print(even_numbers)

# Out: [0, 2, 4, 6, 8]

example:

[x if x % 2 == 0 else None for x in range(10)]

# Out: [0, None, 2, None, 4, None, 6, None, 8, None]

items:

<value-if-condition-is-true> if <condition> else <value-if-condition-is-false>

This becomes more obvious if you combine it with other operators:

[2 * (x if x % 2 == 0 else -1) + 1 for x in range(10)]

# Out: [1, -1, 5, -1, 9, -1, 13, -1, 17, -1]

List Comprehensions with Nested Loops

List Comprehensions can use nested for loops. You can code any number of nested for loops within a list comprehension, and each for loop may have an optional associated if test. When doing so, the order of the for constructs is the same order as when writing a series of nested for statements. The general structure of list

comprehensions looks like this:

[ expression for target1 in iterable1 [if condition1]

for target2 in iterable2 [if condition2]...

for targetN in iterableN [if conditionN] ]

For example, the following code flattening a list of lists using multiple for statements:

data = [[1, 2], [3, 4], [5, 6]]

output = []

for each_list in data:

for element in each_list:

output.append(element)

print(output)

# Out: [1, 2, 3, 4, 5, 6]

can be equivalently written as a list comprehension with multiple for constructs:

data = [[1, 2], [3, 4], [5, 6]]

output = [element for each_list in data for element in each_list]

print(output)

# Out: [1, 2, 3, 4, 5, 6]

 Dictionary Comprehensions

A dictionary comprehension is similar to a list comprehension except that it produces a dictionary object instead of a list.

{x: x * x for x in (1, 2, 3, 4)}

# Out: {1: 1, 2: 4, 3: 9, 4: 16}

which is just another way of writing:

dict((x, x * x) for x in (1, 2, 3, 4))

# Out: {1: 1, 2: 4, 3: 9, 4: 16}

As with a list comprehension, we can use a conditional statement inside the dict comprehension to produce only the dict elements meeting some criterion.

{name: len(name) for name in ('hi', 'hello','python') if len(name) > 6}

# Out: {'Exchange': 8, 'Overflow': 8}

Or, rewritten using a generator expression.

dict((name, len(name)) for name in ('hi', 'hello'python') if len(name) > 6)

# Out: {'Exchange': 8, 'Overflow': 8}

Starting with a dictionary and using dictionary comprehension as a key-value pair filter

initial_dict = {'x': 1, 'y': 2}

{key: value for key, value in initial_dict.items() if key == 'x'}

# Out: {'x': 1}

Merging Dictionaries

Combine dictionaries and optionally override old values with a nested dictionary comprehension.

dict1 = {'w': 1, 'x': 1}

dict2 = {'x': 2, 'y': 2, 'z': 2}

{k: v for d in [dict1, dict2] for k, v in d.items()}

# Out: {'w': 1, 'x': 2, 'y': 2, 'z': 2}

However, dictionary unpacking (PEP 448) may be a preferred.

Python 3.x Version ≥ 3.5

{**dict1, **dict2}

# Out: {'w': 1, 'x': 2, 'y': 2, 'z': 2}