Applying a function to every row in a Pandas DataFrame means executing custom logic on each row individually. For example, if a DataFrame contains columns 'A', 'B' and 'C', and you want to compute their sum for each row, you can apply a function across all rows to generate a new column.

Let’s explore efficient methods to achieve this efficiently.

Using vectorized operations

This is the most efficient method for column-wise operations in Pandas. It directly adds entire columns using built-in vectorized arithmetic, leveraging low-level optimizations. Ideal for simple mathematical computations across columns.

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = df['A'] + df['B'] + df['C']
print(df)

   A  B  C  add
0  1  4  7   12
1  2  5  8   15
2  3  6  9   18

Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using Pandas' vectorized operations by adding a new column 'add' with df['A'] + df['B'] + df['C'].

Using .values()

By converting selected DataFrame columns to a NumPy array using .values, you can perform fast row-wise computations using NumPy functions like np.sum(). This method is very efficient for numerical data.

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = np.sum(df[['A', 'B', 'C']].values, axis=1)
print(df)

   A  B  C  add
0  1  4  7   12
1  2  5  8   15
2  3  6  9   18

Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using NumPy’s sum() on the DataFrame’s .values, with axis=1 to sum across columns. The result is stored in a new column 'add'.

Using itertuples()

This method iterates over DataFrame rows as namedtuples, offering better performance than iterrows(). It’s suitable when you need row-wise logic without sacrificing too much speed.

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
res = []
for row in df.itertuples(index=False):
    res.append(row.A + row.B + row.C)
df['add'] = res
print(df)

   A  B  C  add
0  1  4  7   12
1  2  5  8   15
2  3  6  9   18

Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using itertuples(index=False), summing values for each row and storing the results in a list, which is then assigned to a new column 'add'.

Using list comprehension

A Pythonic and readable way to perform row-wise operations by zipping columns. It’s efficient for small to medium DataFrames where the logic is simple.

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = [a + b + c for a, b, c in zip(df['A'], df['B'], df['C'])]
print(df)

   A  B  C  add
0  1  4  7   12
1  2  5  8   15
2  3  6  9   18

Explanation: We use zip() to iterate over columns 'A', 'B' and 'C' in parallel, summing their values for each row and storing the result in a new column 'add'.

Related articles

• .values()
• Pandas DataFrame
• itertuples()
• List comprehension