Skip to Content

Pandas – fillna with values from another column

In this tutorial, we’ll look at how to fill missing values (using fillna) in one column with values from another column of a pandas dataframe.

The pandas dataframe fillna() function is used to fill missing values in a dataframe. Generally, we use it to fill a constant value for all the missing values in a column, for example, 0 or the mean/median value of the column but you can also use it to fill corresponding values from another column. The following is the syntax:

df['Col1'].fillna(df['Col2'])

Here, we apply the fillna() function on “Col1” of the dataframe df and pass the series df[‘Col2’] as an argument. The above code fills the missing values in “Col1” with the corresponding values (based on the index) from “Col2”. To modify the dataframe in-place, pass inplace=True to the above function.

Let’s look at a use case of filling missing or NA values in a column with values from another column using the above method. First, let’s create a sample dataframe to operate on.

import numpy as np
import pandas as pd

# dataframe of postal and permanent address
df = pd.DataFrame({
    'Postal Address': ['New York', np.nan, 'London', 'Mumbai', np.nan],
    'Permanent Address': ['Miami', 'Amsterdam', 'London', 'Rajkot', 'Sydney']
})

print(df)

Output:

  Postal Address Permanent Address
0       New York             Miami
1            NaN         Amsterdam
2         London            London
3         Mumbai            Rajkot
4            NaN            Sydney

In the above dataframe we have postal and permanent addresses (for simplicity they are just city names) of winners of an online contest.

Note that, some of the postal addresses are missing. The organizer wants to send out the prizes to all the winners. For lack of better information, the organizer decides to send out the prizes to the permanent addresses for winners where you don’t have a postal address.

To fill the missing values in the column “Postal Address” with corresponding values from the column “Permanent Address”:

# fill missing values
df['Postal Address'].fillna(df['Permanent Address'], inplace=True)
print(df)

Output:

  Postal Address Permanent Address
0       New York             Miami
1      Amsterdam         Amsterdam
2         London            London
3         Mumbai            Rajkot
4         Sydney            Sydney

You can see that the values “Amsterdam” and “Sydney” from column “Permanent Address” are filled in the missing values at index 1 and 4 of column “Postal Address” resepectively.

For more on the pandas fillna() function, refer to its documentation. Our other related tutorials:

With this, we come to the end of this tutorial. The code examples and results presented in this tutorial have been implemented in a Jupyter Notebook with a python (version 3.8.3) kernel having numpy version 1.18.5 and pandas version 1.0.5


Subscribe to our newsletter for more informative guides and tutorials.
We do not spam and you can opt out any time.


Author

  • Piyush

    Piyush is a data scientist passionate about using data to understand things better and make informed decisions. In the past, he's worked as a Data Scientist for ZS and holds an engineering degree from IIT Roorkee. His hobbies include watching cricket, reading, and working on side projects.