Last Value in Each Group – Pandas Groupby

You can use Pandas groupby to group the underlying data on one or more columns and estimate useful statistics like count, mean, median, std, min, max etc.  Sometimes knowing the first, last, or the nth value in the group might also be useful. In this tutorial we will look at how to get the last value for each group with the help of some examples.

How to get the last value in each group?

You can use the pandas.groupby.last() function to get the last value in each group. The following is the syntax assuming you want to group the dataframe on column “Col1” and get the last value in the “Col2” for each group.

# using pandas.groupby().last()
df.groupby('Col1')['Col2'].last()

Examples

Let’s look at some examples of using the above syntax. First, we will create a sample dataframe that we will be using throughout this tutorial.

import pandas as pd

# create a dataframe of GRE scores of two students
df = pd.DataFrame({
    'Name': ['Jim', 'Jim', 'Jim', 'Pam', 'Pam'],
    'Attempt': ['First', 'Second', 'Third', 'First', 'Second'],
    'GRE Score': [298, 321, 314, 318, 330]
})
# display the dataframe
df

Output:

We now have a dataframe containing the GRE scores of two students, Jim and Pam, across their multiple attempts at the exam.

Using Pandas Groupby Last

Let’s get the latest GRE Score for each student. This is the last score for each student in the dataframe. For this, we will group the data on column “Name” and then get the last value in column “GRE Score” for each group by applying the last() function.

# the last GRE score for each student
df.groupby('Name')['GRE Score'].last()

Output:

Name
Jim    314
Pam    330
Name: GRE Score, dtype: int64

We get the latest (or the last) score for both the students. We can see that Pam scored better on the last attempt.

For more on the pandas groupby last() function, refer to its documentation.

Using Pandas Groupby nth()

Alternatively, you can use the pandas groupby nth() function. This function is used to return the value of the nth row for each group. To get the last value, pass -1 as the argument. Let’s get the same thing as above, the latest GRE Score for both the students but using this function instead.

# the last GRE score for each student
df.groupby('Name')['GRE Score'].nth(-1)

Output:

Name
Jim    314
Pam    330
Name: GRE Score, dtype: int64

We get the same results as above.

Note that, the last() function returns the first non-Nan value it encounters from the bottom for each group whereas the nth() function returns the value of the passed row irrespective of whether it’s Nan or not.

With this, we come to the end of this tutorial. The code examples and results presented in this tutorial have been implemented in a Jupyter Notebook with a python (version 3.8.3) kernel having pandas version 1.0.5