You can use Pandas groupby to group the underlying data on one or more columns and estimate useful statistics like count, mean, median, std, min, max etc. In this tutorial, we will look at how to get the standard deviation of a column (or columns) for each group in pandas groupby with the help of some examples.

## Pandas Groupby Standard Deviation

To get the standard deviation of each group, you can directly apply the pandas `std()`

function to the selected column(s) from the result of pandas groupby. The following is a step-by-step guide of what you need to do.

- Group the dataframe on the column(s) you want.
- Select the field(s) for which you want to estimate the standard deviation.
- Apply the pandas
`std()`

function directly or pass ‘std’ to the`agg()`

function.

The following is the syntax –

# groupby columns on Col1 and estimate the std dev of column Col2 for each group df.groupby([Col1])[Col2].std() # alternatively, you can pass 'std' to the agg() function df.groupby([Col1])[Col2].agg('std')

## Examples

Let’s look at how to get the standard deviation for each group with the help of some examples. First, we will create a sample dataframe that we will be using throughout this tutorial.

import pandas as pd # create a dataframe of car models by two companies df = pd.DataFrame({ 'Company': ['A', 'A', 'A', 'B', 'B', 'B', 'B'], 'Model': ['A1', 'A2', 'A3', 'B1', 'B2', 'B3', 'B4'], 'Year': [2019, 2020, 2021, 2018, 2019, 2020, 2021], 'Transmission': ['Manual', 'Automatic', 'Automatic', 'Manual', 'Automatic', 'Automatic', 'Manual'], 'EngineSize': [1.4, 2.0, 1.4, 1.5, 2.0, 1.5, 1.5], 'MPG': [55.4, 67.3, 58.9, 52.3, 64.2, 68.9, 83.1] }) # display the dataframe df

Output:

We now have a dataframe containing the specifications of the car models by two different companies. The “EngineSize” column is the size of the engine in litres and the “MPG” column is the mileage of the car in miles-per-gallon.

### 1. Groupby std of a Single Column

Let’s get the standard deviation of the mileage “MPG” column for each “Company” in the dataframe df. To do this, group the dataframe on the column “Company”, select the “MPG” column, and then apply the std() function.

# std dev in MPG for each Company df.groupby('Company')['MPG'].std()

Output:

**Data Science Programs By Skill Level**

**Introductory** ⭐

- Harvard University Data Science: Learn R Basics for Data Science
- Standford University Data Science: Introduction to Machine Learning
- UC Davis Data Science: Learn SQL Basics for Data Science
- IBM Data Science: Professional Certificate in Data Science
- IBM Data Analysis: Professional Certificate in Data Analytics
- Google Data Analysis: Professional Certificate in Data Analytics
- IBM Data Science: Professional Certificate in Python Data Science
- IBM Data Engineering Fundamentals: Python Basics for Data Science

**Intermediate ⭐⭐⭐**

- Harvard University Learning Python for Data Science: Introduction to Data Science with Python
- Harvard University Computer Science Courses: Using Python for Research
- IBM Python Data Science: Visualizing Data with Python
- DeepLearning.AI Data Science and Machine Learning: Deep Learning Specialization

**Advanced ⭐⭐⭐⭐⭐**

- UC San Diego Data Science: Python for Data Science
- UC San Diego Data Science: Probability and Statistics in Data Science using Python
- Google Data Analysis: Professional Certificate in Advanced Data Analytics
- MIT Statistics and Data Science: Machine Learning with Python - from Linear Models to Deep Learning
- MIT Statistics and Data Science: MicroMasters® Program in Statistics and Data Science

**🔎 Find Data Science Programs 👨💻 111,889 already enrolled**

Disclaimer: Data Science Parichay is reader supported. When you purchase a course through a link on this site, we may earn a small commission at no additional cost to you. Earned commissions help support this website and its team of writers.

Company A 6.115826 B 12.736921 Name: MPG, dtype: float64

You can see that std dev in “MPG” for company “A” is less than that of company “B”. This tells us that the mileage of cars from company “A” do not vary as much as the mileage for cars from company “B”.

Alternatively, you can also use the pandas `agg()`

function on the resulting groups.

# std dev in MPG for each Company df.groupby('Company')['MPG'].agg('std')

Output:

Company A 6.115826 B 12.736921 Name: MPG, dtype: float64

We get the same result as above.

Note that the pandas `std()`

function calculates the sample standard deviation by default (normalizing by N-1). To get the population standard deviation, pass `ddof = 0`

to the `std()`

function. To see an example, check out our tutorial on calculating standard deviation in Python. Also, here’s a link to the official documentation.

If you want to see common descriptive stats for each group, like mean, median, standard deviation, etc., you can apply the pandas `describe()`

function on the result of groupby.

# useful statistics on MPG for each Company df.groupby('Company')['MPG'].describe()

Output:

You can see that we get the standard deviation as well as other common descriptive stats like mean, median, min, max, etc. for each group.

### 2. Groupby std of Multiple Columns

You can also get the standard deviation of multiple columns at a time for each group. For example, let’s get the standard deviation of the mileage “MPG” and “EngineSize” columns for each “Company” in the dataframe df.

# std dev in MPG and EngineSize for each Company df.groupby('Company')[['MPG', 'EngineSize']].std()

Output:

Here we selected the columns that we wanted to compute the std dev on from the resulting groupby object and then applied the std() function. We already know that the standard deviation for “MPG” is smaller for company “A”. Here we additionally find that the standard deviation for “EngineSize” is larger for company “A” compared to “B”.

Let’s now do the same thing using the pandas `agg()`

function.

# std dev in MPG and EngineSize for each Company df.groupby('Company')[['MPG', 'EngineSize']].agg('std')

Output:

We get the same result as above.

With this, we come to the end of this tutorial. The code examples and results presented in this tutorial have been implemented in a Jupyter Notebook with a python (version 3.8.3) kernel having pandas version 1.0.5

**Subscribe to our newsletter for more informative guides and tutorials. ****We do not spam and you can opt out any time.**