Skip to Content

Standard Deviation of Numpy Array with NaN Values

The Numpy library in Python comes with a number of useful built-in functions for computing common descriptive statistics like mean, median, standard deviation, etc. In this tutorial, we will look at how to get the standard deviation of values in a Numpy array containing one or more NaN values.

Can you use the numpy.std() function on an array with NaN values?

We use the numpy.std() function to get the standard deviation of values in a Numpy array. But what happens if the array contains one or more NaN values?

Let’s find out.

import numpy as np

# create array
ar = np.array([1, 2, np.nan, 3])
# get array std dev
print(np.std(ar))

Output:

nan

Here, we created a one-dimensional Numpy array containing some numbers and a NaN value. We then applied the numpy.std() function which resulted in nan. This happened because the numpy.std() function wasn’t able to handle the nan value present in the array when computing the standard deviation.

Thus, you cannot use the numpy.std() function to calculate the standard deviation of an array with NaN values.

How to ignore NaN values when calculating the standard deviation of a Numpy array?

standard deviation of numpy array with nan values

You can use the numpy.nanstd() function to calculate the standard deviation of a Numpy array containing NaN values. Pass the array as an argument.

The following is the syntax –

# std dev of array with nan values
numpy.nanstd(ar)

It returns the standard deviation among all the values in the array ignoring all the NaN values.

Let’s look at some examples of using the numpy.nanstd() function.

Example 1 – standard deviation of one-dimensional array with NaN values

Let’s apply the numpy.nanstd() function on the same array used in the example above.

# create array
ar = np.array([1, 2, np.nan, 3])
# get array std dev
print(np.nanstd(ar))

Output:

0.816496580927726

We get the standard deviation in the above array as approximately 0.82. The numpy.nanstd() function ignores the NaN values when computing the standard deviation.

Example 2 – Standard deviation of multi-dimensional array with NaN values

The numpy.nanstd() function is very similar to the numpy.std() function in its arguments. For example, use the axis parameter to specify the axis along which to compute the standard deviation.

First, let’s create a 2-D Numpy array.

# create 2-D numpy array
ar = np.array([[1, np.nan, 3],
               [np.nan, 5, np.nan]])
# display the array
print(ar)

Output:

[[ 1. nan  3.]
 [nan  5. nan]]

Here, we used the numpy.array() function to create a Numpy array with two rows and three columns. You can see that there are some NaN values present in the array.

If you use the Numpy nanstd() function on an array without specifying the axis, it will return the standard deviation of the values inside the array.

# std dev of array
print(np.nanstd(ar))

Output:

1.632993161855452

We get the standard deviation of all the values inside the 2-D array.

Use the numpy.nanstd() function with axis=1 to get the standard deviation for each row in the array.

# std dev of each row in array
print(np.nanstd(ar, axis=1))

Output:

[1. 0.]

We get the standard deviation of each row in the above 2-D array. The standard deviation of values in the first row is 1 and in the second row is 0.

Use the numpy.nanstd() function with axis=0 to get the standard deviation of each column in the array.

# std dev of each column in array
print(np.nanstd(ar, axis=0))

Output:

[0. 0. 0.]

We get the standard deviation of each column in the above 2-D array. In this example, each column has one NaN value and one non-NaN value (thus we get 0 as the standard deviation as there’s only one unique value in the column).

Summary

The following is a short summary of the important points mentioned in this tutorial.

  1. Using the numpy.std() function on an array with NaN values results in NaN.
  2. Use the numpy.nanstd() function to get the standard deviation of values in an array containing one or more NaN values. It computes the standard deviation by taking into account only the non-NaN values in the array.
  3. Similar to the numpy.std() function, you can specify the axis along which you want to compute the standard deviation with the numpy.nanstd() function.

You might also be interested in –


Subscribe to our newsletter for more informative guides and tutorials.
We do not spam and you can opt out any time.


Author

  • Piyush Raj

    Piyush is a data professional passionate about using data to understand things better and make informed decisions. In the past, he's worked as a Data Scientist for ZS and holds an engineering degree from IIT Roorkee. His hobbies include watching cricket, reading, and working on side projects.