Read CSV file as NumPy Array

In this tutorial, we will look at how to read CSV files in Python using the Numpy library.

Numpy functions to read CSV files

You can use the numpy functions genfromtxt() or loadtxt() to read CSV files to a numpy array. The following is the syntax:

import numpy as np

# using genfromtxt()
arr = np.genfromtxt("data.csv", delimiter=",")
# using loadtxt()
arr = np.loadtxt("data.csv", delimiter=",")

Both the functions return a numpy array. Note that the numpy genfromtxt() function is a more versatile reader compared to loadtxt() which aims to be a fast reader for simply formatted files.

Examples

Let’s look at the usage of both these functions and some specific use-cases with the help of examples. For illustrating the examples we will be reading sample data from a CSV file stored locally. This is how the contents of the file look in Notepad –

1. Using np.genfromtxt() to read CSV

Let’s use the genfromtxt() function to read the above CSV file.

import numpy as np

# read csv
arr = np.genfromtxt("sample_data.csv", delimiter=",")
# display the array
print(arr)

Output:

[[1. 4. 7.]
 [2. 5. 8.]
 [3. 6. 9.]
 [0. 2. 4.]
 [7. 7. 7.]]

The values from the CSV file have now been loaded into an array. Let’s go ahead and confirm that it’s a numpy array.

type(arr)

Output:

numpy.ndarray

You can see that it is a numpy array.

2. Using np.loadtxt() to read CSV

You can also use the loadtxt() function to read CSV files to numpy arrays. Let’s read the above file using this function.

# read csv
arr = np.loadtxt("sample_data.csv", delimiter=",")
# display the array
print(arr)

Output:

[[1. 4. 7.]
 [2. 5. 8.]
 [3. 6. 9.]
 [0. 2. 4.]
 [7. 7. 7.]]

We get a numpy array of values from the CSV file.

Is genfromtxt() reading the first value as nan?

A common error users face when reading CSV files with the numpy genfromtxt() function is that the first value in the returned numpy array is missing (nan).

This generally occurs if the file begins with a Byte Order Mark (BOM). For example, for the CSV file below note that its encoding is “UTF-8 with BOM”

Now, if you try to read this CSV file with the np.genfromtxt() function, this is what we get –

# read csv
arr2 = np.genfromtxt("sample_data2.csv", delimiter=",")
# display the array
print(arr2)

Output:

[[nan  4.  7.]
 [ 2.  5.  8.]
 [ 3.  6.  9.]
 [ 0.  2.  4.]
 [ 7.  7.  7.]]

Notice that the first element in the array is a nan value. This happened due to the encoding of the file which included a BOM.

If such an issue occurs, try reading the file with the encoding “utf-8-sig”

# read csv
arr2 = np.genfromtxt("sample_data2.csv", delimiter=",", encoding="utf-8-sig")
# display the array
print(arr2)

Output:

[[1. 4. 7.]
 [2. 5. 8.]
 [3. 6. 9.]
 [0. 2. 4.]
 [7. 7. 7.]]

Now the CSV file is read correctly.

Alternatively, you can also save the file with the basic “UTF-8” encoding using Save As and then read it without specifying the encoding like we did in the earlier examples.

# read csv
arr3 = np.genfromtxt("sample_data3.csv", delimiter=",")
# display the array
print(arr3)

Output:

[[1. 4. 7.]
 [2. 5. 8.]
 [3. 6. 9.]
 [0. 2. 4.]
 [7. 7. 7.]]

You can see that now the file is read correctly.

For more on the numpy genfromtxt() function, refer to its documentation.

With this, we come to the end of this tutorial. The code examples and results presented in this tutorial have been implemented in a Jupyter Notebook with a python (version 3.8.3) kernel having numpy version 1.18.5