In this tutorial, we will look at how to get a count of the distinct values in a column of a Pyspark dataframe with the help of examples. How to count unique values in a Pyspark dataframe column? You can use the Pyspark count_distinct() function to get a count of the distinct values in a […]
Pyspark – Count Distinct Values in a Column Read More »
In this tutorial, we will look at how to get the sum of the distinct values in a column of a Pyspark dataframe with the help of examples. How to sum unique values in a Pyspark dataframe column? You can use the Pyspark sum_distinct() function to get the sum of all the distinct values in
Pyspark – Sum of Distinct Values in a Column Read More »
In this tutorial, we will look at how to convert a datetime type column to a category type column in Pandas and apply common datetime operations. How to convert datetime column to category type column in Pandas? You can use the Pandas astype() function to change the type of a column. To convert a column
Pandas – Category Column with Datetime Values Read More »
In this tutorial, we will look at how to construct schema for a Pyspark dataframe with the help of Structype() and StructField() in Pyspark. Pyspark Dataframe Schema The schema for a dataframe describes the type of data present in the different columns of the dataframe. Let’s look at an example. Output: Here, we created a
Pyspark DataFrame Schema with StructType() and StructField() Read More »
In this tutorial, we will look at how to convert a category type column in Pandas to a list with the help of some examples. How to get the list of values from a Pandas categorical column? You can use the Pandas series tolist() or the Python built-in list() function to get values from a
Pandas Category Column to a List Read More »
In this tutorial, we will look at how to apply string functions to a category type column in Pandas with the help of some examples. How to apply string functions to a category type column? For appropriate category type columns in Pandas, you can use the .str accessor to apply string functions. The following is
Pandas – Apply String Functions to Category Column Read More »
In this tutorial, we will look at how to set a category order in a Pandas category type column with the help of some examples. How to set category order in Pandas? You can use the Pandas categorical set_categories() function to set and order categories in a category type column. Use the .cat accessor to
Pandas – Set Category Order of a Categorical Column Read More »
In this tutorial, we will look at how to change the category order for a category type column in Pandas with the help of examples. How to change the category order in Pandas? To change the category order in an ordered categorical column in Pandas, use the Pandas categorical reorder_categories() function with the help of
Change Category Order of a Pandas Column Read More »
In this tutorial, we will look at how to print the schema of a Pyspark dataframe with the help of some examples. How to get the schema of a Pyspark dataframe? You can use the printSchema() function in Pyspark to print the schema of a dataframe. It displays the column names along with their types.
Print Pyspark DataFrame Schema Read More »
In this tutorial, we will look at how to remove categories from a category type column in Pandas with the help of some examples. How to remove categories from a Pandas Categorical Column? You can use the Pandas remove_categories() method to remove categories from a categorical field in Pandas. For a Pandas series, use the
Pandas – Remove Categories From a Categorical Column Read More »
