Set DatetimeIndex for pandas DataFrame in Python (3 Examples)

 

This article illustrates how to convert a datetime column to the index of a pandas DataFrame in the Python programming language.

The content of the post is structured like this:

Here’s the step-by-step process!

 

Example 1: Adjust DatetimeIndex from Existing datetime Column

In this first example, we already have an existing datetime column, which we want to set as index.

But before we can start, we have to load the pandas library:

import pandas as pd                                  # Import pandas library

As a next step, we need to construct our example data we can work with:

data1 = pd.DataFrame({'datetime':pd.to_datetime(['01-06-2018 23:15:00',   # Creating data
                                                 '02-09-2019 01:48:00',
                                                 '08-06-2020 13:20:00',
                                                 '07-03-2021 14:50:00']),
                      'values':range(0, 4)})
print(data1)                                         # Print example DataFrame
#               datetime  values
# 0 2018-01-06 23:15:00       0
# 1 2019-02-09 01:48:00       1
# 2 2020-08-06 13:20:00       2
# 3 2021-07-03 14:50:00       3

To change the index of our pandas DataFrame we can use the set_index function as you can see in the following code:

data1_new = data1.set_index('datetime')           # Applying the set_index method
print(data1_new)                                  # Print new DataFrame with updated index
#                       values
# datetime                   
# 2018-01-06 23:15:00       0
# 2019-02-09 01:48:00       1
# 2020-08-06 13:20:00       2
# 2021-07-03 14:50:00       3

That’s it! Now the index of our DataFrame has been changed to DatetimeIndex.

 

Example 2: Set DatetimeIndex from Separated Date & Time Columns Using + Operator

In the next two examples, I’ll illustrate how to adjust the DatetimeIndex from divided date and time columns.

As example data for the following two examples, we will use a copy of the DataFrame we create next:

data2 = pd.DataFrame({'date':['09-05-2019', '08-03-2020', '02-06-2021', '01-01-2022'],  # Creating example data
                      'time':['22:40:00', '03:46:00', '21:19:00', '17:35:00'],
                      'values':range(0, 4)})
print(data2)                                         # Print example DataFrame
#          date      time  values
# 0  09-05-2019  22:40:00       0
# 1  08-03-2020  03:46:00       1
# 2  02-06-2021  21:19:00       2
# 3  01-01-2022  17:35:00       3

Before we can set DatetimeIndex, we have to create a new datetime column from our date and time columns.

To achieve this, we can use the + operator within the to_datetime function as shown below:

data2_new = data2.copy()                             # Create copy of pandas DataFrame
data2_new['datetime'] = pd.to_datetime(data2_new['date'] +  # Create datetime column using + operator
                                       ' ' +
                                       data2_new['time'])
print(data2_new)                                     # Print new DataFrame with datetime column
#          date      time  values            datetime
# 0  09-05-2019  22:40:00       0 2019-09-05 22:40:00
# 1  08-03-2020  03:46:00       1 2020-08-03 03:46:00
# 2  02-06-2021  21:19:00       2 2021-02-06 21:19:00
# 3  01-01-2022  17:35:00       3 2022-01-01 17:35:00

As illustrated by the output above, a new datetime column has been created.

Now we can apply the set_index function to set our DatetimeIndex:

data2_new = data2_new.set_index('datetime')          # Applying the set_index method
print(data2_new)                                     # Print new DataFrame with updated index
#                            date      time  values
# datetime                                         
# 2019-09-05 22:40:00  09-05-2019  22:40:00       0
# 2020-08-03 03:46:00  08-03-2020  03:46:00       1
# 2021-02-06 21:19:00  02-06-2021  21:19:00       2
# 2022-01-01 17:35:00  01-01-2022  17:35:00       3

Here we go. The previous output shows our DataFrame with the DatetimeIndex.

Since the date and time columns are no more required, we can simply remove them, as outlined in the code below:

del data2_new['date']                                # Remove unnecessary date column
del data2_new['time']                                # Remove unnecessary time column
print(data2_new)                                     # Print adapted DataFrame
#                      values
# datetime                   
# 2019-09-05 22:40:00       0
# 2020-08-03 03:46:00       1
# 2021-02-06 21:19:00       2
# 2022-01-01 17:35:00       3

Looks perfect!

 

Example 3: Set DatetimeIndex from Separated Date & Time Columns Using the format Parameter

Similar to the second example, we want to set DatetimeIndex from the two separated columns called date and time.

To make life easier, we use a copy of the example DataFrame we have created above:

data3_new = data2.copy()                               # Creating example data

To get a new datetime column and set it as DatetimeIndex we can use the format parameter of the to_datetime function followed by the set_index function.

Consider the following code:

data3_new['datetime'] = pd.to_datetime(data3_new['date'] +  # Create datetime column using the format parameter
                                       data3_new['time'],
                                       format = '%m-%d-%Y%H:%M:%S')
data3_new = data3_new.set_index('datetime')            # Applying the set_index method
print(data3_new)                                       # Print new DataFrame with updated index
#                            date      time  values
# datetime                                         
# 2019-09-05 22:40:00  09-05-2019  22:40:00       0
# 2020-08-03 03:46:00  08-03-2020  03:46:00       1
# 2021-02-06 21:19:00  02-06-2021  21:19:00       2
# 2022-01-01 17:35:00  01-01-2022  17:35:00       3

The output above shows our DataFrame with DatetimeIndex.

To get rid of the unnecessary date and time columns, we can simply remove them:

del data3_new['date']                                # Remove unnecessary date column
del data3_new['time']                                # Remove unnecessary time column
print(data3_new)                                     # Print adapted DataFrame
#                      values
# datetime                   
# 2019-09-05 22:40:00       0
# 2020-08-03 03:46:00       1
# 2021-02-06 21:19:00       2
# 2022-01-01 17:35:00       3

That’s it! Now we have an adapted pandas DataFrame with DatetimeIndex.

 

Video & Further Resources

In case you need further information on the topics of this article, you might want to watch the following video on my YouTube channel. I demonstrate the Python codes of this tutorial in the video:

 

The YouTube video will be added soon.

 

In addition, you may want to have a look at the other posts which I have published on this website. A selection of tutorials can be found below:

 

In this Python tutorial, you have learned how to use date and time columns as index of a pandas DataFrame. If you have further questions and/or comments, please let me know in the comments.

 

Matthias Bäuerlen Python Programmer

This page was created in collaboration with Matthias Bäuerlen. Have a look at Matthias’ author page to get more information about his professional background, a list of all his tutorials, as well as an overview on his other tasks on Statistics Globe.

 

Subscribe to the Statistics Globe Newsletter

Get regular updates on the latest tutorials, offers & news at Statistics Globe.
I hate spam & you may opt out anytime: Privacy Policy.


Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Top