Change language

Processing time with Pandas DataFrame

| |

Let’s discuss all the different ways to handle date and time with a Pandas dataframe.

Split date and time into multiple functions:
Create five dates and times. using pd.date_range which generates sequences of dates with fixed frequency and time intervals. We then use pandas.Series.dt to extract functions.

# Load library

import pandas as pd

 
# call the DataFrame constructor

df = pd.DataFrame ()

 
# Create 6 dates

df [ ’time’ ] = pd.date_range ( ’ 2/5 / 2019’ , periods = 6 , freq = ’ 2H’ )

print (df [ ’time’ ])  # print data

 
# Extract functions - year, month, day, hour and minute

df [ ’year’ ] = df [ ’time’ ]. dt.year

df [ ’month’ ] = df [ ’time’ ]. dt.month

df [ ’day’ ] = df [ ’ time’ ]. dt.day

df [ ’hour’ ] = df [ ’time’ ]. dt.hour

df [ ’minute’ ] = df [ ’time’ ]. dt.minute

 
# Show six lines

df.head ( 6 )

Exit:

 0 2019-02 -05 00:00:00 1 2019-02-05 02:00:00 2 2019-02-05 04:00:00 3 2019-02-05 06:00:00 4 2019-02-05 08:00: 00 5 2019-02-05 10:00:00 Name: time, dtype: datetime64 [ns] time year month day hour minute 0 2019-02-05 00:00:00 2019 2 5 0 0 1 2019-02-05 02:00:00 2019 2 5 2 0 2 2019-02-05 04:00 : 00 2019 2 5 4 0 3 2019-02-05 06:00:00 2019 2 5 6 0 4 2019-02-05 08:00:00 2019 2 5 8 0 5 2019-02-05 10:00:00 2019 2 5 10 0 


Convert strings to timestamps:

We convert string data to datetime format using pd.to_datetime and then we can extract various features from the first datetime method using uisng.

# Load libraries

import numpy as np

import pandas as pd

 
# Create time strings

dt_strings = np.array ([ ’ 04-03-2019 12:35 PM’ ,

’06/22/2017 11:01 AM’ ,

’09/05/2009 07:09 PM’ ])

 
# Convert to date and time format

timestamps = [pd.to_datetime (date, format = "% d-% m-% Y% I:% M% p " ,

  errors = " coerce " ) for date < / code> in dt_strings]

 
# Convert to datetimes

timestamps = [pd.to_datetime (date, format = "% d-% m-% Y% I:% M% p" ,

errors = "coerce" ) for date in dt_strings]

Exit:

[Timestamp (’2019-03-04 12:35:00’), Timestamp (’2017-06-22 11:01:00 ’), Timestamp (’ 2009-09-05 19:09:00 ’)]

Extract d no week from the specified date:
We use Series.dt.weekday_name to find the name of the day of the week from the specified date.

# Load library

import pandas as pd

df = pd.DataFrame ()

 
# Create 6 dates

dates = pd.pd.Series (date_range ( ’ 2 / 5 / 2019’ , periods = 6 , freq = ’M’ ) )

  

print (dates)

 
# Extract days of the week and then print

print (dates.dt.weekday_name)

Exit :

 0 2019-02-28 1 2019-03-31 2 2019-04-30 3 2019-05-31 4 2019-06-30 5 2019-07-31 dtype: datetime64 [ns] 0 Thursday 1 Sunday 2 Tuesday 3 Friday 4 Sunday 5 Wednesday dtype: object 

Extract data in date and time ranges:
We can get rows that lie in a certain time range from a given dataset.

Method # 1: If the dataset is not indexed over time.

# Load library

import pandas as pd

 
# Create data frame

df = pd.DataFrame ()

 
# Create date and time

df [ ’date’ ] = pd.date_range ( ’ 1/1 / 2012’ , periods = 1000 , freq = ’H’ )

 

print (df.head ( 5 ))

 
# Select observations between two dates

x = df [(df [ ’date’ ]"  ’2012-1-1 01: 00: 00’ ) & amp; 

(df [ ’date’ ] " = ’2012-1-1 11:00: 00’ )]

 

print (x)

Exit :

 date 0 2012-01-01 00:00:00 1 2012-01-01 01:00:00 // 5 rows of Timestamps out of 1000 2 2012-01-01 02:00:00 3 2012-01-01 03:00:00 4 2012-01-01 04:00:00 date 2 2012-01-01 02:00 : 00 3 2012-01-01 03:00:00 4 2012-01-01 04:00:00 5 2012-01-01 05:00:00 // Timestamps in the given range 6 2012-01-01 06: 00:00 7 2012-01-01 07:00:00 8 2012-01-01 08:00:00 9 2012-01-01 09:00:00 10 2012-01-01 10:00:00 11 2012- 01-01 11:00:00 

Method # 2: If the dataset is indexed over time

# Load library

import pandas as pd

 
# Create data frame

df = pd.DataFrame ()

 
# Create date and time

df [ ’date’ ] = pd.date_range ( ’ 1/1 / 2012’ , periods = 1000 , freq = ’H’ )

  # Set index

df = df.set_index (df [ ’date’ ])

 

print (df.head ( 5 ))

 
# Select observations between two dates

x = df.loc [ ’ 2012-1-1 04: 00: 00’ : ’2012-1-1 12: 00: 00’ ]

 

print (x)

Exit:

 date date 2 012-01-01 00:00:00 2012-01-01 00:00:00 2012-01-01 01:00:00 2012-01-01 01:00:00 2012-01-01 02:00:00 2012-01-01 02:00:00 2012-01-01 03:00:00 2012-01-01 03:00:00 // 5 rows of Timestamps out of 1000 2012-01-01 04:00:00 2012 -01-01 04:00:00 date date 2012-01-01 04:00:00 2012-01-01 04:00:00 2012-01-01 05:00:00 2012-01-01 05:00: 00 2012-01-01 06:00:00 2012-01-01 06:00:00 2012-01-01 07:00:00 2012-01-01 07:00:00 2012-01-01 08:00: 00 2012-01-01 08:00:00 2012-01-01 09:00:00 2012-01-01 09:00:00 // Timestamps in the given range 2012-01-01 10:00:00 2012- 01-01 10:00:00 2012-01-01 11:00:00 2012-01-01 11:00:00 2012-01-01 12:00:00 2012-01-01 12:00:00 

Shop

Learn programming in R: courses

$

Best Python online courses for 2022

$

Best laptop for Fortnite

$

Best laptop for Excel

$

Best laptop for Solidworks

$

Best laptop for Roblox

$

Best computer for crypto mining

$

Best laptop for Sims 4

$

Latest questions

NUMPYNUMPY

Common xlabel/ylabel for matplotlib subplots

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

12 answers

NUMPYNUMPY

Flake8: Ignore specific warning for entire file

12 answers

NUMPYNUMPY

glob exclude pattern

12 answers

NUMPYNUMPY

How to avoid HTTP error 429 (Too Many Requests) python

12 answers

NUMPYNUMPY

Python CSV error: line contains NULL byte

12 answers

NUMPYNUMPY

csv.Error: iterator should return strings, not bytes

12 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

sin

How to specify multiple return types using type-hints

exp

Printing words vertically in Python

exp

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

cos

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically