Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone

| | |

👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!

You can use the function tz_localize to make a Timestamp or DateTimeIndex timezone aware, but how can you do the opposite: how can you convert a timezone aware Timestamp to a naive one, while preserving its timezone?

An example:

In [82]: t = pd.date_range(start="2013-05-18 12:00:00", periods=10, freq="s", tz="Europe/Brussels")

In [83]: t
Out[83]: 
<class "pandas.tseries.index.DatetimeIndex">
[2013-05-18 12:00:00, ..., 2013-05-18 12:00:09]
Length: 10, Freq: S, Timezone: Europe/Brussels

I could remove the timezone by setting it to None, but then the result is converted to UTC (12 o"clock became 10):

In [86]: t.tz = None

In [87]: t
Out[87]: 
<class "pandas.tseries.index.DatetimeIndex">
[2013-05-18 10:00:00, ..., 2013-05-18 10:00:09]
Length: 10, Freq: S, Timezone: None

Is there another way I can convert a DateTimeIndex to timezone naive, but while preserving the timezone it was set in?


Some context on the reason I am asking this: I want to work with timezone naive timeseries (to avoid the extra hassle with timezones, and I do not need them for the case I am working on).
But for some reason, I have to deal with a timezone-aware timeseries in my local timezone (Europe/Brussels). As all my other data are timezone naive (but represented in my local timezone), I want to convert this timeseries to naive to further work with it, but it also has to be represented in my local timezone (so just remove the timezone info, without converting the user-visible time to UTC).

I know the time is actually internal stored as UTC and only converted to another timezone when you represent it, so there has to be some kind of conversion when I want to "delocalize" it. For example, with the python datetime module you can "remove" the timezone like this:

In [119]: d = pd.Timestamp("2013-05-18 12:00:00", tz="Europe/Brussels")

In [120]: d
Out[120]: <Timestamp: 2013-05-18 12:00:00+0200 CEST, tz=Europe/Brussels>

In [121]: d.replace(tzinfo=None)
Out[121]: <Timestamp: 2013-05-18 12:00:00> 

So, based on this, I could do the following, but I suppose this will not be very efficient when working with a larger timeseries:

In [124]: t
Out[124]: 
<class "pandas.tseries.index.DatetimeIndex">
[2013-05-18 12:00:00, ..., 2013-05-18 12:00:09]
Length: 10, Freq: S, Timezone: Europe/Brussels

In [125]: pd.DatetimeIndex([i.replace(tzinfo=None) for i in t])
Out[125]: 
<class "pandas.tseries.index.DatetimeIndex">
[2013-05-18 12:00:00, ..., 2013-05-18 12:00:09]
Length: 10, Freq: None, Timezone: None

👻 Read also: what is the best laptop for engineering students?

Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone __del__: Questions

How can I make a time delay in Python?

5 answers

I would like to know how to put a time delay in a Python script.

2973

Answer #1

import time
time.sleep(5)   # Delays for 5 seconds. You can also use a float value.

Here is another example where something is run approximately once a minute:

import time
while True:
    print("This prints once a minute.")
    time.sleep(60) # Delay for 1 minute (60 seconds).

2973

Answer #2

You can use the sleep() function in the time module. It can take a float argument for sub-second resolution.

from time import sleep
sleep(0.1) # Time in seconds

Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone __del__: Questions

How to delete a file or folder in Python?

5 answers

How do I delete a file or folder in Python?

2639

Answer #1


Path objects from the Python 3.4+ pathlib module also expose these instance methods:

How do I filter query objects by date range in Django?

2 answers

I"ve got a field in one model like:

class Sample(models.Model):
    date = fields.DateField(auto_now=False)

Now, I need to filter the objects by a date range.

How do I filter all the objects that have a date between 1-Jan-2011 and 31-Jan-2011?

299

Answer #1

Use

Sample.objects.filter(date__range=["2011-01-01", "2011-01-31"])

Or if you are just trying to filter month wise:

Sample.objects.filter(date__year="2011", 
                      date__month="01")

Edit

As Bernhard Vallant said, if you want a queryset which excludes the specified range ends you should consider his solution, which utilizes gt/lt (greater-than/less-than).

299

Answer #2

You can use django"s filter with datetime.date objects:

import datetime
samples = Sample.objects.filter(sampledate__gte=datetime.date(2011, 1, 1),
                                sampledate__lte=datetime.date(2011, 1, 31))

We hope this article has helped you to resolve the problem. Apart from Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone, check other __del__-related topics.

Want to excel in Python? See our review of the best Python online courses 2022. If you are interested in Data Science, check also how to learn programming in R.

By the way, this material is also available in other languages:



Jan Galleotti

New York | 2022-12-05

Thanks for explaining! I was stuck with Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone for some hours, finally got it done 🤗. Will get back tomorrow with feedback

Marie Schteiner

Warsaw | 2022-12-05

Thanks for explaining! I was stuck with Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone for some hours, finally got it done 🤗. Will use it in my bachelor thesis

Frank Jackson

Moscow | 2022-12-05

Thanks for explaining! I was stuck with Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone for some hours, finally got it done 🤗. I am just not quite sure it is the best method

Shop

Learn programming in R: courses

$

Best Python online courses for 2022

$

Best laptop for Fortnite

$

Best laptop for Excel

$

Best laptop for Solidworks

$

Best laptop for Roblox

$

Best computer for crypto mining

$

Best laptop for Sims 4

$

Latest questions

NUMPYNUMPY

Common xlabel/ylabel for matplotlib subplots

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

12 answers

NUMPYNUMPY

Flake8: Ignore specific warning for entire file

12 answers

NUMPYNUMPY

glob exclude pattern

12 answers

NUMPYNUMPY

How to avoid HTTP error 429 (Too Many Requests) python

12 answers

NUMPYNUMPY

Python CSV error: line contains NULL byte

12 answers

NUMPYNUMPY

csv.Error: iterator should return strings, not bytes

12 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

sin

How to specify multiple return types using type-hints

exp

Printing words vertically in Python

exp

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

cos

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically