How to deal with SettingWithCopyWarning in Pandas

Michael Zippo

👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!

Background

I just upgraded my Pandas from 0.11 to 0.13.0rc1. Now, the application is popping out many new warnings. One of them like this:

E:FinReporterFM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df["TVol"]   = quote_df["TVol"]/TVOL_SCALE

I want to know what exactly it means? Do I need to change something?

How should I suspend the warning if I insist to use quote_df["TVol"] = quote_df["TVol"]/TVOL_SCALE?

The function that gives errors

def _decode_stock_quote(list_of_150_stk_str):
    """decode the webpage and return dataframe"""

    from cStringIO import StringIO

    str_of_all = "".join(list_of_150_stk_str)

    quote_df = pd.read_csv(StringIO(str_of_all), sep=",", names=list("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg")) #dtype={"A": object, "B": object, "C": np.float64}
    quote_df.rename(columns={"A":"STK", "B":"TOpen", "C":"TPCLOSE", "D":"TPrice", "E":"THigh", "F":"TLow", "I":"TVol", "J":"TAmt", "e":"TDate", "f":"TTime"}, inplace=True)
    quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]
    quote_df["TClose"] = quote_df["TPrice"]
    quote_df["RT"]     = 100 * (quote_df["TPrice"]/quote_df["TPCLOSE"] - 1)
    quote_df["TVol"]   = quote_df["TVol"]/TVOL_SCALE
    quote_df["TAmt"]   = quote_df["TAmt"]/TAMT_SCALE
    quote_df["STK_ID"] = quote_df["STK"].str.slice(13,19)
    quote_df["STK_Name"] = quote_df["STK"].str.slice(21,30)#.decode("gb2312")
    quote_df["TDate"]  = quote_df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10])
    
    return quote_df

More error messages

E:FinReporterFM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df["TVol"]   = quote_df["TVol"]/TVOL_SCALE
E:FinReporterFM_EXT.py:450: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df["TAmt"]   = quote_df["TAmt"]/TAMT_SCALE
E:FinReporterFM_EXT.py:453: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df["TDate"]  = quote_df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10])

👻 Read also: what is the best laptop for engineering students?

We hope this article has helped you to resolve the problem. Apart from How to deal with SettingWithCopyWarning in Pandas, check other abc Python module-related topics.

Want to excel in Python? See our review of the best Python online courses 2023. If you are interested in Data Science, check also how to learn programming in R.

By the way, this material is also available in other languages:

Carlo OConnell

San Francisco | 2023-03-24

Simply put and clear. Thank you for sharing. How to deal with SettingWithCopyWarning in Pandas and other issues with warnings Python module was always my weak point 😁. I am just not quite sure it is the best method

Carlo Lehnman

Paris | 2023-03-24

Thanks for explaining! I was stuck with How to deal with SettingWithCopyWarning in Pandas for some hours, finally got it done 🤗. Will get back tomorrow with feedback

Anna Krasiko

Milan | 2023-03-24

Simply put and clear. Thank you for sharing. How to deal with SettingWithCopyWarning in Pandas and other issues with StackOverflow was always my weak point 😁. Will get back tomorrow with feedback