Python | Pandas.melt ()

melt | Python Methods and Functions

To make it easier to parse the data in the table, we can transform the data into a more computer-friendly form using Pandas in Python. Pandas.melt () — one of the functions that does this ..
Pandas.melt () expands the DataFrame from wide format to long format. 
The melt () function is useful for converting a DataFrame to a format in which one or more columns are identifier variables, while all other columns considered as measured variables are not. are rotated to the row axis, leaving only two columns with no identifier, a variable and a value. 
Syntax:

 pandas.melt (frame, id_vars = None, value_vars = None, var_name = None, value_name = 'value', col_level = None) 

Parameters:

frame: DataFrame
id_vars [tuple, list, or ndarray, optional]: Column (s) to use as identifier variables.
value_vars [tuple, list, or ndarray, optional]: Column (s) to unpivot. If not specified, uses all columns that are not set as id_vars.
var_name [scalar]: Name to use for the 'variable' column. If None it uses frame.columns.name or 'variable'.
value_name [scalar, default 'value']: Name to use for the 'value' column.
col_level [int or string, optional]: If columns are a MultiIndex then use this level to melt.

Example:

# Create a simple data frame

 
# import pandas as pd

import pandas as pd

 
# create data frame

df = pd.DataFrame ({ 'Name ' : { 0 : < code class = "string"> 'John' , 1 : 'Bob' , 2 : ' Shiela' },

' Course' : { 0 : 'Masters' , 1 : 'Graduate' , 2 : 'Graduate' },

'Age' : { 0 : 27 , 1 : 23 , 2 : 21 }})

df

# Name is id_vars and rate is value_vars.

pd.melt (df, id_vars = [ 'Name' ], value_vars = [ 'Course' ])

# multiple undo columns

pd.melt (df, id_vars = [ 'Name' ], value_vars = [ ' Course' , 'Age' ])

# Variable and value column names can be changed

pd.melt (df, id_vars = [ 'Name' ], value_vars = [ 'Course' ],

  var_name = 'ChangedVarname' , value_name = 'ChangedValname' )





Python | Pandas.melt (): StackOverflow Questions

Answer #1

UPDATE
From v0.20, melt is a first order function, you can now use

df.melt(id_vars=["location", "name"], 
        var_name="Date", 
        value_name="Value")

  location    name        Date  Value
0        A  "test"    Jan-2010     12
1        B   "foo"    Jan-2010     18
2        A  "test"    Feb-2010     20
3        B   "foo"    Feb-2010     20
4        A  "test"  March-2010     30
5        B   "foo"  March-2010     25

OLD(ER) VERSIONS: <0.20

You can use pd.melt to get most of the way there, and then sort:

>>> df
  location  name  Jan-2010  Feb-2010  March-2010
0        A  test        12        20          30
1        B   foo        18        20          25
>>> df2 = pd.melt(df, id_vars=["location", "name"], 
                  var_name="Date", value_name="Value")
>>> df2
  location  name        Date  Value
0        A  test    Jan-2010     12
1        B   foo    Jan-2010     18
2        A  test    Feb-2010     20
3        B   foo    Feb-2010     20
4        A  test  March-2010     30
5        B   foo  March-2010     25
>>> df2 = df2.sort(["location", "name"])
>>> df2
  location  name        Date  Value
0        A  test    Jan-2010     12
2        A  test    Feb-2010     20
4        A  test  March-2010     30
1        B   foo    Jan-2010     18
3        B   foo    Feb-2010     20
5        B   foo  March-2010     25

(Might want to throw in a .reset_index(drop=True), just to keep the output clean.)

Note: pd.DataFrame.sort has been deprecated in favour of pd.DataFrame.sort_values.

Answer #2

To save some folks some time, here is a list I extracted from a small corpus. I do not know if it is complete, but it should have most (if not all) of the help definitions from upenn_tagset...

CC: conjunction, coordinating

& "n and both but either et for less minus neither nor or plus so
therefore times v. versus vs. whether yet

CD: numeral, cardinal

mid-1890 nine-thirty forty-two one-tenth ten million 0.5 one forty-
seven 1987 twenty "79 zero two 78-degrees eighty-four IX "60s .025
fifteen 271,124 dozen quintillion DM2,000 ...

DT: determiner

all an another any both del each either every half la many much nary
neither no some such that the them these this those

EX: existential there

there

IN: preposition or conjunction, subordinating

astride among upon whether out inside pro despite on by throughout
below within for towards near behind atop around if like until below
next into if beside ...

JJ: adjective or numeral, ordinal

third ill-mannered pre-war regrettable oiled calamitous first separable
ectoplasmic battery-powered participatory fourth still-to-be-named
multilingual multi-disciplinary ...

JJR: adjective, comparative

bleaker braver breezier briefer brighter brisker broader bumper busier
calmer cheaper choosier cleaner clearer closer colder commoner costlier
cozier creamier crunchier cuter ...

JJS: adjective, superlative

calmest cheapest choicest classiest cleanest clearest closest commonest
corniest costliest crassest creepiest crudest cutest darkest deadliest
dearest deepest densest dinkiest ...

LS: list item marker

A A. B B. C C. D E F First G H I J K One SP-44001 SP-44002 SP-44005
SP-44007 Second Third Three Two * a b c d first five four one six three
two

MD: modal auxiliary

can cannot could couldn"t dare may might must need ought shall should
shouldn"t will would

NN: noun, common, singular or mass

common-carrier cabbage knuckle-duster Casino afghan shed thermostat
investment slide humour falloff slick wind hyena override subhumanity
machinist ...

NNP: noun, proper, singular

Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos
Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA
Shannon A.K.C. Meltex Liverpool ...

NNS: noun, common, plural

undergraduates scotches bric-a-brac products bodyguards facets coasts
divestitures storehouses designs clubs fragrances averages
subjectivists apprehensions muses factory-jobs ...

PDT: pre-determiner

all both half many quite such sure this

POS: genitive marker

" "s

PRP: pronoun, personal

hers herself him himself hisself it itself me myself one oneself ours
ourselves ownself self she thee theirs them themselves they thou thy us

PRP$: pronoun, possessive

her his mine my our ours their thy your

RB: adverb

occasionally unabatingly maddeningly adventurously professedly
stirringly prominently technologically magisterially predominately
swiftly fiscally pitilessly ...

RBR: adverb, comparative

further gloomier grander graver greater grimmer harder harsher
healthier heavier higher however larger later leaner lengthier less-
perfectly lesser lonelier longer louder lower more ...

RBS: adverb, superlative

best biggest bluntest earliest farthest first furthest hardest
heartiest highest largest least less most nearest second tightest worst

RP: particle

aboard about across along apart around aside at away back before behind
by crop down ever fast for forth from go high i.e. in into just later
low more off on open out over per pie raising start teeth that through
under unto up up-pp upon whole with you

TO: "to" as preposition or infinitive marker

to

UH: interjection

Goodbye Goody Gosh Wow Jeepers Jee-sus Hubba Hey Kee-reist Oops amen
huh howdy uh dammit whammo shucks heck anyways whodunnit honey golly
man baby diddle hush sonuvabitch ...

VB: verb, base form

ask assemble assess assign assume atone attention avoid bake balkanize
bank begin behold believe bend benefit bevel beware bless boil bomb
boost brace break bring broil brush build ...

VBD: verb, past tense

dipped pleaded swiped regummed soaked tidied convened halted registered
cushioned exacted snubbed strode aimed adopted belied figgered
speculated wore appreciated contemplated ...

VBG: verb, present participle or gerund

telegraphing stirring focusing angering judging stalling lactating
hankerin" alleging veering capping approaching traveling besieging
encrypting interrupting erasing wincing ...

VBN: verb, past participle

multihulled dilapidated aerosolized chaired languished panelized used
experimented flourished imitated reunifed factored condensed sheared
unsettled primed dubbed desired ...

VBP: verb, present tense, not 3rd person singular

predominate wrap resort sue twist spill cure lengthen brush terminate
appear tend stray glisten obtain comprise detest tease attract
emphasize mold postpone sever return wag ...

VBZ: verb, present tense, 3rd person singular

bases reconstructs marks mixes displeases seals carps weaves snatches
slumps stretches authorizes smolders pictures emerges stockpiles
seduces fizzes uses bolsters slaps speaks pleads ...

WDT: WH-determiner

that what whatever which whichever

WP: WH-pronoun

that what whatever whatsoever which who whom whosoever

WRB: Wh-adverb

how however whence whenever where whereby whereever wherein whereof why

Answer #3

I was asked the same question recently, and came up with several answers. I hope it"s OK to revive this thread, as I wanted to elaborate on a few of the use cases mentioned, and add a few new ones.

Most metaclasses I"ve seen do one of two things:

  1. Registration (adding a class to a data structure):

    models = {}
    
    class ModelMetaclass(type):
        def __new__(meta, name, bases, attrs):
            models[name] = cls = type.__new__(meta, name, bases, attrs)
            return cls
    
    class Model(object):
        __metaclass__ = ModelMetaclass
    

    Whenever you subclass Model, your class is registered in the models dictionary:

    >>> class A(Model):
    ...     pass
    ...
    >>> class B(A):
    ...     pass
    ...
    >>> models
    {"A": <__main__.A class at 0x...>,
     "B": <__main__.B class at 0x...>}
    

    This can also be done with class decorators:

    models = {}
    
    def model(cls):
        models[cls.__name__] = cls
        return cls
    
    @model
    class A(object):
        pass
    

    Or with an explicit registration function:

    models = {}
    
    def register_model(cls):
        models[cls.__name__] = cls
    
    class A(object):
        pass
    
    register_model(A)
    

    Actually, this is pretty much the same: you mention class decorators unfavorably, but it"s really nothing more than syntactic sugar for a function invocation on a class, so there"s no magic about it.

    Anyway, the advantage of metaclasses in this case is inheritance, as they work for any subclasses, whereas the other solutions only work for subclasses explicitly decorated or registered.

    >>> class B(A):
    ...     pass
    ...
    >>> models
    {"A": <__main__.A class at 0x...> # No B :(
    
  2. Refactoring (modifying class attributes or adding new ones):

    class ModelMetaclass(type):
        def __new__(meta, name, bases, attrs):
            fields = {}
            for key, value in attrs.items():
                if isinstance(value, Field):
                    value.name = "%s.%s" % (name, key)
                    fields[key] = value
            for base in bases:
                if hasattr(base, "_fields"):
                    fields.update(base._fields)
            attrs["_fields"] = fields
            return type.__new__(meta, name, bases, attrs)
    
    class Model(object):
        __metaclass__ = ModelMetaclass
    

    Whenever you subclass Model and define some Field attributes, they are injected with their names (for more informative error messages, for example), and grouped into a _fields dictionary (for easy iteration, without having to look through all the class attributes and all its base classes" attributes every time):

    >>> class A(Model):
    ...     foo = Integer()
    ...
    >>> class B(A):
    ...     bar = String()
    ...
    >>> B._fields
    {"foo": Integer("A.foo"), "bar": String("B.bar")}
    

    Again, this can be done (without inheritance) with a class decorator:

    def model(cls):
        fields = {}
        for key, value in vars(cls).items():
            if isinstance(value, Field):
                value.name = "%s.%s" % (cls.__name__, key)
                fields[key] = value
        for base in cls.__bases__:
            if hasattr(base, "_fields"):
                fields.update(base._fields)
        cls._fields = fields
        return cls
    
    @model
    class A(object):
        foo = Integer()
    
    class B(A):
        bar = String()
    
    # B.bar has no name :(
    # B._fields is {"foo": Integer("A.foo")} :(
    

    Or explicitly:

    class A(object):
        foo = Integer("A.foo")
        _fields = {"foo": foo} # Don"t forget all the base classes" fields, too!
    

    Although, on the contrary to your advocacy for readable and maintainable non-meta programming, this is much more cumbersome, redundant and error prone:

    class B(A):
        bar = String()
    
    # vs.
    
    class B(A):
        bar = String("bar")
        _fields = {"B.bar": bar, "A.foo": A.foo}
    

Having considered the most common and concrete use cases, the only cases where you absolutely HAVE to use metaclasses are when you want to modify the class name or list of base classes, because once defined, these parameters are baked into the class, and no decorator or function can unbake them.

class Metaclass(type):
    def __new__(meta, name, bases, attrs):
        return type.__new__(meta, "foo", (int,), attrs)

class Baseclass(object):
    __metaclass__ = Metaclass

class A(Baseclass):
    pass

class B(A):
    pass

print A.__name__ # foo
print B.__name__ # foo
print issubclass(B, A)   # False
print issubclass(B, int) # True

This may be useful in frameworks for issuing warnings whenever classes with similar names or incomplete inheritance trees are defined, but I can"t think of a reason beside trolling to actually change these values. Maybe David Beazley can.

Anyway, in Python 3, metaclasses also have the __prepare__ method, which lets you evaluate the class body into a mapping other than a dict, thus supporting ordered attributes, overloaded attributes, and other wicked cool stuff:

import collections

class Metaclass(type):

    @classmethod
    def __prepare__(meta, name, bases, **kwds):
        return collections.OrderedDict()

    def __new__(meta, name, bases, attrs, **kwds):
        print(list(attrs))
        # Do more stuff...

class A(metaclass=Metaclass):
    x = 1
    y = 2

# prints ["x", "y"] rather than ["y", "x"]

 

class ListDict(dict):
    def __setitem__(self, key, value):
        self.setdefault(key, []).append(value)

class Metaclass(type):

    @classmethod
    def __prepare__(meta, name, bases, **kwds):
        return ListDict()

    def __new__(meta, name, bases, attrs, **kwds):
        print(attrs["foo"])
        # Do more stuff...

class A(metaclass=Metaclass):

    def foo(self):
        pass

    def foo(self, x):
        pass

# prints [<function foo at 0x...>, <function foo at 0x...>] rather than <function foo at 0x...>

You might argue ordered attributes can be achieved with creation counters, and overloading can be simulated with default arguments:

import itertools

class Attribute(object):
    _counter = itertools.count()
    def __init__(self):
        self._count = Attribute._counter.next()

class A(object):
    x = Attribute()
    y = Attribute()

A._order = sorted([(k, v) for k, v in vars(A).items() if isinstance(v, Attribute)],
                  key = lambda (k, v): v._count)

 

class A(object):

    def _foo0(self):
        pass

    def _foo1(self, x):
        pass

    def foo(self, x=None):
        if x is None:
            return self._foo0()
        else:
            return self._foo1(x)

Besides being much more ugly, it"s also less flexible: what if you want ordered literal attributes, like integers and strings? What if None is a valid value for x?

Here"s a creative way to solve the first problem:

import sys

class Builder(object):
    def __call__(self, cls):
        cls._order = self.frame.f_code.co_names
        return cls

def ordered():
    builder = Builder()
    def trace(frame, event, arg):
        builder.frame = frame
        sys.settrace(None)
    sys.settrace(trace)
    return builder

@ordered()
class A(object):
    x = 1
    y = "foo"

print A._order # ["x", "y"]

And here"s a creative way to solve the second one:

_undefined = object()

class A(object):

    def _foo0(self):
        pass

    def _foo1(self, x):
        pass

    def foo(self, x=_undefined):
        if x is _undefined:
            return self._foo0()
        else:
            return self._foo1(x)

But this is much, MUCH voodoo-er than a simple metaclass (especially the first one, which really melts your brain). My point is, you look at metaclasses as unfamiliar and counter-intuitive, but you can also look at them as the next step of evolution in programming languages: you just have to adjust your mindset. After all, you could probably do everything in C, including defining a struct with function pointers and passing it as the first argument to its functions. A person seeing C++ for the first time might say, "what is this magic? Why is the compiler implicitly passing this to methods, but not to regular and static functions? It"s better to be explicit and verbose about your arguments". But then, object-oriented programming is much more powerful once you get it; and so is this, uh... quasi-aspect-oriented programming, I guess. And once you understand metaclasses, they"re actually very simple, so why not use them when convenient?

And finally, metaclasses are rad, and programming should be fun. Using standard programming constructs and design patterns all the time is boring and uninspiring, and hinders your imagination. Live a little! Here"s a metametaclass, just for you.

class MetaMetaclass(type):
    def __new__(meta, name, bases, attrs):
        def __new__(meta, name, bases, attrs):
            cls = type.__new__(meta, name, bases, attrs)
            cls._label = "Made in %s" % meta.__name__
            return cls 
        attrs["__new__"] = __new__
        return type.__new__(meta, name, bases, attrs)

class China(type):
    __metaclass__ = MetaMetaclass

class Taiwan(type):
    __metaclass__ = MetaMetaclass

class A(object):
    __metaclass__ = China

class B(object):
    __metaclass__ = Taiwan

print A._label # Made in China
print B._label # Made in Taiwan

Edit

This is a pretty old question, but it"s still getting upvotes, so I thought I"d add a link to a more comprehensive answer. If you"d like to read more about metaclasses and their uses, I"ve just published an article about it here.

Answer #4

If I may provide some historical context :

The term "slug" has to do with casting metal—lead, in this case—out of which the press fonts were made. Every paper then had its fonts factory regularly re-melted and recast in fresh molds, since after many prints they became worn out. Apprentices like me started their career there, and went all the way to the top (not anymore).

Typographs had to compose the text of an article in a backward manner with lead characters stacked in a wise. So at printing time the letters would be straight on the paper. All typographs could read the newspaper mirrored as fast as the printed one. Therefore the slugs, (like snails) also the slow stories (the last to be fixed) were many on the bench waiting, solely identified by their fist letters, mostly the whole title generally more readable. Some "hot" news were waiting there on the bench, for possible last minute correction, (Evening paper) before last assembly and definitive printing.

Django emerged from the offices of the Lawrence journal in Kansas. Where probably some printing jargon still lingers. A-django-enthusiast-&-friendly-old-slug-boy-from-France.

Tutorials