pandas 之 索引重塑

import numpy as np 
import pandas as pd

There are a number of basic operations for rearanging tabular data. These are alternatingly referred to as reshape or pivot operations.

多层索引重塑

Hierarchical indexing provides a consistent way to rearrange data in a DataFrame. There are two primary actions:

stack - 列拉长index
​ This "rotates" or pivots from the columns in the data to the rows.

unstack
​ This pivots from the rows into the columns.
​
I'll illustrate these operations through a series of examples. Consider a small DataFrame with string arrays as row and column indexes:
​

data = pd.DataFrame(np.arange(6).reshape((2, 3)),
    index=pd.Index(['Ohio', 'Colorado'], name='state'),
    columns=pd.Index(['one', 'two', 'three'],
    name='number'))

data

Using the stack method on this data pivots the columns into the rows, producing a Series.

"stack 将每一行, 叠成一个Series, 堆起来"
result = data.stack()

result

'stack 将每一行, 叠成一个Series, 堆起来'






state     number
Ohio      one       0
          two       1
          three     2
Colorado  one       3
          two       4
          three     5
dtype: int32

From a hierarchically indexed Series, you can rearrage the data back into a DataFrame with unstack

"unstack 将叠起来的Series, 变回DF"

result.unstack()

'unstack 将叠起来的Series, 变回DF'

By default the innermost level is unstacked(same with stack). You can unstack a different level by passing a level number or name.

result.unstack(level=0)

result.unstack(level='state')

Unstacking might introduce missing data if all of the values in the level aren't found in each of the subgroups.

s1 = pd.Series([0, 1, 2, 3], index=['a', 'b', 'c', 'd'])

s2 = pd.Series([4, 5, 6], index=['c', 'd', 'e'])

data2 = pd.concat([s1, s2], keys=['one', 'two'])

data2

one  a    0
     b    1
     c    2
     d    3
two  c    4
     d    5
     e    6
dtype: int64

data2.unstack()  # 外连接哦

%time data2.unstack().stack()

Wall time: 5 ms





one  a    0.0
     b    1.0
     c    2.0
     d    3.0
two  c    4.0
     d    5.0
     e    6.0
dtype: float64

%time data2.unstack().stack(dropna=False)

Wall time: 3 ms





one  a    0.0
     b    1.0
     c    2.0
     d    3.0
     e    NaN
two  a    NaN
     b    NaN
     c    4.0
     d    5.0
     e    6.0
dtype: float64

When you unstack in a DataFrame, the level unstacked becomes the lowest level in the result:

df = pd.DataFrame({'left': result, 'right': result + 5},
columns=pd.Index(['left', 'right'], name='side'))

df

df.unstack("state")

When calling stack, we can indicate the name of the axis to stack:

%time df.unstack('state').stack('side')

Wall time: 118 ms

长转宽形

A common way to store multiple time series in databases and CSV is in so-called long or stacked format. Let's load some example data and do a small amonut of time series wrangling and other data cleaning:

%%time

data = pd.read_csv("../examples/macrodata.csv")

data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 203 entries, 0 to 202
Data columns (total 14 columns):
year        203 non-null float64
quarter     203 non-null float64
realgdp     203 non-null float64
realcons    203 non-null float64
realinv     203 non-null float64
realgovt    203 non-null float64
realdpi     203 non-null float64
cpi         203 non-null float64
m1          203 non-null float64
tbilrate    203 non-null float64
unemp       203 non-null float64
pop         203 non-null float64
infl        203 non-null float64
realint     203 non-null float64
dtypes: float64(14)
memory usage: 22.3 KB
Wall time: 142 ms

data.head()

periods = pd.PeriodIndex(year=data.year, quarter=data.quarter, name='date')

columns = pd.Index(['realgdp', 'infl', 'unemp'], name='item')

# 修改列索引名
data = data.reindex(columns=columns)

data.index = periods.to_timestamp('D', 'end')

ldata = data.stack().reset_index().rename(columns={0:'value'})


ldata[:10]

This is so-called long format for multiple time series, or other observational data with two or more keys. Each row in the table represents a single observation.

Data is frequently stored this way in relational databases like MySQL, as a fixed schema allows the number of distinct values in the item columns to change as data is added to the table. In the previous example, date and keys offering both relational integrity and easier joins. In some cases, the data may be more difficult to work with in this format; you might prefer to have a DataFrame containing one column per distinct item value indexed by timestamps in the date column. DataFrame's pivot method performs exactly this transformation:

pivoted = ldata.pivot('date', 'item', 'value')

pivoted[:5]

The first two values passed are the columns to be used respectively as the row and column index, then finally an optional value column to fill the DataFrame. Suppose you had two value columns that you wanted to reshape simultaneously:

ldata['valu2'] = np.random.randn(len(ldata))

ldata[:10]

By omitting the last argument, you obtain a DataFrame with hierarchical columns:

Wide to Long

An inverse operation to pivot for DataFrame is pandas.melt. Rather than transroming one columns into many in a new DataFrame, it merges multiple columns into one, producing a DataFrame that is longer than the input, Let's look at an example:

df = pd.DataFrame({
    'key': ['foo', 'bar', 'baz'],
    'A':[1,2,3],
    'B':[4,5,6],
    'C':[7,8,9]
})

df

The 'key' columns may be a group indicator, and the other columns are data values. When using pandas.melt, we must indicate which colmuns are group indicators Let's use 'key' as the only group indicator here:

melted = pd.melt(df, ['key'])

melted

Using pivot, we can reshape back to the original layout:(布局)

reshaped = melted.pivot('key', 'variable', 'value')

reshaped

Since the result of pivot creats an index from the column used as the row labels, we may want to use reset_index to move the data back into a column:

reshaped.reset_index()

You can also specify a subset of columns to use as value columns:

pd.melt(df, id_vars=['key'], value_vars=['A', 'B'])

pandas.melt can be used without any group identifiers, too:

pd.melt(df, value_vars=['A', 'B', 'C'])

pd.melt(df, value_vars=['key', 'A', 'B'])

小结

Now that you have some pandas basics for data import, clearning, and reorganization under your belt, we are ready to move on to data visualization with matplotlib. We will return to pandas later in the book when we discuss more advance analytics.

原文地址：https://www.cnblogs.com/chenjieyouge/p/11945169.html