发帖
雷达卡
n R community, there’s this one guy, Hadley Wickam, who by himself made R great again. One of the many, many things he came up with - so many they call it a hadleyverse - is the dplyr package, which aims to make data analysis easy and fast. It works by allowing a user to take a data frame and apply to it a pipeline of operations resulting in a desired outcome (an example in just a minute). This approach is a good match for the mental model some data scientists have and turned out to be successful. Then people have ported key pieces to Pandas.
This is a draft. Come back later for the final version.
There’s an interesting story about how Hadley invented all those things. It goes like this. An angel - some say it was a daemon, but don’t believe them, it was an angel - visited our hero in a dream and said: ”I will give you some ideas that will make you rich and famous - well, rich intellectually and famous in the R community - but there’s a catch. For reasons I won’t disclose, for a mortal like you wouldn’t really understand, you must make the piping operator as bad as you possibly can, but without rendering it outright ridiculous. No more than three chars, and remember - as ugly and hard to type as you can.”
Hadley agreed and woke up. Being a smart guy that he is, in the morning he constructed an appropriate bundle of characters. Reportedly, the thought process unfolded as follows:
Okay, three characters. Let’s invert that old <-, and elongate it: -->. Meh, waaay too pretty and you type it all with one hand.
I know… ~~> is good. ~>~ even better. One needs to press tilde key twice, then delete one, go to >, repeat, all with Shift pressed. Sweet. But dang, too pretty. I need ugly.
Think, man, think! Let’s see. #>#. Yeah. Nah, that looks half-reasonable. Wait… wait… yes… %>%. That’s it!
[color=rgb(255, 255, 255) !important]
Image credit: Dexter’s Laboratory
And the rest is history:
carriers_db2 %>% summarise(delay = mean(arr_delay)) %>% collect()But seriously, Hadley says that’s all because infix operators in R they must have the form %something%.
In PandasWe know of three modules for piping in Pandas: pandas-ply, dplython and dfply. People porting dplyr to Python didn’t have Hadley’s obligations, so all three use reasonable operators.
pandas-ply, from Coursera, is the simplest of them and closest to the Pandas spirit. It uses a normal dot for chaining and just adds a few methods to the DataFrame. Here’s their motivating example, copied from the dplyr intro:
grouped_flights = flights.groupby(['year', 'month', 'day']) output = pd.DataFrame() output['arr'] = grouped_flights.arr_delay.mean() output['dep'] = grouped_flights.dep_delay.mean() filtered_output = output[(output.arr > 30) & (output.dep > 30)] # instead: (flights .groupby(['year', 'month', 'day']) .ply_select( arr = X.arr_delay.mean(), dep = X.dep_delay.mean()) .ply_where(X.arr > 30, X.dep > 30))Less typing and no intermediate artifacts. Notice how you refer to the transformed dataframe inside the pipeline by X.
dplython is closer to dplyr. The module provides verbs (functions) similar to the R counterpart, but the pipeline operator is a handsome >>:
(diamonds >> sample_n(10) >> arrange(X.carat) >> select(X.carat, X.cut, X.depth, X.price)) (diamonds >> mutate(carat_bin=X.carat.round()) >> group_by(X.cut, X.carat_bin) >> summarize(avg_price=X.price.mean()))By the way, what’s with the outer parens? Is this Lisp or something? We hate parens in Lisp.
Let us mention that in R, all functions are pipable. In Python, you need to make them pipable. dlpython has a special decorator for it, @DelayFunction.
Finally, there is dfply, inspired by dlpython, but with even more functions. It appears less mature than the previous two - pip install won’t work here.
diamonds >> drop_endswith('e','y','z') >> head(2)What these modules provide is mostly syntactic sugar, and using them depends on a personal taste. For example, while the pandas-ply flights example above is convincing, is one of these lines better than the others?
diamonds >> sift(X.carat > 4) >> select(X.carat, X.cut, X.depth, X.price) diamonds.ply_where(X.carat > 4).ply_select('carat', 'cut', 'depth', 'price') diamonds[diamonds.carat > 4]['carat', 'cut', 'depth', 'price']
提升卡
置顶卡
沉默卡
变色卡
抢沙发
千斤顶
显身卡
京公网安备 11010802022788号
