Dataframe groupby aggregation for percentile
up vote
0
down vote
favorite
Suppose I have a dataframe with columns as:
JobTitle, Age, Stats and Date. Goal is to group it by JobTitle and Age and apply the Aggregation functions to Stats and Date. Stats column will be read from a config file. If it is empty, the default will be 'mean', else it will take the user defined number for percentile.
This is what I have done:
import pandas as pd
import numpy as np
aggregate_dict='Stats':'Mean', 'Date':'min'
for i in range(0,df.shape[0]):
if df.Stats[i]:
temp_StatsName = df.Stats[i]
aggregate_dict='Stats':'percentile('+temp_StatsName+')', 'Date':'min'
df_final=df.groupby(['JobTitle','Age']).agg(aggregate_dict).reset_index()
Besides this, I have also tried creating my own percentile function as so and used it in the aggregate_dict definition but was not successful:
def percentile(n):
def percentile_(x):
return np.percentile(x, n)
percentile_.__name__ = 'percentile_%s' % n
return percentile_
If you may have a suggestion how to implement user defined percentiles when the Stats column is not empty, that would be very helpful. My code may not be ideal, I am relatively new in Python. Thanks!
python pandas aggregate pandas-groupby
add a comment |
up vote
0
down vote
favorite
Suppose I have a dataframe with columns as:
JobTitle, Age, Stats and Date. Goal is to group it by JobTitle and Age and apply the Aggregation functions to Stats and Date. Stats column will be read from a config file. If it is empty, the default will be 'mean', else it will take the user defined number for percentile.
This is what I have done:
import pandas as pd
import numpy as np
aggregate_dict='Stats':'Mean', 'Date':'min'
for i in range(0,df.shape[0]):
if df.Stats[i]:
temp_StatsName = df.Stats[i]
aggregate_dict='Stats':'percentile('+temp_StatsName+')', 'Date':'min'
df_final=df.groupby(['JobTitle','Age']).agg(aggregate_dict).reset_index()
Besides this, I have also tried creating my own percentile function as so and used it in the aggregate_dict definition but was not successful:
def percentile(n):
def percentile_(x):
return np.percentile(x, n)
percentile_.__name__ = 'percentile_%s' % n
return percentile_
If you may have a suggestion how to implement user defined percentiles when the Stats column is not empty, that would be very helpful. My code may not be ideal, I am relatively new in Python. Thanks!
python pandas aggregate pandas-groupby
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Suppose I have a dataframe with columns as:
JobTitle, Age, Stats and Date. Goal is to group it by JobTitle and Age and apply the Aggregation functions to Stats and Date. Stats column will be read from a config file. If it is empty, the default will be 'mean', else it will take the user defined number for percentile.
This is what I have done:
import pandas as pd
import numpy as np
aggregate_dict='Stats':'Mean', 'Date':'min'
for i in range(0,df.shape[0]):
if df.Stats[i]:
temp_StatsName = df.Stats[i]
aggregate_dict='Stats':'percentile('+temp_StatsName+')', 'Date':'min'
df_final=df.groupby(['JobTitle','Age']).agg(aggregate_dict).reset_index()
Besides this, I have also tried creating my own percentile function as so and used it in the aggregate_dict definition but was not successful:
def percentile(n):
def percentile_(x):
return np.percentile(x, n)
percentile_.__name__ = 'percentile_%s' % n
return percentile_
If you may have a suggestion how to implement user defined percentiles when the Stats column is not empty, that would be very helpful. My code may not be ideal, I am relatively new in Python. Thanks!
python pandas aggregate pandas-groupby
Suppose I have a dataframe with columns as:
JobTitle, Age, Stats and Date. Goal is to group it by JobTitle and Age and apply the Aggregation functions to Stats and Date. Stats column will be read from a config file. If it is empty, the default will be 'mean', else it will take the user defined number for percentile.
This is what I have done:
import pandas as pd
import numpy as np
aggregate_dict='Stats':'Mean', 'Date':'min'
for i in range(0,df.shape[0]):
if df.Stats[i]:
temp_StatsName = df.Stats[i]
aggregate_dict='Stats':'percentile('+temp_StatsName+')', 'Date':'min'
df_final=df.groupby(['JobTitle','Age']).agg(aggregate_dict).reset_index()
Besides this, I have also tried creating my own percentile function as so and used it in the aggregate_dict definition but was not successful:
def percentile(n):
def percentile_(x):
return np.percentile(x, n)
percentile_.__name__ = 'percentile_%s' % n
return percentile_
If you may have a suggestion how to implement user defined percentiles when the Stats column is not empty, that would be very helpful. My code may not be ideal, I am relatively new in Python. Thanks!
python pandas aggregate pandas-groupby
python pandas aggregate pandas-groupby
edited Nov 11 at 0:20
asked Nov 10 at 20:54
shaucha
46111
46111
add a comment |
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53243319%2fdataframe-groupby-aggregation-for-percentile%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown