Python - pandas explode rows by turns
I have a dataframe as below.
df = DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
Now I extracted letters from B1-B3 and add to new columns U1-U3 get:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A A
2 41A 28A 3A A A A
and I want to let the row to explode like this:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A
2 3A 1A A
3 41A 28A 3A A
4 41A 28A 3A A
5 41A 28A 3A A
Thanks in advance
python pandas dataframe
add a comment |
I have a dataframe as below.
df = DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
Now I extracted letters from B1-B3 and add to new columns U1-U3 get:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A A
2 41A 28A 3A A A A
and I want to let the row to explode like this:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A
2 3A 1A A
3 41A 28A 3A A
4 41A 28A 3A A
5 41A 28A 3A A
Thanks in advance
python pandas dataframe
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22
add a comment |
I have a dataframe as below.
df = DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
Now I extracted letters from B1-B3 and add to new columns U1-U3 get:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A A
2 41A 28A 3A A A A
and I want to let the row to explode like this:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A
2 3A 1A A
3 41A 28A 3A A
4 41A 28A 3A A
5 41A 28A 3A A
Thanks in advance
python pandas dataframe
I have a dataframe as below.
df = DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
Now I extracted letters from B1-B3 and add to new columns U1-U3 get:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A A
2 41A 28A 3A A A A
and I want to let the row to explode like this:
B1 B2 B3 U1 U2 U3
0 1C C
1 3A 1A A
2 3A 1A A
3 41A 28A 3A A
4 41A 28A 3A A
5 41A 28A 3A A
Thanks in advance
python pandas dataframe
python pandas dataframe
edited Nov 15 '18 at 9:39
xiumpt
asked Nov 15 '18 at 7:34
xiumptxiumpt
565
565
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22
add a comment |
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22
add a comment |
1 Answer
1
active
oldest
votes
I think, it needs 3 step solution of
1) extracting the Alphabates from data and creating new columns,
2) duplicating the rows w.r.t values and
3) masking with identity matrix.
df = pd.DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
1) Extracting the Alphabates from the rows and assigning as columns
df = df.merge(df.apply(lambda x: x.str.extract('([A-Za-z])')).add_prefix('U_'), left_index=True,right_index=True,how='outer')
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C NaN NaN
1 3A 1A A A NaN
2 41A 28A 3A A A A
2) You can try of duplicating the rows
of dataframe wherever it has more than 1 value
# Duplicating the rows of dataframe
val = df[['U_B1','U_B2','U_B3']].notnull().sum(axis=1)
df1 = df.loc[np.repeat(val.index,val)]
-> 3) then by grouping with index, pick only masked values of identity matrix
(np.identity) w.r.t length of each group.
df1[['U_B1','U_B2','U_B3']] = df1.groupby(df1.index)['U_B1','U_B2','U_B3'].apply(lambda x: x.dropna(axis=1).mask(np.identity(len(x))==0))
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C
1 3A 1A A
1 3A 1A A
2 41A 28A 3A A
2 41A 28A 3A A
2 41A 28A 3A A
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the linedf = df.merge
it showsIf using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
|
show 2 more comments
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53314441%2fpython-pandas-explode-rows-by-turns%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think, it needs 3 step solution of
1) extracting the Alphabates from data and creating new columns,
2) duplicating the rows w.r.t values and
3) masking with identity matrix.
df = pd.DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
1) Extracting the Alphabates from the rows and assigning as columns
df = df.merge(df.apply(lambda x: x.str.extract('([A-Za-z])')).add_prefix('U_'), left_index=True,right_index=True,how='outer')
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C NaN NaN
1 3A 1A A A NaN
2 41A 28A 3A A A A
2) You can try of duplicating the rows
of dataframe wherever it has more than 1 value
# Duplicating the rows of dataframe
val = df[['U_B1','U_B2','U_B3']].notnull().sum(axis=1)
df1 = df.loc[np.repeat(val.index,val)]
-> 3) then by grouping with index, pick only masked values of identity matrix
(np.identity) w.r.t length of each group.
df1[['U_B1','U_B2','U_B3']] = df1.groupby(df1.index)['U_B1','U_B2','U_B3'].apply(lambda x: x.dropna(axis=1).mask(np.identity(len(x))==0))
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C
1 3A 1A A
1 3A 1A A
2 41A 28A 3A A
2 41A 28A 3A A
2 41A 28A 3A A
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the linedf = df.merge
it showsIf using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
|
show 2 more comments
I think, it needs 3 step solution of
1) extracting the Alphabates from data and creating new columns,
2) duplicating the rows w.r.t values and
3) masking with identity matrix.
df = pd.DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
1) Extracting the Alphabates from the rows and assigning as columns
df = df.merge(df.apply(lambda x: x.str.extract('([A-Za-z])')).add_prefix('U_'), left_index=True,right_index=True,how='outer')
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C NaN NaN
1 3A 1A A A NaN
2 41A 28A 3A A A A
2) You can try of duplicating the rows
of dataframe wherever it has more than 1 value
# Duplicating the rows of dataframe
val = df[['U_B1','U_B2','U_B3']].notnull().sum(axis=1)
df1 = df.loc[np.repeat(val.index,val)]
-> 3) then by grouping with index, pick only masked values of identity matrix
(np.identity) w.r.t length of each group.
df1[['U_B1','U_B2','U_B3']] = df1.groupby(df1.index)['U_B1','U_B2','U_B3'].apply(lambda x: x.dropna(axis=1).mask(np.identity(len(x))==0))
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C
1 3A 1A A
1 3A 1A A
2 41A 28A 3A A
2 41A 28A 3A A
2 41A 28A 3A A
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the linedf = df.merge
it showsIf using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
|
show 2 more comments
I think, it needs 3 step solution of
1) extracting the Alphabates from data and creating new columns,
2) duplicating the rows w.r.t values and
3) masking with identity matrix.
df = pd.DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
1) Extracting the Alphabates from the rows and assigning as columns
df = df.merge(df.apply(lambda x: x.str.extract('([A-Za-z])')).add_prefix('U_'), left_index=True,right_index=True,how='outer')
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C NaN NaN
1 3A 1A A A NaN
2 41A 28A 3A A A A
2) You can try of duplicating the rows
of dataframe wherever it has more than 1 value
# Duplicating the rows of dataframe
val = df[['U_B1','U_B2','U_B3']].notnull().sum(axis=1)
df1 = df.loc[np.repeat(val.index,val)]
-> 3) then by grouping with index, pick only masked values of identity matrix
(np.identity) w.r.t length of each group.
df1[['U_B1','U_B2','U_B3']] = df1.groupby(df1.index)['U_B1','U_B2','U_B3'].apply(lambda x: x.dropna(axis=1).mask(np.identity(len(x))==0))
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C
1 3A 1A A
1 3A 1A A
2 41A 28A 3A A
2 41A 28A 3A A
2 41A 28A 3A A
I think, it needs 3 step solution of
1) extracting the Alphabates from data and creating new columns,
2) duplicating the rows w.r.t values and
3) masking with identity matrix.
df = pd.DataFrame(['B1': '1C', 'B2': '', 'B3': '', ,
'B1': '3A', 'B2': '1A', 'B3': '',
'B1': '41A', 'B2': '28A', 'B3': '3A'])
B1 B2 B3
0 1C
1 3A 1A
2 41A 28A 3A
1) Extracting the Alphabates from the rows and assigning as columns
df = df.merge(df.apply(lambda x: x.str.extract('([A-Za-z])')).add_prefix('U_'), left_index=True,right_index=True,how='outer')
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C NaN NaN
1 3A 1A A A NaN
2 41A 28A 3A A A A
2) You can try of duplicating the rows
of dataframe wherever it has more than 1 value
# Duplicating the rows of dataframe
val = df[['U_B1','U_B2','U_B3']].notnull().sum(axis=1)
df1 = df.loc[np.repeat(val.index,val)]
-> 3) then by grouping with index, pick only masked values of identity matrix
(np.identity) w.r.t length of each group.
df1[['U_B1','U_B2','U_B3']] = df1.groupby(df1.index)['U_B1','U_B2','U_B3'].apply(lambda x: x.dropna(axis=1).mask(np.identity(len(x))==0))
Out:
B1 B2 B3 U_B1 U_B2 U_B3
0 1C C
1 3A 1A A
1 3A 1A A
2 41A 28A 3A A
2 41A 28A 3A A
2 41A 28A 3A A
edited Nov 15 '18 at 10:13
answered Nov 15 '18 at 8:38
Naga KiranNaga Kiran
2,3271516
2,3271516
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the linedf = df.merge
it showsIf using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
|
show 2 more comments
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the linedf = df.merge
it showsIf using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
1
1
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
In the original df, columns U1 ec. doesn't exist
– Rahul Agarwal
Nov 15 '18 at 8:41
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
@RahulAgarwal i just edited my response to incorporate the change :-)
– Naga Kiran
Nov 15 '18 at 9:09
at the line
df = df.merge
it shows If using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
at the line
df = df.merge
it shows If using all scalar values, you must pass an index
– xiumpt
Nov 15 '18 at 9:19
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
I checked it again, can you pass the json form of your dataframe ?
– Naga Kiran
Nov 15 '18 at 9:25
1
1
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
And I found that other line of code is ok to run, using the second block as df in my question which has extracted alphabates
– xiumpt
Nov 15 '18 at 9:49
|
show 2 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53314441%2fpython-pandas-explode-rows-by-turns%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Your original df will only have columns B1 B2 & B3, I suppose?
– Rahul Agarwal
Nov 15 '18 at 8:19
@RahulAgarwal Yes
– xiumpt
Nov 15 '18 at 8:22