What is the difference between $“Col1”, 'Col1 and “Col1” in Spark SQL?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I am currently referring Spark in Action Book in that, I came across using same column in different ways.
val postsIdBody = postsDf.select('id, 'body)
val postsIdBody = postsDf.select($"id", $"body")
val postsIdBody = postsDf.select("id", "body")
we are able to get similar results. Is there any much difference between those? Can anyone clearly explain in what situations we need to implement each type of those.
Thanks in advance
scala apache-spark apache-spark-sql
edited Nov 16 '18 at 18:36
moe
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
add a comment |
I am currently referring Spark in Action Book in that, I came across using same column in different ways.
val postsIdBody = postsDf.select('id, 'body)
val postsIdBody = postsDf.select($"id", $"body")
val postsIdBody = postsDf.select("id", "body")
we are able to get similar results. Is there any much difference between those? Can anyone clearly explain in what situations we need to implement each type of those.
Thanks in advance
scala apache-spark apache-spark-sql
edited Nov 16 '18 at 18:36
moe
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27
add a comment |
I am currently referring Spark in Action Book in that, I came across using same column in different ways.
val postsIdBody = postsDf.select('id, 'body)
val postsIdBody = postsDf.select($"id", $"body")
val postsIdBody = postsDf.select("id", "body")
we are able to get similar results. Is there any much difference between those? Can anyone clearly explain in what situations we need to implement each type of those.
Thanks in advance
scala apache-spark apache-spark-sql
edited Nov 16 '18 at 18:36
moe
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
I am currently referring Spark in Action Book in that, I came across using same column in different ways.
val postsIdBody = postsDf.select('id, 'body)
val postsIdBody = postsDf.select($"id", $"body")
val postsIdBody = postsDf.select("id", "body")
we are able to get similar results. Is there any much difference between those? Can anyone clearly explain in what situations we need to implement each type of those.
Thanks in advance
scala apache-spark apache-spark-sql
scala apache-spark apache-spark-sql
edited Nov 16 '18 at 18:36
moe
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
edited Nov 16 '18 at 18:36
moe
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
edited Nov 16 '18 at 18:36
moe
574519
edited Nov 16 '18 at 18:36
moe
574519
edited Nov 16 '18 at 18:36
moe
574519
574519
asked Nov 16 '18 at 11:25
user2815076user2815076
94
asked Nov 16 '18 at 11:25
user2815076user2815076
94
asked Nov 16 '18 at 11:25
user2815076user2815076
94
94
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27
add a comment |
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27
add a comment |
1 Answer
1
active
oldest
votes
I'm sure the book includes this, but by importing the implicits package in Scala, you can use these symbols to create Column objects without otherwise typing out new Column(name)
You would use column objects rather than strings because you can do ordering and aliasing easier within the dataframe API
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53336930%2fwhat-is-the-difference-between-col1-col1-and-col1-in-spark-sql%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I'm sure the book includes this, but by importing the implicits package in Scala, you can use these symbols to create Column objects without otherwise typing out new Column(name)
You would use column objects rather than strings because you can do ordering and aliasing easier within the dataframe API
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
add a comment |
I'm sure the book includes this, but by importing the implicits package in Scala, you can use these symbols to create Column objects without otherwise typing out new Column(name)
You would use column objects rather than strings because you can do ordering and aliasing easier within the dataframe API
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
add a comment |
I'm sure the book includes this, but by importing the implicits package in Scala, you can use these symbols to create Column objects without otherwise typing out new Column(name)
You would use column objects rather than strings because you can do ordering and aliasing easier within the dataframe API
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
I'm sure the book includes this, but by importing the implicits package in Scala, you can use these symbols to create Column objects without otherwise typing out new Column(name)
You would use column objects rather than strings because you can do ordering and aliasing easier within the dataframe API
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
answered Nov 16 '18 at 14:59
cricket_007cricket_007
84.1k1147119
84.1k1147119
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53336930%2fwhat-is-the-difference-between-col1-col1-and-col1-in-spark-sql%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Using the $ sign is a spark-specific way of scala string interpolation. see stackoverflow.com/questions/35885702/sqlcontext-implicits and bzhangusc.wordpress.com/2015/03/29/the-column-class
– moe
Nov 16 '18 at 14:27