UNIX - Add column at the end of CSV with double quotes using SED
I have data in following format :
"1";"abc"
"2";"dfg"
"3";"hij"
I used the following command to add a column:
sed "s/$/;"newc"/" file.csv
but i get the last column without quotes:
"1";"abc";newc
"2";"dfg";newc
"3";"hij";newc
can not figure out how to update it to add double quotes
and get:
"1";"abc";"newc"
"2";"dfg";"newc"
"3";"hij";"newc"
linux sed csv
add a comment |
I have data in following format :
"1";"abc"
"2";"dfg"
"3";"hij"
I used the following command to add a column:
sed "s/$/;"newc"/" file.csv
but i get the last column without quotes:
"1";"abc";newc
"2";"dfg";newc
"3";"hij";newc
can not figure out how to update it to add double quotes
and get:
"1";"abc";"newc"
"2";"dfg";"newc"
"3";"hij";"newc"
linux sed csv
Surrounding sed script with single quotes doesn't work?'s/$/;"newc"/'
– Paulo
Feb 7 at 17:03
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18
add a comment |
I have data in following format :
"1";"abc"
"2";"dfg"
"3";"hij"
I used the following command to add a column:
sed "s/$/;"newc"/" file.csv
but i get the last column without quotes:
"1";"abc";newc
"2";"dfg";newc
"3";"hij";newc
can not figure out how to update it to add double quotes
and get:
"1";"abc";"newc"
"2";"dfg";"newc"
"3";"hij";"newc"
linux sed csv
I have data in following format :
"1";"abc"
"2";"dfg"
"3";"hij"
I used the following command to add a column:
sed "s/$/;"newc"/" file.csv
but i get the last column without quotes:
"1";"abc";newc
"2";"dfg";newc
"3";"hij";newc
can not figure out how to update it to add double quotes
and get:
"1";"abc";"newc"
"2";"dfg";"newc"
"3";"hij";"newc"
linux sed csv
linux sed csv
asked Feb 7 at 13:42
ChompisChompis
31
31
Surrounding sed script with single quotes doesn't work?'s/$/;"newc"/'
– Paulo
Feb 7 at 17:03
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18
add a comment |
Surrounding sed script with single quotes doesn't work?'s/$/;"newc"/'
– Paulo
Feb 7 at 17:03
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
– Paulo
Feb 7 at 17:03
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
– Paulo
Feb 7 at 17:03
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18
add a comment |
1 Answer
1
active
oldest
votes
The issue was solved in comments without proper explanation:
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
yes, it worked!
This answer is going to shed some light on what happened and why the solution works.
In your original command quoting goes like this:
sed "s/$/;"newc"/" file.csv
# ^ ^ a matching pair of quotes
# ^ ^ another pair of quotes
# s/$/; / these fragments are quoted
# newc this fragment is not quoted at all
The quotes you used are consumed by the shell. Their presence tells the shell to treat quoted strings somewhat differently than non-quoted ones, e.g. the quoted semicolon (;
) is not a command separator; then they disappear, i.e. the shell doesn't pass them to sed
.
Note newc
contains no characters special to the shell, it behaves in the same way whether it's quoted or not. This means newc
might as well be quoted, like this:
sed "s/$/;""newc""/" file.csv
# ^ ^ added pair of quotes
But this is equivalent to
sed "s/$/;newc/" file.csv
and after the shell consumes the quotes sed
gets these arguments: s/$/;newc/
, file.csv
. As you can see the tool gets no quotes at all.
To pass quotes to sed
you need to make them "survive" parsing done by the shell. There are few ways to do this. Two common approaches:
Escaping with
. Inside double quotes you can escape a double quote character, so it is treated as a part of the quoted string, not as a closing quote. In your case:
sed "s/$/;"newc"/" file.csv
Mixing quotes. A double quote inside single quotes remains. The mentioned solution uses this fact:
sed 's/$/;"newc"/' file.csv
A single quote inside double quotes also remains. E.g. if you need to pass a literal
'"
argument toecho
, this will work:
echo "'"'"'
# ^ ^ # a pair of double quotes that make the single quote survive
# ^ ^ # a pair of single quotes that make the double quote survive
Sometimes it's good to invoke set -x
before a "misbehaving" command to learn what is left after the shell parses it. Your original command and two fixed ones generate this (output from sed
omitted for clarity):
$ sed "s/$/;"newc"/" file.csv # original command
+ sed s/$/;newc/ file.csv
$ # the above line contains what sed really got
$ sed "s/$/;"newc"/" file.csv # fixed
+ sed s/$/;"newc"/ file.csv
$ # this time sed got the right string
$ sed 's/$/;"newc"/' file.csv # also fixed
+ sed s/$/;"newc"/ file.csv
$ # again the right string
$
Note: at the end invoke set +x
to revert what set -x
did.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1403129%2funix-add-column-at-the-end-of-csv-with-double-quotes-using-sed%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The issue was solved in comments without proper explanation:
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
yes, it worked!
This answer is going to shed some light on what happened and why the solution works.
In your original command quoting goes like this:
sed "s/$/;"newc"/" file.csv
# ^ ^ a matching pair of quotes
# ^ ^ another pair of quotes
# s/$/; / these fragments are quoted
# newc this fragment is not quoted at all
The quotes you used are consumed by the shell. Their presence tells the shell to treat quoted strings somewhat differently than non-quoted ones, e.g. the quoted semicolon (;
) is not a command separator; then they disappear, i.e. the shell doesn't pass them to sed
.
Note newc
contains no characters special to the shell, it behaves in the same way whether it's quoted or not. This means newc
might as well be quoted, like this:
sed "s/$/;""newc""/" file.csv
# ^ ^ added pair of quotes
But this is equivalent to
sed "s/$/;newc/" file.csv
and after the shell consumes the quotes sed
gets these arguments: s/$/;newc/
, file.csv
. As you can see the tool gets no quotes at all.
To pass quotes to sed
you need to make them "survive" parsing done by the shell. There are few ways to do this. Two common approaches:
Escaping with
. Inside double quotes you can escape a double quote character, so it is treated as a part of the quoted string, not as a closing quote. In your case:
sed "s/$/;"newc"/" file.csv
Mixing quotes. A double quote inside single quotes remains. The mentioned solution uses this fact:
sed 's/$/;"newc"/' file.csv
A single quote inside double quotes also remains. E.g. if you need to pass a literal
'"
argument toecho
, this will work:
echo "'"'"'
# ^ ^ # a pair of double quotes that make the single quote survive
# ^ ^ # a pair of single quotes that make the double quote survive
Sometimes it's good to invoke set -x
before a "misbehaving" command to learn what is left after the shell parses it. Your original command and two fixed ones generate this (output from sed
omitted for clarity):
$ sed "s/$/;"newc"/" file.csv # original command
+ sed s/$/;newc/ file.csv
$ # the above line contains what sed really got
$ sed "s/$/;"newc"/" file.csv # fixed
+ sed s/$/;"newc"/ file.csv
$ # this time sed got the right string
$ sed 's/$/;"newc"/' file.csv # also fixed
+ sed s/$/;"newc"/ file.csv
$ # again the right string
$
Note: at the end invoke set +x
to revert what set -x
did.
add a comment |
The issue was solved in comments without proper explanation:
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
yes, it worked!
This answer is going to shed some light on what happened and why the solution works.
In your original command quoting goes like this:
sed "s/$/;"newc"/" file.csv
# ^ ^ a matching pair of quotes
# ^ ^ another pair of quotes
# s/$/; / these fragments are quoted
# newc this fragment is not quoted at all
The quotes you used are consumed by the shell. Their presence tells the shell to treat quoted strings somewhat differently than non-quoted ones, e.g. the quoted semicolon (;
) is not a command separator; then they disappear, i.e. the shell doesn't pass them to sed
.
Note newc
contains no characters special to the shell, it behaves in the same way whether it's quoted or not. This means newc
might as well be quoted, like this:
sed "s/$/;""newc""/" file.csv
# ^ ^ added pair of quotes
But this is equivalent to
sed "s/$/;newc/" file.csv
and after the shell consumes the quotes sed
gets these arguments: s/$/;newc/
, file.csv
. As you can see the tool gets no quotes at all.
To pass quotes to sed
you need to make them "survive" parsing done by the shell. There are few ways to do this. Two common approaches:
Escaping with
. Inside double quotes you can escape a double quote character, so it is treated as a part of the quoted string, not as a closing quote. In your case:
sed "s/$/;"newc"/" file.csv
Mixing quotes. A double quote inside single quotes remains. The mentioned solution uses this fact:
sed 's/$/;"newc"/' file.csv
A single quote inside double quotes also remains. E.g. if you need to pass a literal
'"
argument toecho
, this will work:
echo "'"'"'
# ^ ^ # a pair of double quotes that make the single quote survive
# ^ ^ # a pair of single quotes that make the double quote survive
Sometimes it's good to invoke set -x
before a "misbehaving" command to learn what is left after the shell parses it. Your original command and two fixed ones generate this (output from sed
omitted for clarity):
$ sed "s/$/;"newc"/" file.csv # original command
+ sed s/$/;newc/ file.csv
$ # the above line contains what sed really got
$ sed "s/$/;"newc"/" file.csv # fixed
+ sed s/$/;"newc"/ file.csv
$ # this time sed got the right string
$ sed 's/$/;"newc"/' file.csv # also fixed
+ sed s/$/;"newc"/ file.csv
$ # again the right string
$
Note: at the end invoke set +x
to revert what set -x
did.
add a comment |
The issue was solved in comments without proper explanation:
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
yes, it worked!
This answer is going to shed some light on what happened and why the solution works.
In your original command quoting goes like this:
sed "s/$/;"newc"/" file.csv
# ^ ^ a matching pair of quotes
# ^ ^ another pair of quotes
# s/$/; / these fragments are quoted
# newc this fragment is not quoted at all
The quotes you used are consumed by the shell. Their presence tells the shell to treat quoted strings somewhat differently than non-quoted ones, e.g. the quoted semicolon (;
) is not a command separator; then they disappear, i.e. the shell doesn't pass them to sed
.
Note newc
contains no characters special to the shell, it behaves in the same way whether it's quoted or not. This means newc
might as well be quoted, like this:
sed "s/$/;""newc""/" file.csv
# ^ ^ added pair of quotes
But this is equivalent to
sed "s/$/;newc/" file.csv
and after the shell consumes the quotes sed
gets these arguments: s/$/;newc/
, file.csv
. As you can see the tool gets no quotes at all.
To pass quotes to sed
you need to make them "survive" parsing done by the shell. There are few ways to do this. Two common approaches:
Escaping with
. Inside double quotes you can escape a double quote character, so it is treated as a part of the quoted string, not as a closing quote. In your case:
sed "s/$/;"newc"/" file.csv
Mixing quotes. A double quote inside single quotes remains. The mentioned solution uses this fact:
sed 's/$/;"newc"/' file.csv
A single quote inside double quotes also remains. E.g. if you need to pass a literal
'"
argument toecho
, this will work:
echo "'"'"'
# ^ ^ # a pair of double quotes that make the single quote survive
# ^ ^ # a pair of single quotes that make the double quote survive
Sometimes it's good to invoke set -x
before a "misbehaving" command to learn what is left after the shell parses it. Your original command and two fixed ones generate this (output from sed
omitted for clarity):
$ sed "s/$/;"newc"/" file.csv # original command
+ sed s/$/;newc/ file.csv
$ # the above line contains what sed really got
$ sed "s/$/;"newc"/" file.csv # fixed
+ sed s/$/;"newc"/ file.csv
$ # this time sed got the right string
$ sed 's/$/;"newc"/' file.csv # also fixed
+ sed s/$/;"newc"/ file.csv
$ # again the right string
$
Note: at the end invoke set +x
to revert what set -x
did.
The issue was solved in comments without proper explanation:
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
yes, it worked!
This answer is going to shed some light on what happened and why the solution works.
In your original command quoting goes like this:
sed "s/$/;"newc"/" file.csv
# ^ ^ a matching pair of quotes
# ^ ^ another pair of quotes
# s/$/; / these fragments are quoted
# newc this fragment is not quoted at all
The quotes you used are consumed by the shell. Their presence tells the shell to treat quoted strings somewhat differently than non-quoted ones, e.g. the quoted semicolon (;
) is not a command separator; then they disappear, i.e. the shell doesn't pass them to sed
.
Note newc
contains no characters special to the shell, it behaves in the same way whether it's quoted or not. This means newc
might as well be quoted, like this:
sed "s/$/;""newc""/" file.csv
# ^ ^ added pair of quotes
But this is equivalent to
sed "s/$/;newc/" file.csv
and after the shell consumes the quotes sed
gets these arguments: s/$/;newc/
, file.csv
. As you can see the tool gets no quotes at all.
To pass quotes to sed
you need to make them "survive" parsing done by the shell. There are few ways to do this. Two common approaches:
Escaping with
. Inside double quotes you can escape a double quote character, so it is treated as a part of the quoted string, not as a closing quote. In your case:
sed "s/$/;"newc"/" file.csv
Mixing quotes. A double quote inside single quotes remains. The mentioned solution uses this fact:
sed 's/$/;"newc"/' file.csv
A single quote inside double quotes also remains. E.g. if you need to pass a literal
'"
argument toecho
, this will work:
echo "'"'"'
# ^ ^ # a pair of double quotes that make the single quote survive
# ^ ^ # a pair of single quotes that make the double quote survive
Sometimes it's good to invoke set -x
before a "misbehaving" command to learn what is left after the shell parses it. Your original command and two fixed ones generate this (output from sed
omitted for clarity):
$ sed "s/$/;"newc"/" file.csv # original command
+ sed s/$/;newc/ file.csv
$ # the above line contains what sed really got
$ sed "s/$/;"newc"/" file.csv # fixed
+ sed s/$/;"newc"/ file.csv
$ # this time sed got the right string
$ sed 's/$/;"newc"/' file.csv # also fixed
+ sed s/$/;"newc"/ file.csv
$ # again the right string
$
Note: at the end invoke set +x
to revert what set -x
did.
answered Feb 8 at 23:16
Kamil MaciorowskiKamil Maciorowski
28k156184
28k156184
add a comment |
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1403129%2funix-add-column-at-the-end-of-csv-with-double-quotes-using-sed%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Surrounding sed script with single quotes doesn't work?
's/$/;"newc"/'
– Paulo
Feb 7 at 17:03
yes, it worked! sorry, was simpler than I thought.Thanks!
– Chompis
Feb 7 at 19:18