SSIS error “UTF8” has no equivalant in encoding “WIN1252”
I'm using SSIS package to extract the data from a Postgres database, but I'm getting following error one of the tables.
Character with byte sequence 0xef 0xbf 0xbd in encoding "UTF8" has no
equivalant in encoding "WIN1252
I have no idea how to resolve it. I made all the columns in the sql table to NVARCHAR(MAX) but still no use. Please provide the solution.
sql postgresql utf-8 ssis
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
add a comment |
I'm using SSIS package to extract the data from a Postgres database, but I'm getting following error one of the tables.
Character with byte sequence 0xef 0xbf 0xbd in encoding "UTF8" has no
equivalant in encoding "WIN1252
I have no idea how to resolve it. I made all the columns in the sql table to NVARCHAR(MAX) but still no use. Please provide the solution.
sql postgresql utf-8 ssis
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29
add a comment |
I'm using SSIS package to extract the data from a Postgres database, but I'm getting following error one of the tables.
Character with byte sequence 0xef 0xbf 0xbd in encoding "UTF8" has no
equivalant in encoding "WIN1252
I have no idea how to resolve it. I made all the columns in the sql table to NVARCHAR(MAX) but still no use. Please provide the solution.
sql postgresql utf-8 ssis
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
I'm using SSIS package to extract the data from a Postgres database, but I'm getting following error one of the tables.
Character with byte sequence 0xef 0xbf 0xbd in encoding "UTF8" has no
equivalant in encoding "WIN1252
I have no idea how to resolve it. I made all the columns in the sql table to NVARCHAR(MAX) but still no use. Please provide the solution.
sql postgresql utf-8 ssis
sql postgresql utf-8 ssis
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
edited Nov 14 '18 at 7:13
jarlh
28.9k52138
28.9k52138
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
asked Nov 14 '18 at 6:30
Shiva KumarShiva Kumar
13
13
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29
add a comment |
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29
add a comment |
1 Answer
1
active
oldest
votes
The full Unicode character set (as encoded in UTF8) contains tens of thousands of different characters. WIN1252 contains 256. Your data contains characters that cannot be represented in WIN1252.
You either need to export to a more useful character encoding, remove the "awkward" characters from the source database or do some (lossy) translation with SSIS itself (I believe "character map translation" is what you want to search for).
I would recommended first though spending am hour or so googling around the subject of Unicode, it's utf encodings and its relationship to the ISO and WIN character sets. That way you will understand which of the above to choose.
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53294313%2fssis-error-utf8-has-no-equivalant-in-encoding-win1252%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The full Unicode character set (as encoded in UTF8) contains tens of thousands of different characters. WIN1252 contains 256. Your data contains characters that cannot be represented in WIN1252.
You either need to export to a more useful character encoding, remove the "awkward" characters from the source database or do some (lossy) translation with SSIS itself (I believe "character map translation" is what you want to search for).
I would recommended first though spending am hour or so googling around the subject of Unicode, it's utf encodings and its relationship to the ISO and WIN character sets. That way you will understand which of the above to choose.
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
add a comment |
The full Unicode character set (as encoded in UTF8) contains tens of thousands of different characters. WIN1252 contains 256. Your data contains characters that cannot be represented in WIN1252.
You either need to export to a more useful character encoding, remove the "awkward" characters from the source database or do some (lossy) translation with SSIS itself (I believe "character map translation" is what you want to search for).
I would recommended first though spending am hour or so googling around the subject of Unicode, it's utf encodings and its relationship to the ISO and WIN character sets. That way you will understand which of the above to choose.
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
add a comment |
The full Unicode character set (as encoded in UTF8) contains tens of thousands of different characters. WIN1252 contains 256. Your data contains characters that cannot be represented in WIN1252.
You either need to export to a more useful character encoding, remove the "awkward" characters from the source database or do some (lossy) translation with SSIS itself (I believe "character map translation" is what you want to search for).
I would recommended first though spending am hour or so googling around the subject of Unicode, it's utf encodings and its relationship to the ISO and WIN character sets. That way you will understand which of the above to choose.
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
The full Unicode character set (as encoded in UTF8) contains tens of thousands of different characters. WIN1252 contains 256. Your data contains characters that cannot be represented in WIN1252.
You either need to export to a more useful character encoding, remove the "awkward" characters from the source database or do some (lossy) translation with SSIS itself (I believe "character map translation" is what you want to search for).
I would recommended first though spending am hour or so googling around the subject of Unicode, it's utf encodings and its relationship to the ISO and WIN character sets. That way you will understand which of the above to choose.
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
answered Nov 14 '18 at 7:22
Richard HuxtonRichard Huxton
15.2k12236
15.2k12236
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
add a comment |
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
"regexp_replace(vendor, E'[\u200e\u200f\u202f]+', ' ', 'gi' ) as vendor" what does this expression do?
– Shiva Kumar
Nov 14 '18 at 9:25
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53294313%2fssis-error-utf8-has-no-equivalant-in-encoding-win1252%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The problem is on the client side. Can you get SSIS to use a different encoding? If you cannot change that, you'll have to modify the data.
– Laurenz Albe
Nov 14 '18 at 7:29