Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [encoding]

Encoding is a set of rules used to represent data in a form that can be stored and transmitted to another process or system. Character encoding (e.g. Windows-1252, ISO-8859-1, UTF-8, UTF-16) refers to the way character data is represented as a series of bytes. Binary encoding (e.g. Base64) refers to the way binary data is transformed into a series of characters.

126 questions
32
votes
1 answer

How do I set a SQL Server Unicode / NVARCHAR string to an emoji or Supplementary Character?

I want to set a Unicode string variable to particular character based on its Unicode code point. I want to use a code point beyond 65535, but the SQL Server 2008 R2 database has a collation of SQL_Latin1_General_CP1_CI_AS. According to Microsoft's…
Riley Major
  • 1,761
  • 1
  • 14
  • 28
26
votes
3 answers

Using PostgreSQL 8.4, how to convert bytea to text value in postgres?

In my application I insert data in database using C code, since the strings I receive from an untrusted source I have escaped them using PQescapeByteaConn of libpq library. Which is working perfectly fine i.e. results in Octet format String. See…
Amit
  • 411
  • 2
  • 5
  • 10
21
votes
4 answers

SQLCMD command is not able to insert accents

I am trying to run sqlcmd.exe in order to setup a new database from command line. I am using SQL SERVER Express 2012 on Windows 7 64 bits. Here's the command I use: SQLCMD -S .\MSSQLSERVER08 -V 17 -E -i %~dp0\aqualogyDB.sql -o…
Oskytar
  • 313
  • 1
  • 2
  • 7
19
votes
4 answers

SQL Server 2005/2008 UTF-8 Collation/Charset

I can't find option(s) directly to set UTF-8 rellated Collations/Charsets in SQL Server 2005/2008, same as is possible to set in another SQL engines, but in SQL Server 2005/2008 are there only Latin and SQL collations. Is there some option to…
mKorbel
  • 597
  • 2
  • 12
  • 25
18
votes
5 answers

Why does the varchar datatype allow unicode values?

I have a table with a varchar column. It is allowing Trademark(™), copyright(©) and other Unicode characters as shown below. Create table VarcharUnicodeCheck ( col1 varchar(100) ) insert into VarcharUnicodeCheck (col1) values ('MyCompany') insert…
Shiva
  • 765
  • 3
  • 6
  • 17
17
votes
4 answers

How To Strip Hebrew Accent Marks

I need a Char Encoding Trick to Strip Hebrew Accent Marks. Sample Before בְּרֵאשִׁ֖ית בָּרָ֣א אֱלֹהִ֑ים אֵ֥ת הַשָּׁמַ֖יִם וְאֵ֥ת הָאָֽרֶץ Sample After בראשית ברא אלהים את השמים ואת הארץ
Decrypted
  • 273
  • 1
  • 6
17
votes
1 answer

Error starting SQL Server 2017 service. Error Code 3417

I have SQL Server 2017 installed on my computer. This is what SELECT @@VERSION returns: Microsoft SQL Server 2017 (RTM-GDR) (KB4293803) - 14.0.2002.14 (X64) Jul 21 2018 07:47:45 Copyright (C) 2017 Microsoft Corporation Enterprise Edition…
Beginner
  • 273
  • 3
  • 7
13
votes
1 answer

Postgresql: set default psql client encoding

When I connect to a Postgresql DB using psql, I often get these messages: => SELECT * FROM question_view ; ERROR: character with byte sequence 0xd7 0x9e in encoding "UTF8" has no equivalent in encoding "LATIN1" Following this SO answer, I…
Adam Matan
  • 10,129
  • 27
  • 75
  • 94
12
votes
1 answer

Postgres - Encoding, Collation and CType

List of databases Name | Owner | Encoding | Collate | Ctype …
Rob P.
  • 223
  • 2
  • 6
12
votes
1 answer

Encoding issue with SQL Server VARCHAR column retrieved in Python

We recently had an issue with encoding related to a field that's being stored as a varchar(120) in SQL Server. In SSMS, the varchar appears as: "Who Killed JonBen‚t?" However, when it's brought into python, it appears as: I've researched this…
Eric
  • 223
  • 1
  • 2
  • 9
10
votes
3 answers

PostgreSQL: difference between collations 'C' and 'C.UTF-8'

In PostgreSQL, what is the difference between collations C and C.UTF-8? Both show up in rows of pg_collation. Is it perhaps the case that C.UTF-8 is the same as C with encoding UTF-8 regardless or what the actual encoding of a database is?
rookie099
  • 258
  • 1
  • 2
  • 8
9
votes
1 answer

Find non-UTF-8 data in mysql

I have a MySQL database containing data that is input by our customers. Customers are told their data should only be encoded as UTF-8, however this is currently not being enforced and some people are using characters not encoded in UTF-8. We intend…
Bret
  • 93
  • 1
  • 1
  • 3
9
votes
3 answers

Is it possible to use OPENROWSET to import fixed width UTF8 encoded files?

I have an example data file with following contents and saved with UTF8 encoding. oab~opqr öab~öpqr öab~öpqr The format of this file is fixed width with columns 1 to 3 each being allocated 1 character and column 4 reserved 5 characters. I have…
Martin Smith
  • 77,689
  • 15
  • 224
  • 316
9
votes
1 answer

MySQL: Illegal mix of collations

I've tried using a stored procedure to create indices, and got the following error: ERROR 1267 (HY000): Illegal mix of collations (utf8_general_ci,IMPLICIT) and (utf8_unicode_ci,IMPLICIT) for operation '=' There's no line…
Adam Matan
  • 10,129
  • 27
  • 75
  • 94
8
votes
2 answers

Automatic Translation when Converting Unicode to non-Unicode / NVARCHAR to VARCHAR

Unicode code point 9619 is a character called "Dark shade": ▓ (http://unicode-table.com/en/search/?q=9619). Using the SQL_Latin1_General_CP1_CI_AS collation and 1252 code page, I would expect that casting / converting that Unicode character to…
Henry Lee
  • 1,236
  • 1
  • 10
  • 16
1
2 3
…
8 9