Character Encoding Issues
Questions
- Why
- How
- Troubleshooting
Answers
Why
Anchor | ||||
---|---|---|---|---|
|
...
- Java Servlet Specification 4.0
- HTTP 1.1 Protocol: Message Syntax and Routing, HTTP 1.1 Protocol: Semantics and Content …
- URI Syntax
- ARPA Internet Text Messages
- HTML 4, HTML 5
Default encoding for request and response bodies
...
- ISO-8859-1 and ASCII are compatible for character codes 0x20 to 0x7E, so they are often used interchangeably.
- Modern browsers encoding URIs using UTF-8. Some browsers appear to use the encoding of the current page to encode URIs for links.
- HTML 4.0 recommends the use of UTF-8 to encode the query string.
- When in doubt, use POST for any data you think might have problems surviving a trip through the query string.
Default Encoding for POST
...
- RFC 2616 Section 3.4.1 stated that recipients of an HTTP message must respect the character encoding specified by the sender in the
Content-Type
header if the encoding is supported. A missing character allows the recipient to "guess" what encoding is appropriate. - Most web browsers today do not specify the character set of a request, even when it is something other than ISO-8859-1. This seems to be in violation of the HTTP specification. Most web browsers appear to send a request body using the encoding of the page used to generate the POST (for instance, the <form> element came from a page with a specific encoding... it is that encoding which is used to submit the POST data for that form).
Percent Encoding for application/x-www-form-urlencoded
...
Section 3.1 of the ARPA Internet Text Messages spec states that headers are always in US-ASCII encoding. Anything outside of that needs to be encoded. See the section above regarding query strings in URIs.
How
Anchor | ||||
---|---|---|---|---|
|
...
- Set the
URIEncoding
attribute on the <Connector> element in server.xml to something specific (e.g.URIEncoding="UTF-8"
). - Set the
useBodyEncodingForURI
attribute on the <Connector> element in server.xml totrue
. This will cause the Connector to use the request body's encoding for GET parameters.
In Tomcat 8 starting with 8.0.0 (8.0.0-RC3, to be specific), the default value of URIEncoding
attribute on the <Connector> element depends on "strict servlet compliance" setting. The default value (strict compliance is off) of URIEncoding
is now UTF-8
. If "strict servlet compliance" is enabled, the default value is ISO-8859-1
.
...
- Set
URIEncoding="UTF-8"
on your <Connector> inserver.xml
. References: HTTP Connector, AJP Connector. - Set the default request character encoding either in the Tomcat
conf/web.xml
file or in the web appweb.xml
file; either by setting<request-character-encoding>
or by using a character encoding filter. - Change all your JSPs to include charset name in their contentType. For example, use
<%@page contentType="text/html; charset=UTF-8" %>
for the usual JSP pages and<jsp:directive.page contentType="text/html; charset=UTF-8" />
for the pages in XML syntax (aka JSP Documents). - Change all your servlets to set the content type for responses and to include charset name in the content type to be UTF-8. Use
response.setContentType("text/html; charset=UTF-8")
orresponse.setCharacterEncoding("UTF-8")
. - Change any content-generation libraries you use (Velocity, Freemarker, etc.) to use UTF-8 and to specify UTF-8 in the content type of the responses that they generate.
- Disable any valves or filters that may read request parameters before your character encoding filter or jsp page has a chance to set the encoding to UTF-8. For more information see http://www.mail-archive.com/users@tomcat.apache.org/msg21117.html.
...
You have to encode them in some way before you insert them into a header. Using url-encoding (%
+ high byte number + low byte number) would be a good idea.
Troubleshooting
Anchor | ||||
---|---|---|---|---|
|
...