URLEncoding and HTML/XML Encoding

Recently one of our client applications were passing XML data as a POST parameter to the server. e.g. xmlreqdoc=<xml request here>.They were using XML/HTML encoding to replace special characters in the XML e.g. '&' was replaced with '&amp;amp;amp;' and they
thought it should work.But on the server side whenever we were trying to get the XML string as a request parameter, we were getting a truncated string. After careful debugging, we found out the reason and the solution.

Main reason: we all were getting confused between URL encoding and XML encoding. As we know Servlet distinguishes different parameters with & sign. And if one of the
parameter’s values contains “&” then servlet will think that she got another
parameter. e.g if the intension is to send 2 parameters x=Rock&Roll and Y=onlyRoll e.g. - http://myservlet/service?x=Rock&amp;amp;amp;Roll&y=onlyRoll
Whether it is GET or POST, Servlet still needs to parse different parameters. This request will be parsed as 3 parameters,

Parameter1 : x = Rock

Parameter 2 : Roll = null

Parameter 3: y = onlyRoll

So what needs to be done so that we get the values correctly --- URLEncode comes to the rescue here. Either you use java.net.URLEncode.encode (“Rock&Roll”) or you can hardcode the encoded value yourself.

Basically when we use this API, each unsafe special character is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. So the
“&” becomes “%26”;

So the request with encoded value looks like

http://myservlet/service?x=Rock%26Roll&y=onlyRoll

Now the request will be parsed as 2 parameters

Parameter1 : x = Rock%26Roll

Parameter 2 : y = onlyRoll

But if you use the HTTPServletRequest API getParameter(“x”), it does decode behind the scene for you so you will get decoded value

getParameter(“x”) will return Rock&Roll

getParameter(“y”) will return onlyRoll

So far so good…. But what if this Rock&Roll is getting passed inside an XML tag to the XML Parser for further processing….It will fail because “&” is also a special character in XML.

e.g.

http://myservlet/service?xmlreqdoc=<?xml version="1.0"
encoding="UTF-8"?><request>Rock&Roll</request>

so if you do request.getParameter(“xmlreqdoc”) it will return truncated value <?xml
version="1.0" encoding="UTF-8"?><request>Rock and Servlet will think she got
another parameter named “Roll</request>” with no value

So of course it will fail.

Our client was passing with XML encoded value and they thought it should do the trick….

http://myservlet/service?xmlreqdoc=<?xml
version="1.0" encoding="UTF-8"?><request>Rock&amp;Roll</request>

still do request.getParameter(“xmlreqdoc”) it will return truncated value <?xml
version="1.0" encoding="UTF-8"?><request>Rock

Servlet will think she got another parameter named “amp;Roll</request>” with no value. So still we had problems..

Only way you can get around if we encode the value in request which has special characters.
http://myservlet/service?xmlreqdoc=<?xml%20version="1.0"
encoding="UTF-8"?><request>Rock%26Roll</request>

With above request request.getParameter(“xmlreqdoc”) will get us the =

<?xml version="1.0" encoding="UTF-8"?><request>Rock&Roll</request> (since Servlet does decode behind the scene)

But still if you pass this request to XML parser it will fail..because “&” is special character for
XML as well. And it needs to be sent as “&amp;amp;amp;amp;” to the parser.

So the correct URL should be

http://myservlet/service?xmlreqdoc=<?xml%20version="1.0"%20encoding="UTF-8"?><request>Rock%26amp;Roll</request>;

now request.getParameter(“xmlreqdoc”) will get us the =<?xml version="1.0"
encoding="UTF-8"?><request>Rock&amp;Roll</request>
and XML parser will not complain…...and they will live happily ever after.
Another approach is not to read the request as a parameter using request.getParameter()API and use getInputStreamAPI..in that case it does not need URLEncoding only XML encoding
should be sufficient.