Recently I had some problems with the internationalization support in an application. Apparently some single quotes kept disappearing. Not all of them though...The faulty web page even had a mix of good and bad behaviour.
Let the digging begin...
I came across this using Spring's
ReloadableResourceBundleMessageSource. But apparently this comes directly from Java's
MessageFormat class.
The Docs
It seems that if you need to add proper support for internationalization to your application via property files, you have to be really careful with the specs. They are not at all obvious.
In fact, just by skimming the docs, you get to this point: "
Warning: The rules for using quotes within message format patterns unfortunately have shown to be somewhat confusing. In particular, it isn't always obvious to localizers whether single quotes need to be doubled or not."
Ok, fair warning, what about the Spring docs? Nothing on ReloadableResourceBundleMessageSource, but if we go deeper, you have a mention of this issue on
AbstractMessageSource, the parent class of our reloadable bundle of joy: "
Note: By default, message texts are only parsed through MessageFormat if arguments have been passed in for the message. In case of no arguments, message texts will be returned as-is. As a consequence, you should only use MessageFormat escaping for messages with actual arguments, and keep all other messages un-escaped. If you prefer to escape all messages, set the "alwaysUseMessageFormat" flag to "true". ".
...ok, whatever... I said to myself the first time I read this, and like any other good developer, I forgot I ever red that and moved on.
Back to our problem, the application was exhibiting strange behaviour inconsistencies, on apparently similar use cases. Then it hit me... parameters!!
If you use parameters in the internationalization string, every single quote
has to be escaped by another single quote,
else it does
not have to be escaped
- ...you said what now?
Moreover, by not escaping single quotes in strings that have params, the injection will not work, so it's even more crappy.
As it later occurred to me, by reading the Spring docs, if you enable the "
alwaysUseMessageFormat", it will always try to format the message, so you could just default to escaping every single quote and you are done.
The Tests
It was still not 100% clear to me, so I decided to write some unit tests, covering these cases, just to be sure.
And now for the coding part of the post, I'm going to go through each test, and explain shortly what is going on.
A little context, before beginning: I'm using JUnit 4, and Spring's ReloadableResourceBundleMessageSource, bundled together with a small test property file. I'll also attach the entire project at the end, for your testing pleasure.
The property file:
simple=page
simple_param=page {0}
quoted=page d'accueil
quoted_escaped_noparam=page d''accueil
quoted_unescaped_param=page d'accueil est {0}
quoted_escaped_param=page d''accueil est {0}
The test class, hopefully the comments I added to each method makes for an easier understanding of the code
public class SingleQuoteTest {
private ReloadableResourceBundleMessageSource source;
/**
* initialize the bundle to make sure that we have a clean state
*/
@Before
public void init() {
source = new ReloadableResourceBundleMessageSource();
source.setBasename("test");
source.setDefaultEncoding("UTF-8");
}
/**
* no surprises here, just making sure the basic mechanism works
*/
@Test
public void testSimple() {
String code = "simple";
String expectedValue = "page";
String actualValue = source.getMessage(code, null, null);
assertEquals(expectedValue, actualValue);
}
/**
* just making sure the basic mechanism works even with parameters
*/
@Test
public void testSimpleParam() {
String code = "simple_param";
String[] params = { "d'accueil" };
String expectedValue = "page d'accueil";
String actualValue = source.getMessage(code, params, null);
assertEquals(expectedValue, actualValue);
}
/**
* basic functional test on a un-scaped string, with no params
*/
@Test
public void testQuotedNoParams() {
String code = "quoted";
String expectedValue = "page d'accueil";
String actualValue = source.getMessage(code, null, null);
assertEquals(expectedValue, actualValue);
}
/**
* This will <b>fail</b> because we passed a parameter, even though it is
* not used
*
* junit.framework.ComparisonFailure: null expected:<page d[']accueil> but
* was:<page d[]accueil>
*
*/
@Test
public void testQuotedUselessParams() {
String code = "quoted";
String[] params = { "d'accueil" };
String expectedValue = "page d'accueil";
String actualValue = source.getMessage(code, params, null);
assertEquals(expectedValue, actualValue);
}
/**
* This will <b>fail</b> by returning two single quotes, because we did not
* pass any parameters
*
* junit.framework.ComparisonFailure: null expected:<page d'[]accueil> but
* was:<page d'[']accueil>
*/
@Test
public void testQuotedEscapedNoParams() {
String code = "quoted_escaped_noparam";
String expectedValue = "page d'accueil";
String actualValue = source.getMessage(code, null, null);
assertEquals(expectedValue, actualValue);
}
/**
* apparently if you enable the 'alwaysUseMessageFormat' flag, this will
* always get escaped
*
*/
@Test
public void testQuotedEscapedNoParamsAlwaysFormatOn() {
source.setAlwaysUseMessageFormat(true);
String code = "quoted_escaped_noparam";
String expectedValue = "page d'accueil";
String actualValue = source.getMessage(code, null, null);
assertEquals(expectedValue, actualValue);
}
/**
* <b>fail</b> again
*
* Having params and un-escaped single quotes makes for a nasty output
*
* junit.framework.ComparisonFailure: null expected:<page d['accueil est
* ici]> but was:<page d[accueil est {0}]>
*
*/
@Test
public void testQuotedUnescapedParams() {
String code = "quoted_unescaped_param";
String[] params = { "ici" };
String expectedValue = "page d'accueil est ici";
String actualValue = source.getMessage(code, params, null);
assertEquals(expectedValue, actualValue);
}
/**
* the way to go: params and escaped single quotes
*/
@Test
public void testQuotedEscapedParams() {
String code = "quoted_escaped_param";
String[] params = { "ici" };
String expectedValue = "page d'accueil est ici";
String actualValue = source.getMessage(code, params, null);
assertEquals(expectedValue, actualValue);
}
}
If you remember the initial starting point of this shenanigan, my web page had one message that received an unused parameter, that was causing the escaping problem, thus all the weirdness.
As you can see, this was meant to document a strange (at first sight) behaviour, so that maybe future-me will save half a day of digging through the internet.
I really hope you too learned something today!
Download the test project: click here (the link to archive uploaded in google docs)
