The error wasn't very clear since it only gave the following exception:
Can't find resource for bundle java.util.PropertyResourceBundle, key 'somekey'
Looking in my project, I could clearly see that the resource bundle was there and after a quick peek at the resource bundle file itself, I could see that the requested key was also present.
After trying some different options I came to the conclusion that my web application was unable to read the actual .properties file from the classpath. By searching some more, I found out that the Java compiler and other Java tools can only process files which contain Latin-1 and/or Unicode-encoded (\udddd notation) characters. Since I was seeing Japanese characters when opening the properties file, it was clearly the case that this file did not meet those requirements.
Solving this issue was quite simple in the end, since the Sun JDK comes with a utility to help you out with files that contain characters, which are not Latin1. The utility is called: 'native2ascii' and can be run from the command-line quite easily by typing:
$ native2ascii [inputfile] [outputfile]
Once I did that the application was working like a charm again!