Performance problems in Locale.Charset.*.low_convert
Imported from http://bugzilla.roxen.com/bugzilla/show_bug.cgi?id=5636
Reported by Martin Karlgren marty@roxen.com
Locale.Charset.*.low_convert are very inefficient when they need to fallback to replacement strings/callbacks. This becomes noticeable when the input string is relatively large and contains many wide characters.
$ time pike -e 'Locale.Charset.encoder ("latin1", "")->feed ("\x100" * 100000);'
real 0m0.794s user 0m0.788s sys 0m0.000s
$ time pike -e 'Locale.Charset.encoder ("latin1", "")->feed ("\x100" * 500000);'
real 1m15.714s user 0m24.762s sys 0m50.815s
$ time pike -e 'Locale.Charset.encoder ("latin1", "")->feed ("\xff" * 500000);'
real 0m0.191s user 0m0.116s sys 0m0.008s