mro.c: Correct utf8 and bytes concatenation

The previous commit introduced some code that concatenates a pv on to
an sv and then does SvUTF8_on on the sv if the pv was utf8.

That can’t work if the sv was in Latin-1 (or single-byte) encoding
and contained extra-ASCII characters.  Nor can it work if bytes are
appended to a utf8 sv.  Both produce mangled utf8.

There is apparently no function apart from sv_catsv that handle
this.  So I’ve modified sv_catpvn_flags to handle this if passed the
SV_CATUTF8 (concatenating a utf8 pv) or SV_CATBYTES (cancatenating a
byte pv) flag.

This avoids the overhead of creating a new sv (in fact, sv_catsv
even copies its rhs in some cases, so that would mean creating two
new svs).  It might even be worthwhile to redefine sv_catsv in terms
of this....

1 files changed, 6 insertions, 0 deletions

diff --git a/sv.h b/sv.h
index 534535204c..bbf41c84f6 100644
--- a/sv.h
+++ b/sv.h
@@ -1750,6 +1750,12 @@ Like sv_utf8_upgrade, but doesn't do magic on C<sv>
 /* if (after resolving magic etc), the SV is found to be overloaded,
  * don't call the overload magic, just return as-is */
 #define SV_SKIP_OVERLOAD	8192
+/* It is not yet clear whether we want this as an API, or what the
+ * constants should be named. */
+#ifdef PERL_CORE
+# define SV_CATBYTES		16384
+# define SV_CATUTF8		32768
+#endif
 
 /* The core is safe for this COW optimisation. XS code on CPAN may not be.
    So only default to doing the COW setup if we're in the core.