The json_encode one is particularly annoying as you can't even consider sending JSON to the browser and using it in JS without all the flags on. especially if you're dealing with user data or scraping websites.
We send back a json_encoded array containing a property "html" with html markup to be injected back into a contenteditable div. Originally I had a ton of problems if there happened to be the invisible unicode character in there (php's str functions could not find/replace this character no matter what I tried). However, this is the first I've seen those flags the GP mentioned.
function AsJSON($arr, $cruft=null){
//convmap since 0x80 char codes so it takes all multibyte codes (above ASCII 127).
//So such characters are being "hidden" from normal json_encoding
array_walk_recursive($arr, function (&$item, $key) {
if (is_string($item)){
$item = mb_encode_numericentity($item, array (0x80, 0xffff, 0, 0xffff), 'UTF-8');
}
});
$JSON = mb_decode_numericentity(
json_encode($arr, JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP),
array (0x80, 0xffff, 0, 0xffff),
'UTF-8'
);
if(json_last_error() === JSON_ERROR_NONE){
return $cruft.$JSON;
}
}
You can identify them by pack("CCC",0xef,0xbb,0xbf), encoding to ""
But then again, my name gives me special powers.
Modifying the content by default would be completely counter-intuitive, specially with a function that is used mostly in API and thus being encoded / decoded in different places and languages.
IMO content filtering is specific to the view where you output it, and should be done there. Any hazardous content should always go through a sanitize function when you echo it in the middle of HTML.