Use the unicode entity with createTextNode. For example, to append an ampersand, you would use \u0026 like this:
document.createTextNode("\u0026");
When using innerHTML, you can use the HTML ASCII entity:
document.getElementById("myDiv").innerHTML += "&";
For those interested: