Dynamically Creating Icons in Lightning Design System

EDIT

I see you are creating a VF page - this is quite different. Since you are creating SVG, this is essentially a different namespace, so you would create an element as follows.

First, define a div to append the svg element to:

<div id="svgContainer"/>

Then in call this in JavaScript:

var xmlns = "http://www.w3.org/2000/svg";
var boxWidth = 100;
var boxHeight = 100;

var element = document.createElementNS (xmlns, "svg");
element.setAttributeNS (null, "viewBox", "0 0 " + boxWidth + " " + boxHeight);
element.setAttributeNS (null, "width", boxWidth);
element.setAttributeNS (null, "height", boxHeight);
element.style.display = "block";

var svgContainer = document.getElementById ("svgContainer");
svgContainer.appendChild (element); 

Original Answer (might still be useful if you are using Lightning)

I suggest you use the custom SVG provided by Lightning here. It shows you how you can dynamically specify all the parameters of an SVG, using a component.

I think your specific issue is linked to how lightning renders components, so the key is probably here, in the renderer:

// Calls the helper function to append the SVG icon
helper.renderIcon(component);
return ret;

I was able to reproduce the issue just by dynamically creating a button which contains SVG tag inside it using javascript.Strangely the dynamic button markup while inspecting the element was same as the static ones included in the page(as you described in the question).

Few trail and errors, I found that the DOM API appendChild() behaved strangely only for SVG elements.

 actionButton.appendChild(delSvg); // Not working if child is svg 

So I tried to set the innerHTML of the button tag to the SVG's HTML content and it solved the issue.

Here's what I tried:

<apex:page standardStylesheets="false" sidebar="false" showHeader="false">
    <link href="{!URLFOR($Resource.SLDS0121, 'assets/styles/salesforce-lightning-design-system-vf.css')}" rel="stylesheet"/>

   <body class="slds" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
        <button class="slds-button slds-button--icon">
            <svg aria-hidden="true" class="slds-button__icon">
                <use xlink:href="{!URLFOR($Resource.SLDS202, '/assets/icons/utility-sprite/svg/symbols.svg#delete')}"></use>
            </svg>
            <span class="slds-assistive-text">Settings</span>
        </button>

     <div id="svgBtn"/>

   </body>

    <script type="text/javascript">
        var svgLink = "{!URLFOR($Resource.SLDS202, '/assets/icons/utility-sprite/svg/symbols.svg#delete')}";
        var delSvg = document.createElementNS ("http://www.w3.org/2000/svg", "svg");
            delSvg.setAttribute('aria-hidden', 'true');
            delSvg.setAttribute('class', 'slds-button__icon');

        var useLink = document.createElement('use');
            useLink.setAttribute("xlink:href",svgLink);
            useLink.setAttribute("xmlns:xlink", "http://www.w3.org/1999/xlink");
            delSvg.appendChild(useLink);


        var button = document.createElement('button');
            button.setAttribute('aria-hidden', 'true');
            button.setAttribute('class', 'slds-button slds-button--icon');
            button.innerHTML = delSvg.outerHTML;

        document.getElementById("svgBtn").appendChild(button);

    </script>
</apex:page>