Ironpython 2.6 .py -> .exe

This is a long standing question about which there is very little information on the internet. The only known solution I can find is at http://community.sharpdevelop.net/blogs/mattward/archive/2010/03/16/CompilingPythonPackagesWithIronPython.aspx which uses SharpDevelop. However, this solution is impractical because any semi-complex python project will do a LOT of module imports, and the SharpDevelop solution requires you to create a project per import. I started at it and gave up after about thirty new projects, better to write an automated solution!

So here's my solution, and I'll warn you right now it's not being released as a proper project for good reason:

#!/usr/bin/env python
# CompileToStandalone, a Python to .NET ILR compiler which produces standalone binaries
# (C) 2012 Niall Douglas http://www.nedproductions.biz/
# Created: March 2012

import modulefinder, sys, os, subprocess, _winreg

if len(sys.argv)<2:
    print("Usage: CompileEverythingToILR.py <source py> [-outdir=<dest dir>]")
    sys.exit(0)

if sys.platform=="cli":
    print("ERROR: IronPython's ModuleFinder currently doesn't work, so run me under CPython please")
    sys.exit(1)

sourcepath=sys.argv[1]
destpath=sys.argv[2][8:] if len(sys.argv)==3 else os.path.dirname(sys.argv[0])
ironpythonpath=None
try:
    try:
        keyh=_winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, "SOFTWARE\\IronPython\\2.7\\InstallPath")
        ironpythonpath=_winreg.QueryValue(keyh, None)
    except Exception as e:
        try:
            keyh=_winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, "SOFTWARE\\Wow6432Node\\IronPython\\2.7\\InstallPath")
            ironpythonpath=_winreg.QueryValue(keyh, "")
        except Exception as e:
            pass
finally:
    if ironpythonpath is not None:
        _winreg.CloseKey(keyh)
        print("IronPython found at "+ironpythonpath)
    else:
        raise Exception("Cannot find IronPython in the registry")

# What we do now is to load the python source but against the customised IronPython runtime
# library which has been hacked to work with IronPython. This spits out the right set of
# modules mostly, but we include the main python's site-packages in order to resolve any
# third party packages
print("Scanning '"+sourcepath+"' for dependencies and outputting into '"+destpath+"' ...")
searchpaths=[".", ironpythonpath+os.sep+"Lib"]
searchpaths+=[x for x in sys.path if 'site-packages' in x]
finder=modulefinder.ModuleFinder(searchpaths)
finder.run_script(sourcepath)
print(finder.report())
modules=[]
badmodules=finder.badmodules.keys()
for name, mod in finder.modules.iteritems():
    path=mod.__file__
    # Ignore internal modules
    if path is None: continue
    # Ignore DLL internal modules
    #if '\\DLLs\\' in path: continue
    # Watch out for C modules
    if os.path.splitext(path)[1]=='.pyd':
        print("WARNING: I don't support handling C modules at '"+path+"'")
        badmodules.append(name)
        continue
    modules.append((name, os.path.abspath(path)))
modules.sort()
print("Modules not imported due to not found, error or being a C module:")
print("\n".join(badmodules))
raw_input("\nPress Return if you are happy with these missing modules ...")

with open(destpath+os.sep+"files.txt", "w") as oh:
    oh.writelines([x[1]+'\n' for x in modules])
cmd='ipy64 '+destpath+os.sep+'pyc.py /main:"'+os.path.abspath(sourcepath)+'" /out:'+os.path.splitext(os.path.basename(sourcepath))[0]+' /target:exe /standalone /platform:x86 /files:'+destpath+os.sep+'files.txt'
print(cmd)
cwd=os.getcwd()
try:
    os.chdir(destpath)
    retcode=subprocess.call(cmd, shell=True)
finally:
    os.chdir(cwd)
sys.exit(retcode)

This was written against IronPython v2.7.2 RC1 using its new standalone binary feature, and indeed it does work. You get a standalone .exe file which is totally self-contained - it needs nothing else installed. The script works by parsing the imports for the supplied script and sending the entire lot to pyc.py. That's the good news.

The bad news is as follows:

  1. IronPython v2.7.2 RC1's ModuleFinder doesn't appear to work, so the above script needs to be run using CPython. It then uses CPython's ModuleFinder but against IronPython's customised runtime library. Yeah, I'm amazed it works as well ...
  2. The binaries output start at about 8Mb. A simple unit test weighed in at 16Mb. There's a lot of stuff that doesn't need to be in there e.g. it throws it Wpf support and a bit more, but still they aren't small.
  3. Load times are much slower than non-standalone. Think about forty seconds for a standalone unit test on a fast Intel Core 2 versus about three seconds for non-standalone. If compiled just for x86, that drops to ten seconds.
  4. Run time performance is slower than non-standalone by about 40%. If compiled just for x86, performance approximately doubles. This is why I left in the /platform:x86 above.
  5. There is a well known bug in CPython's encodings and codecs support where ModuleFinder doesn't include any codec support at all unless you manually specify it. So, for example, if you are using UTF-8 with codecs.open() then you NEED a "from encodings import utf_8 as some_unique_identifier" to force the dependency.
  6. The above assumes a modified pyc.py which can take a /files parameter as the command line length limit easily gets exceeded. You can modify your own pyc.py trivially, if not I've submitted the enhancement for inclusion into the next IronPython.

So there you go. It works, but the solution still needs a lot more maturing. Best of luck!


You can use pyc.py, the Python Command-Line Compiler, which is included in IronPython since version 2.6 to compile a Python script to an executable. You find it at %IRONPYTONINSTALLDIR%\Tools\Scripts\pyc.py on your hard disk.

Example

Let's assume you have a simple script test.py that just prints out something to console. You can turn this into an executable with the following command-line (assuming that the IronPython directory is the current directory and that test.py is in there, too):

ipy.exe Tools\Scripts\pyc.py /main:test.py /target:exe

Note: If you are using forms and don't want a console window to open, you want to use /target:winexe instead of /target:exe.

The result will be two files, test.dll and test.exe. test.dll will contain your actual script code, while test.exe is just a launcher for test.dll. You can distribute this EXE and DLL to other computers which do not have IronPython installed if you include the files

  • IronPython.dll,
  • Microsoft.Dynamic.dll,
  • Microsoft.Scripting.Core.dll,
  • Microsoft.Scripting.Debugging.dll,
  • Microsoft.Scripting.dll,
  • Microsoft.Scripting.ExtensionAttribute.dll and
  • IronPython.Modules.dll (sometimes needed).

Also see the blog entry IronPython - how to compile exe.

Tags:

Compilation

Ironpython

Recent Posts