Porting C Modules to Jython

CPython includes many library modules, some of which are written in pure Python (which is great because these will work in Jython (hopefully) without modification), but others are written in C, which means they must be rewritten in Java in order to work with Jython. Take the csv module for example. It is a Python library module so it should be possible to use it in Jython as-is without any extra work. If only it were that that simple! You see, if you look at the code for csv.py you'll notice it uses another module called _csv for most of it's behaviour and it just so happens that _csv is written in C. Therefore, it is necessary to port this module to Jython. It's worth noting that there are many cases where a python library module is just a wrapper for an underlying C module, but not always - for example, cStringIO and cPickle are first-class library modules implemented in C.

Before actually creating the _csv module, it's always helpful to see things fail first then you get a nice feeling of satisfaction when you get the test to pass (or fail less!). So to prove that csv doesn't work in current Jython builds I ran the following test:

bash# jython dist/Lib/test/test_csv.py

which resulted in the following predictable error:

Traceback (innermost last):
  File "dist/Lib/test/test_csv.py", line 8, in ?
  File "/work/jython/dist/Lib/csv.py", line 7, in ?
ImportError: no module named _csv

As expected csv.py is unable to import the _csv module because it doesn't exist. To create it I followed the guidelines by Charlie Groves in the wiki (which - funnily enough - uses the csv module as an example - what a coincidence!). I created a _csv.java file in $JYTHON_HOME/src/org/python/modules" then added "_csv" to the list of modules in Setup.java. After building Jython and running test_csv.py again I saw the following error:

Traceback (innermost last):
  File "dist/Lib/test/test_csv.py", line 8, in ?
  File "/work/jython/dist/Lib/csv.py", line 7, in ?
ImportError: cannot import names Error, __version__, 
  writer, reader, register_dialect, unregister_dialect, 
  get_dialect, list_dialects, QUOTE_MINIMAL, QUOTE_ALL, 
  QUOTE_NONNUMERIC, QUOTE_NONE, __doc__

Now, I have a different error, but the fact that the first error has disappeared means that Jython has recognised my new _csv module! The new error is just Jython complaining because _csv doesn't define any of the identifiers that it is expecting.

Some of the missing identifiers are simple to resolve, like __doc__ which is just a string. Others are more difficult and will require further investigation like Error which is an exception and I don't know how to do exceptions in Jython yet. For now, I will just add everything as a PyObject to get past the error, then I will revisit each in turn. Here's _csv.java as it looks after adding all the missing identifiers:

public class _csv {
    public static PyObject Error;
    public static String __version__ = "1.0";
    public static PyObject Dialect;
    public static PyObject writer;
    public static PyObject reader;
    public static PyObject register_dialect;
    public static PyObject unregister_dialect;
    public static PyObject get_dialect;
    public static PyObject list_dialects;
    public static PyObject QUOTE_MINIMAL;
    public static PyObject QUOTE_ALL;
    public static PyObject QUOTE_NONNUMERIC;
    public static PyObject QUOTE_NONE;
    public static String __doc__;
}

Now, when I run test_csv.py I get the following error:

Traceback (innermost last):
  File "dist/Lib/test/test_csv.py", line 8, in ?
  File "/work/jython/dist/Lib/csv.py", line 87, in ?
TypeError: call of non-function ('NoneType' object)

Although the error isn't very helpful, I can go to line 87 in csv.py (just by clicking on the error in Eclipse) and see that Jython is unhappy because register_dialect is supposed to be a method yet I have defined it as a PyObject, so now I can forget about the other identifiers and focus on getting this method to work.

This is where this entry ends while I go and figure out how methods and dynamic arguments work in Jython!