Lightweight literate programming

The purpose of literate programming is to erase the distinction between source code and documentation. The author writes a single narrative, describing algorithms, data structures, and design philosophy, and the source code is embedded at appropriate locations in this narrative.

The idea is that you do not document programs (after the fact), but write documents that contain the programs. [John Max Skaller]

The primary early proponent of literate programming was Dr. Donald E. Knuth. For a good summary of the history and current work in literate programming, see the Literate Programming website.

The author was inspired by recent work of his colleague, Dr. Allan M. Stavely, in devising lightweight approaches to literate programming. We have a peer-reviewed journal article on the subject:

Stavely, Allan, Lynda Walsh, and John Shipman. Lightweight literate programming: a documentation practice. Technical Communication 55(1), Feb. 2008.

In the author's current toolchain, the input is written in a slightly customized dialect of DocBook. Automated tools produce the two primary forms of output:

• The ordinary DocBook toolchain processes the source file as a document, producing both Web and PDF formats. For details of this toolchain, see Writing documentation with Docbook-XML 5.0.
• A small Python script extracts one or more source files from the document, and these are the executable form of the program. See A source extractor for lightweight literate programming. This script is itself done in the literate style, and the link here is to the generated document. See also the source for litlxml, the generated source file.
• For source extraction from DocBook 5.0 documents, see litdocbook5: A source extractor for lightweight literate programming.

Following is a list of numerous completed projects done by the author in the literate style, with comments about points of interest in that product. All these projects also use The Cleanroom software development methodology.

• Python: Basic examples
• Tkinter: Graphic user interfaces in Python
• XML: Python applications with eXtended Markup Language
• XSL-FO: Generating PDF with XSL Formatting Objects
• Literate programming of LaTeX templates
• Zoological applications of Python
• Projects for the New Mexico Tech Computer Center (TCC)

Python: Basic examples

• rational.py: An example Python class: An abstract data type for rational numbers.
• A skip list container class in Python: An abstract data type for sorting and searching ordered sets in O(log n) time.
• Graphics transformations with homogeneous coordinates. Examples of matrix algebra using numpy, Numeric Python; includes formulas written in LaTeX.
• A Python module to generate a Key Word In Context (KWIC) index
• sidereal.py: A Python package for astronomical calculations
• deduper: Find large duplicate files. A disk management utility that uses a small SQLAlchemy database built on SQLite.
• tracecase: Trace table coverage checker.
• PathInfo: A file information object in Python. A small object-oriented systems programming example.
• Scanning objects for Python: Scan and Log. Support for small recursive descent compilers, and a base class for the Singleton design pattern.
• A sudoku solver: A study in recursive puzzle-solving.
• kkck: A solver for Kriss Kross puzzles. A more sophisticated puzzle solver combining iterative and recursive stages.

Tkinter: Graphic user interfaces in Python

• huey: Internal maintenance specification. A sizeable Tkinter application for selecting colors and fonts.
• FontSelect: A Tkinter widget to select fonts
• ScrolledList: A Tkinter scrollable list widget. Example of a compound widget.
• pageturner.py: A Tkinter widget for turning pages
• mazeratty: A rattier maze generator. A novel algorithm for generating pencil mazes of satisfying complexity.

XML: Python applications with eXtended Markup Language

• Python XML processing with lxml. Primarily a manual, but includes a literate module to help build XML files.
• The ISO 9573-2003 Unicode entity group. Demonstrates reading and writing XML.
• sox.py: Streaming output of XML for Python. For generating XML files too large to fit in memory.
• pyrang: A single-sourcing tool for Python-XML applications. Extracts tag and attribute names from an XML schema for use in Python scripts.

XSL-FO: Generating PDF with XSL Formatting Objects

• docbookindex: ID indexer for DocBook. Direct generation of PDF using XSL Formatting Objects. See also the DocBook 5.0 version.
• listext2: A text file listing utility. Generates PDF.
• fohelpers: XSL-FO helper functions for Python. Used by docbookindex and listext2.
• fosox: Generating XSL Formatting Objects in Python. Uses the sox.py module, for situations where memory size is limited.

Literate programming of LaTeX templates

• NMT Thesis Stylesheet: Internals: Stylesheet for our local thesis style, documented in Writing a New Mexico Tech thesis with LaTeX.
• A LaTeX template for NMT Student Research Symposium Extended Abstracts

Zoological applications of Python

• xnomo3: A system for representing bird taxonomy. Infrastructure for many ornithology-related projects. The code is exhibited in xnomo3.py: Internal maintenance specification.
• nomcompile3: Build a set of nomenclature files. Compiler for the bird taxonomy system.
• pycbc: A Python interface to the Christmas Bird Count database. Demonstrates an object-relational database mapper.
• Christmas Bird Count history report system. CGI front end for pycbc: generates a report showing the complete history of any circle or circles.
• A system for encoding bird field notes, using XML.
• noteweb: Rendering bird field notes for the Web. Generates my online birdwatching field notes.
• The abalist program: computing American Birding Association list totals. Bookkeeping for bird records.
• birdnotes.py: Objects to represent bird note files. Infrastructure for the above.
• catweb: A Web-based bird photo catalog. Your author is a birdwatcher and has thousands of catalogued bird photos. You can see thumbnails of all of them.
• An XML-based bird image cataloging system. Infrastructure for the above application.
• archx: A program to index a photo archive. Uses the Relax NG schema language.
• iband7: IBP banding data entry system: A complete data entry application with multiple scripts. The code is presented in IBP banding data system: Internal maintenance specification.
• IBP banding data system: emacs customizations. Emulation of certain IBM 029 keypunch functions in the emacs text editor.

Projects for the New Mexico Tech Computer Center (TCC)

• homelist2: A Python script to generate a list of user homepages. Clickable list of all locally hosted homepages; uses the KWIC module.
• manweb: A CGI script to display man pages
• Courseware Banner integration tools: Imports Banner registration information into the Moodle courseware server.
• Customization of the 4.3 DocBook XSL Stylesheets for the TCC
• cmsadds: A GUI for CMS course creation
• hwscan3: TCC processor inventory report; the actual code is in hwscan3 IMS: TCC processor inventory report.
• tccwebstats 4.0: TCC web server statistics: Specification for a Web-based report showing access statistics for locally hosted Web pages. The code is presented in Webstats 4.0: Internal Maintenance Specification.
• tccpage2.py: Dynamic generation of TCC-style web pages with lxml

Last updated: 2014/05/09 20:39:32
URL: http://www.nmt.edu/~shipman/soft/litprog/homepage.html