Quest in Time

Sunday, January 16, 2005

Domain Specific Languages - a reflection on past projects

I've yet to get my hands on the Microsoft DSL Tools beta, as I first have to get the Visual Studio 2005 beta and its only available on CD (as far as I can tell).

I have managed to read up a bit, and at least it turns out not be something that Microsoft dreamed up. I'm still feeling my way here, and articles like this one in Martin Fowlers bliki, give me a sense of the size, and complexity, of the subject without making me feel I understand it.

It has caused me to reflect however on the last major project I worked on (in a previous organisation). This was a suite of bespoke applications for use in UK electricity market. In order to give some indication of scale here are a few statistics.

  • Length - 2 years to live plus 2 years on-going development
  • Team - on average about 6 Java developers, 1 ASP developer, 2 Analysts, and 8 testers/test automation developers
  • Size - 3 applications comprising about 5000 Java classes
  • Technology - Java (1.6.1), ASP, CORBA, SQL Server

This project started when Java was very young, and J2EE was still years away, but the idea was to deliver an enterprise application, with a B2B messaging infrastructure, workflow, distributed processing, and a web based interface. In the absence of standard solutions for much of this the project had to roll its own.

The project built 4 key infrastructure components on which the entire solution was based.

  • A Database Agent to manage persistence, object relational mapping, instance management, and compiled queries
  • A Job component to manage distributed processes over CORBA
  • A Message component to handle parsing and validation of all incoming and outgoing messages
  • A Workflow component to handle consequential processing of incoming messages, exception handling, manual steps, and escalation

I believe that in each case Domain Specific Languages played their part even if I didn't think of it this way.

The project used most of the Rational tools suite, Rose, RequisitePro, and SQA, and UML throughout the life cycle.

The message catalog and business workflow documents were centrally produced for industry wide use. Each was produced by a separate body and they did not synchronize their issue. So we could get a new version of the catalog that defined messages that were not used in any process, and new versions of the processes that referred to messages that were not in the catalog. As the rate of change was significant, we found that we were repeatedly throwing away are Use Cases and starting again. We defined some use case stereotypes which separated use cases in to message handling, and process types. After we started using these we never threw the model away again. We also standardised on a way of laying out the use case diagrams. Both these standards and the stereotypes were only used on diagrams of the messaging use cases, so I believe that using the extensibility mechanisms in UML we had created a DSL for the B2B massaging domain.

The Database agent design originally required that the developer implement a lot of code for each persistent object, and prepare the SQL tables by hand. A redesign introduced a metadata database, and a code generator which would automate this process (outputting about half the Java files mentioned above). I think that this metadata database was implicitly a DSL for our Persistence domain.

As Activity charts were not part of either UML or Rose at this time, we used state charts to diagram workflows in analysis, but had build a data driven workflow engine to implement and execute them. The mapping between the UML statecharts and the workflow metadata was explicit and may well have had DSL aspects.

In the first case above the DSL was formed through the extensibility mechanisms inherent in UML, a more extreme example can be found in the work of Jim Conallen in modeling web based applications in UML.

In the latter 2 cases while UML was still used at some point a non-uml tool took over that was designed deal with the specific domain.

I may still be missing the point, so I'll keep looking for further enlightenment.