Archive for the ‘Model-Driven Architecture’ Category
Text, Expressivity and Culture-Oriented Programming
Following up on my reflexion about what could software development look like a few years or decades from now, there is this big problem that has been bugging me for years now and that I have never found the time to really tackle: expressivity. In the same way as files appear to me as the biggest obstacle to collaboration, I think the main barrier in the way of expressivity is TEXT.
It’s hard to admit, but we’re still building software like cavemen. We don’t have spoken language, just a bunch of noises, we don’t conceptualize much but we do communicate with a few gestures and more importantly some colored drawings on cave walls. The way I see it, we are not much more advanced than that, but it’s normal, software is still relatively young as a discipline and although it has already changed our lives, we have to imagine that it’s just the beginning. And the good news is that we are headed in the right direction.
We started off with most elementary way of storing information and communicating with a machine: zeros and ones. Binary. It was too elementary, more like noises coming out of our mouths, so we started to group bits in octets corresponding to hardware instructions and characters. In fact, we added gestures to noises. Then we grouped instructions into statements and procedures, and we designed a way to translate those into the most elementary form of language that machines could understand. We started drawing on walls. But as procedures multiplied like crazy, we needed to conceptualize some more, talking about classes, objects, methods, properties, and so on. Spoken language was born. And with higher level concepts like services, components and multiple programming languages, we added written language. OK, the analogy is not that good, but you get my point: I’m convinced we’re still very early in the overall evolution of communication with machines, and although this evolution is somewhat slow and creates a lot of inertia, I believe that if we want computers to really expand our capabilities (note that I didn’t say “replace us”), we need to go further in abstraction levels.
So what’s next? Binary, assembly, procedural, object-oriented (yes, and functional, if you want), then what? Model-driven? I’ve tried that, it’s just replacing the constraints of text with the constraints of visual representations. It sure makes it easier to conceptualize, but at some point we’re still translating those visual models into text code, which we have to compile. The roundtrip is just too long. What about domain-specific languages? Well, I’m more into that right now. It looks like communicating is naturally based on languages, collections of concepts that relate with one another to describe what a software is and what it does. So focusing on making it easier to define new languages definitely goes in the right direction. That’s why it’s so linked with meta-programming: instead of statically defining layers upon layers of fixed concepts to describe systems with even higher level abstractions, let’s define the root concepts we will use to describe languages that will allow us to describe our systems. That’s why I’m so interested in Groovy at the moment for internal DSLs, although I prefer the more elegant idea of external DSLs and language workbenches, like Jetbrains MPS does.
All of this evolution makes me think of a video I saw recently, that tried to make String theory more accessible:
Let’s say binary is our dimension 0. Assembly is dimension 1: a line. Procedural programming is dimension 2: a plane. Object-Oriented Programming is then 3rd dimension: a volume. And it’s very hard for us to leave it, as it is our most natural way of seeing things. But that’s where I find this explanation of String theory particulary interesting (although not rigorous as some math geek friends of mine pointed out): there starts a cycle. Dimension 4 is a line again, formed by 2 different Object-Oriented Languages, like Java and C++ for example. Still there? Good! That’s where the fun starts: dimension 5 is a plane composed by all the parallel universes that are created by our own choices. Functional programming and OOP can be considered as forming such a plan. Now what if we could design a way to go directly from one of those paradigms to another one, to fold the plane in the 6th dimension: please welcome language-oriented (or meta-) programming! See the cycle? Now most of us are stuck in the 3rd dimension, and some of us are already experiencing the 6th dimension.
So there we are. Seventh dimension is the line joining the set of all possible timelines starting from our software big bang, which is the binary transistor, to another set of possible timelines, starting from another big bang. Quantum computing can be another option, but it’s a hardware one. What about software? Isn’t virtualization a way to forget about the physical hardware? And there we go 8th dimension: going from binary transistors to quantum computing is one line in the seventh dimension. Choosing to go to virtualization instead creates an branching line in the 8th dimension. Which means that if we want to create our ninth dimension, we need to fold the eighth in order to jump from quantum to virtual computing. And that’s where I locate what I call Culture-Oriented Programming. The third stage of the third 3-stage cycle. The final frontier? The next step? Hooooo… my head hurts.
But wait a second, I only talked about computing here. A virtual reality. What if dimension 10 was the line uniting computing with the real world in the purest possible way. Direct communication between human beings and computers. Who said “scary”?
PS: I didn’t intend this post to be so “theoretical”. I only thought with my keyboard and let my imagination go. But I’d love to know what you think about that crazy analogy? Do you think we are limited to 11 dimensions like in string theory?
PPS: That might be my geekiest post EVER!
2008 flashback
That’s it. Holidays are here, this year is almost over, time slows down for a few days so that we can look back, then enjoy and start again.
This post is probably not be of great use to anyone but me, but it’s a blog after all, sometimes I can use it as a personal journal too, like all those teenagers.
This year was actually very rich to me. I got to discover a few very interesting technologies, starting with OSGi and its ability to break up with big monolithic enterprise applications. And I really dived into Flex and how it can integrate with Java, which culminated in this tutorial, which was even republished on the Adobe Developer Connection. And finally, in my quest to discover better ways to develop applications, I came across Jetbrains MPS. And when I realized it was still far from being ready for prime time, I decided to keep it on my radar but focus on a more realistic alternative on a shorter term: Groovy/Grails. Now I’ve widened my technological scope beyond just JEE and AndroMDA, and I feel quite ready for the challenges ahead.
More importantly, this year will have been a great initiatic journey. Last year at about the same period, I decided I wanted to turn one of my ideas into a company, to create my own startup. The road was bumpy, my colleagues at Axen could feel it, friends who joined me on this journey could feel it too. I put a lot of things in question: my job, my ambitions, my strengths and weaknesses. And the project failed… so far! But they say there’s always hope at the end of the tunnel.
I had the idea, I had the skills, I had the will, time and energy, and yet there was this big barrier to entry, this thing that seems so important to everybody that nations and societies, people and governments are falling apart just because of it: money. I’ve always felt bad about money, like it was the poisonous blood flowing in the veins of an otherwise healthy man, eating him from the inside out, like something that is here to transport what makes us live, but that we’ve come to confuse for life itself. And it happened, crisis happened, the disease hit us all. We’re still far from the bottom since we still believe that the best cure against poisonous blood is more blood. But I don’t want to wait. My failed project and the recent course of events have made me realize something: THERE’S GOTTA BE A BETTER WAY!
2009 is full of promises. I have already started to understand many things about myself, about why I failed on this project, about how I live, how I can improve. I moved to a new place too, more comfortable, more like me. And now I have this new project, this game changer, this new way of thinking about how creative people can turn ideas into something real. 2009 is going to be fan… wait for it… tastic!
Talk Less, Do More
I’ve been a Java developer for almost 8 years now, I’ve used many frameworks and tools, Spring, Hibernate, EJB’s, Swing, etc. And a few years back, I started to look for a less verbose way to express concepts like business entities, services, workflows and so on. For some time, Model-Driven Architecture (MDA) has been the best answer I could find. But the learning curve and the difficulty to customize transformation patterns for specific setups, like a Flex front-end on a Spring back-end for example, have made me realize that this approach was somewhat limited by its UML foundation.
So I started looking for a better answer, and I thought I had found it with Language-Oriented Programming, the idea to decouple the structure, the syntax and the semantics of a language and ease the creation of Domain-Specific Languages in a very generic way. Unfortunately, as promising as it looks, it’s still in early stages of research and development and I needed a more immediate way to improve my productivity, my ability to do more with less words.
So I came back to something I had totally pushed aside till then: scripting languages. I’ve been “raised” with strongly and statically-typed languages since day 1 of my programming carrier: first Pascal, then Delphi, C++ and Java. And I always thought that scripting languages like PHP, and later Ruby were just tinkerer stuff for lazy hobbyist programmers, that didn’t take into account constraints like maintainability, robustness, readability, performance, etc. All those constraints I faced as a professional consultant. But a few weeks ago, I started to question my certainties and began wandering around PHP, Ruby, Python and others.
PHP, I’m using it at work on You And The World, it’s too hard to debug in a Flex/AMF setup and it’s really too permissive. As for Ruby, I like the ability to design DSLs but the syntax is definitely not intuitive enough for me. So now, I’m trying to learn more about Python, whose syntax seemed more familiar and maturity is reassuring. Everything seemed good until I read this single sentence in the documentation:
[...] classes in Python do not put an absolute barrier between definition and user, but rather rely on the politeness of the user not to “break into the definition.”
Come ooooon! Politeness of the user? You must be kidding right. In a corporate environment, when your code is likely to be reused and shared and refactored? I thought “hey, that’s probably just a provocative way to say that we Java developers tend to overdesign, and I know it’s true… so let’s read on…”. But unfortunately it proved to be much more than just a provocative sentence, since a few lines later:
In fact, nothing in Python makes it possible to enforce data hiding — it is all based upon convention.
OK. It took me a while to accept the argument that static typing can be too much of a cost in terms of verbosity compared to the poor protection it offers in terms of code correctness. But I said “OK, we’ll see how it looks when we come to real code”. But now they want me to forget about encapsulation too! What is the purpose of even writing a class then? What are you describing? A library of functions with a hidden first parameter? Is that it?
I mean, typing might be a unnecessary hurdle when writing unit test becomes much easier, but giving up on the behavior/state balance, the ability to describe what a class can do AND what it looks like, this is too much to ask.
So now what? Am I stuck between two extreme alternatives: either describe everything and waste energy doing it, or describe nothing a pray/ask for politeness?
And then I thought about LOP, and this interesting idea to decouple structure, syntax and semantics. After all, programming languages have entangled those three aspects for decades. And now both mainstream platforms (Java and .Net) are offering a variety of languages running on the same Virtual Machine. Isn’t that a first step in decoupling syntax on one side, structure and semantics on the other side?
Suddenly I’m starting to dream of a new syntax, far less verbose than the one of the Java programming language but with the same rigorous Object-Oriented structure, and the ability to run on all runtimes with different semantics…
What if the following Java code:
{code type=java}
public class TodoItem extends Item implements Synchronizable{
private String title;
public String getTitle(){return title;}
public void setTitle(String t){title=t;}
protected boolean done;
public TodoItem(String title){this.title=title;}
public void check(){done=true;}
}
{/code}
Could be rewritten like this:
{code}
+ class TodoItem > Item @ Synchronizable
- title
+ <-
return title
+ -> t
title = t
#done
init t
title = t;
+ check
done = true
{/code}
…or something similar ;o)
The key ideas being:
- Keep all the key concepts of OOP: encapsulation, inheritance, polymorphism, composition
- Remove static type declaration: I don’t care that my hammer has a wood or a steel handle, but I need to know it has a handle.
- Replace long keywords by well-identified signs
- Replace “{}” and “;” by indentation (Python-like)
- Add simple property syntax (I love the way ActionScript does it)
In other words, I’m looking for a middle-path alternative between Java and Python, between waste and pray, between overdesign and politeness. Does anyone know of a language that offers that kind of compromise? Does anyone know if it has already been tested but has proven to be a dead-end for some reason?
The End of an Era, the Beginning of a New One
I’m a pure product of the Object-Oriented Programming generation. I’ve experimented several different paradigms when I was in engineering school, including formal programming in B, functional programming in OCaml, logical programming with PROLOG and Eclipse (the programming language, not the IDE). But when I started writing software in high school, it was with TurboPascal and then Delphi. And the main language I’ve been using during my studies was Java.
During all that time, I essentially worked on relatively simple problematics, and my end-of-study project was the first one where I started to look for better ways to handle a more complex environment, namely web applications and web services. The revelation came from AndroMDA, an Open-Source Model-Driven Architecture framework that has been my main specialty and domain of research during the past three years or so. As a professional IT consultant, I’ve used it and promoted it on at least 3 customer projects with rather great success.
But today, I think this time is over. I’ll move on to the next step, my next toy. And if you’ve read my previous entry about the limitations of OOP and its comments, you might guess what it is. I was looking for a better way to tackle today’s software issues and go further in my philosophical goal to make technology useful, and I found it. It’s called Language-Oriented Programming.
If you want to know more about it and wonder why I moved onto this so quickly, here are the articles that hit me as another revelation during the past couple of days:
- Language Oriented Programming, by Sergey Dimitriev
- Language Workbenches, by Martin Fowler
- The bridge between MDA and LOP, by Martin Fowler
Now I think I’ll have a much closer look at the practical side of things:
All of my personal projects, including MobiMap, will certainly benefit from this new enlightenment. And I sincerely hope that my day-to-day job will give me the opportunity to work with those concepts…
What’s there after Objects?
As far as I can tell, there has been 3 major “revolutions” in software development since it all started:
- From binary to assembly language
- From assembly language to procedural languages
- From procedural languages to Object-Oriented Programming
Of course I’m not even talking about marginal evolutions like functional programming, scripting or formal programming.
And behind a revolution, there’s always a massive “there’s gotta be a better way” feeling. For assembly language people probably thought, “since we’re working with registers and memory addresses and instructions, let’s have readable forms of these concepts rather than just hexadecimal code, obviously we’re going to be more productive”. And then it was probably the same with procedural vs. OOP: Objects, classes, fields and methods were just closer to the kind concepts we were manipulating at some point and we couldn’t just keep tinkering with old paradigms, even on steroids (read structures and things like that).
Now I’m starting to wonder whether we’ve not come to the same point with OOP. At least I have come to this point myself. Objects, packages, components, whatever the granularity I’m still stuck having to tinker with old stuff to realize higher level tasks at some point. Most of the time, I’m not thinking about adding a field or a method in an object, but instead I want to add a property to an entity in my domain, or expose a business operation via a webservice. That’s the kind of tasks that make my day-to-day work. And still, let’s say I want to add a property to an entity in my domain, then I have to add the corresponding field in a class, write a getter and a setter, update the DDL script for the database, add the mapping to the Hibernate mapping file, add a validation directive somewhere… there’s gotta be a better way.
Now I realize that’s the main reason I love AndroMDA so much: because I can do high-level modifications in my UML models and boom! The generator just takes it over from there and updates the code almost everywhere it needs to be, which improves my productivity by a tremendous factor. Others use other tricks for the very same purpose: some use annotations, others turn to active record implementations. But whatever the trick, I feel more and more like it’s a dead-end, like… there’s gotta be a better way.
The closer the tools I’m using are to the concepts I manipulate, the less work I have to do to implement those concepts, the better it is for the whole project:
- it reduces the probability for errors
- it improves time-to-market
- it frees up some time to focus on the real business at hand
- it gives me a better sense of achievement
- it dramatically improves return on investment since basically you’re doing more with less
Tonight I had a very interesting conversation with a very smart guy from the AndroMDA team, and it just appeared to me like magic: AndroMDA is just one way to cope with limitations of the underlying object-oriented paradigms, and we have to start thinking about how we can reach the same goals, but in a more native way. We need a brand new paradigm, a new way to build software closer to what we try to achieve everyday at a higher level.
What do you guys think? Have you heard of anything like that? Because I would be really interested in knowing about it. Otherwise, does anyone have any idea about what could be this new paradigm? And do you feel we need it? I’m really interested in your feedback on that one.
Goal-Driven Development
It’s always amazing to see how people find it hard to compromise, to nuance their opinions about things. And I’m probably the first one finding it hard. But when it comes to software practices, boy it’s annoying.
I was talking to a colleague of mine the other day, we were talking about unit testing and other best practices, and I boldly and shamelessly said to him: “I haven’t written any unit test on this project.” He stared at me with a bit of surprise, then with a bit of disappointment and finally with a bit of something like… disgust!
“What!? No tests, but why? How can you pretend to be a software architect if you don’t practice systematic unit testing?! Oh my god!…”
My first reaction was a furious urge to hit him in the face. But then I remembered myself a few years back, and I did nothing but leave him in his panic state and disappointment.
When I started working as a consultant, I had already written a few applications in engineering school, but strangely enough they didn’t really talked to us about software quality, agile methodologies and other modern best practices. We did formal programming, Prolog, we reprogrammed a memory management unit and wrote a compiler for a subset of the Java language, but nothing useful ;o).So when I discovered that some people were actually doing things in a rational way, to write better software, I was at the same time impressed and overwhelmed by all of that. Design patterns, continuous integration, test-driven development… I had never seen those in the real world, but I read a lot of books and websites and it seemed so nice that I came to think that it was the only way. And of course I was wrong.Fortunately, I started working on real-world projects, dealing with all sorts of external constraints. And no matter how hard I tried to impose “my” great software best practices, everybody looked at me like an alien because they just didn’t understand why I was talking about all this stuff. So when I was fed up with complaining about no one understanding what should be done, I realized that they were at least right to wonder. Why are things like TDD, CI or Design Patterns so important?
When you read so many articles explaining to you how to write unit tests, they all seem to assume that unit-testing is good in itself, as if it was some kind of axiom. But the truth is that no matter how good they are, writing them and maintaining them is a real pain, right? Lesson learned: when something is painful but good for you, there’s gotta be a better way. But a better way to what?
Because for me that’s the real problem: losing sight of the objective. Unit-testing and TDD are not good in themselves, they are just one best practice which allows you to improve the overall quality and robustness of your code. Why so? Because when you write your tests before the corresponding functional code, you considerably reduce the probability of a bug in the functional code. And robustness comes from the fact that whenever the implementation of a given functionality is modified over time, unit tests are there to ensure that the overall result will remain the same. Fine! But how about a better way?
First, let’s talk about reducing the probability of bugs. The main problem with code is that it’s written by us, human beings. And like it or not, we are faillible, we make mistakes, and the more we think we don’t, the more we actually do. Now rather than testing everything that we write, isn’t there a way to reduce the amount of code that we have to write? I can tell you that: there is a way!
I already see my colleague smiling if he reads this, because I’m so obsessive about this “way” that it becomes ridiculous. Take care, here comes the dirty words: code generation. Of course I’m not talking about the kind of code generation that just displaces the problem elsewhere, that is in diagrams that are the ont-to-one mapping of your code. I’m talking of a higher level of code generation, I’m talking of Model-Driven Architecture.
I know that many programmers don’t like it, have read many articles about and introductions to MDA without ever going further, mostly because programmers like to… program, not draw. And I can understand that. And if you’re in that situation, then you’re stuck with test-driven development… or any other “way” you can find. But if you like to have a better view of the big picture, if you want to accelerate your development and increase the robustness and quality of your code, then have a look at this (my favourite), this and this. You might be surprised.
Will it completely replace unit tests? I don’t think so, but it can create a situation in which you have so dramatically reduced the amount of code to test, that it can be interesting to wonder whether it’s still worth the effort.
My main point is, software best practices are nothing without the goal they allow you to reach, and forgetting that is the best way to stop looking for better ways, to stop innovation and progress.
AndroMDA, an Open Source MDA toolkit
I gave a quickie at Javapolis today about AndroMDA. It was a first for me, and boy it was hard! My goal with this presentation was to act as a counterbalance to last year’s scripting language hype, to show that yes, you CAN be productive with traditional Java frameworks. It’s just a matter of tools and methodology.
And because I wanted to demonstrate that, I actually challenged myself into developing a full-blown JSF/Spring/Hibernate application in front of the audience in just 15 minutes. And I must admit that I had my eyes bigger than my stomach on this one. I’ve repeated my demo something like ten times, and I only managed to do it once in less than 15 minutes. So that one was risky. And as a matter of fact, I made just a small mistake in my modeling that actually led me to fail showing my running application.
But just for the sake of precision, and in case you need to see it working, I’ve actually attached the final application to this post: TodoList sample application. And here is what you need to do to build it:
- You have to install Java 5, Maven 2.0.7 (I’ve not tested it with Maven 2.0.8), MySQL 5, JBoss 4.0.5 GA
- Download mysql-connector-java-bin-5.1.5.jar and copy it to server/default/lib subdirectory of your JBoss installation
- Set JAVA_HOME, JBOSS_HOME and M2_HOME environment variables accordingly
- Create a database called ‘todolist’, and allow a user named ‘todolist’ with a password as ‘todolist’ to do everything on this database
- Run the following commands in the unzipped directory:
mvn -f core/pom.xml andromdapp:schema -Dtasks=create mvn install -Ddeploy
- Run JBoss
- Browse to http://localhost:8080/todolist
Please let me know if you have any issue or question related to this sample project. Comments are here for that.
And I’ll probably use this sample application as a starting point for a more complete learning thread so stay tuned.
And by the way, if you need more information about AndroMDA itself, please refer to their official website. The AndroMDA forum is very active as well, and if you can’t wait to get started, you will definitely love the official Getting Started Guide, which exists in Java and .Net versions.
Javapolis, D-Day minus 2
We’re saturday and I just finished to prepare the code for my quickie about AndroMDA. Hopefully I’ll be able to make it in 15 minutes. I make it a personal challenge.
Of course, the code will be published here after the presentation, but that’s not all. Now that the code is ready, I intend to prepare a small screencast, not limited to 15 minutes, so with more time to actually explain what’s going on behind the scenes.
But there’s even more. I think I’m going to use this demo application as a starting point for a whole training thread in videos, maybe even a video podcast, and I hope it’s gonna help people get started with the tool in real life.
See You at JavaPolis
That’s official: my AndroMDA Quickie proposal has been accepted and I’ll be giving a 15-minute demonstration of AndroMDA power on Wednesday, December 12th. That’s really great! Now I’ll just have to figure out the best possible way to make my point. For now, my objective is to realize some sort of todo-list demonstration in just 15 minutes. That’s short but I think that’s the only way I can get people to understand that scripting languages are not the only way to be productive with Java.
Now I still need to further test AndroMDA 4 to see if it’s ready for such a demonstration.
Anyway, if you’re curious about MDA in general and AndroMDA in particular, feel free to come by and ask plenty of embarrassing questions.
