Rich Internet Applications

My Case for DTO’s

In many of my posts about Grails and Flex integration, I take for granted that I use Data Transfer Objects to transfer data between my Grails backend and my Flex frontend. Put simply, Data Transfer Object are pure data containing classes different from the domain entity classes used to store data in the backend. I take it for granted because I’m deeply convinced that it’s the best way to do things and so far, experience has never proved me wrong. But I often get this question in comments or by mail (this is for you Martijn): why bother create an entirely separate class structure and copy data from entities to DTO’s and back instead of just using entities?

I’ve expressed my arguments a couple of times across various posts but I thought it would be nice to sum things up in here for future reference.

Where does it come from?

When I first started to work on enterprise applications and ORM-based architectures, it was with a Model-Driven Architecture framework called AndroMDA. AndroMDA was absolutely key in helping me getting started with Spring and Hibernate and I was especially inspired by one paragraph in their “getting started” tutorial, which I quote here:

Data Propagation Between Layers

In addition to the concepts discussed previously, it is important to understand how data propagates between various layers of an application. Follow along the diagram above as we start from the bottom up.

As you know, relational databases store data as records in tables. The data access layer fetches these records from the database and transforms them into objects that represent entities in the business domain. Hence, these objects are called business entities.

Going one level up, the data access layer passes the entities to the business layer where business logic is performed.

The last thing to discuss is the propagation of data between the business layer and the presentation layer, for which there are two schools of thought. Some people recommend that the presentation layer should be given direct access to business entities. Others recommend just the opposite, i.e. business entities should be off limits to the presentation layer and that the business layer should package necessary information into so-called “value objects” and transfer these value objects to the presentation layer. Let’s look at the pros and cons of these two approaches.

The first approach (entities only, no value objects) is simpler to implement. You do not have to create value objects or write any code to transfer information between entities and value objects. In fact, this approach will probably work well for simple, small applications where the the presentation layer and the service layer run on the same machine. However, this approach does not scale well for larger and more complex applications. Here’s why:

  • Business logic is no longer contained in the business layer. It is tempting to freely manipulate entities in the presentation layer and thus spread the business logic in multiple places — definitely a maintenance nightmare. In case there are multiple front-ends to a service, business logic must be duplicated in all these front-ends. In addition, there is no protection against the presentation layer corrupting the entities – intentionally or unintentionally!
  • When the presentation layer is running on a different machine (as in the case of a rich client), it is very inefficient to serialize a whole network of entities and send it across the wire. Take the example of showing a list of orders to the user. In this scenario, you really don’t need to transfer the gory details of every order to the client application. All you need is perhaps the order number, order date and total amount for each order. If the user later wishes to see the details of a specific order, you can always serialize that entire order and send it across the wire.
  • Passing real entities to the client may pose a security risk. Do you want the client application to have access to the salary information inside the Employee object or your profit margins inside the Order object?

Value objects provide a solution for all these problems. Yes, they require you to write a little extra code; but in return, you get a bullet-proof business layer that communicates efficiently with the presentation layer. You can think of a value object as a controlled view into one or more entities relevant to your client application. Note that AndroMDA provides some basic support for translation between entities and value objects, as you will see in the tutorial.

Because of this paragraph, I started writing all my business services with only data transfer objects (what they call “value objects”) as input and output. And it worked great. Yes it did require a little bit of coding, especially as I had not discovered Groovy yet, but it was worth the time, for all the following reasons.

The conceptual argument: presentation/storage impedance mismatch

Object-relational mapping is what Joel Spolsky calls a “Leaky Abstraction“. It’s supposed to hide away the fact that your business entities are in fact stored in a relational database, but it forces you to do all sorts of choices because of that very fact. You have to save data in a certain order in order not to break certain integrity constraints, certain patterns are to be avoided for better query performance, and so on and so forth. So whether we like it or not, our domain model is filled with “relational choices”.

Now the way data is presented involves a whole different set of constraints. Data is very often presented in a master/detail format, which means you first display a list of items, with only a few fields for each item, and possible some of those fields are calculated based on data that is stored in the database. For example, you may store a country code in your database, but you will display the full country name in the list. And then when the user double-clicks an item, he can see all the fields for that item. This pattern is totally different from how you actually store the data.

So even though some of the fields in your DTO’s will be mere copies of their counterparts in the entity, that’s only true for simple String-typed fields. As soon as you start dealing with dates, formatted floats or enum codes, there is some transformation involved, and doing all that transformation on the client-side is not always the best option, especially when you have several user interfaces on top of your backend (a Flex app and an iPhone app for example), in which case you’re better off doing most of these transformations on the server.

In anyway, if you change the way you store data, it should not influence too much the way you present the same data, and vice-versa. This decoupling is very important for me.

The bandwidth argument: load just the data you need

In the master/data use case, when you display the list of items, you just need a subset of the fields from your entities, not all of them. And even though you’re using Hibernate on the backend with lazy-loading enabled, fields are still initialized and transferred over the wire. So if you use entity classes for data transfer, you will end up transferring a whole bunch of data that may never be used. Now it might not be very important for hundreds of records, but it starts being a problem with thousands of records, especially when there is some parsing involved. The less data you transfer the better.

The security argument: show only the data you want to show

Let’s say you’re displaying a list of users, and in the database, each user has a credit card number. Now of course when you display a list of users, you might not want everyone to see the list of credit card numbers. You might want to expose this data only in detail view for certain users with certain privileges. DTO’s allow you to tailor your API to expose just the data you need.

The error-prone argument: argh! Yet another LazyInitializationException!

Of course there are associations between your business entities, and by default, those associations are lazy-loaded, which means they are not initialized until you actually query them. So if you just load a bunch of instances from your entity manager and send them over to your client, the client might end up with null collections. Now of course you can always pay attention, or use some tricks to initialize associations up to a certain level before you send your data, but this process is not automatic and it’s very error-prone. As for using things like dpHibernate, I think it just adds too much complexity and uncontrolled server requests.

The laziness argument: Come on! It’s not that hard!

I think that most of the time, the real reason why people don’t want to use DTO’s is because they’re lazy. Creating new classes, maintaining code that does “almost” the same as existing code, adding some code to service implementation to copy data back and forth, all of that takes time and effort. But laziness has never been a good reason for ditching a design pattern altogether. Yes, sometimes, best practices force us to do more stuff for the sake of maintainability and robustness of our code, and for me the solution is certainly not to shortcut the whole practice, but just to find the best tools to minimize the added work. With its property support and collection closures, Groovy makes both creating, maintaining and feeding DTO’s as simple and fast as it can be. AndroMDA had converters. There are even some DTO-mapping frameworks like Dozer to help you. No excuse for laziness.

For me, all the reasons above largely overcome the added work to maintain a parallel DTO structure.

Now of course, this is a very opinionated topic and you will probably have a different view. So all your comments are welcome as long as they remain constructive and argumented.

By Sébastien, ago

The Real Reasons Behind the Flash Debate

In my humble opinion, the main reason why the debate around Flash versus HTML5 (and Adobe versus Apple) has gone haywire during the past few weeks is that everyone is lying about their true motives (like in any apparently inextricable conflict):

  • Apple doesn’t want Flash on its closed devices like the iPhone, iPod Touch and iPhone because Flash would be an alternate content channel that would hinder Apple’s very lucrative iTunes business. And of course they won’t admit it because it would mean that they put business strategy before innovation.
  • Adobe has some technical issues with Flash because the codebase is very old and costs a lot to maintain. And they won’t admit it because it would make Flash look weak and unreliable on the long term.
  • Most HTML5 proponents started as amateur web developers, tinkering for years with poor technologies like HTML, CSS and Javascript, building entire businesses around their capacity to figure out all this mess out of trial and error.
  • Flash developers (including yours truly) have invested a lot of time, money and effort in tools, training and experience with the Flash platform, and are not so enthusiastic about throwing all of that down the garbage for yet another fashion tech. But of course we won’t admit that because we’re afraid to look like dinosaurs who refuse to evolve.

And since everybody uses pretexts, everyone will use every contradiction and approximation possible to maintain their cover. Like I said, wars work the same way: ideology hides facts. It motivates your troops to fight the other side until they’re dead, because the big guys, the ones who know about the real reasons simply can’t find a peaceful solution that would allow everyone to win.

It reminds me of a European civil servant who asked me this simple question once when I started a mission for the European Parliament: “Why do you think the European Union was created?” Naively I answered “so that one day we can be on equal foot with the United States.” “Wrong!” he said. “That’s not what Shuman and its friends had in mind. After World War II, they realized that the true source of the conflict between France and Germany, and by extension between their respective allies, was the repartition of natural resources in general, and most notably at this time, coal. But it’s hard to get people to fight for that kind of cause, so powers had to disguise it with ideology so that people would feel invested and do terrible things in the name of it. And they understood that if they didn’t cure the real sickness instead of fighting the symptoms like they had done after World War I, they could not prevent it from happening again. That’s the real reason why CECA was created, and then CEE, then EU. It was not a matter of power, it was a matter of peace.”

And that answer struck me because the same pattern repeats itself over and over again in a lot of contexts. Behind decades of war between Israel and Palestine, there’s a fight for water. Behind the war in Irak, there was a fight for gas. Behind the war for the future of web technologies, there’s a fight for customers. And the conflict is even more violent as it happens on 2 different levels: company-wise between Apple and Adobe, and community-wise between Flash developers and HTML developers.

The good news is that there’s a way for everyone to win peacefully once we reveal the real issues. There are enough customers for everyone. It’s not a zero-sum fight. The other side doesn’t have to lose so that we can win. The question is: will we be strong enough to reject the ideology and find a common ground at the community-level? Or will we let the big guys at Adobe and Apple hide behind ideology and use us as cannon fodder.

By Sébastien, ago

Grails BlazeDS 4 Integration Plugin

One of the main goals I’ve been pursuing for a few months is the integration of Grails with Flex 4. I need to rework ConferenceGuide‘s administration backend to make it more ergonomic so that we can cover more events, and ever since I discovered Flex 4 niceties, I couldn’t think of doing that with anything else. The problem is that none of the existing plugins suited my needs. All of them cover Flex 3 only, some of them introduce a lot of complexity for CRUD generation, some of them use GraniteDS instead of BlazeDS, and the simplest plugin, grails-flex has never gone further than the experimental stage. I did a lot of experiments, talked a lot about it on Grails mailing lists, until Tomas Lin kindly explained to me that maybe I was approaching it the wrong way. I wanted a plugin that would set up a complete environment for Flex developement right in the middle of my Grails application. And I wanted that because I wanted to avoid using Flash Builder (you know… Eclipse… ierk!), mainly because the only advantage of Flex Builder 3 over IntelliJ Idea was the visual designer. But he was right, I was wrong. Flash Builder 4 DOES change everything. It does include a lot of very interesting features that greatly improve Flex development productivity, especially when it comes to integration with backend technologies. And for those features, I can only admit that IntelliJ is not up to par yet. I’m still gonna use it for the Grails part, but Flash Builder will be my environment of choice for Flex 4.

So, once I learnt more about Flex 4, BlazeDS 4 and Flash Builder 4 beta 2, it was time to reconsider my approach and develop a much simpler Grails plugin so that any Grails application could be used in combination with Flash Builder. I just released grails-blazeds plugin to do just that. Here is how it works:

  1. Install grails-blazeds plugin: “grails install-plugin blazeds”. This plugin copies a couple of configuration files into your application and imports all of the required libraries, including BlazeDS 4 and Spring-BlazeDS integration 1.5, both in nightly snapshot versions, since they have not been officially released yet
  2. Create a service to be exposed to your Flex application over AMF, or choose an existing one
  3. Add @RemotingDestination annotation to your service class, and @RemotingInclude annotation to all of the methods in this service that you wish to expose
  4. Edit web-app/WEB-INF/flex-servlet.xml (created when you installed the plugin): uncomment the context:component-scan element and set the base-package corresponding to your service class
  5. Make sure your exposed service methods don’t use Groovy classes, either as argument or return types. This is a known limitation I’m still working on, but if there are some Groovy classes here, Flash Builder doesn’t manage to generate ActionScript counterparts.
  6. Run your Grails application using “grails run-war” instead of “grails run-app”. Once again this is a known limitation: Flash Builder BlazeDS data connection plugin relies on a classical web app layout and doesn’t understand Grails dynamic layout (that is until someone manages to create a Grails data connection wizard for Flash Builder 4)
  7. In Flash Builder 4 beta 2, create a new Flex project with a J2EE server. Here are what your parameters shoud look like, “conferenceguide” being the name of my Grails application and “sarbogast” being my home directory:

  8. Click “Data” menu, then “Connect to BlazeDS…”
  9. In the “Authentication required” dialog box that appears, check “no password required” box, and click OK
  10. You should see your service appear, and you can select it and click Finish.

  11. Your service should appear in the Data/Services view. You can then compose your user interface and drag and drop your service methods to the relevant components to connect them with your Grails backend.
  12. Don’t forget to configure a channelset on your service:
    [code=xml]
    <adminservice:AdminService id=”adminService” fault=”Alert.show(event.fault.faultString + ‘\n’ + event.fault.faultDetail)” showBusyCursor=”true”>
    <adminservice:channelSet>
    <s:ChannelSet>
    <s:AMFChannel uri=”http://localhost:8080/conferenceguide/messagebroker/amf”/>
    </s:ChannelSet>
    </adminservice:channelSet>
    </adminservice:AdminService>
    [/code]

And there you go. Special thanks to James Ward, whose screencasts really helped me get it right. Now the only thing that this plugin misses, beyond the 2 known limitations, is integration with Spring Security, but this is just a start.

Enjoy!

By Sébastien, ago

Fear, Uncertainty and Doubt

I have hesitated for a long time… a few hours that is. But given my geek reputation (of which I’m still proud by the way), I could just not avoid it: I have to say something about the iPad. Of course I was following the keynote live yesterday with a bunch of geeks in Café Numérique, in Brussels. And of course I was very excited about it. Now instead of writing long sentences about what I like and don’t like about it, I’ll just go over my impressions quickly:

  • I don’t really like the name, but iSlate would have been worst and Macbook Touch was clearly not adapted
  • I love the device itself, and yes I’m gonna get one as soon as it’s released. I’m planning a trip to WWDC in June and hopefully I can bring one back
  • it really is yet another game changer. Apple did with the iPad to the Tablet PC segment what they did with the iPhone to the smartphone segment: bypass the professional market, make it a general public appliance. Brilliant!
  • it will create a whole new market for new applications on the App Store, I can think of a few ones myself
  • I don’t care that it doesn’t have multi-tasking, I’ve never needed it and reactivity is too important for me
  • demonstrations in a couch, what a great communication symbol!
  • I don’t care about the batteries being sealed, so long as it gets me more battery-life
  • I hope there will be an Apple Care on this one
  • Absence of SMS and phone capabilities: it’s not a phone anyway
  • No front camera: who really uses video-conference in the general public anyway? Video-conference in a couch? Come on!
  • The price tag is just awesome. The top one is cheaper than my 3GS
  • I’m so glad I didn’t get a Kindle DX or a Nook as I intended to. By the way, even though e-ink is more comfortable to use, I think the generic aspect of the device and the availability of the iBooks store are going to marginalize specialize eBook readers
  • Stop it with the “giant iPhone” complaint. And laptops are mini-desktops, so what?

Now in parallel to all those impressions, I couldn’t help to see those floating images in my head, of websites with big blank areas and a blue logo, I could even hear those blank areas whisper in my ears: “No Flash support…. ouuuuuuuuhh… No Flash support”. Yes, I know, I’m going nuts. My first thought was “how are you going to explain your daughter that she cannot use the iPad to access her favourite color painting site with her fingers because there’s no Flash?” For such a general public family appliance, it just doesn’t make sense.

But then I started reading blog posts and comments about the announcement, and the frustration turned into anger. Anger against Apple and Adobe who can’t seem to find a common ground on this issue. But more importantly angry against all those self-proclaimed death prophets, all those open standard ayatollahs claiming that they don’t care about Flash since Flash is dead anyway, and Flash is closed and proprietary, and Adobe is all evil, and HTML5 is going to rule the world. And it kind of woke up the Flex developer beast in me, I turned all green, I tore my shirt apart, going all…

FUCK HTML5 !!!

And then I started punching around.

First off, Flash has evolved a lot in the past few years: Flash is not just used for ads anymore. It powers the vast majority of videos on the web, plus a lot of multimedia websites that we love and use everyday (Deezer for example, Google Finance, etc.)

Second, Flash is not completely open, but it is far less closed than what a lot of people know: Tamarin, the Flash virtual machine, the basis of the Flash plugin, has been donated as Open Source to the Mozilla foundation 3 years ago, SWF (Flash file format), AMF (Flash remoting protocol), RTMP (Flash realtime communication protocol) are all open specifications that allow anyone to write their own Flash plugin (with a licence, but still) or generator. Plus Adobe has gone a long way in opening up its tools and processes for the Flash platform as a whole by open sourcing the Flex SDK, creating the Open Screen Project, and I could go on and on. I’ve met some of the openness evangelists inside Adobe and I can tell you that they’re doing a great job opening up what used to be a very protective and old-school company. And it’s just the beginning.

Third: being an open standard is not a f***ing feature for Steve’s sake! If using committee standards means I have to wait for 10 years before any evolution becomes available (how long has W3C been working on HTML5? how long before it is finalized), if it means going back in time on problems we thought had been solved for good (like the video codec hell coming back from the dead), if it means having to spend hours tweaking my web applications so that they look and behave the same in all browsers, then I don’t give a sh*t about open standards. Where is the added value?

Fourth, I can already hear you yell at me about the last argument: “we just can’t let one (evil) company have so much control about a web technology!”. And still that’s exactly what Sun has been doing with another omnipresent web technology: Java. And very few people ever complained about it. And what about Google with Android? The truth is that, from a developer standpoint, having one company orchestrating the evolution of such a huge technology is very good: it guarantees a certain level of consistency, so that we don’t have to deal with compatibility issues between different implementations. It’s also a good point for stability, knowing that you will always have backwards compatibility and professional support on the long term, and that you can invest safely in the technology. And of course it’s excellent for efficiency, because they don’t have to waste time on endless arguments about who’s got the bigger video codec or whatever, so it evolves fast.

So that’s it. I hate HTML, Javascript and CSS, I do it when I have to, but it’s not development, it’s tinkering. And I hate all those people spreading FUD about Flash without knowing what they’re talking about. And I love the Flash platform, and what Adobe is doing with it. I just hope Adobe and Apple will eventually reach an agreement to bring Flash to the iPhone and iPad. And I hope Adobe will do some PR to fix their image because there’s a big problem there.

Just my 2 cents…

By Sébastien, ago

Spring, Flex, BlazeDS Full Stack is Back!

After trying for weeks to integrate Flex and BlazeDS into Grails, in vain, I’ve decided to come back to vanilla Spring/Hibernate for Conference Guide server. And I took the opportunity to upgrade my todolist sample application, the one featured in the article published both on this blog and on the Adobe Developer Connection.

Here are the improvements compared to the original version:

  • I replaced the old SpringFactory by Spring BlazeDS Integration (spring-flex) library
  • I’ve added basic security using Spring Security, based on Cristophe Coenraets’ article on DZone
  • The domain now has 2 entities with a one-to-many relationship: you can manage several todolists, each having several items
  • I’m now using proper DTO’s for data transfer between the client and the server, which is for me the cleaner way to do things
  • Flexmojos Maven plugin has been upgraded to 3.4.2 (the latest stable version today).

I won’t go into detail about every aspect of this project because I think it can be much more useful to use it as a basis rather than try to reproduce it from scratch. Here is what you need to do to make it work:

  • Download the project source code on GitHub
  • Install Maven 2.2.0 if not already done
  • Run “mvn install” at the root of your project until you get a BUILD SUCCESSFUL message
  • Copy server/target/todolist.war to the webapps directory of your Tomcat server, or startup Tomcat and run “mvn org.codehaus.mojo:tomcat-maven-plugin:1.0-beta-1:redeploy” from within the server module
  • Open your browser on http://localhost:8080/todolist and login as john/john
  • Enjoy!

Capture d’écran 2009-11-07 à 19.12.52

If you have any suggestion for improvements, of if you have any question, feel free to leave a comment here. Cheers!

By Sébastien, ago

Grails Flex Integration, version 1.0

Ever since I discovered Grails, I’ve never stopped looking for the best way to make it work with Flex (I guess for me, the search was NOT over). Why so? Simply because the less time we spend connecting components, mapping objects with the database and dealing with boilerplate code, the more time we have for building gorgeous user interfaces. As for usual web suspects like JSF, GWT, GSP and other HTML/JS-generators, they have never been the best solution for me.

So how do we get Grails and Flex to work together? Well, Grails uses Spring behind the scenes and it is maintained by SpringSource, who also happen to be behind Spring BlazeDS integration in partnership with Adobe. So everything seems to be there… but there’s a problem. The best way to integrate any technology with Grails is via a plugin. Unfortunately the Grails Flex plugin is very old (it does not use Spring BlazeDS integration, but an old custom workaround) and only experimental. There is also a GraniteDS plugin for Grails, but GraniteDS is an alternative to BlazeDS, so it’s not the mainstream way of doing things. And Graeme Rocher has started working on a Spring BlazeDS integration plugin based on the old Flex plugin, but it was never released. Hmmm…

I know, when you’re unhappy with an open source project, when you feel it’s missing something, the better way to help is to do it yourself and share it with the community. But the problem is, although I love using Grails, I’ve never developed a plugin for it before. And this Flex plugin doesn’t seem like an easy one to start with. That’s why I decided to do things differently.

A few months ago, I published an article about Spring, Hibernate, BlazeDS and Flex. This article was very popular, both on my blog and on Adobe Developer Network. But new versions of Maven, Flex and Flexmojos have been released since then, so the article is a little bit outdated now. So why not use this opportunity to do an update?

So here we go. In the following file, there is the full todolist-grails project, that you can also find on GitHub (this is my first Git project by the way). This first version is merely a proof-of-concept. It’s not a plugin, it’s a traditional Grails application with Flex infrastructure added to it.  And because I didn’t find a way to integrate Flex compiler with Grails yet, I’m still using IntelliJ Idea to build the Flex part. Still, I’m publishing it as it is because I hope people will help me improve it incrementally.

So if you can help me improve this project and create a Grails plugin out of it in order to automate Flex compilation, integrate Spring Security, generate DTO’s automatically, you’re more than welcome. Let’s get this thing rolling.

By Sébastien, ago

Oops! They seem to be doing it again!

I’ve seen a lot of job offers lately for Flex developers in London banks (including investment banks like Goldman Sachs this morning). I love Flex development and I’m starting to look for freelancing opportunities around this technology but technology is not everything to me: I also thrive on purpose, I love to work on stuff that matters to me. And the fact that the banking sector seems to be hiring rich internet application developers so much kind of worries me somehow.

Because one of the main interests of technologies like Flex is the quality of data visualization it allows. Now what worries me is that software has already been a major enabler in the subprime crisis, with algorithms able to calculate and mitigate the risks for very complex financial products. Now it’s like they’re saying “OK, let’s do the same shit, but put more human factor in it, so let’s give them more accurate data!”. Now this is just an external impression and maybe I’m wrong about what they’re doing with Flex. But maybe I’m not. Does anyone work with Flex in banking and can tell us what you’re doing with it?

As far as I’m concerned, I’d love to work on a “new energy”-related project, like what Roundarch did for the Tesla Model S. Or I’d love to work for a project like Better Place, or Desertec. I’m sure they could use some modern , productive and good-looking software too. But banking? Hum, not a good option for me until they integrate more human data in their decisions.

By Sébastien, ago

Software Architecture Cheatsheet (Part 3/3)

In the previous post, I tried to think of the business constraints that intervene in the choices of a software architect. In this one, I’ll take a feww shots at guessing which technologies are important nowadays to build software solutions for these constraints.

I see… I see…

There are so many technologies out there that I will not risk myself in designing some sort of female magazine test like “tell me about your application, I’ll tell you what technologies you should use”. That’s a very exciting part of what I perceive as what is the job of a software architect: finding the right combination of tools and techniques for a specific business context in order to develophigh-quality, high-value and robust software for customers.

That said, there are a few important areas that seem very important to explore or even master in this world, and more specifically in this new economy we’re facing.

Productive dynamic Java

Java is a very mature and popular technology, so much so that many people have predicted its death times and times again. But in my view, it’s very much alive, especially with recent developments that made Java development much more productive. Of course, SpringSource-originated frameworks like Spring and its galaxy have changed the enterprise Java environment for a long time.

But even more recently, inspiration has come from the “casual programmer” side with Ruby on Rails and Python/Django yielding even more interesting developments like Groovy and Grails that combine the flexibility of a dynamic language with the incredible power and richness of the Java platform.

In my opinion, Groovy/Grails are about to rejuvenate enterprise development in an incredible way.

Modular Java

There has been a lot of marketing fuzz a few months ago about something called Service-Oriented Architecture. Unfortunately, although it was based on common sense, marketers and tool vendors completely killed the concept in the egg, but still, some important aspects have emerged and remain limitations of the most popular technology platforms. One of them is the importance of modularity: the ability to change one part of a system without touching anything else, whether it is to adapt them or to restart them.

OSGi (Open Service Gateway initiative) is a standard that has made a remarkable progression on the server side in the past few months, and with its massive adoption by major vendors, it’s definitely going to be something to watch.

Server-agnostic Rich Internet Applications

RIA-enabling technologies compose a very competive landscape: Adobe Flex, Microsoft Silverlight, Sun JavaFX, and even more niche technologies like OpenLaszlo, Curl. And I’m not even considering all those Javascript frameworks and AJAX-generating techniques that I personally don’t see as viable alternatives in an enterprise environment.

My technology of choice is definitely Adobe Flex: it’s open (and it’s even become one of the most impressive examples of Open Source development lately), it’s robust, it’s server-agnostic (it works with Java, .Net, PHP, Python, what have you), it offers desktop integration capabilities, making it possible to cover many of the use cases mentioned above, and it’s very elegant by design. More importantly it was one of the first RIA technologies out there, which makes it both very mature AND very popular.

Native Mobile Development

Mobile development has always been a hobby. Taking useful applications with you is an old fantasy. For a long time, it’s been so poor that it was difficult to turn this hobby of mine into a professional activity. That was until I came in touch with iPhone SDK development, which really blew me away. For the first time we have some great mobile hardware with unique usability capabilities, and we have the software development platform to use those capabilities like never before. And it’s going to be even better with the release of iPhone OS 3.0.

Of course, it’s about to become a very competitive area too, with the release of Palm WebOS, Google Android and Nokia Qt. But for now, the iPhone SDK is by far the most advanced native mobile development option.

What’s my point?

The purpose of this series is double:

1. try to show why software in general, and software architecture in particular are such exciting fields
2. wake up people who tend to have only one single hammer in their toolbox

Now if in addition to that, it can create a debate, then I have a few questions for you guys (and hopefully gals :oP) So, what technologies do you think are important to know in the current and future software world?

By Sébastien, ago

Software Architecture Cheatsheet (Part 2/3)

In the previous post in this series, I tried to enumerate the most frequent kinds of applications. The question I’m going to ask myself here is what are the constraints that intervene in choosing the right paradigm and the correponding technologies to implement it.

Environment! Environment! Environment!

Before we start answering that question, let’s just be clear with something. We live in a world where there are plenty of free and Open Source libraries and frameworks and tools of all kinds. It doesn’t mean that free is always good, but at least it’s an option, and if you have a commercial option that can add some value somewhere, then go for it, it’ll be worth it. So I won’t consider tooling cost as a parameter here.

Performance (high computational power and low bandwidth)

Whenever you hear your customer say “I need it to handle several million transactions per second”or “I quickly want to make decisions based on thousands and thousands of records”, you know that you will have to think about performance. There can be several kinds of performance: memory consumption, CPU cycles, disk space, network bandwidth, hardware cost, etc. And all those metrics very often play together, which means that any change to one of these metrics has an impact on all of the others. For instance, it’s very common that you have to increase memory consumption to optimize CPU or disk access (caching).

Another important characteristics of performance is that optimization requires you to dig deeper into low-level details, because most of the performance is lost when abstracting machine constructs to be closer to human users. That’s why optimizing performance requires more work than doing things naively, and it’s very important to weigh the benefits of this work compared to the cost.

Moreover, it’s sometimes tempting to think of performance very early on and to focus on that more than the business value the application is supposed to create. But experience proves that you can quickly end up with very fast systems that don’t do what they are supposed to do because the closer you are to the machine, the harder it is to develop on it or maintain it. That’s why Donald Knuth said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Distributivity (number and spreading of end-users)

Nowadays, it seems like all applications are meant to be web applications, all the more so with the recent fashion for cloud-based applications that attempt to “webify” traditionally desktop-based apps like word processors, worksheets, and so on.  And there has been so much effort spent in web apps in the past 10 years or so, that everyone knows about the technologies to build them. Yet it’s always important to ask yourself a few questions: will the application be accessible to the general public? Will it be extranet or intranet? How many users are likely to access the application at the same time? Are potential end users ALWAYS online? What would be the impact of the browser crashing in the middle of a session?

Sometimes, having to think about data access concurrency, network bandwidth or security is a useless hurdle that you can avoid just by developing a desktop application.

Automation (launch it regularly and in the background)

What if your application doesn’t need a complex user interface but requires just a few parameters to do its job? What if, on the other hand, it needs to be easily automated and integrated into a batch processing system? When you face such a business context, it’s important to consider the option of a CLI app, because then it can also be easier to integrate with other kernel system apps through scripting.

Whenever you hear your customer say words like “data analysis”, “system check” or “automatic synchronization”, you’d better think twice about your web app idea.

Ergonomics (easy and quick data input and visualization)

At the other hand of the data analysis pipeline, there is data input. And the more data there is to input, the higher the risk of rejection of the application by end users. And since end users generally wait a long time to get theirs hands on the application, this rejection traditionally happens very late in the development process. Combine that with the fact that people who ask for the application are not the ones using it, and the very special mindset of developers and you have all the chance in the world to miss your target and have the project fail before it reaches the finish line.

Of course, technology is not the primary solution to this problem. The first thing is to consider end users, consult them, talk with them, even if the business owner doesn’t think it’s useful. Then of course, methodology goes a long in putting the application into end users hands as soon and often as possible. But as soon as you realize the specificity of what users are expecting, you understand that you need a technology that gives you all the freedom to implement very complex use cases, without forgetting about the conventions and paradigms that people are used to.

Integration (with operating system and external systems)

Web apps have another big drawback in addition to ergonomic limitations, which is desktop integration. This issue comes from the security model of the web. Because it’s so easy to access a web application, because you don’t have anything to install and because the application is directly connected, it also creates a huge opportunity for malicious use. Which is why web apps usually work in what is called a sandbox: network access is limited to the originating domain (unless specified otherwise), no direct access to the file system is allowed, no native API access to things like system tray icons, drag and drop and so on.

And if your application has tom import or export very big files, or notify the user on a regular basis, those limitations can be a killer. There are some technologies now that create some sort of a bridge between a runtime plugin in your browser and a runtime app on your machine, but portability of this bridge across systems and across browsers is sometimes limited.

Productivity (getting things done and adapting fast)

How stable are the business rules you’re asking me to implement? How sure do you know what you expect from this application? If your customer answers “not very” to any of these questions, you might think twice about using this low-level highly-optimized programming language. Because if it takes you weeks to implement any change or new feature, your application might quickly end very far away from the business value is was supposed to create.

Fortunately, with the maturity of web application development, there has been a lot of very interesting developments in the area of development productivity lately. Development tools like integrated development environments certainly go a long way in making developers more productive, but when this concern is dealt with at the programming language level, it’s even better.

Maintenance cost (number and quality of resources)

Whatever technologies you plan to use, you definitely must consider the constraint of resources. There are so many techniques out there that it’s impossible for everyone to know all of them. Some of those technologies are very mature and popular, thus making it easier to find people to maintain and evolve your application on the long term. But the more mature the less innovative they often are. So finding the ideal compromise between the benefits of innovation versus the cost of resources to maintain your application is very important. Thus is might require some insight and technology watch in order to anticipate which of these innovative techniques will grow fast and be there for a long time.

And if you really need one of these innovative technologies that is not very popular yet, then don’t forget to include training costs in your plan. Last but not least, don’t forget to consider company-wide policies: IT architecture departments can create substantial impediments on your way, which might lead you to weigh in the cost of those impediments.

Continuity (robustness and evolutivity)

Beyond people able to maintain it, there is another thing that is very important for the longevity of your application: the intrinsic software quality assets of the technologies that you use. Testability, decoupling, Domain Specific Language support, portability, internationalization support, integration capabilities with other technologies and platforms, extensibility, modularity. All those characteristics can be very important to consider if your application is supposed to stay there for more than 5 years and evolve with the business at hand.

A lot of money is spent and sometimes wasted in reegineering entire applications just to keep up with current technologies or new business constraints, so much so that choosing robust and evolutive techniques can greatly reduce the long term ownership cost of the application.

In the final issue, I’ll risk myself into making some predictions about the technologies that seem very important in order to implement applications with that kind of constraints. But before I do that, do you see other business constraints that might be important to consider before choosing the best tools for the job?

By Sébastien, ago

Software Architecture Cheatsheet (Part 1/3)

What I really like about being a software artist is the richness of tools and techniques you have at your disposal. And the more tools you have, the harder it is to use the right ones, the more tempting it is to limit yourself to a few of them. But to me it’s like analogic versus digital DJing: given that your ultimate purpose is to create sounds that make people move, why limit yourself to sync-and-scratch when you can have effects, loops, samples and a virtually unlimited library of tracks?

But I’m sort of missing my point here. Let’s get back to software. I’ve recently come to work on a new project that has been in the works for almost 2 years. For 2 years, wanna-be software developers have tried to solve a very difficult problem with very usual tools. It’s like Maslow said:

When you know how to use a hammer, everything starts to look like a nail.

Well, guess what! Everything is NOT a nail. And I’m gonna try to go over the reasons why in this post.

Software is one big family…

…and each member of the family has its own personality.

The most popular right now is certainly web applications. And by web applications, I mean traditional ones. HTML, CSS, throw is a little bit of Javascript, and maybe generate all of that with some server-side scripting like PHP, Python, JSF or whatever. Heavy load on the server, but very lightweight on the client. The interface is somehow poor because it relies heavily on technologies that were designed for documents gathered in websites, rather than for full-blown applications, with all the interactivity that it implies. Yes, some progress has been made in the past few years with all this AJAX stuff, but bear with me, this all seems like tinkering to me.

The mirror opposite of lightweight clients are certainly fat clients, aka desktop applications. Those applications are based on a composition of graphical widgets the user interacts with, throwing events around and interacting with the operating system. Contrary to their web counterpart, they usually require quite a procedure for deployment and maintenance, because they are physically running on the user’s machine and only check in with the server if they need to. But damn they’re fast.

More recently, a new compromise solution has shown up, offering the best of both worlds: the great ergonomy of desktop clients combined with the ease of deployment and maintenance of web clients. That’s what marketing guys have lovingly called Rich Internet Applications. Now behind this lovely RIA thing, there are a few technologies that make it a lot easier to write rich user interfaces that run within the confines of a web browser. But still, those have limitations compared to their pure desktop brethren: poor integration with the operating system, security constraints all over the place, heavily rely on server-side business code.

Now if Rich Internet Applications are web applications that solve the ergonomy problem, there is of course the other side of the compromise: desktop applications that solve the deployment and maintainability issues. Those are sometimes called smart clients: local database, offline mode, online synchronization, automatic updates, easy one-click installation.

Even though, those seem to fulfill the family picture, there are a few weird cousins out there that are good to be known. Command-Line Interface (CLI) applications have poor to no user interface at all. Their main purpose is to be run on the command-line by some geeky system administrator somewhere, or to be part of batch scripts running automatically every night. Very useful for maintenance apps, and for all long tasks like data analysis or system checks.

And of course there are mobile applications and all kinds of embedded systems. The user interface simply cannot be rich here, because the display is so small, and the computing resources are so limited. Small memory, small keyboard. The iPhone is certainly changing the landscape here, but you still have to manage memory!

Don’t forget extension apps, like SAP modules, CMS plugins, MS Access applications. Those are applications of their own. Usually highly specialized but very fast to develop for simple use cases, to get things done quickly.

Finally, even though, they’re less and less popular, there are still many mainframe applications out there. Now I won’t go into much details here because I’ve never set foot on that ground. But it certainly doesn’t harm to remember that it exists.

Now there certainly are a few other kinds of software applications out there that I didn’t think of, but you get my point. There are a lot of different tools out there, and very different techniques to use those tools in order to create software solutions to very different problems. And what makes those problems so different, you might ask. Well, it’s all about the business context. In the second part of this series, I’ll focus on the characteristics of a business environment can influence the tools you choose to implement the solution to a problem.

But before we get there, do you see other kinds of applications that I forgot to mention?

By Sébastien, ago