Software Architecture Cheatsheet (Part 3/3)

In the previous post, I tried to think of the business constraints that intervene in the choices of a software architect. In this one, I’ll take a feww shots at guessing which technologies are important nowadays to build software solutions for these constraints.

I see… I see…

There are so many technologies out there that I will not risk myself in designing some sort of female magazine test like “tell me about your application, I’ll tell you what technologies you should use”. That’s a very exciting part of what I perceive as what is the job of a software architect: finding the right combination of tools and techniques for a specific business context in order to develophigh-quality, high-value and robust software for customers.

That said, there are a few important areas that seem very important to explore or even master in this world, and more specifically in this new economy we’re facing.

Productive dynamic Java

Java is a very mature and popular technology, so much so that many people have predicted its death times and times again. But in my view, it’s very much alive, especially with recent developments that made Java development much more productive. Of course, SpringSource-originated frameworks like Spring and its galaxy have changed the enterprise Java environment for a long time.

But even more recently, inspiration has come from the “casual programmer” side with Ruby on Rails and Python/Django yielding even more interesting developments like Groovy and Grails that combine the flexibility of a dynamic language with the incredible power and richness of the Java platform.

In my opinion, Groovy/Grails are about to rejuvenate enterprise development in an incredible way.

Modular Java

There has been a lot of marketing fuzz a few months ago about something called Service-Oriented Architecture. Unfortunately, although it was based on common sense, marketers and tool vendors completely killed the concept in the egg, but still, some important aspects have emerged and remain limitations of the most popular technology platforms. One of them is the importance of modularity: the ability to change one part of a system without touching anything else, whether it is to adapt them or to restart them.

OSGi (Open Service Gateway initiative) is a standard that has made a remarkable progression on the server side in the past few months, and with its massive adoption by major vendors, it’s definitely going to be something to watch.

Server-agnostic Rich Internet Applications

RIA-enabling technologies compose a very competive landscape: Adobe Flex, Microsoft Silverlight, Sun JavaFX, and even more niche technologies like OpenLaszlo, Curl. And I’m not even considering all those Javascript frameworks and AJAX-generating techniques that I personally don’t see as viable alternatives in an enterprise environment.

My technology of choice is definitely Adobe Flex: it’s open (and it’s even become one of the most impressive examples of Open Source development lately), it’s robust, it’s server-agnostic (it works with Java, .Net, PHP, Python, what have you), it offers desktop integration capabilities, making it possible to cover many of the use cases mentioned above, and it’s very elegant by design. More importantly it was one of the first RIA technologies out there, which makes it both very mature AND very popular.

Native Mobile Development

Mobile development has always been a hobby. Taking useful applications with you is an old fantasy. For a long time, it’s been so poor that it was difficult to turn this hobby of mine into a professional activity. That was until I came in touch with iPhone SDK development, which really blew me away. For the first time we have some great mobile hardware with unique usability capabilities, and we have the software development platform to use those capabilities like never before. And it’s going to be even better with the release of iPhone OS 3.0.

Of course, it’s about to become a very competitive area too, with the release of Palm WebOS, Google Android and Nokia Qt. But for now, the iPhone SDK is by far the most advanced native mobile development option.

What’s my point?

The purpose of this series is double:

1. try to show why software in general, and software architecture in particular are such exciting fields
2. wake up people who tend to have only one single hammer in their toolbox

Now if in addition to that, it can create a debate, then I have a few questions for you guys (and hopefully gals :oP) So, what technologies do you think are important to know in the current and future software world?

Software Architecture Cheatsheet (Part 2/3)

In the previous post in this series, I tried to enumerate the most frequent kinds of applications. The question I’m going to ask myself here is what are the constraints that intervene in choosing the right paradigm and the correponding technologies to implement it.

Environment! Environment! Environment!

Before we start answering that question, let’s just be clear with something. We live in a world where there are plenty of free and Open Source libraries and frameworks and tools of all kinds. It doesn’t mean that free is always good, but at least it’s an option, and if you have a commercial option that can add some value somewhere, then go for it, it’ll be worth it. So I won’t consider tooling cost as a parameter here.

Performance (high computational power and low bandwidth)

Whenever you hear your customer say “I need it to handle several million transactions per second”or “I quickly want to make decisions based on thousands and thousands of records”, you know that you will have to think about performance. There can be several kinds of performance: memory consumption, CPU cycles, disk space, network bandwidth, hardware cost, etc. And all those metrics very often play together, which means that any change to one of these metrics has an impact on all of the others. For instance, it’s very common that you have to increase memory consumption to optimize CPU or disk access (caching).

Another important characteristics of performance is that optimization requires you to dig deeper into low-level details, because most of the performance is lost when abstracting machine constructs to be closer to human users. That’s why optimizing performance requires more work than doing things naively, and it’s very important to weigh the benefits of this work compared to the cost.

Moreover, it’s sometimes tempting to think of performance very early on and to focus on that more than the business value the application is supposed to create. But experience proves that you can quickly end up with very fast systems that don’t do what they are supposed to do because the closer you are to the machine, the harder it is to develop on it or maintain it. That’s why Donald Knuth said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Distributivity (number and spreading of end-users)

Nowadays, it seems like all applications are meant to be web applications, all the more so with the recent fashion for cloud-based applications that attempt to “webify” traditionally desktop-based apps like word processors, worksheets, and so on.  And there has been so much effort spent in web apps in the past 10 years or so, that everyone knows about the technologies to build them. Yet it’s always important to ask yourself a few questions: will the application be accessible to the general public? Will it be extranet or intranet? How many users are likely to access the application at the same time? Are potential end users ALWAYS online? What would be the impact of the browser crashing in the middle of a session?

Sometimes, having to think about data access concurrency, network bandwidth or security is a useless hurdle that you can avoid just by developing a desktop application.

Automation (launch it regularly and in the background)

What if your application doesn’t need a complex user interface but requires just a few parameters to do its job? What if, on the other hand, it needs to be easily automated and integrated into a batch processing system? When you face such a business context, it’s important to consider the option of a CLI app, because then it can also be easier to integrate with other kernel system apps through scripting.

Whenever you hear your customer say words like “data analysis”, “system check” or “automatic synchronization”, you’d better think twice about your web app idea.

Ergonomics (easy and quick data input and visualization)

At the other hand of the data analysis pipeline, there is data input. And the more data there is to input, the higher the risk of rejection of the application by end users. And since end users generally wait a long time to get theirs hands on the application, this rejection traditionally happens very late in the development process. Combine that with the fact that people who ask for the application are not the ones using it, and the very special mindset of developers and you have all the chance in the world to miss your target and have the project fail before it reaches the finish line.

Of course, technology is not the primary solution to this problem. The first thing is to consider end users, consult them, talk with them, even if the business owner doesn’t think it’s useful. Then of course, methodology goes a long in putting the application into end users hands as soon and often as possible. But as soon as you realize the specificity of what users are expecting, you understand that you need a technology that gives you all the freedom to implement very complex use cases, without forgetting about the conventions and paradigms that people are used to.

Integration (with operating system and external systems)

Web apps have another big drawback in addition to ergonomic limitations, which is desktop integration. This issue comes from the security model of the web. Because it’s so easy to access a web application, because you don’t have anything to install and because the application is directly connected, it also creates a huge opportunity for malicious use. Which is why web apps usually work in what is called a sandbox: network access is limited to the originating domain (unless specified otherwise), no direct access to the file system is allowed, no native API access to things like system tray icons, drag and drop and so on.

And if your application has tom import or export very big files, or notify the user on a regular basis, those limitations can be a killer. There are some technologies now that create some sort of a bridge between a runtime plugin in your browser and a runtime app on your machine, but portability of this bridge across systems and across browsers is sometimes limited.

Productivity (getting things done and adapting fast)

How stable are the business rules you’re asking me to implement? How sure do you know what you expect from this application? If your customer answers “not very” to any of these questions, you might think twice about using this low-level highly-optimized programming language. Because if it takes you weeks to implement any change or new feature, your application might quickly end very far away from the business value is was supposed to create.

Fortunately, with the maturity of web application development, there has been a lot of very interesting developments in the area of development productivity lately. Development tools like integrated development environments certainly go a long way in making developers more productive, but when this concern is dealt with at the programming language level, it’s even better.

Maintenance cost (number and quality of resources)

Whatever technologies you plan to use, you definitely must consider the constraint of resources. There are so many techniques out there that it’s impossible for everyone to know all of them. Some of those technologies are very mature and popular, thus making it easier to find people to maintain and evolve your application on the long term. But the more mature the less innovative they often are. So finding the ideal compromise between the benefits of innovation versus the cost of resources to maintain your application is very important. Thus is might require some insight and technology watch in order to anticipate which of these innovative techniques will grow fast and be there for a long time.

And if you really need one of these innovative technologies that is not very popular yet, then don’t forget to include training costs in your plan. Last but not least, don’t forget to consider company-wide policies: IT architecture departments can create substantial impediments on your way, which might lead you to weigh in the cost of those impediments.

Continuity (robustness and evolutivity)

Beyond people able to maintain it, there is another thing that is very important for the longevity of your application: the intrinsic software quality assets of the technologies that you use. Testability, decoupling, Domain Specific Language support, portability, internationalization support, integration capabilities with other technologies and platforms, extensibility, modularity. All those characteristics can be very important to consider if your application is supposed to stay there for more than 5 years and evolve with the business at hand.

A lot of money is spent and sometimes wasted in reegineering entire applications just to keep up with current technologies or new business constraints, so much so that choosing robust and evolutive techniques can greatly reduce the long term ownership cost of the application.

In the final issue, I’ll risk myself into making some predictions about the technologies that seem very important in order to implement applications with that kind of constraints. But before I do that, do you see other business constraints that might be important to consider before choosing the best tools for the job?

Software Architecture Cheatsheet (Part 1/3)

What I really like about being a software artist is the richness of tools and techniques you have at your disposal. And the more tools you have, the harder it is to use the right ones, the more tempting it is to limit yourself to a few of them. But to me it’s like analogic versus digital DJing: given that your ultimate purpose is to create sounds that make people move, why limit yourself to sync-and-scratch when you can have effects, loops, samples and a virtually unlimited library of tracks?

But I’m sort of missing my point here. Let’s get back to software. I’ve recently come to work on a new project that has been in the works for almost 2 years. For 2 years, wanna-be software developers have tried to solve a very difficult problem with very usual tools. It’s like Maslow said:

When you know how to use a hammer, everything starts to look like a nail.

Well, guess what! Everything is NOT a nail. And I’m gonna try to go over the reasons why in this post.

Software is one big family…

…and each member of the family has its own personality.

The most popular right now is certainly web applications. And by web applications, I mean traditional ones. HTML, CSS, throw is a little bit of Javascript, and maybe generate all of that with some server-side scripting like PHP, Python, JSF or whatever. Heavy load on the server, but very lightweight on the client. The interface is somehow poor because it relies heavily on technologies that were designed for documents gathered in websites, rather than for full-blown applications, with all the interactivity that it implies. Yes, some progress has been made in the past few years with all this AJAX stuff, but bear with me, this all seems like tinkering to me.

The mirror opposite of lightweight clients are certainly fat clients, aka desktop applications. Those applications are based on a composition of graphical widgets the user interacts with, throwing events around and interacting with the operating system. Contrary to their web counterpart, they usually require quite a procedure for deployment and maintenance, because they are physically running on the user’s machine and only check in with the server if they need to. But damn they’re fast.

More recently, a new compromise solution has shown up, offering the best of both worlds: the great ergonomy of desktop clients combined with the ease of deployment and maintenance of web clients. That’s what marketing guys have lovingly called Rich Internet Applications. Now behind this lovely RIA thing, there are a few technologies that make it a lot easier to write rich user interfaces that run within the confines of a web browser. But still, those have limitations compared to their pure desktop brethren: poor integration with the operating system, security constraints all over the place, heavily rely on server-side business code.

Now if Rich Internet Applications are web applications that solve the ergonomy problem, there is of course the other side of the compromise: desktop applications that solve the deployment and maintainability issues. Those are sometimes called smart clients: local database, offline mode, online synchronization, automatic updates, easy one-click installation.

Even though, those seem to fulfill the family picture, there are a few weird cousins out there that are good to be known. Command-Line Interface (CLI) applications have poor to no user interface at all. Their main purpose is to be run on the command-line by some geeky system administrator somewhere, or to be part of batch scripts running automatically every night. Very useful for maintenance apps, and for all long tasks like data analysis or system checks.

And of course there are mobile applications and all kinds of embedded systems. The user interface simply cannot be rich here, because the display is so small, and the computing resources are so limited. Small memory, small keyboard. The iPhone is certainly changing the landscape here, but you still have to manage memory!

Don’t forget extension apps, like SAP modules, CMS plugins, MS Access applications. Those are applications of their own. Usually highly specialized but very fast to develop for simple use cases, to get things done quickly.

Finally, even though, they’re less and less popular, there are still many mainframe applications out there. Now I won’t go into much details here because I’ve never set foot on that ground. But it certainly doesn’t harm to remember that it exists.

Now there certainly are a few other kinds of software applications out there that I didn’t think of, but you get my point. There are a lot of different tools out there, and very different techniques to use those tools in order to create software solutions to very different problems. And what makes those problems so different, you might ask. Well, it’s all about the business context. In the second part of this series, I’ll focus on the characteristics of a business environment can influence the tools you choose to implement the solution to a problem.

But before we get there, do you see other kinds of applications that I forgot to mention?

2008 flashback

That’s it. Holidays are here, this year is almost over, time slows down for a few days so that we can look back, then enjoy and start again.

img_0076This post is probably not be of great use to anyone but me, but it’s a blog after all, sometimes I can use it as a personal journal too, like all those teenagers.

This year was actually very rich to me. I got to discover a few very interesting technologies, starting with OSGi and its ability to break up with big monolithic enterprise applications. And I really dived into Flex and how it can integrate with Java, which culminated in this tutorial, which was even republished on the Adobe Developer Connection. And finally, in my quest to discover better ways to develop applications, I came across Jetbrains MPS. And when I realized it was still far from being ready for prime time, I decided to keep it on my radar but focus on a more realistic alternative on a shorter term: Groovy/Grails. Now I’ve widened my technological scope beyond just JEE and AndroMDA, and I feel quite ready for the challenges ahead.

More importantly, this year will have been a great initiatic journey. Last year at about the same period, I decided I wanted to turn one of my ideas into a company, to create my own startup. The road was bumpy, my colleagues at Axen could feel it, friends who joined me on this journey could feel it too. I put a lot of things in question: my job, my ambitions, my strengths and weaknesses. And the project failed… so far! But they say there’s always hope at the end of the tunnel.

I had the idea, I had the skills, I had the will, time and energy, and yet there was this big barrier to entry, this thing that seems so important to everybody that nations and societies, people and governments are falling apart just because of it: money. I’ve always felt bad about money, like it was the poisonous blood flowing in the veins of an otherwise healthy man, eating him from the inside out, like something that is here to transport what makes us live, but that we’ve come to confuse for life itself. And it happened, crisis happened, the disease hit us all. We’re still far from the bottom since we still believe that the best cure against poisonous blood is more blood. But I don’t want to wait. My failed project and the recent course of events have made me realize something: THERE’S GOTTA BE A BETTER WAY!

2009 is full of promises. I have already started to understand many things about myself, about why I failed on this project, about how I live, how I can improve. I moved to a new place too, more comfortable, more like me. And now I have this new project, this game changer, this new way of thinking about how creative people can turn ideas into something real. 2009 is going to be fan… wait for it… tastic!

My OSGi Learning Path (Part 1)

I’m more and more convinced that OSGi is going to change the way we develop enterprise applications. In a sense it’s funny because it’s been around for a while in embedded and client-side environment, and it’s only been reaching to us server-side developers for a few months. I think that the fact that the most popular Java component framework, ie Spring, jumped in with Spring OSGi (now Spring Dynamic Modules) has a lot to do with this new awareness. That and the growing interest for modularity and reusability encouraged by the Service-Oriented Architecture fashion. Yes, I agree that SOA has become yet another buzzword encompassing everything and nothing, depending on what software vendors sell to their naive customers. But if customers are ultimately willing to believe false promises, it might be because there is a gap in current technologies. Some see this gap as an opportunity for sales, developers see it as an opportunity for technology improvement. And I really see OSGi as THE technological answer to that and many other problems. Yes, OSGi might be server-side development’s 42!

Now, once you realize the importance OSGi is going to have in the years to come, the question is… sorry, the questions are why, what and how? Of course you could read the entire OSGi specification, but it would be like using a bulldozer to plant a flower. Fortunately for us, there are plenty of articles, blog posts and documents out there to help us understand how to deal with our delicate flower. Here is a possible learning path that will take you through a journey in OSGi world, why it’s useful, what it is, and how you can use it in your own projects.

Why is OSGi interesting?

Kirk Knoernschild wrote two excellent blog posts with sample code that is as simple as it can be and really gives a practical idea of what OSGi is all about.

    • The first one introduces the notion of a service in OSGi
    • The second one shows how you can really decouple the interface and its different implementations and dynamically switch from one implementation to another one

What I really like about those articles is that they use no external tools like Maven or Eclipse. Just plain text and Ant to compile. Just one thing that can be useful: you might want to download the latest version of Felix (1.0.4 at the time of this writing) and customize osgi/HelloWorld/config.properties (lines 31 to 34):

felix.auto.start.1= \
 file:/Library/Apache/felix-1.0.4/bundle/org.apache.felix.shell-1.0.1.jar \
  file:/Library/Apache/felix-1.0.4/bundle/org.apache.felix.shell.tui-1.0.1.jar \
 file:/Library/Apache/felix-1.0.4/bundle/org.apache.felix.bundlerepository-1.0.3.jar

Those are my own modifications because I downloaded Felix and installed it in /Library/Apache/felix-1.0.4 on my Mac. And of course you will have to adapt startfelix.sh (or startfelix.bat) as well:

java -Dfelix.config.properties=file:./config.properties -jar /Library/Apache/felix-1.0.4/bin/felix.jar

Finally, in the second article, at the time of this writing there is a mistake in the code of the manifest for service2. I’ve left a comment and hopefully Kirk will fix his code in the repository. HelloWorldSpec/service2/META-INF/Manifest.mf should look like this:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Hello Service Impl 2
Bundle-SymbolicName: helloserviceimpl2
Bundle-Version: 1.0.0
Bundle-Activator: com.extensiblejava.hello.service2.impl.HelloServiceImpl
Import-Package: org.osgi.framework, com.extensiblejava.hello.service

Then, following those two articles, Kirk modified HelloWorldSpec to integrate Spring Dynamic Modules and show what it brings to the table. More specifically, you can see that Spring Dynamic Modules makes it possible to externalize service registration and consumption in Spring configuration files and have service and client classes as simple POJO’s.

That’s it for the “why?”. I guess you understand why it can be interesting to use OSGi on the server-side by now. In the next article, I’ll try to give you a few links to identify what elements can make up your OSGi development environment.

You dreamt of it? SpringSource did it!

3 months ago, the day SpringSource announced their acquisition of Covalent, the company behind Tomcat, I knew they were up to something. I even talked about it with Andrew Glover for JavaWorld. And now it’s real: SpringSource has just announced SpringSource Application Platform.

First off, a few links around this announcement:

Now this is very exciting. That’s the kind of inspiring technologies that make me think of tens of ideas for new projects, new tools that were just impossible before and that could now become reality. Adobe Flex had the same effect of unleashing my software artist imagination. Now I’m starting to think about combining the user experience of Flex applications on top of the ease of management, deployment and extensibility of OSGi with SSAP. Jeez, we’re living an incredible period!

My thoughts about OSGi on JavaWorld

A few weeks ago, I’ve been interviewed by Andrew Glover for JavaWorld’s Java Technology Insider podcast and it’s been published today. So without further ado, feel free to check out our talk here.

Now if you want to read it rather than listen to it, I’ve prepared a transcript for you guys. I’ve added a few links inside the transcript for the references I mention.

(more…)

JBoss is working on OSGi too

home_callout_jbossosgi.gifI wanted to download the latest version of JBoss AS to deploy it to a new server, and while I was wandering round their site to see what is going on, I found an interview of Ales Justing by Mark Newton about the work they’re doing with OSGi.

Apparently they are not waiting for it to happen. They’re really taking part in it, which is refreshing because ever since Redhat acquired them, I’ve always been afraid of the possibility that they could rest on their achievements and have trouble keeping up with innovation. Obviously I was wrong and since they even have some people inside the Enterprise Expert Group, we can expect some pretty good integration of OSGi into future releases of JBoss.

They are even going as far as reengineering a key part of their architecture, which is the microcontainer, to integrate OSGi. That’s really an excellent thing because I’ve always found that JMX is really a pain to manage. According to the interview, they are totally changing their core classloader:

We could probably use the classloading features of existing OSGi frameworks but it would again mean bending around things to make them work. As we wanted to have a bullet proof implementation, where all the nasty details were hidden away under private/protected modifiers, it was important that we could tightly control access through policies and delegation. From this perspective it made more sense to implement our own classloading layer.

Concerning integration with Spring, apparently they are still taking their distances. I guess it has something to do with the fact they Spring/Hibernate competes with EJBs and has thus encouraged many developers to choose a simple Tomcat server instead of a full-blown JBoss for their deployment. But as long as I can freely deploy my Spring/Hibernate on JBoss and still benefit from features like SOAP, JNDI, JMS and so on, that’s fine with me.

So that’s one other major actor of the enterprise application server market who is moving towards OSGi. Did I already say that it’s going to be big? And OSGi DevCon is certainly going to be very important this year.

Towards a mainstream Open Source OSGi application server?

That’s official: SpringSource has just acquired Covalent. Or as I explained it to one of my colleagues, “the company behind our IoC framework has bought the one behind our application server”.

My first reaction was satisfaction, because it’s another step forward in the direction of corporate Open Source adoption. It’s always amazed me to see how big companies can be afraid of Open Source. And the fact that there is now one bigger support service offer behind two of the most popular Open Source technologies in the enterprise will certainly reassure some skepticals.

The second “kiss cool effect” was undoubtedly about something I’ve really been playing with lately, since Javapolis: OSGi. The fact that componentization-related JSR’s are so fragmented and so alpha, plus the recent works of Spring Dynamic Modules for OSGi, added to Peter Kriens’ presentation at Javapolis, all of that really got my attention. And now I’m dreaming of building a collaboration platform using Flex for the front-end and OSGi, Spring and Hibernate JPA for the back-end. The only component that’s missing in my big picture is a deployment target platform, i.e. an OSGi application server. Of course, JOnAS is working on that but there’s no documentation on their 5.0 server whatsoever. And I’ve heard Websphere and Weblogic are using OSGi too, but hey, I’m talking about Open Source here! Now have a look at the documentation of both Tomcat and Spring. They’re probably amongst the best Open Source documentations on the planet. Now imagine that quality of documentation for a brand new Tomcat 7 server using OSGi as a core deployment mechanism, and integrating Spring DM libraries to ease the development of web application bundles.

Yummy!