I’m so amazed when I see how so few companies are using Groovy and Grails right now, and are still using old stuff like Spring and Hibernate, that I thought I would jump in and do my share of educating. And why not give in to the fashion of top lists while I’m at it? So here it goes: if you are an enterprise software architect and you have a lot of Java in your world, you might want to read carefully what follows.
Why I switched back from Heroku to CloudBees
I used to have several grails applications deployed on CloudBees. I liked the fact that they were Java all along, I liked the smooth integration between Jenkins CI and the deployment environment. I really liked the fact that you could hide an application behind a username and password during testing. I just hated their design (seriously guys, hire a good designer) and I was not thrilled by their catalog of third-party services. So when Heroku announced that they supported Java applications, and then Grails applications, it was not long before I migrated all my apps over to their servers.
But more recently, I’ve had issues with a more plugin-rich application. And tonight, after several weeks of fighting, I migrated this app to CloudBees for one general reason: Heroku was really designed with RoR in mind, and even though they made a new stack for java apps, the old rules still apply:
- if an application takes more than 60 seconds to boot, it is crashed. There is no way to adjust this timeout, and we all know that it can be pretty common for a java app to take a little more than that.
- another consequence of this slow boot intolerance is that an application is automatically put in sleep mode after several minutes of not being used. Consequence: when someone comes to the site after such a period, say, in the morning, the app is rebooted out of sleep mode… and sometimes it crashes again. Terrible for availability.
And apparently, even if you pay, there is no way to prevent that sleep behavior and to keep your app alive all the time.Actually to be fair, there is a way to disable idling: once you scale up to 2 dynos, idling is disabled.
- since your app can be sleeping, implementing long-running background processes is very complicated too. You have to use their worker processes, but there is no documentation on how to do that in a java application, let alone in a grails app.
- last but not least, even though I tried to limit the amount of memory of my app, it kept going over 512MB, which is the absolute limit. Once again no way to change that. It doesn’t crash my app, but it clutters my log with plenty of annoying warning messages.
That plus the fact that they refused to answer my last support request about memory consumption and marked it as solved when it got hard. The fact that they don’t have an easy continuous integration feature was not a deal breaker, but it adds to the rest.
Now I don’t know if CloudBees will be better on all these points, but it looks good on paper. Unfortunately, we java devs don’t have a lot of choice when it comes to cloud deployment.Â CloudFoundry is way too low-level, AppFog is still in private beta, and Amazon Elastic Beanstalk is awful for deployment (40MB take a long time to upload). What other options do we have that I’ve never heard of?
My Case for DTO’s
In many of my posts about Grails and Flex integration, I take for granted that I use Data Transfer Objects to transfer data between my Grails backend and my Flex frontend. Put simply, Data Transfer Object are pure data containing classes different from the domain entity classes used to store data in the backend. I take it for granted because I’m deeply convinced that it’s the best way to do things and so far, experience has never proved me wrong. But I often get this question in comments or by mail (this is for you Martijn): why bother create an entirely separate class structure and copy data from entities to DTO’s and back instead of just using entities?
I’ve expressed my arguments a couple of times across various posts but I thought it would be nice to sum things up in here for future reference.
Where does it come from?
When I first started to work on enterprise applications and ORM-based architectures, it was with a Model-Driven Architecture framework called AndroMDA. AndroMDA was absolutely key in helping me getting started with Spring and Hibernate and I was especially inspired by one paragraph in their “getting started” tutorial, which I quote here:
Data Propagation Between Layers
In addition to the concepts discussed previously, it is important to understand how data propagates between various layers of an application. Follow along the diagram above as we start from the bottom up.
As you know, relational databases store data as records in tables. The data access layer fetches these records from the database and transforms them into objects that represent entities in the business domain. Hence, these objects are called business entities.
Going one level up, the data access layer passes the entities to the business layer where business logic is performed.
The last thing to discuss is the propagation of data between the business layer and the presentation layer, for which there are two schools of thought. Some people recommend that the presentation layer should be given direct access to business entities. Others recommend just the opposite, i.e. business entities should be off limits to the presentation layer and that the business layer should package necessary information into so-called “value objects” and transfer these value objects to the presentation layer. Let’s look at the pros and cons of these two approaches.
The first approach (entities only, no value objects) is simpler to implement. You do not have to create value objects or write any code to transfer information between entities and value objects. In fact, this approach will probably work well for simple, small applications where the the presentation layer and the service layer run on the same machine. However, this approach does not scale well for larger and more complex applications. Here’s why:
- Business logic is no longer contained in the business layer. It is tempting to freely manipulate entities in the presentation layer and thus spread the business logic in multiple places — definitely a maintenance nightmare. In case there are multiple front-ends to a service, business logic must be duplicated in all these front-ends. In addition, there is no protection against the presentation layer corrupting the entities – intentionally or unintentionally!
- When the presentation layer is running on a different machine (as in the case of a rich client), it is very inefficient to serialize a whole network of entities and send it across the wire. Take the example of showing a list of orders to the user. In this scenario, you really don’t need to transfer the gory details of every order to the client application. All you need is perhaps the order number, order date and total amount for each order. If the user later wishes to see the details of a specific order, you can always serialize that entire order and send it across the wire.
- Passing real entities to the client may pose a security risk. Do you want the client application to have access to the salary information inside the Employee object or your profit margins inside the Order object?
Value objects provide a solution for all these problems. Yes, they require you to write a little extra code; but in return, you get a bullet-proof business layer that communicates efficiently with the presentation layer. You can think of a value object as a controlled view into one or more entities relevant to your client application. Note that AndroMDA provides some basic support for translation between entities and value objects, as you will see in the tutorial.
Because of this paragraph, I started writing all my business services with only data transfer objects (what they call “value objects”) as input and output. And it worked great. Yes it did require a little bit of coding, especially as I had not discovered Groovy yet, but it was worth the time, for all the following reasons.
The conceptual argument: presentation/storage impedance mismatch
Object-relational mapping is what Joel Spolsky calls a “Leaky Abstraction“. It’s supposed to hide away the fact that your business entities are in fact stored in a relational database, but it forces you to do all sorts of choices because of that very fact. You have to save data in a certain order in order not to break certain integrity constraints, certain patterns are to be avoided for better query performance, and so on and so forth. So whether we like it or not, our domain model is filled with “relational choices”.
Now the way data is presented involves a whole different set of constraints. Data is very often presented in a master/detail format, which means you first display a list of items, with only a few fields for each item, and possible some of those fields are calculated based on data that is stored in the database. For example, you may store a country code in your database, but you will display the full country name in the list. And then when the user double-clicks an item, he can see all the fields for that item. This pattern is totally different from how you actually store the data.
So even though some of the fields in your DTO’s will be mere copies of their counterparts in the entity, that’s only true for simple String-typed fields. As soon as you start dealing with dates, formatted floats or enum codes, there is some transformation involved, and doing all that transformation on the client-side is not always the best option, especially when you have several user interfaces on top of your backend (a Flex app and an iPhone app for example), in which case you’re better off doing most of these transformations on the server.
In anyway, if you change the way you store data, it should not influence too much the way you present the same data, and vice-versa. This decoupling is very important for me.
The bandwidth argument: load just the data you need
In the master/data use case, when you display the list of items, you just need a subset of the fields from your entities, not all of them. And even though you’re using Hibernate on the backend with lazy-loading enabled, fields are still initialized and transferred over the wire. So if you use entity classes for data transfer, you will end up transferring a whole bunch of data that may never be used. Now it might not be very important for hundreds of records, but it starts being a problem with thousands of records, especially when there is some parsing involved. The less data you transfer the better.
The security argument: show only the data you want to show
Let’s say you’re displaying a list of users, and in the database, each user has a credit card number. Now of course when you display a list of users, you might not want everyone to see the list of credit card numbers. You might want to expose this data only in detail view for certain users with certain privileges. DTO’s allow you to tailor your API to expose just the data you need.
The error-prone argument: argh! Yet another LazyInitializationException!
Of course there are associations between your business entities, and by default, those associations are lazy-loaded, which means they are not initialized until you actually query them. So if you just load a bunch of instances from your entity manager and send them over to your client, the client might end up with null collections. Now of course you can always pay attention, or use some tricks to initialize associations up to a certain level before you send your data, but this process is not automatic and it’s very error-prone. As for using things like dpHibernate, I think it just adds too much complexity and uncontrolled server requests.
The laziness argument: Come on! It’s not that hard!
I think that most of the time, the real reason why people don’t want to use DTO’s is because they’re lazy. Creating new classes, maintaining code that does “almost” the same as existing code, adding some code to service implementation to copy data back and forth, all of that takes time and effort. But laziness has never been a good reason for ditching a design pattern altogether. Yes, sometimes, best practices force us to do more stuff for the sake of maintainability and robustness of our code, and for me the solution is certainly not to shortcut the whole practice, but just to find the best tools to minimize the added work. With its property support and collection closures, Groovy makes both creating, maintaining and feeding DTO’s as simple and fast as it can be. AndroMDA had converters. There are even some DTO-mapping frameworks like Dozer to help you. No excuse for laziness.
For me, all the reasons above largely overcome the added work to maintain a parallel DTO structure.
Now of course, this is a very opinionated topic and you will probably have a different view. So all your comments are welcome as long as they remain constructive and argumented.
Flex on Grails, Take 2: Part 3
At the end of the second article in this series, we ended up with a working application but it was not really ready for the real world because it had one major flaw: the URL of the AMF endpoint was hardcoded in the client in such a way that it was impossible to change after compilation and very hard to handle several environments (dev, test, prod). The solution to that problem is to integrate dependency injection into the mix.
Now there are a lot of such frameworks for Flex/ActionScript applications, including Parsley, Swiz, Cairngorm, etc. But I’ve never been a big fan of those big MVC frameworks that impose their own interpretation of the MVC pattern and completely limit the initial capabilities of Flex itself. For me, the Flex framework itself is clean enough that you don’t need all that overhead and it’s better to use a non-intrusive framework like Spring ActionScript. So that’s what we are going to do.
Flex on Grails: Take 2, Part 2
This is a follow-up post to Flex on Grails, Take 2.
Now that we have a basic working application, let’s improve it by adding some security in there. If your Grails application is still running, you can leave it that way as you won’t need to restart your backend every time you modify it. That’s the whole beauty and productivity of Grails and this plugin.
Flex on Grails: Take 2
A little bit of history
When I first discovered Flex, one of my first obsessions was how to make it work with a Java backend. I’m a java developer at heart and my Java backend stack of choice back then was Spring/Hibernate-based. That’s why I published a series of full-stack articles that became quite popular. But another obsession of mine has always been productivity so when I discovered Grails, it became my new preferred environment and I started looking for ways to plug a Flex frontend into a Grails backend. All of this work culminated in the release of my Grails BlazeDS plugin which worked great but had a few limitations (only Java DTO’s, run-war instead of run-app, etc.). I mean, it worked great… until it didn’t. For some obscure reason, my plugin didn’t work at all with Grails 1.3.x. I fought with this for months, but there were just too many technologies involved (Groovy, Grails, BlazeDS, Spring-Flex, Spring, etc.) and my knowledge of some of those technologies was too shallow to really understand everything happening under the hood. That’s why I called upon SpringSource and/or Adobe to help me or provide the community with a decent Flex support for Grails. And guess what! They listened. A couple of months ago, I got in touch with Burt Beckwith, from SpringSource, who intended to work on that. So he asked me for feedback and really that’s all I did: I explained to him some of the issues I had with the plugin, the typical environment that we Flex developers work with, etc. And today… TADAAAA! The new Flex support plugins are here.
Adobe, SpringSource, Please Help us with Grails/Flex integration
I’ve been using Grails extensively for a year or so, and I love it, I really do. I even wrote a couple of plugins for it, including one for integrating Grails with Apple Push Notification Service. But since I’m always looking for productive ways to develop, I always thought that Grails and Flex were the best combination ever… until I hit the brick wall!
There’s a Grails Flex plugin, but it has 3 major issues:
- The Flex plugin has not been maintained for years, and it uses the old-fashioned way of integrating BlazeDS into Spring instead of the more modern, robust and easy to configure Spring BlazeDS integration library
- It doesn’t deal at all with authentication and authorization which is for me a critical issue in any remoting setup. And the Flex plugin doesn’t really offer a solution to that.
- The Flex plugin integrates BlazeDS3 and suggests you should mix your Flex sources with Grails project, which doesn’t make it easy to work on the Flex part using Flash Builder and its great data connectivity wizards.
Those are the 3 reasons why I’ve been working on the Grails BlazeDS plugin which works great with Grails up to version 1.2.x. Unfortunately I’m tackling some challenges with Grails 1.3. And even with Grails 1.2.x, there are still some limitations with Flash Builder, mainly due to the fact that
- Flash Builder requires a standard WAR layout, which “grails run-app” doesn’t create
- Flash Builder doesn’t like artifacts that Groovy adds to classes at compile time, so it’s not able to generate client-side stubs
So from a Grails/Flex integration standpoint, we end up being stuck between an old solution that is working but incomplete and not integrated in tools, and a more modern solution that does not work anymore and forces us to use some workarounds.
All the community needs to make this work is a little coordinated help from Adobe and SpringSource so that we can have:
- a BlazeDS 4 + Spring BlazeDS 1.5 + Spring Security 3 integration plugin for Grails 1.3 and above
- a data connectivity wizard that does not require a standard WAR layout and is capable of generating client stubs based on Groovy classes.
I’ve left a couple of messages on Grails mailing lists, Adobe forums and SpringSource forums, but so far, all my calls for help have remained unanswered. So if anyone is willing to help or support this ongoing initiative, it’ll be greatly appreciated. Just leave a comment if you need such a better Grails /Flex integration and maybe it will trigger a red light somewhere at Adobe and/or SpringSource. And if someone from Adobe or SpringSource is reading this, please help!
Grails/BlazeDS/Flex/iPhone Full Stack Part 2/3
In the previous episode, we built a simple Grails backend for the todolist application. In this installment, we will create a simple Flex 4 front-end for this backend.
The following assumes that you have already installed Flash Builder 4 (formerly known as Flex Builder), either in standalone mode or as an Eclipse plug-in.
Grails/BlazeDS/Flex/iPhone Full Stack Part1/3
A couple of years ago, I published an article on this blog entitled “Flex, Spring and BlazeDS: the full stack!” and this article became very popular. Actually it broke my daily visits record. Today I’m gonna try to break this record again.
In the last couple of years, I’ve worked a lot with Flex and Spring. But in my eternal quest for productivity and user experience, I discovered Grails. Based on the same ideas as Ruby on Rails or Django, it combines a dynamic language – Groovy – with the power of “convention over configuration” to make it possible to create web applications in no time, thus allowing you to spend more time on your user experience.
Of course it’s not the only thing that has changed since then. Flex 4 is now finally out in the open. BlazeDS 4 too. And Flash Builder is finally productive enough for me to use it… in combination with IntelliJ IDEA of course.
All those evolutions in my toolset needed to be integrated, so I ended up building a Grails plugin to do just that: Grails BlazeDS plugin. And since this plugin could not be officially released without a proper demonstration, here we are.
I prepared this tutorial for a BeJUG talk I gave last week. So I want to thank Stephan Janssen and the BeJUG for inviting me… and for staying with me without throwing vegetables at me, given all the failures I had during the live coding session. For those of you who were there: I clearly identified PermGen space as the guilty part responsible for all these blank screens. In fact, since Grails migrated from Jetty to Tomcat as their runtime server, default java memory settings are not enough anymore, so I always configure my projects for that, but of course, in live, I forgot…
Anyway. I’m going to publish this tutorial in three episodes, each one of them being accompanied by its very own screencast on Vimeo (damn Youtube and their 10-minute limit!). But I’ll also publish the tutorial transcript right here for those who want to go faster.
Important note: this tutorial covers the following set of technologies in the specified versions:
- Grails 1.2.2
- Grails Spring Security plugin 0.5.3
- Grails BlazeDS plugin 1.0
- Flex 4
- BlazeDS 4
- IntelliJ IDEA 9.0.2
- Flash Builder 4
- Spring BlazeDS Integration 1.0.3
- Spring Security 2.0.4
- Spring 3.0.0
In particular, it seems that Grails 1.3 that was just released a couple of days ago breaks Grails BlazeDS plugin. I’ll update both my plugin and this tutorial when I can make them all work together again.
Grails BlazeDS 4 Integration Plugin
One of the main goals I’ve been pursuing for a few months is the integration of Grails with Flex 4. I need to rework ConferenceGuide‘s administration backend to make it more ergonomic so that we can cover more events, and ever since I discovered Flex 4 niceties, I couldn’t think of doing that with anything else. The problem is that none of the existing plugins suited my needs. All of them cover Flex 3 only, some of them introduce a lot of complexity for CRUD generation, some of them use GraniteDS instead of BlazeDS, and the simplest plugin, grails-flex has never gone further than the experimental stage. I did a lot of experiments, talked a lot about it on Grails mailing lists, until Tomas Lin kindly explained to me that maybe I was approaching it the wrong way. I wanted a plugin that would set up a complete environment for Flex developement right in the middle of my Grails application. And I wanted that because I wanted to avoid using Flash Builder (you know… Eclipse… ierk!), mainly because the only advantage of Flex Builder 3 over IntelliJ Idea was the visual designer. But he was right, I was wrong. Flash Builder 4 DOES change everything. It does include a lot of very interesting features that greatly improve Flex development productivity, especially when it comes to integration with backend technologies. And for those features, I can only admit that IntelliJ is not up to par yet. I’m still gonna use it for the Grails part, but Flash Builder will be my environment of choice for Flex 4.
So, once I learnt more about Flex 4, BlazeDS 4 and Flash Builder 4 beta 2, it was time to reconsider my approach and develop a much simpler Grails plugin so that any Grails application could be used in combination with Flash Builder. I just released grails-blazeds plugin to do just that. Here is how it works:
- Install grails-blazeds plugin: “grails install-plugin blazeds”. This plugin copies a couple of configuration files into your application and imports all of the required libraries, including BlazeDS 4 and Spring-BlazeDS integration 1.5, both in nightly snapshot versions, since they have not been officially released yet
- Create a service to be exposed to your Flex application over AMF, or choose an existing one
- Add @RemotingDestination annotation to your service class, and @RemotingInclude annotation to all of the methods in this service that you wish to expose
- Edit web-app/WEB-INF/flex-servlet.xml (created when you installed the plugin): uncomment the context:component-scan element and set the base-package corresponding to your service class
- Make sure your exposed service methods don’t use Groovy classes, either as argument or return types. This is a known limitation I’m still working on, but if there are some Groovy classes here, Flash Builder doesn’t manage to generate ActionScript counterparts.
- Run your Grails application using “grails run-war” instead of “grails run-app”. Once again this is a known limitation: Flash Builder BlazeDS data connection plugin relies on a classical web app layout and doesn’t understand Grails dynamic layout (that is until someone manages to create a Grails data connection wizard for Flash Builder 4)
- In Flash Builder 4 beta 2, create a new Flex project with a J2EE server. Here are what your parameters shoud look like, “conferenceguide” being the name of my Grails application and “sarbogast” being my home directory:
- Click “Data” menu, then “Connect to BlazeDS…”
- In the “Authentication required” dialog box that appears, check “no password required” box, and click OK
- You should see your service appear, and you can select it and click Finish.
- Your service should appear in the Data/Services view. You can then compose your user interface and drag and drop your service methods to the relevant components to connect them with your Grails backend.
- Don’t forget to configure a channelset on your service:
<adminservice:AdminService id=”adminService” fault=”Alert.show(event.fault.faultString + ‘\n’ + event.fault.faultDetail)” showBusyCursor=”true”>
And there you go. Special thanks to James Ward, whose screencasts really helped me get it right. Now the only thing that this plugin misses, beyond the 2 known limitations, is integration with Spring Security, but this is just a start.