Archive for the ‘Geek Culture’ Category
Knowledge Management: It’s All About Granularity
In my pursuit of the ideal collaboration platform, I’ve tested a few knowledge management systems lately: Knowledge Plaza, Seemy, a combination of del.icio.us and Twitter. And those tests were very interesting because they allowed me to spot the main common problem they all share.
How many times have I heard that the Google Wave presentation is too long, leading people to simply not watch it at all? How many times have my friends complained to me about the length of my own blog posts? The granularity of information on the web is simply too big. The web is all about resources, and there are billions and billions of these resources out there, and what makes it even harder to process and integrate them is that each resource mixes a lot of different information items.
And for me, THIS is the nightmare for my technology watch, and for knowledge management as a whole. You can comment on or share whole web pages through links, whole Youtube videos through embed codes, whole discussions through podcasts. But what if you want to extract what is to you the essential part of a blog post, the funniest moment in a video? Well, let’s say I don’t know any solution for that.
For my everyday technology watch, what I would really need is a knowledge management platform that allows me to select small chunks of information in text, video, audio or images, and then tag those chunks, comment on them, and store them somewhere in the cloud for sharing them with my friends or colleagues, or simply keep them for myself for later reference. All of that while keeping a link to the full original resource of course. That would be awesome!
Now of course because I love to solve problems, my next move is to think about a solution. I don’t know any existing system that does that, so if you do, please tell me about it. Now if it doesn’t exist, we have to invent it. And the way I see it, there are two main aspects to this system.
The first issue is how do we capture excerpts out of web resources. If we want to make it as simple as possible, we need to integrate deeply with a web browser in order to create a natural user experience based on drag-and-drop selection, keyboard shortcuts and so on. This is why I’ve tweeted about me looking for a Firefox extension developer to help me out: I’ve never developed any Firefox extension myself, and I could learn but (a) it would take much time and (b) I’m not fond of Javascript. So once again, if someone out there is a Firefox extension developer and would like to collaborate on this experiment, you are welcome. Let’s try first with text, we’ll see later for other kinds of multimedia content.
And the second issue is how do we store and present all this information in a highly usable and intuitive way, without being too disruptive, without inventing too many new concepts. This part I can handle. I already have a few ideas.
I think before the Internet, there were technology watch departments in companies, whose job consisted in cutting out paper pieces in newspaper, pasting them and composing press reviews with comments and writing reports about what competitors were doing. Nowadays, it’s as if we just gathered full articles or newspaper pages, videotapes, full interview transcripts and just put small post-it notes on them. It’s just too rough, not pre-chewed enough, not efficient enough. And as always, there’s gotta be a better way.
What do you think?
Text, Expressivity and Culture-Oriented Programming
Following up on my reflexion about what could software development look like a few years or decades from now, there is this big problem that has been bugging me for years now and that I have never found the time to really tackle: expressivity. In the same way as files appear to me as the biggest obstacle to collaboration, I think the main barrier in the way of expressivity is TEXT.
It’s hard to admit, but we’re still building software like cavemen. We don’t have spoken language, just a bunch of noises, we don’t conceptualize much but we do communicate with a few gestures and more importantly some colored drawings on cave walls. The way I see it, we are not much more advanced than that, but it’s normal, software is still relatively young as a discipline and although it has already changed our lives, we have to imagine that it’s just the beginning. And the good news is that we are headed in the right direction.
We started off with most elementary way of storing information and communicating with a machine: zeros and ones. Binary. It was too elementary, more like noises coming out of our mouths, so we started to group bits in octets corresponding to hardware instructions and characters. In fact, we added gestures to noises. Then we grouped instructions into statements and procedures, and we designed a way to translate those into the most elementary form of language that machines could understand. We started drawing on walls. But as procedures multiplied like crazy, we needed to conceptualize some more, talking about classes, objects, methods, properties, and so on. Spoken language was born. And with higher level concepts like services, components and multiple programming languages, we added written language. OK, the analogy is not that good, but you get my point: I’m convinced we’re still very early in the overall evolution of communication with machines, and although this evolution is somewhat slow and creates a lot of inertia, I believe that if we want computers to really expand our capabilities (note that I didn’t say “replace us”), we need to go further in abstraction levels.
So what’s next? Binary, assembly, procedural, object-oriented (yes, and functional, if you want), then what? Model-driven? I’ve tried that, it’s just replacing the constraints of text with the constraints of visual representations. It sure makes it easier to conceptualize, but at some point we’re still translating those visual models into text code, which we have to compile. The roundtrip is just too long. What about domain-specific languages? Well, I’m more into that right now. It looks like communicating is naturally based on languages, collections of concepts that relate with one another to describe what a software is and what it does. So focusing on making it easier to define new languages definitely goes in the right direction. That’s why it’s so linked with meta-programming: instead of statically defining layers upon layers of fixed concepts to describe systems with even higher level abstractions, let’s define the root concepts we will use to describe languages that will allow us to describe our systems. That’s why I’m so interested in Groovy at the moment for internal DSLs, although I prefer the more elegant idea of external DSLs and language workbenches, like Jetbrains MPS does.
All of this evolution makes me think of a video I saw recently, that tried to make String theory more accessible:
Let’s say binary is our dimension 0. Assembly is dimension 1: a line. Procedural programming is dimension 2: a plane. Object-Oriented Programming is then 3rd dimension: a volume. And it’s very hard for us to leave it, as it is our most natural way of seeing things. But that’s where I find this explanation of String theory particulary interesting (although not rigorous as some math geek friends of mine pointed out): there starts a cycle. Dimension 4 is a line again, formed by 2 different Object-Oriented Languages, like Java and C++ for example. Still there? Good! That’s where the fun starts: dimension 5 is a plane composed by all the parallel universes that are created by our own choices. Functional programming and OOP can be considered as forming such a plan. Now what if we could design a way to go directly from one of those paradigms to another one, to fold the plane in the 6th dimension: please welcome language-oriented (or meta-) programming! See the cycle? Now most of us are stuck in the 3rd dimension, and some of us are already experiencing the 6th dimension.
So there we are. Seventh dimension is the line joining the set of all possible timelines starting from our software big bang, which is the binary transistor, to another set of possible timelines, starting from another big bang. Quantum computing can be another option, but it’s a hardware one. What about software? Isn’t virtualization a way to forget about the physical hardware? And there we go 8th dimension: going from binary transistors to quantum computing is one line in the seventh dimension. Choosing to go to virtualization instead creates an branching line in the 8th dimension. Which means that if we want to create our ninth dimension, we need to fold the eighth in order to jump from quantum to virtual computing. And that’s where I locate what I call Culture-Oriented Programming. The third stage of the third 3-stage cycle. The final frontier? The next step? Hooooo… my head hurts.
But wait a second, I only talked about computing here. A virtual reality. What if dimension 10 was the line uniting computing with the real world in the purest possible way. Direct communication between human beings and computers. Who said “scary”?
PS: I didn’t intend this post to be so “theoretical”. I only thought with my keyboard and let my imagination go. But I’d love to know what you think about that crazy analogy? Do you think we are limited to 11 dimensions like in string theory?
PPS: That might be my geekiest post EVER!
Software Development and the Whiteboard Paradigm
For years we have been using computers in a way that mimics the office environment. Most of the concepts that we are manipulating were designed as virtual emulations of real-world artifacts: workspace, desktop, folder, file, button, menu, (e)mail message, inbox. The problem with any metaphor is that, at some point, it just doesn’t make any sense anymore. And if you want to stick with it, you’re forced either to stretch the metaphor to a point where people lose it (like I’m giving you a file, even though I’m keeping an exact copy for myself), or to invent even more metaphoric or exotic concepts to fill in the holes (like pokes and docks and so on).
More specifically we software developers have been fighting with the very metaphors we have invented some decades ago. All of our applications are made up of files that need to be compiled and linked together so that the computer can understand them and run them. Yet the computer world has changed a lot in a few decades, and it’s very interesting to wonder whether all those concepts are still valid, or whether they don’t hinder our very creativity and the magnitude of the problems we can solve. Indeed, we’re talking more and more about cloud and parallel computing, language-oriented programming, business process modeling, a lot of higher-level concepts that we are still forced to anchor into low-level files and computers. Language workbenches generate traditional code, cloud computing platforms need a lot of middleware to interface with our programs, and BPM uses a lot of black-box magic.
What bothers me the most at the moment is how these old school metaphors impact our way to collaborate. For me it all resides in the FILE concept. Files were not invented to collaborate. Files were created for storage. You put some information into a single unit of content and then you store it for later use. And if someone needs to reuse your work, then you must find the right file (using some ordering mechanism like folders), copy it and send it, which means that many things can go wrong in the process. The copy can go wrong, someone can intercept or interrupt the transfer of the file and more importantly, the file that your collaborator will be using is merely a copy. Which means that if you need to reuse his work, then he needs to send you a copy of his own work, and you’re back where you started. Now imagine that process happening several tens of times a day for hundreds of collaborating software developers, and you can quickly understand why there is such a huge market for coordination software. Keeping track of modifications, resolving conflicts, linking code with business requests and so on. All of that because of what? Because our primary collaboration artifacts are FILES!
Files don’t make sense in this world anymore. All our computers are almost permanently connected, forming one giant virtual workspace in which we can forget about physical borders of files. Instead of exchanging data in the form of files, we are now able to exchange actions and services on common artifacts. Doesn’t it ring a bell? It’s like we are all standing up in front of a big whiteboard with our keyboard and mouse as a marker. We can all write together on the whiteboard, we can erase, we can write, we can draw, we can compose, we can link, we can step back to see the big picture. And since the whiteboard is virtual we can even do amazing things like actually moving elements from place to place, we can keep track of the evolution of the whiteboard. When you’ve done a couple of Walt Disney creative sessions, or experimented with SCRUM boards, you can easily feel the incredible power the whiteboard has on collaboration.
What’s interesting is that Google has already started to implement this paradigm shift for communication with Google Wave. A wave is really just a whiteboard where everyone can collaborate. Sure you can use it in a very linear way in which everyone writes stuff on it, one at a time. But this is just a very small subset of what you can do with it. That’s why Google Wave is truly groundbreaking: it is redefining the rules of collaboration by getting rid of all the useless constraints and freeing up our creativity. Now what Google is doing for generic communication, I would love to apply it to software development.
Version control, unit tests, continuous integration, issue tracking, release management… all those are clever patches that heal the symptoms of a much deeper disease. What if we could work on the disease itself? What if a software application was made up of real functional modules, and each module could be implemented by a separate team on a whiteboard? What if we could do that online without having to be in the same physical place at the same time?
I’m dreaming here. I don’t have any concrete solution to offer yet. I have a lot of ideas but not a lot of time to implement them because… well… I’m still doing a living writing FILES! But… one day… a software development environment without files… Zzzzzzz…
Oops! They seem to be doing it again!
I’ve seen a lot of job offers lately for Flex developers in London banks (including investment banks like Goldman Sachs this morning). I love Flex development and I’m starting to look for freelancing opportunities around this technology but technology is not everything to me: I also thrive on purpose, I love to work on stuff that matters to me. And the fact that the banking sector seems to be hiring rich internet application developers so much kind of worries me somehow.
Because one of the main interests of technologies like Flex is the quality of data visualization it allows. Now what worries me is that software has already been a major enabler in the subprime crisis, with algorithms able to calculate and mitigate the risks for very complex financial products. Now it’s like they’re saying “OK, let’s do the same shit, but put more human factor in it, so let’s give them more accurate data!”. Now this is just an external impression and maybe I’m wrong about what they’re doing with Flex. But maybe I’m not. Does anyone work with Flex in banking and can tell us what you’re doing with it?
As far as I’m concerned, I’d love to work on a “new energy”-related project, like what Roundarch did for the Tesla Model S. Or I’d love to work for a project like Better Place, or Desertec. I’m sure they could use some modern , productive and good-looking software too. But banking? Hum, not a good option for me until they integrate more human data in their decisions.
A New Experiment : eXperts UnLimited
I love technology watch. I could do that all day. Browsing my RSS feeds, discovering new technologies, reading strong opinions, commenting here and there. And I love answering questions too. Do you know a way to download any Flash video? Which solution would you choose for our messaging problem: Oracle queues or a custom messaging system? What are the pros and cons of Flex, Silverlight, JavaFX and AJAX? What do you think about Unit Testing? I would love to answer technological questions all day.
But I’m a geek. I don’t do that all day. My day job is to develop software. And even though I plan to change that very soon, more often than not, I’m not really convinced what I’m doing is needed. And a lot of customers I work for wouldn’t think that reading blogs, writing articles and chatting with fellow geeks is valuable work… that is until they come to me with a “what’s the better way to …?” question.
What if I could do that all day, or at least earn some of my living doing that? And I know a few guys who would really love to do that too. That’s why I’ve just launched a new public experiment called eXperts UnLimited (XUL). If you want to know more, or if you have questions or suggestions, feel free to leave a comment there.
Let’s Solve Problems
Let me paraphrase Clay Shirky here. Let’s talk about problem solving. When it comes to solving big problems, we assume that problems really worth solving can hardly be solved by one person alone. We need to join forces, energy, knowledge and skills of many people and coordinate the efforts of those people towards the collective solution of the problem. And there’s nothing wrong with that logic.
Now the question is, how do you coordinate such a collective effort when people live on their own? Well, the traditional answer is creating an institution: a common ground where all the persons involved in the problem solving effort can be at the same place at the same time so that communication cost is reduced to a minimal. That’s what companies, governments, associations are made for.
The problem is that those institutions tend to create 3 major side-effects. First, since people are used to live their life on their own in a very individual way, it’s not natural for them to coordinate. Hence you need to hire some more people who don’t participate directly in the problem solving effort, but merely help them to come to a consistent and viable solution. Second, because institutions are like some sort of engine, you don’t only need fuel to run it, and lubricant to compensate for the friction, but you also need a structure to hold it, a vehicle to move forward. And that is administration, taxes, financial stuff. Finally, in addition to management and administration byproducts, maybe the most important side-effect is goal shift: in some sort of fractal way, most institutions eventually shift their goal from whatever it was supposed to be at the beginning, to survival. It’s as if the institution became a being of its own, trying to survive at all costs, and making it necessary to group institutions into even bigger institutions.
The consequences of these evolutions are many, but most of them revolve around inefficiency. Since you literally move people from wherever they’re living to the institution, assuming it’s impossible to be at several places at the same time (right?) then it’s very hard to belong to several institutions. Hence institutions are highly exclusive, which means they keep hold on people from other institutions who might need their help, and keep people for themselves even when they don’t really need them. This ends up with a massive waste of force, energy, knowledge and skills. Another aspect of the problem is that management, administration and politics (institutional manifestation of survival instinct) create such overhead that some choices have to be made in terms of the scope of the problem the institution was created to solve. In other words, because of this overhead, most of the time it is just unrealistic to solve all of the problem, so you end up using 20% of the resources that are enough to solve 80% of the problem. And of course the next question is: can a problem considered to be solved when it’s only 80% solved? Maybe some problems more than others. Maybe we think that it’s sufficient because we were told it’s enough. What if climate change, financial crises and all those big problems we’re facing today were the byproduct of those 20% we left off of all our solutions?
In his presentation, Clay Shirky correctly notices that it might be time to reevaluate the roots of the institutional response to problem solving, that is coordination costs. Now that we have the Internet, VOIP, cellular networks and so on, communication costs have never been so low. In this era, do we still need to be at the same place at the same time in order to join forces? Of course not! Those changes have already influenced our professional world with video-conference, corporate collaboration systems like emails. But what if this is just the beginning of a massive transition, comparable to the one initiated by the printing press, or the one started by the mechanization of agriculture? Well, Clay’s vision (that I totally share) is that the professional world is slowly moving away from institutions and towards more collaboration for problem solving. No need for people to be at the same place at the same time anymore, so no more administration overhead. And since the beast is only virtual, there is less chance for it to turn into a monster who wants to survive at the cost of its initial purpose. And since such collaborative groups would not be exclusive anymore, maybe we could get back those wasted resources in order to actually solve those 20% of the solution we were missing.
Now of course, there’s no miracle, only evolution. And if we have to recruit managers today in order to help people coordinate their efforts, it’s very likely that we will need to find ways to help people collaborate more efficiently tomorrow. My assumption is that this issue is part of those 20% we left off yesterday because it was cheaper to teach a few managers how to coordinate efforts for others, rather than actually bringing this knowledge to every single individual. What if even that became possible tomorrow? Instead of creating specialized institutions for teaching some people how to solve problems, others how to coordinate problem solvers, and yet others to run the problem-solving structure, what if we educated people through the same collaborative groups, the purpose of which being to expand the knowledge and skills of people as well as teach them everything they need in order to gradually evolve towards solving concrete problems?
That’s what I thought when I read Bruce Eckel’s article about Edupunk (and other stuff, but this part struck me the most). Now I don’t know how or when, but this could very well become a major breakthrough in my personal and professional life somehow. So if you agree with me, what do you say we join our forces and collaborate on that? Let’s solve the problem of expanding our global knowledge and skills in a collaborative way?
About Inertia in Corporate Software
When you’re passionate about software, it can get really frustrating to notice how corporate customers are sometimes suffering from software inertia. You talk to them about all those wonderful technologies that you’ve come to master, all those innovative solutions you can offer… and they tell you that they’re looking for AS/400 developers, or people with 10+ years of experience with EJB’s. As a consulting company, it can be tempting to give in to that kind of request. As a software consultant, I always feel like it’s my duty to improve the level of information of the person I’m talking to, and be a little more aggressive.
Why so much inertia?
There are a lot of factors that play a role in this situation, proportions varying depending on the company and its history. An important one is certainly vendor lock-in. Old-school software vendors like Microfost, IBM, Oracle, Serena and others have become masters in selling big expensive licenses for their tools. Their strategy is not based on the quality of what they’re selling, but on things like their long-time reputation, the belief that when the price is high then it cannot be bad, the coherence principle (you never question your own decision when it requires a big sacrifice from you) and so-called integrated offers: operating system + development environment, reporting+collaboration, tools+methodology. And they always add something exclusive to the mix so that switching to another vendor is as hard as it can be.
Very often combined with vendor lock-in is the belief that coherence lies in unity, that if you go for everything Microsoft or everything IBM, you’ll never have to make big decisions again. Whenever there is a new need in your software infrastructure, you’ll just take what your “preferred partner” has to offer for that, and you will end up with choosing tools not because they are the best at what they do, but only because they integrate nicely with what you already have.
Another dangerous quest is the one of the silver-bullet technology, the tool/architecture/methodology that will solve all of your problems today and tomorrow. Once again, big software vendors have traditionally played an important role in sustaining this myth, abusing buzzwords and repackaging old offers in new shiny boxes. And what’s even worse is that very interesting principles have been completely perverted because of this quest for the one technology that will solve all problems. Unfortunately, like most myths, this one was born in laziness and experience shows you that it’s very important to characterize each problem in order to always choose the right tool for the job.
Another motivation for sticking with your current infrastructure is to capitalize on your expertise. With time, your people start to master your infrastructure, know its weaknesses, learn how to work around them. All this experience is definitely precious and it is for me the first good reason to maintain a certain level of inertia. That and the fact that all this internal knowledge can make you save some budget on training as people can train one another and as you can build your own knowledge base.
Last but not least, there is a very important aspect, that is not always admitted explicitly, but is usually very present in big old companies: fear of chaos. IT in general and software in particular are very dynamic fields, and this trend has been accentuated by the Internet in the past decade. Combine that with the fact that most young IT professionals have a tendency to play with the newest toys on the market, and you quickly notice that things can get a little out of control when people try to incorporate immature technologies and then move backwards when they realize the cost of this immaturity in terms of maintainability, durability, integration, training and so on. This fear is perfectly understandable, but as always it’s the extreme response to this fear that can have dangerous effects.
What are the problems?
The first issue that comes to mind is the lack of agility (Titanic effect). The older the tools are, the more likely it is that they were designed at a time when the business conditions didn’t evolve as fast as they do today. Hence it becomes harder and harder to adapt to those changing business conditions, to collaborate with partners, to merge infrastructures, to expand business models and so on.
Behind lack of agility, there is a more general symptom that shows that your software infrastructure is bloated: people lose an awful amount of time solving very old problems that have long been solved by newer technologies. Productivity is key both for the motivation of your software team and for your ability to quickly implement new requirements. But mature technologies are also getting older and older, and as time goes, it becomes exponentially harder for them to integrate new features, to the point when this evolution is just not possible anymore, forcing you to do the Big Switch.
It’s a well-known disease in corporate IT: instead of making it evolve continuously, inertia often leads companies to evolving in eras, the transition for one era to the next one being often marked by big rewritings. The cost of these efforts is usually extremely high, not only because of the resources you need to allocate to this rewriting, but also because of the time during which you won’t be able to make your features evolve. Some of this cost might be compensated by the fact that you take the opportunity to improve performance or add new features. But it can also be amplified by bad business decisions like trusting Indians to do the porting or something like that.
Another important issue is the growing cost of maintenance of legacy systems. Even when you realize that rewriting everything on each transition is risky, maintaining legacy software becomes more and more expensive as it is harder to find resources with the knowledge of your technologies. Combine that with the point mentioned above about productivity and you understand that it takes more time to more expensive people to maintain your system.
Finally, as I said before, technology passionate people often tend to like playing with new toys. Now even if letting them completely free to do so can be very dangerous for you as a decision maker, forcing them to use only old technologies that they will probably never need anywhere else can also frustrate them to the point when your will lose their intelligence and insight. That’s when the cost of turnover hits you in the face.
So what can I do?
That’s the question you might ask yourself if you have some decision power in a company with too much software inertia and too many of the issues mentioned above. Here are a few leads that can help you improve the situations. Most of them are based on the generic principle that you have to find the right balance between capitalizing on your existing infrastructure and make it evolve continuously.
The first key for me is technology watch. RSS feeds and conferences are a great way to keep in touch with what’s happening outside of your organization, and even what’s coming in the next few years. When you’re in darkness and you don’t have a clue about what’s around you, it’s always safer not to move at all. But since you need to move, you also must learn more about your industry and what others are using, even if you don’t incorporate all of them.
The next step is to be honest with yourself about the limitations of your current infrastructure. As I said before, a lot of software vendors rely on the fact that you won’t discuss your own decisions when they are very expensive. So it takes some courage to be lucid about the weaknesses of your current system and monitoring those weaknesses in time is crucial, either to find local solutions to those limitations, or to decide when it’s time to replace a technology that is too limited. This is especially hard to do in bureaucratic environments where decisions tend go from the top down, and bottom feedback is not often listened to. So my advice: listen more to the people who are using the technologies you choose on a day-to-day basis, and you might realize that your situation is not as perfect as you currently think it is (Ostrich-style).
When you hear about interesting new stuff, don’t hesitate to try it out, to allocate some resources to develop a proof-of-concept for it, or go through a getting-started tutorial. Very important: do not base your opinion solely on rumors, analyst reports and opinion of others. Don’t forget that in the same way as silver-bullet is a myth, the technology that is inherently and universally bad does not exist. So what might be the right tool for the job of other might not be good for you, and vice-versa. That’s why making your own opinion is very important, even if it will never be comprehensive.
Another important thing to understand is that innovation starts in opposition. Opposition against the status quo, against existing technologies, which often leads innovators to restart from scratch in very original ways, making it harder to integrate those innovations with existing technologies. But if those are successful, there is very often a second phase in which those innovations are incorporated into existing technologies, or at least some work is done to ease their integration with mature tools. For almost every new technique, version 1 is a proof-of-concept, version 2 turns into a more complete solution, version 3 gets integrated with existing technologies. So when it gets to that version 3, it’s always interesting to try it out.
Next, let’s talk about Open Source. It’s incredible to see all the limiting beliefs that circulate around Open Source. Beyond the fact that people tend to associate free and low-quality (a belief that is of course sustained by big software vendors, even when they benefit from Open Source themselves), a lot of IT decision-makers also think that if it’s Open Source, they won’t have any support for it. My answer is that if it doesn’t have any support, it is in fact not worth it, but make sure there really is no support, because one of the most successful business models for Open Source is corporate support. Hence most interesting Open Source technologies are backed by at least one company offering high-grade professional support. And maybe the most unfair argument is that Open Source is dangerous because it can disappear over night. When you understand how most Open Source licenses work, you see that they simply forbid project owners to hide their code, and beyond that, communities are often much more reliable than software vendors who can bankrupt in no time. I’m not saying that Open Source software is more reliable and durable than commercial software, but it is certainly not less so. Now beyond this reliability and durability issue, by definition, Open Source projects don’t rely on vendor lock-in, but on the quality and usefulness of their technology, which leaves you with more choice to go from one to another and to combine technologies coming from different providers. Now I just want to mitigate what I just said: Open Source can definitely help you to get rid off your inertia more than you think, but it might also create new problems like free lock-in, in which you end up using a tool not because it’s the best for the job, but simply because it’s free. Fortunately for us, not all commercial software vendors are as rotten as big old ones.
Architecture can also play an important role in gaining more agility with your software infrastructure. Try to avoid big monolithic solutions and favor modular architectures allowing you to change parts of your systems without rewriting everything. Virtualization and componentization are definitely going to help you in that endeavor, as well as getting your software architects more experienced with composing their own toolkits for specific problems, instead of forcing them to always apply the same bloated infrastructure.
Finally, important enemies of agility are over-confidence and self-sufficiency. Of course I’m preaching for my church here but I think that bringing new blood to your teams in the form of external consultants can really help you make your software technologies evolve continuously, especially when you don’t just use them to fill in boxes but instead empower them to help you get more agile… and avoid the iceberg (shameless plug inside).
How Open Source can be a Game Changer
A few years back, I read a very interesting article (that I can’t find anymore) by John Newton, the CEO of Alfresco, explaining how Open Source changed their strategy. Of course, one of the main interests of having an Open Source version of your product is to expand your market by making it more accessible, thus creating opportunities for high-value services such as training, customization and so on. But beyond this shift in “how do we make money?”, he explained that it totally changed the nature of their salesforce. When you sell that kind of enterprise system, the best way to get potential customers to know and see your product is to invest a whole lot of money in marketing and sales. And you end up with a sales cycle of several months between the first prospection call and the actual sale… when it goes through. No wonder why license fees are so expensive for that kind of product, making it even harder to sell, increasing the length of the sales cycle, and here goes the vicious cycle.
Having at least an Open Source version of your product completely changes that because people who don’t have the checkbook but will eventually use your product can try it on their own. Maybe they’re working on personal projects, maybe even Open Source projects, and they can install and use your product for free. Then if they like it, they are more likely to go see their boss at work and recommend your product. In other words, you can replace a big chunk of your very expensive salesforce by free happy end-users and their recommendations. Hence not only does Open Source change how you make money, it makes it possible for you to save money where it’s not that necessary, and focus a lot more on your core business: building a great product.
But the way I see it, there’s another very important change with Open Source. As I said before, when your business model is based only on license fees, you focus most of your attention on your salesforce, which costs a lot of money, which makes your license fees so high that it is very likely to represent a big investment for your customers. So your salesmen end up attacking prospects at high levels of management. The problem is that the guys who have the checkbook also have very different concerns and priorities compared to the people who will eventually use your product. Let’s say you are building a software project tool suite with things like issue tracking, source control and so on. You try to sell your product to companies who do in-house software development, and the people who will eventually use your software are developers. But you don’t talk to those guys directly because they have almost no decision power, especially when it comes to spending several hundreds of thousands of euros for software tools. So you talk to their Program Manager or VP of Software Engineering or CTO of some sort. And all of a sudden, you realize that those people have concerns like keeping things under control whatever happens, minimizing risk (whatever that means), reporting, making sure that no developer does anything on their own, and so on. So of course, you will implement features to satisfy those needs because you want the big guys to be happy.
Now let’s go back to the Open Source alternative. If you hope that developers will use your software on their own project, find it cool and then go back to their boss and say “we should have that”, you’re not talking to the same people. It really makes things straight again because you’re back to consider end-users’ concerns first. Developers want tools that don’t get in their way, that allow them to save some time, not waste it, that are integrated in their development environment, and so on. My point is that an Open Source model does not only change where you make money, or how you balance your own investments. It can also change your whole product itself by changing whom you’re talking to.
Now of course, I’m opposing 2 strategies here: the traditional top-down approach that forces you to focus on upper-management priorities versus the Open Source bottom-up strategy in which end-users are the ones you have to convince. And I know that this kind of 2-way alternative is likely to create controversy because project managers will read this article and think “but we need reporting, we need control, we need power!”. Now of course you know that I don’t agree with this obsession for power and control as it is associated to the software engineering mirage. It’s like the big guys couldn’t stand the IT guys anymore, with their strange language and culture and so on. So they irrationally tried to isolate us from them with layers and layers of project management and control. But they lost a great deal of intelligence in the process and they’ve created a whole generation of frustrated developers who just execute what they have been asked to, even when decisions are made by people who don’t understand a damn thing about what software is and how powerful it can really be. And of course, big old-school software vendors have created opportunities out of this mess and we’re all forced to work with their products now. But hopefully, Open Source is already changing that, it is participating in a movement of IT empowerment, together with Agile methodologies, that will lead to more intelligent uses of technology and fill in the gap between business and IT.
Now to conclude, I’m not saying that management is totally useless. YES, we have a strange culture and a weird way to look at things. So YES, we need people to help us interface with business people and teach us how to do it ourselves. NO, the solution is certainly not to reduce our field of action to the minimum but YES, we need people who understand both the power that we have AND the big picture of what is necessary, and are able to make them coincide. And YES, this mission might require additional features in the tools we’re using, for things like reporting, prioritization and so on. But those features should never conflict with our concerns as end-users, they should never get in our way to building better software. But the good news is that there are very clever ways to do just that: have a look at Mylyn, or FogBugz. And of course, I myself am thinking about bringing my own contribution. So stay tuned…
Now I would love to try an experiment and use comments on this post as an informal survey. So if you are a software developer, what tools are you using at work for things like source control, issue tracking, documentation management, and so on? Do you like those tools and why?
Software Engineering Is NOT Dead!
There’s been a few reactions lately about the Tom DeMarco article. To be honnest with you, I didn’t know the guy but apparently he’s quite respected. And the reason why I didn’t know the guy might be related to the fact that he wrote a software-related book entitled “Controlling Software Projects: Management, Measurement and Estimation”. In other words, he’s the one (OK, one of the guys) responsible with all those project managers asking us for sharp estimates, negotiating them down with us, and committing on them without our agreement, leaving us with no choice but to produce crappy software and/or fail to meet the deadlines. For me, Tom DeMarco looks a lot like this Dr. Winston D. Royce dude, who first described the theory of Waterfall methodologies and forgot to write the following sentence in bold font: “the implementation described above is risky and invites failure“. No kidding!
My reaction is about the same to what DeMarco took 40 years to realize. Software Engineering is not dead, it has never lived! There has been a lot of controversy about that, some people arguing it’s more of an art, others saying it’s science, yet others that it’s more like craftsmanship. At the end of the day, the only fact that there is a controversy shows that it doesn’t fit in the hole we’re trying to stick it into. And yes, this blog is called Software Artist, but I have to tell you that it’s more as a reaction to those who are fiercely trying to “industrialize IT processes”. Oh man, that makes me angry! In the end, I have to say that the more I think about it, the more I like the idea of craftsmanship.
Because craftsmanship is all about finding the right balance between the techniques that you’ve learnt and the intuition that you have about what you’re doing. And finding this right balance requires a lot of practice. Note that I didn’t say “time”, I said “practice”. I like it also because for any craftsman, using the right tools for the job is fundamental, and once again, note that I said “the right tools”, not “the tools that everyone else is using in here” or “the tools we invested a lot of money on and we need to make up for”. And finally I like it because when a joiner tells you that he doesn’t know exactly when this wardrobe will be finished, you don’t bother him with that because the fact that it corresponds exactly to what you need, and the fact that it will last a very long time, largely makes up for this little uncertainty. And if he tells you he’s going to need another 4 days, you don’t negotiate, because if you know better how to do it faster, why in the world didn’t you do it yourself in the first place? Or why didn’t you go and buy one at IKEA?
Now after re-reading the above paragraphs, I realize that my statements might seem a little harsh and extremist (as usual). But I don’t want to sound negative here. So if you manage software people and you feel disturbed by this perspective to lose control, let me just tell you this. Software can be incredibly powerful, it can really create tremendous value in a very flexible way. And I’m not talking about allowing you to just save money. I’m talking about the possibility to develop your business and create value from scratch. But the more value you expect to create, the more you have to ask yourself whether you absolutely need to control every aspect of the software creation process. Remember, you’ve tried it before, and it most probably didn’t work as expected. But fortunately for you there’s a different approach. Empower your software team. Don’t lead them, but make their ride smoother. Don’t make them work FOR you, work WITH them. Don’t consider them as standard office workers. Instead, hire the best craftsmen you can find and trust them with your project. Make it theirs as well as yours. Surrender on this idea of controlling everything and you’ll be surprised at the result.
Software engineering is not dead, because it has never existed. Hence there is no need to mourn it. Let’s just roll up our sleeves because we have a few palaces to build for the years to come.
Why Do I Hate Eclipse?
As a consultant, I’m very often forced to use Eclipse as a development environment, and every time I do, it’s such a pain for me that I can’t help complaining about the poorness of this thing. And every time I do, most of my team mates, who have been brainwashed by the monopolistic propaganda of Eclipse, just keep asking me what’s wrong with it. And sometimes it’s hard to explain because it’s really a matter of user experience. And each time I find a specific example, I get answers like “yeah, but that’s just one thing”, or “I’ve never had that, you’re not lucky”, or “this is just because you’re not used to the Eclipse way of doing things”, or even the worst one “maybe yes, but it’s free!”. Since when is “free” a feature?
Right now, I’m reading the SpringSource dm Server getting started guide, and I was very surprised to read that SpringSource guys, who aren’t exactly stupid, and seem very experienced with Eclipse itself as they have based all their development tools on it (Spring IDE, STS, etc.), talk about what they call the “Eclipse Dance”. I didn’t know about the expression but I’ve definitely danced it more than once: every now and then, Eclipse views get all mixed up, some views indicate errors in a file, while other views on the same file say everything is OK. Or you get a message saying that it cannot find a class where you have the source in front of your eyes. Or like now, I have 2 maven projects at the same level referencing the same parent POM, and one of the projects says it can’t find the parent artifact, whether the other one seems to find it without problem. And when that kind of things happen, the only thing to do is to try a combination of closing all projects and reopening them, clean all projects to force a clean build, or even restart the whole Eclipse workbench. WTF?
How can SpringSource support such a poorly designed environment while admitting such unacceptable bugs? Oh yeah right! It’s free, so everybody uses it. This is really the perfect example of when Open Source can also kill innovation instead of fostering it. It’s free so everybody uses it, including corporate customers, so all tool vendors base their tools on it (Spring IDE, Flex Builder, Weblogic Portal Workshop, etc.), so even more people use it (even if they have better tools in their bag), and we’re screwed.
I would love that framework vendors focus first on command-line integration with tools like Maven and Ant, and then provide IDE integration for a few popular environments, including Eclipse, Netbeans, and my personal choice, IntelliJ IDEA. This would reinforce competition between IDE vendors instead of killing it while considerably lowering the barrier to entry to their frameworks. Right now, SpringSource is lucky I really need to understand more about dm Server, because if it had been only for cusriosity’s sake, I would have given up already just because of the tight integration with this crappy Eclipse thing and all the pain I have to make it work consistently.
So if you’ve already been in that situation, and you start to think there’s gotta be a better way, try out IntelliJ IDEA.
PS: I’m not related to Jetbrains in any way. I just happen to be a very happy customer of theirs, happy to pay a few hundred bucks every year to get their latest version, because as a Jetbrains guy said it last year at Devoxx, “IntelliJ iDEA is the only IDE worth paying for.”
