I had a fun chat today with Coté on his “make all” podcast. Here are some of the topics I remember discussing:
Happy listening!
Shortly before winter vacation, an IBM colleague contacted me to learn more about how extensibility – particularly web UI extensibility – works in the IBM Rational Jazz Platform on which I work. After typing up my response I thought it might be interesting to a broader audience so I’ve reposted it here (with some minor tweaks for clarity). Note that some but not all of the links below require Jazz.net registration (sorry).
—–
Hi <redacted>,
You are correct that as of Jazz Foundation 1.0.x (shipped this year), one contributes “viewlets” (a Jazz-specific widget mechanism comparable to OpenSocial Gadgets and iWidgets) to Jazz Dashboards via a server-side Eclipse extension point.
However, the Jazz extensibility story is more complex than OSGi bundles and Eclipse extension points. I will explain this below, with the assumption that you are interested.
When we first started the Jazz server work in 2005 we made a design decision that the Jazz server would be based on Java/OSGi and you contribute capabilities to the server by adding additional OSGi bundles. I’m guessing you’re very familiar with this architecture; it’s of course very similar to the Eclipse workbench except instead of contributing actions and views (e.g. to implement a Java IDE) you contributed web services and data definitions (e.g. to implement a bug tracking system). We went so far as to enable this style of programming for Dojo-based Ajax code so that developers could add “web bundles” and have the Jazz Web UI framework discover and surface their web capabilities in the Jazz Web UI. The Dashboard viewlets are one example of an extension-point-based extension. There are other extensions also enabled by extension-points (as in Eclipse, this extensibility model is open-ended).
Around 2007 we came to the conclusion that the “extend the server” model wouldn’t enable us to achieve our broad goal of “enabling integration of heterogenous tools across the software lifecycle”. There were several blocking problems with this architecture. I will list a few below:
We came to the conclusion that the RPC style service APIs were too fragile over time and would make it difficult to allow for independent upgrades of different tools in an environment containing many tools.
We could only support a narrow range of technologies (i.e. OSGi-ified Java and JavaScript code built on top of our server architecture). This limitation is readily apparent when you compare it to something like OpenSocial Gadgets or iWidget where you need only specify a gadget/widget URL to achieve integration.
This caused us to take a step back and ultimately decided to use the REST architectural style to achieve integration across tools. For instance this year we enabled several integrations between our Requirements Composer, Team Concert, and Quality Manager products using RESTful protocols and APIs. For an example of these integrations, see this video on Jazz.net that demonstrates how we use REST and Discovery to drive linking and “picker” integration between two tools.
The shift to the RESTful architecture left a gap w/r/t extensibility. I.e. how do we meet extensibility needs if we can’t assume a homogeneous OSGi/Java/JavaScript-based programming model? The solution we came up with was something that we generically call “Discovery” (an overloaded term, I know). Basically each tool in a Jazz environment has a single “root services” document that describes the basic capabilities of the tool and has pointers to other documents that contain more details about a specific service. You configure tools to know about each others’ service documents, and from there they can discover and (if appropriate) consume each others’ capabilities using spec’d protocols (as opposed to physically installing code from Tool A within Tool B’s environment).
For instance, we have a “viewlet provider” service type that announces that the tool in question has viewlets that can be embedded in Dashboards. Here’s an example of how it works:
The interesting thing about this is that Tool A and Tool B could be implemented in wildly different ways… e.g. OSGi/Java vs. PHP.
* As an aside, we are currently working to deprecate Jazz Viewlets and adding the ability to host iWidgets and OpenSocial gadgets in Jazz Dashboards. This is in plan for 2010.
Even though we have moved to a REST-centric architecture, OSGi and Extension Points are still important, but less so than they used to be. We will continue to ship and use the OSGi and Extension Point-based Ajax frameworks and building blocks for the near future, but our major development investments and the Jazz SDK‘s focus will only assume fundamental web technologies like HTTP, HTML, JavaScript, and CSS and provide optional higher-level frameworks around emerging web standards like OAuth and OpenSocial.
I hope this was useful for you.
– Bill
Between early 2006 and early this year, my team at IBM Rational and I built a framework for component-based Ajax development for use in the Rational Jazz platform. The framework’s capabilities aren’t the focus of this entry, so I’ll just list some of them briefly:
org.eclipse.core.runtime extension registry Java APIs, allowing for open-ended extensibility using the Eclipse extensibility modelWe did a fairly good job of keeping our Ajax framework decoupled from the rest of the Jazz Platform both for the purpose of design cohesiveness but also to allow for possible future use by teams who weren’t building full-blown application lifecycle management applications like Jazz SCM or Iteration Planning.
Over time, other IBM teams heard about some of our capabilities and saw the positive results in our Rational Team Concert web UIs [2] and contacted us to explore whether they could make use of our code in their products. Each conversation went like this:
Initially this didn’t really bother me – after all my job was to deliver the foundation for our Jazz web user interfaces, not to create a general purpose Ajax framework for IBM, but as I’ve thought about it more over time and seen this anti-pattern from other teams and other frameworks, I’ve decided that we should make a conscious effort to make our useful functionality available as simple building blocks (in the form of libraries) and then provide frameworks that layer on top of these building blocks.
Let me take a step back and explain what I mean.
A library is a set of reusable functionality where your application uses parts of the library as necessary. For instance, most programming languages have a library for creating and manipulating dates. Frameworks also provide functionality to the application programmer, but instead of the application programmer making simple function calls, the framework runs the show and calls into the application at well defined hook points. For example, GUI frameworks provide ways to wire up code to run in response to users clicking on buttons.
Both the library and the framework provide useful functionality intended to make the application developer more productive. Though it’s dangerous to make a general statement, it feels to me that the big difference between the two is that frameworks generally provide more power, but require you to make a bigger commitment to the framework’s way of working, while libraries provide generally less power, but make few (if any) demands about how you structure your application.
What’s unfortunate is when you’ve got some useful bit of functionality that could be made available as a simple library but it’s only available in the context of a framework. This is where we were with our Ajax framework. This stuff is too abstract, so here’s an analogy: Imagine that you heard about a new refrigerator which provided every feature and characteristic you ever dreamed about having in a refrigerator. The catch however was that you couldn’t just buy the refrigerator – you had to move into a house on the other side of town that included the new refrigerator. After about 10 seconds of thought you realize that even though it’s the fridge of your dreams, you’re sure as hell not going to move into a new house across town in order to get it. This situation (switching back to software from refrigerators) is shown in the diagram below.
Useful building blocks locked inside a framework
My recent realization (which seems obvious in hindsight) is that the useful functionality provided by frameworks and libraries need not be mutually exclusive. For instance, in our Ajax framework’s dynamic build system, rather than requiring applications to run within our framework to enjoy this capability, we could have created a simple callable library to perform dynamic optimization on a set of files, and then created a framework that simply used this same library. This approach is shown in the diagram below:
Useful building blocks used by a framework but also available independently of the framework
Over the past month or so we’ve been refactoring our Ajax framework to extract the useful building blocks into simple callable libraries and making the framework proper smaller by delegating to these libraries. We’ve done this in the hopes that our code will be useful to other IBM teams but as a result of the exercise, we’ve gained a deeper knowledge of our software and the software’s quality has improved as we’ve decoupled the framework aspects from the building blocks aspects.
Going forward, it’s my intention that our team will generally start with building blocks first and then consider if we should provide some higher-level framework that uses these building blocks. I only wish we had taken this approach from the beginning but you know, live and learn.
Footnotes
Updates
Changed “other than writing good code” to “other than writing good application code”
On Monday there were two notable changes at the Jazz.net web site I manage:
I wrote the first entry on the Jazz team blog, and it outlines the sorts of things we’re going to write about over there.
Also, I created a video that provides a tour of Jazz.net in case you’re ambivalent about registering but curious about what’s behind the curtain.
Finally, I’m guessing that I’ll probably write many, if not most, of my work-related blog entries (Jazz, Ajax, etc.) over on the Jazz Team Blog, so you may want to subscribe to that if you’re into my writing on those topics. I’m currently working on an entry over there that explains how some of the early lessons learned developing the Jazz Web UI technology led to my UI design opinions that I captured in my “Uncanny Valley of UI design” post.
As I mentioned on Twitter, as of this week I’ve got a new job of managing the Jazz.net community site. I’ll continue to lead the development on the framework-y pieces of the Jazz Platform web UI stack.
This is going to be a fun job since we’re trying something relatively new with the Jazz Project that we’re calling “Open Commercial Development“. On traditional commercial software projects, there’s a pretty high wall between the development team and everyone else. With the Jazz Platform and the products we’re building on top of it, we want to create a much higher bandwidth channel between the development team and people who use Jazz-based tools and people who build their own tools on top of the Jazz Platform.
So my job is to make it easier for the development team, users, and extenders to collaborate. This of course means good docs and good tools for collaboration, but more than anything, it means getting to know more folks who are interested in learning more about Jazz technology and influencing its direction.
So if you have questions or concerns about Jazz.net, feel free to drop me a line. My work email address is bhiggins@us.ibm.com and my cell phone number is 919-564-6862. Please don’t be shy 🙂
Update: Changed title; original one sounded too presumptuous (was “what development tools do you need besides Jazz?”)
Aaron Cohen of Rational asked an interesting question in the Jazz.net forum the other day. I’ve reprinted his question and my answer here verbatim, with permission from Aaron and the Jazz Project.
Question (Cohen):
Right now Jazz.net uses Wikis and newsgroups to supplement Jazz. If a team is deploying Jazz. What type of Wiki or other collaborative software should we use with Jazz?
Answer (Higgins):
Aaron, it really depends on the needs of your team. With Jazz, our goal is not to reinvent every known software and/or collaboration tool as a Jazz component. Basically we look for opportunities where deep integration with other Jazz components could produce a much more productive experience.
Just implementing a Wiki on top of the Jazz repository is not interesting; the world doesn’t need another Wiki implementation. But it is interesting to observe how we tend to use our own Wiki and then explore how we could provide new functionality in Jazz to replace Wikis for these needs. For example, every Jazz component lead used to create an iteration plan page in the Wiki. Observing this, the Agile Planning team provided an “Overview” page for each iteration plan, where a team can write semi-structured markup as well as linking forward to important work items. Now most component leads don’t feel the need to create a separate plan overview in the Wiki.
In the case of newsgroups, we needed a lightweight way to make announcements to a subset of the project, and newsgroups have been doing this for years so we used newsgroups. This continues to work really well, so… I don’t know why we’d ever change it.
But getting back to your question, I’d rephrase it as “What non-Jazz tools do I find my team using to support our software development?” And if you notice yourself using some non-Jazz tools for large chunks of your development, it’s worth asking “Could a Jazz-based component provide a richer, more productive, more integrated experience?” If the answer to the second question is “Yes”, then we encourage you to consider either building your own Jazz component to fulfill the need or to raise enhancement requests against us.
Thanks for your question and I hope this helps.
I’ll be in Beijing, China next week giving a couple of talks on the Jazz Platform and Rational Team Concert:
Rational Team Concert: Agile Team Development with Jazz
Inside the Jazz Technology Platform
I was in Santa Clara this last week at EclipseCon 2007. I participated, along with several other Jazz component leads, in a demo Wednesday night called “Jazz in Action: Building Jazz with Jazz”. Last year, we demo’d Jazz on a special demo server, this year we demo’d using our self-hosting server – i.e. the Jazz server we use to support our Jazz development (very meta, I know). The demo went well and the attendees seemed to enjoy it; if you wish to hear more, this CRN article describes it fairly well.
Some personal highlights from the week (ordered chronologically):
One FYI on upcoming events: Erich will be giving the “From Eclipse to Jazz” talk as a keynote at TheServerSide.com Java Symposium on Wednesday, March 21st in Las Vegas.
In the last week of the most recent Jazz milestone, I realized that I was running out of time to give one of the web features a UI overhaul. Jen Hayes, the primary web UI designer, had sent me a spec more than a month before but I hadn’t implemented it because I was overwhelmed by the many lines, measurements, and notes specifying the changes I was to make. So I moved on to other stuff and forgot about the UI work until the UI designers brought up my procrastination on a UI design review call with Erich. Doh!
Jen pointed out the most pressing problem (font style in form elements, if memory serves) and I said “Oh, I can change that” and one line of CSS later, the major problem was fixed. Hey, this was pretty easy! Jen mentioned a few other things and I realized that they were easy as well. Erich and the other folks dropped off the call and we decided to start a NetMeeting and try to knock out as many UI polish items we could. Two hours later we were completely done – not only had we fulfilled the spirit of the spec, we’d rethought a few spec’d decisions and tweaked a few additional items in the process of our extremely rapid iterations.
A few days later, I reflected on this and wondered why the heck we had the UI designers sending us super-detailed specs in the first place. If someone were to send me a highly-detailed technical spec on how I should code a particular feature, I’d ask them if I were being Punk’d. Yet with regards to UI design, this is standard practice. Not anymore, at least for me. The UI designers are welcome to draw up whatever specs they like, but I’m only going to do major UI work via pair programming with a UI designer.
Pair programming with a UI designer has the same benefits as pair programming something in [fill in your programming language of choice]. You can experiment, discuss, learn, and refine very quickly, and the real-time verbal and visual communication is much higher bandwidth than email and documents. This is especially true in a web environment, where you can hack CSS literally in real-time via EditCss and a screen sharing program.
This experience and another recent email conversation with James Governor on my reticence to publish a new Ajax article “before it is completely ready” has made me decide to reflect on how I can apply the principles of experiment/discuss/learn/refine to everything I do.
The other day, I thought of a clever solution to speed up an important Jazz function by perhaps 20-40% (depending on a set of environmental factors). I eagerly documented the solution in a bug report and CC’d several teammates. Several times over the past week, I almost went ahead and just coded the solution, because I was so happy with its cleverness.
But I haven’t done this.
Fortunately for users but unfortunately for my idea, the function in question already runs as fast (1-4 seconds) as users expect it to run, given the nature of the function. So although the 20-40% improvement is certainly relatively significant, it isn’t of any practical significance to users. After all, who cares if something that currently runs in a ‘speedy’ 2 seconds were reduced to 1.5 seconds?
So it’s been with great discipline (and a bit of angst) that I’ve abstained from writing this clever code, basically because of the YAGNI principle, which the team as a whole rigorously observes. The benefits (0.2 – 1.6 second reduced response time) simply don’t justify these costs:
The reason I bring this up is because it demonstrates the powerful temptation to implement a clever or elegant solution despite the fact that it lacks a compelling problem. I consider YAGNI to be a fundamental principle of my personal development philosophy, yet I still found myself tempted to implement the solution once I had fallen in love with its cleverness and efficiency.
So … I’ll be patient for now and my solution will remain unimplemented, but documented in a bug report. If, over time, other system factors cause the response time to degrade, perhaps the solution’s value may increase enough to justify its costs.
Or not.
