Thursday, July 31, 2008

Walking away from bugs

One of the most frustrating things that can happen to an open source committer is to meet someone who tried your software, ran into some problem, and just walked away. Did you check the error log? Did you ask on the newsgroup/mailing list/forum? Did you file a bug? As a committer, these questions roll off your tongue. After all, open source is a collaborative effort, and success depends in large part to the collective effort of the community. Walking away like that is just not good citizenship!

Or is it?

Recently I needed to evaluate some plug-in code that was written on a Ganymede-based target platform. For reasons not important here, my development environment is based on Europa at the moment. Eclipse has great support for these cases, however, and setting up various target platforms is routine for me. So, I go and set up the necessary target platform, load the evaluation code into my workspace, and set up a launch configuration. Right – so let's run these plug-ins and see how they work. The splash screen appears and shortly after – bang! It dies with an error* Strange. I've been developing open and closed source plug-ins for Eclipse since V1, and have dealt with all sorts of platform configurations and error conditions: never saw this one before... Mess around with the launch configuration, read config.ini (like the error message suggests); nothing helps. Ok – off to an internet and Bugzilla search for the issue: clearly I must be doing something stupid, because Europa should be able to launch Ganymede-based target platforms (right?). Hmmm... some discussion about this type of error, but nothing seems to fit my case. Try launching the target platform directly, and it works fine. A few more minutes of searching, a few more minutes of messing around with settings. Finally, I walked away.

And then I thought about it. Why did I walk away, and what does that tell me about open source users who do the same? Well, in my case, and I suspect this is common, the reason was pretty simple: my task at hand was to evaluate the plug-in code, not to debug an Eclipse launch failure. I was trying to use Eclipse to work on another problem, not on Eclipse itself. I could get the information I needed in another fashion, and that's what I did. Now, being an open source committer, I felt a little guilty: surely I'm not walking away, but rather I'll get back to that problem. I'll do The Right Thing and check all possible avenues: I'll read everything I can about it on the net, I'll ask on the newsgroup, I'll file a bug and follow up. Yes, yes I will! I will get back to it, right after I get these other things (that I'm paid for) done... Then again, maybe not (to be realistic).

I even felt guilty about writing this blog entry. Who I am to complain when I haven't done my homework? But then I realized that something larger was interesting: my attitude toward the whole situation. Maybe a kind reader will point out an answer for my problem, one which will probably seem entirely obvious, if I had looked in the right place. If so, thanks! But fixing this specific issue is really not the point. Maybe you'll flame me: You don't understand how open source works, you've got to engage the community! To this I'll answer: You don't understand your users! How many of them have walked away silently?

When I was in graduate school, a friend of mine was doing his Ph.D. In the physics department at MIT. Maybe it's a measure of what it takes to operate at that elite level, but he once told me (without pride or arrogance, but simply matter of fact) that he was never shy about asking a question in class, because if he didn't understand something, it was the professor's fault for not being clear.

Just to be clear (because I don't mind being flamed, but prefer to be flamed for what I actually meant): I'm not blaming the Eclipse platform or any other Eclipse project/committer for the error I encountered or my inability to fix it quickly. Rather I am just pointing out the observation that I feel compelled to do a lot of work before raising an issue, and I wonder if this “barrier to the question” is the reason why people walk away. If so, what can be done?

*java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

19 comments:

trisa said...

interesting points and in detail analysis! I observed myself many times when I just walked away and I would have liked to give some effort in return to what's being offered. The main reason for that behaviour was that there's pressure on time and money when you're working in a project off-house and they're not willing to pay your sense of community. The only thing that counts in this situation is the 'return of investement' That's a real pitty and it's almost impossible to teach managers a sense of community. Community only pays back in the long term and project management think in short terms (companies sometimes not, though.

Adrian Taylor said...

Hi, I know it's not the point, but I've seen the same problem and I too walked away. If you find a fix please post a blog entry!

Tiran Kenja said...

Personally I am close to stop reporting issues on some projects. As some project teams seem like they couldn't care less about what the users think, and will just ignore our input.

For me I need to feel someone cares that I report stuff. I have been in a host of beta tests of commercial software (and games). And pretty consistently I loose the interest of helping the developers if they cannot show appreciation for what I do to try and help them create a better product.

Michael Neale said...

Good post. I feel similarly, although in my case how much I am prepared to dig in depends on:

1) What mood I am in
2) How desparate I am for the feature
3) How much of a hurry I am in
4) How much I know about the area it is in

(in that order). Yep, its mostly what mood I am in most of the time !

Anonymous said...

the sad thing about eclipse it seems project is focusing on new cool features but does not have time to fix the bugs, its getting slower and slower with each release and even having decent PC I have to disable some problematic features so I could work more or less comfortably. It is getting so frustrating that I am thinking to go to less feature rich but more stable NetBeans.
Also some plugins that has good ideas have so many bugs it is not possible to use without loosing nerves. Mylyn for instance. What a crap.
I could understand filling a bug if it is something annusual. But if a bug is so clear and happens all the time it should be fixed by developer before release in the first place. I am using Eclipse for many years but I am getting fed ut with it. The thing is we do not have much choice when it comes to IDE do we?
Eclipse, Netbeans, InteliJ Idea and that's it !
Sorry for my English.

kiwi said...

I think it would be useful to be allowed to file a bug report having done a brief google for the bug and a solution and having not found one. As a developer I'd prefer to get multiple copies of bugs rather than no notion there was anything wrong and have people walk away. But then that would require a little more patience in some members of the open source community. Also, with multiple copies of bug reports, each person's use of language to explain the problem is slightly different, and may show up the next time someone does a search using the same terms, whereby they can be redirected to the original bug.

Anthony Rabaa said...

What's poorly lacking in the IT industry is good debug/feedback frameworks that can be easily incorporated into applications. Yes, all sorts of applications have some little gadget that will catch an error and ask to file it with some black-hole website, but that's about it.

When an exception occurs, the application should display a meaningful error and immediately provide hints on rectifying the problem. It should also track how many times the user has encountered that error and once a threshold is reached, provide an option to open a dialog with the development team.

I don't think a user should submit a report and then be forced to sit there wondering how long it may take before the next release of the application is produced. Instead, the user should immediately be provided with information about what was submitted (ex: a bug number which the user can track on the bugzilla page, etc...). With that, the user will feel apart of the community and truly helping not only to resolve his or her own issue but that which many more people may stumble upon.

And if it turns out the issue has already been resolved by another bug, the act of a developer reading his or her report and associating it with the existing resolved bug will go a long way to saving the user time and money. In turn companies would promote their employees providing such feedback knowing there may be a cost benefit to that action in the future.

Sorry for the long comment, dealing with mundane and misleading error messages that have little or nothing to do with the task at hand but absorb much of my precious time has become a pet peeve of mine. Cheers!

Anonymous said...

why i walk away:


i almost alway 'walk away' from bugs, regardless of whether from open source or a vendor

i regard a bug in released software as a major failure; don't care if it's 'community' developed or whatever. fine if software is described as release candidate or beta, but if described as 'stable' or is a product offering from a vendor, then i expect no bugs

now, this is not about very minor, subtle, edge-case bugs, but i am talking about when a major feature just doesnt work, or when when the application crashes, etc. and sadly this happens way too much.

my attitude is that if it doesn't just work, out of the box, then it's crap

i've been developing software a very long time. bugs are almost always due to haste. developers need to stand up to management pressure in commercial companies, and need to learn to focus on quality in open source, and not be in a hurry for their changes to be committed (committers, too, need to slow down)

btw, i agree with one of the comments re: eclipse getting too bloated

Peter Arrenbrecht said...

I used to be more hesitant but I now am rather quick to ask on IRC. Even at the risk of looking stupid. I think it's still less stupid than wasting time when someone else who is online and willing has the answer. Much like your friend's attitude in class. I pay back by helping others on IRC quite a bit too. But, again, only for the things where I know an answer right away. (And sometimes with trickier issues when I need a break anyway.)

I'm more hesitant on the mailing lists, but also less so that at the start. Here, though, I feel some qualms because it's a larger audience with a less clear commitment to listen to quick questions.

So I think a key element could be educating potential users about IRC and how to ask there. For instance, right within error messages.

The "how to" above is about basic rules, like in our project: just ask; be polite; wait for at least 15 mins for a response before giving up an logging out; use the pastebin.

I had been a professional software developer using some open source for about 15 years before I realized how valuable IRC is. I only got into it once I started contributing in earnest. Such a shame. I now think this should be something anyone looking at an open source project should be pointed to right away.

Maybe, though, this view is too biased by the healthy way IRC seems to work on projects like Eclipse and Mercurial. It takes a sufficiently large body of monitors sufficiently spread out in time zones to work, I guess.

Diogo Oliveira said...

My position is just like tiran said. I like to help, but in order to help the community you need some feedback. Sometimes I like the idea of some open source project and try to dig into it and help with known issues and try to find and solve new ones, but I quickly get tired if the people involved don't demonstrate how useful my input could be, or at least some gratitude words for my effort.
After all, we all have some other projects that we're paid for, and that by itself at least makes us feel we spend our effort for something.

Paul Bakker said...

I recognize this too. I also walked away many times, simply because because it costs too much time (for the project) and is too much of a distraction of the work I'm supposed to do.

I do make an exception however for projects that I really use a lot and that I'm sure the dev team takes bugs serious. Examples of this are Spring and Seam.

Guus Bonnnema said...

Hi John,

For each and every piece of software there are a few types of users:

1. Use, as in one of a choice, but work-a-rounds around.
2. Use, and difficult to replace or maybe just your favorite

Now look at the bigger picture. Every software has each category filled to a degree. Depending on the number of users in each category, you will get more active support as a developer by the community or less.

The other way around, for each piece of software you use, you fit into one of the certain category. If you are just using the software, but have many workarounds at your disposal, why dive into faulty software, and be punished in the time you need to produce what does count? Ofcourse, you walk away and use the other software that does work.

AFAICS this is a basic proces that ultimately decides whether a piece will survive: if no one is interested in it, it will die and rightly so.

You are in no way obliged to support the developer, and neither is the developer obliged to support you (even though (s)he may be more motivated to do so).

I think that walking away signals the importance of the software to you.

Carlos Elizondo said...

Wouldn't it be great if you could earn gold points for finding a bug in open source code and you could use those points to download software or documentation of licensed products that started as open source.

Paul Murray said...

Weird! How on earth could anyone suppose that it's their personal job to fix every little thing that's wrong with everybody else's code? Would you apply it in any other sphere of life? If your car was dripping oil all over the road, and I pointed it out, would it be fair - or even reasonable - for you to reply "Oh yeah? And why haven't you fixed it, then?"

Life it too short, the world is too big, and my boss wants me to make these feature enhancements by tomorrow.

Would I walk away if I downloaded a plugin, and it just flat out did not work at all? Hell yeah.

Denis Roy said...

Paul Murray, your analogy makes absolutely no sense in the Open Source world. It should read as such:

If you were a car builder, and you gave cars out for free, and I told you, 'thanks for the free car, but it has a flaw: it's leaking oil'.

Would it be reasonable for you to say, "Oh yeah? I'm sure you, and millions of other people are getting great benefits out of the free cars. If you don't mind, can you help me by trying to fix the problem? It would benefit us all!"

Sounds more than reasonable to me, because as we speak, Arthur is working on the leaky radiator and Jill is improving the seats, and Antoinette is putting in an improved stereo.

RafaelChaves said...

> Europa should be able to launch
> Ganymede-based target platforms
> (right?)

WRONG!

Eclipse aims to provide backward compatibility, and what you want is forward compatibility.

How can PDE in Europa know how to launch an application based on a future version of Eclipse?

But you certainly should be able to do the opposite, i.e. using Ganymede as your development environment, have a target based on Europa (or older).

John Graham said...

@RafaelChaves

We're talking about configuration primarily here, not simply API, for which I could easily see a backward/forward compatibility argument. Eclipse let's us define launch configurations and doesn't "know" about the n strange RCP/product configurations that I have working right now. But somehow this doesn't work from Europa --> Ganymede. I'd accept having to change the launch configuration -- that's why it is there. Just saying it is strange that others have found this problem, and no solution, other than your claim that it shouldn't work.

Anonymous said...

Your blog keeps getting better and better! Your older articles are not as good as newer ones you have a lot more creativity and originality now keep it up!

Anonymous said...

i'm new... expectancy to register nearly more time after time!