Please give us back our stats

I’m a little disappointed with aspects of drupal.org at the moment.

After the relaunch under the “new" D7 architecture, project pages show download and install counts frozen on their values of 20 October 2013, while projects released since then have no counts or graphs at all.

Project stats are important. As mentioned here, project stats not only register a module’s growth, they also help make it grow. Are you likely to try out a module that after 2 months shows (incorrectly) that nobody’s interested in it? Probably not. Will the author of the module therefore receive much useful feedback that helps them improve the module at this critical stage of its early development? Nope. Which then in return would have made that module more useful to a larger, more appreciative audience. Which would have resulted in even more participation, more feedback and a better quality module for all to enjoy?

None of that momentum build-up is happening for hundreds of modules created after 20 October. Spanking new, they already look like efforts that never caught on. It’s especially disheartening for newbie developers who after, no doubt weeks of work, have just released their first modules on drupal.org — let’s hope that not getting any feedback will not make them throw in the towel.

Some commenters have already suggested to temporarily switch off stats altogether so as to give all module maintainers the same unfair disadvantage.

But stats are only one example from this aftermath of regressions and UX issues we, members of drupal.org, currently live with.

I salute all the hard-working individuals behind what was and still appears to be a mammoth effort.
It’s a big site. It’s a big deal.

To the majority of us, drupal.org is the prime communication tool and community fostering vehicle. It’s where it all happens. drupal.org could potentially send a great message to the world about Drupal as an enviable platform. It could be the poster child of what is possible with Drupal, in particular in terms of the UX for its 1,000,000 members.

It could be. What are your thoughts?

File under: 

Planet Drupal

Comments

Submitted by George DeMet (not verified) on Wed, 11/12/2013 - 4:05am

This post does a great job of outlining the problem and why people should care about it, but doesn't have a clear call to action. Here are my questions:

1. What are the blockers to getting this issue fixed?
2. What have you done to try to help?
3. What can others in the community do to help?

Submitted by admin on Wed, 11/12/2013 - 6:25am

The suggestion that many bugs still aren't fixed 6 weeks after launch is due to the fact that I or others not involved in the project should have put more effort in it is a cop-out I don’t buy.

As a thought experiment imagine for one second that drupal.org was commissioned commercially or non-commercially by some big third party. 6 weeks after launch they still find the issues we’re talking about: https://drupal.org/project/issues/d7qa?text=&status=All&priorities=All&c...

Now imagine the D.A's response to their client being those 3 lines you wrote, i.e. “What have you do done to try to help?"
Would that go down well with the client?

Submitted by Cameron Eagans ... (not verified) on Wed, 11/12/2013 - 7:05am

Unfortunately, you don't seem to realize that the bulk of the work on Drupal.org is done by volunteers. It's not a commercial venture. If it's a big problem for you, then spend time or money to get it fixed. If it's not that critical, then you'll have to deal with it until somebody can get to it. Right now, there are more pressing matters, like this one: https://drupal.org/node/2125269, as well as these: https://drupal.org/project/issues/search/d7qa?status%5B%5D=1&status%5B%5...

In short, writing a blog post and whining about how things aren't working doesn't fix anything. Chip in, or wait it out.

Submitted by admin on Wed, 11/12/2013 - 9:20am

Thank you for your feedback. At least you realise that that statistics issue was meant as an introductory symptom of something bigger. This piece is not a whinge, it is constructive criticism to make us think or perhaps rethink aspects of the relaunch of drupal.org.

For the record I work 32 paid hours per week on Drupal and roughly 20 hours/week unpaid as a volunteer on various Drupal things, modules, issue queues both my own and other people’s. I’d love to help out more still, but there’s only so many hours in the day...

I dismiss the statement that because work is done by volunteers, it must be accepted for what it is.

Rather than getting emotional about volunteer work let’s have a look at this.

Drupal pitches itself all the way up to the enterprise level. With some great successes. drupal.org serving 1,000,000 members has itself become an enterprise level operation. It requires enterprise level approaches, methodologies and tools. drupal.org may well have employed those — I'd love to learn more.

But the fact is that months before the delayed release of D8, drupal.org the “face” of Drupal, decides to eat its own dog food (as it should) and gets ported from D6 to D7 and relaunched. 6 weeks later we have a long issue queue about regressions and UX issues.

If that’s an enterprise outcome we all hoped for then fine.

If it is not then maybe we should have a look at our "dog food", the dollars, the methods, the hours etc. A proper post-launch evaluation.

Volunteers made Drupal huge. But I wonder whether volunteering alone are enough to keep it huge. I would love to see more money flow into drupal.org and towards contributors. I donate. But I believe that for a long-term sustainable model that works for the community and at the enterprise level we need more than volunteers and Gittip. Personally I feel TSM is the community's best bet so far. Have a read.

Submitted by Phil (not verified) on Wed, 11/12/2013 - 7:18am

In which case the client would respond, 'hey, I gave you all those bags of money, remember?' The point is, in your thought experiment the client has made a considerable contribution to the project, and has a clear incentive to work with the vendor to sort the problem out.

Here's a thought experiment for you: You build a site for a client. They are a NFP, so you do it at cost. The site has a bug. The client finds the bug. Instead of coming to you and saying, 'the site you built for me has a bug, what can be done to fix it?' , they publish an article in the Sydney Morning Herald telling the story of what a poor job Flink did. Surely you would prefer if they had approached you directly?

Yes there are still bugs on d.o , and I'm pretty sure no one is happy about them. I'm also pretty sure no one is blaming you for not fixing them. They're just suggesting that the next logical step is: what can be done to help fix this?

Submitted by admin on Wed, 11/12/2013 - 9:26am

The real issue is how these bugs could have been avoided in the first place. Bugs are not an act of God that we cannot prevent... Was any testing done prior to going live? As far as telling the newspaper is concerned. Some embittered person could send in an article to the newspaper titled "Famous CMS relaunches using own technology, fails". Oh but wait... they don't have to. All the info is online on drupal.org for the world to see already!

Submitted by George DeMet (not verified) on Wed, 11/12/2013 - 7:41am

I'm still not sure what the call to action is here. Yes, there are a variety of outstanding issues with drupal.org in the wake of the D7 migration, and they are being actively worked on by both staff members and community volunteers, including this one.

I understand that you're unhappy that this issue hasn't been resolved yet, but I'm unclear what your suggested course of action is.

Submitted by YesCT (not verified) on Wed, 11/12/2013 - 7:31am

I think https://drupal.org/node/2128619 "Project usage/download/install statistics not being generated in a timely manner" is part of the work toward a solution.

Long term:
Drupal.org Improvements Group https://groups.drupal.org/drupalorg
Specifics on how decisions will be made https://groups.drupal.org/node/368148
DA prioritizing d.o in the budget https://association.drupal.org/node/18893
There is also work to make it easier for changes to d.o, providing a trimmed sanitized version of d.o for people to work with...

It's definitely on peoples' minds that we need to make changes to d.o easier to do and easier to ensure are not regressions, both for staff paid to work on d.o and for people that want to volunteer and fix things that are bothering them.

Submitted by admin on Wed, 11/12/2013 - 6:00am

This is far from solved. See the link mentioned in the article above: https://drupal.org/comment/8255729#comment-8255729

1) While the graph (once you get to it) at drupal.org/project/usage/SOMEMODULE shows near up-to-date grand totals there are no figures for point releases of SOMEMODULE since 20 Oct.

2) The grand total downloads and install counts on the main project page are frozen on their 20 Oct values

3) For modules released after 20 Oct there are NO stats on the project page and NO graph

The last one I find the most disheartening for the reasons mentioned in the article.

Submitted by kreynen (not verified) on Wed, 11/12/2013 - 9:25am

While usage numbers are important, https://drupal.org/node/2132659 was just closed yesterday. As far as priorities, would it really matter if the usage numbers are correct if developers can't create new projects or package new releases?

From the project packaging perspective, the Drupal.org D7 update was a disaster costing the community hundreds of hours of developer time by both people trying and failing to update their contrib projects and the people trying to work through/around the issues. I'd really like see the D.A. to post a summary retrospective about this project. Why did the D.A. go ahead with the D7 upgrade when there were so many issues? Why did these issues take so long to resolve after the upgrade? Is @dww really the only person on the planet who knows how the project module and packaging works?

The Software Working Group had a call earlier today where they discussed hiring/appointing people to fill some key roles in managing Drupal.org's issue queues https://groups.drupal.org/node/368148

If the project leads are hired/appointed based on their contributions to the current "eating our own dog food" approach, Drupal.org will likely continue with the status quo of trying to reinvent the wheel instead of putting effort into moving to something like Github, Gitlab, or Bitbucket.

In https://association.drupal.org/node/18858, Holly Ross indicated a decision about moving some of the less CMS related functions off of Drupal.org would be made by the Software Working Group and the new Association CTO. The people who become those project leads are going to play a critical role in the sustainability of Drupal.org moving forward.

It's really too late for a call to action about the stats specifically. @drumm has already identified the root issue (disk space/Mongo DB). This issue will be fixed and the status updated in a few days.

Drawing attention to a specific issues like this can raise the priority, but everyone needs to understand that it's just shifting the attention of the same handful of people who are working on dozens of other issues.

If there's going to be a call to action, it really should be about ensuring that the way Drupal.org is managed is sustainable without burning out volunteers or bankrupting the D.A.

One way to do that is going to be to actually pay these project leads well and make a real effort to find people who have had success outside of our community.

Submitted by jthorson (not verified) on Wed, 11/12/2013 - 2:28pm

I just got home after wrapping up a 9.5 hour workday at $dayjob, and, like *so many* nights of late, once again sat down at the computer intending to dig into either the D7QA or Testbot issue queues.

As always, I started with a review of Drupal Planet, and came across this blog post.

Frankly, posts like this one are extremely detrimental to the upgrade effort. Personally, my level of 'let's fix drupal.org' motivation dropped to absolute ZERO after reading it. I volunteer my time because I enjoy enabling others to do great things, and have put in hundreds (if not thousands) of hours for the community over the last four years ... enough that my family calls it my 'second job'. To be honest, as second jobs go, I've never known it to be more thankless than during the last six weeks.

Are you sure you want to choose download statistics as the flag to rally the troops around? The main root cause was identified on November 11th, in comment #4 of the thread, 6 days after it was first reported. It was re-iterated in comment #15, 6 days later. All it needed was a volunteer to port forward an obsolete database query ... and following the link to the referenced issue, oh! There's a revised query that was 90% of the way to the solution, also posted November 11th!

I guess that wasn't enough ... none of those *hundreds of developers* who are missing their stats stepped forward to write the actual patch. Instead, it was left to those "that were involved with the project" to fix things, and "to suggest otherwise is a copout". You're disappointed with how the drupal.org upgrade was handled? To be frank, I'm pretty disappointed with how the *community* has handled itself ... Drupal has always been a do-ocracy; not a "do-it-for-me!"-ocracy; and the level of expectation that the community is placing on a tiny handful of volunteers after this upgrade defies logic.

If you want to draw parallels to an enterprise site upgrade, I can tell you exactly where this one fell apart: The end customer (i.e. the community) skipped out on their portion of the responsiblity. When it came time to comment, evaluate, trial, test, and QA the release candidate site, the upgrade team heard ... crickets.

So yeah, things didn't go smoothly. No upgrade of this magnitude ever goes 100%, especially one performed primarily by community volunteers. We can't expect the same level of stability out of a 6-week old site as we had in our 6-year old site ... and frankly, I'd suggest we get used to it. Since day one of this upgrade, the goal was to offload our technical debt and deliver a site which would make it much easier for folks to hack on and develop new functionality. We're now in a spot where we can finally start to realize the benefits of that work ... but a more dynamic drupal.org (feature-wise) means that things are sometimes going to break, and occasionally in a spectacular fashion.

So let's try to stop dwelling on your "aftermath of regressions", step back, and consider both i) *why* we did this, and ii) what we are gaining over the long run in exchange for this short-term pain. And next time you grow frustrated with drupal.org, instead of a blog post, write a patch. Sure, I know you think that's a cop-out, and you dismiss the statement that "because work is done by volunteers, it must be accepted for what it is"; but the fact is that volunteers is all you've got right now. Try not to drive them away.

In other words, to quote my 3-year-old daughter ... "You get what you get, so don't get upset." And no amount of wishful thinking about 'enterprise' upgrades is going to change that.

Submitted by admin on Wed, 11/12/2013 - 10:05pm

First of all, jthorson, I’m so, so sorry that you feel this bad. I can relate to you as I also spend a substantial number of hours voluntarily on Drupal every week (although patches for drupal.org are not my strength). I too cop some less than grateful reactions that make us wonder why we're going through the trouble.

I hope that on reflection, you agree that I wasn’t attacking you or any other volunteer. You and the team deserve so much better, In fact, if drupal.org indeed relies on a too-small team of volunteers working overtime to the point of burn-out, then there is something really wrong, in terms of project management, systems & processes. And with how we appreciate what it takes to maintain a site the size of drupal.org.
As you say "the level of expectation that the community is placing on a tiny handful of volunteers after this upgrade defies logic” — indeed.

drupal.org is essential oil for the community as well as an advertising vehicle for Drupal as an enviable platform preferable over the competition.

I made a simple bird’s eye observation just like any outsider or industry observer or potential client with $$$ to spare could make. And what they’ve seen is that “Drupal” relaunched their world-facing drupal.org flagship as a site looking very much the same, but with a list of issues that after 6 weeks of operation has not got much shorter.

Those observers who also look at, say Adobe, have every right to raise an eyebrow and ask the odd question. Is Drupal’s own dog food bad medicine?

After my anecdotal opening about the stats issue, I stopped short of stating the question bluntly. I wanted to see if anyone sensed what was the real issue. kreynen and YesCT certainly did. kreynen who seems to be a lot closer to the coal face than I am also poignantly pointed out that my stats issue is in fact one of the SMALLER issues the team was dealing with.

I am totally with kreynen in looking forward to the D.A publishing a summary retrospective.
He then continues to hit the nail on the head:

"If there's going to be a call to action, it really should be about ensuring that the way Drupal.org is managed is sustainable without burning out volunteers or bankrupting the D.A.”

Long term, quality for a web site that 1) serves a 1,000,000 membership base and 2) competes with the likes of Adobe is, in my opinion, simply not possible without injecting some cash for starters via some funding model You, jthorson and colleagues deserve it more than anyone.

Submitted by jthorson (not verified) on Thu, 12/12/2013 - 2:30am

When faced with a large problem, there are always some who choose to yell and scream, some who choose to withdraw and despair, and some who choose to hunker down, turn to face it, and try to tackle it head-on. We need less of the first two types, and more of the third.

"... “Drupal” relaunched their world-facing drupal.org flagship as a site looking very much the same, but with a list of issues that after 6 weeks of operation has not got much shorter."

These types of comments are the root cause of my frustration ... not only do they sensationalize and overstate the true status of the situation, they also trivialize the near heroic efforts of the dedicated volunteers who have been on the metaphorical front lines, battling it out in the trenches day in and day out for the last six weeks.

Reality? The D7QA queue critical issue count is currently sitting at 3. Only one of those is a real 'bug', and even that label is debatable, as it's really a scalability issue with views pushing our db beyond it's limit. The other two are a feature request and a one-time data corruption issue. Go back 5 weeks, and that critical list was over a page and a half long ... to suggest that the list of issues is not getting shorter is doing an incredible dis-service to those actually doing the work.

As for the call to action, it's pretty clear from the DA's 2014 Leadership plan that they have recognized this issue and are taking immediate steps to rectify it; by staffing up a full team that is 100% dedicated to drupal.org. Unfortunately, those complaining the loudest are either ignoring this fact, or spinning it as 'bankrupting the DA' ... one the one hand, they criticizing the DA's attempt to add to the dedicated resource pool, and on the other, they demand a retrospective audit which is simply going to require even more time from the already overextended resources that are already there.

Is it really fair to draw parallels between the DA and Adobe; comparing a 3 person technical team to a company of over 10k employees? Seems to me that there are some significant fundamental attribution errors in play here, and perhaps just a bit of self-serving bias ...

Submitted by admin on Thu, 12/12/2013 - 7:16am

Drupal is competing on a world stage with world standards to match whether you like it or not.
The fact that we have only a 3 person technical team is precisely the problem -- not a justification.
Don't shoot the messenger.

Submitted by Jeff (not verified) on Wed, 11/12/2013 - 8:14pm

Stats are dependent on the update status module which should not be used on a production site and in fact is auto turned off periodically by many managed Drupal hosts, including Omega8.cc (https://omega8.cc/modules-enabled-or-disabled-automatically-117) because of performance issues - the update status module will slow down your site during a cron run especially if you have lots of modules enabled.

From the Drupal stats page: "These statistics are incomplete; only Drupal websites using the Update Status module are included in the data. "

So the stats are very misleading to begin with. There should be a lightweight core function that does this without all the problems attendant with the update status module.

Submitted by Ian Thomas (not verified) on Thu, 12/12/2013 - 12:37am

I think it all boils down to trying to do too much, with too little. I'm sure there was pressure to ship since the upgrade had been dragging on for so long, so perhaps that lead to too little testing being done?

Thankfully the DA do seem to be working on the remaining the fundamental problems:
- Too little employee time: They are working to strengthen the drupal.org team
- Too little volunteer time: They are working to make it easier for people to contribute
- Too much to do: There have been noises that indicate they are considering using a different issue tracker, possibly GitHub

As a community we realise that there are great CMSes so it's a waste of time to roll our own CMS. Well, there a loads of great bug trackers too, so why do we spend so much time maintaining our own? Migrating to a dedicated bug tracker won't be easy, as we would still need to decide how to configure it, migrate our data into it and integrate it with our other systems, but once that was done we'd have a better tool with a lower maintenance cost.

Submitted by ianthomas_uk (not verified) on Thu, 12/12/2013 - 8:52am

I'm arguing that just as I as a web developer leave the creation of my CMS to the experts (i.e. the Drupal community or competitors), we as the Drupal community should leave the creation of our bug tracker to the experts (whoever they may be).

Submitted by Dio (not verified) on Thu, 12/12/2013 - 12:42am

This is a good blog and a good discussion. It was a botched project because maybe too few are doing too little.

That said, my current impression Drupal is certainly not one of an open source community. The project applcation review process I endured was one of the most demotivating exercises I have undertaken in 30 years of writing code.

The Drupal website needs a major overhaul. A lot of this does not involve code changes, only how we use what is available. If Drupal wants to be a great open source project, it should be more, uh, OPEN.

Submitted by George DeMet (not verified) on Thu, 12/12/2013 - 1:59am

For those who aren't aware of what's being done to address the ongoing governance and process issues with Drupal.org, here's a brief (and probably incomplete) list:

- Earlier this year, the Drupal Association chartered three community working groups to provide governance and act as product owners for anything relating to infrastructure, features, and content on Drupal.org: https://drupal.org/governance/drupalorg-working-groups

- The Association has been adding new staff specifically to work on issues relating to the Drupal.org family of sites.

- The Association is currently in the process of hiring a CTO, who will be responsible for developing, managing, and coordinating the implementation of a strategic plan and roadmap for Drupal.org: https://association.drupal.org/node/18778

- The 2014 Leadership Plan and Budget is focused on providing sufficient budget and resources to address issues with Drupal.org and to make it an awesome place for both the developer community and for outside evaluators to learn more about the project: https://association.drupal.org/node/18893

- The Bluecheese theme is being opened up to enable more people to work on it and to contribute improvements back to Drupal.org: https://association.drupal.org/node/18878

Change is not going to happen overnight, but improving Drupal.org for everyone is now a central focus of the DA, and significant resources are being devoted to that effort. We still need help from the community though; even if you can't contribute code or review patches, supporting the DA is a great way to make sure that issues like this happen less often in the future.

Submitted by kreynen (not verified) on Thu, 12/12/2013 - 4:50am

"Frankly, posts like this one are extremely detrimental to the upgrade effort."

I couldn't disagree more. Insisting there is nothing to see here and that users should just carry on is a lost opportunity to do more of what @jthorson started doing... pointing out that if users don't like the current state of Drupal.org, they can do X, Y, and Z to contribute to changing it.

The problem is very few people know what X, Y, and Z are. If you think it was clear before the D7 upgrade or is clearer now, you are wrong. I've been involved in distribution packaging since it was deployed on Drupal.org and I had no idea what I should have been testing or how to test it to make sure the D7 update didn't have this effect.

I never saw a "Holy shit! We're X days away from updating Drupal.org to D7 and no one has tested X, Y, or Z post".

At no time in the 6 weeks while [META] Problems with Project Release Packaging was active did anyone make specific suggestions on how the people posting there could contribute.

Everyone has a breaking point for both getting involved and giving up.

Tweets of the Acquia's lunch time dance party video (http://www.youtube.com/watch?v=-w8WroFmwLg) was what drove me to get involved. While I was trying and failing to update a distribution, other developers were dancing?!? Seeing this as a Nero moment, I could have just said fuck it and moved the distributions to GitHub. Instead I broke involved and started monitoring the queues, consolidating issues, and trying to get a handle on what was actually wrong.

My breaking point for no longer being involved in trying to find solutions came after the 7.24 update. It had been more than 30 days since the D7 upgrade and while many of the module and theme level issues had been resolved, distribution maintainers were still struggling to publish releases. Users were repeatedly asking me why the security updates hadn't been made to distributions I'm involved in and my response had been that it would be fixed soon.

Additional life issues (our first baby!!) added to my stress level and my breaking point in dealing with distribution users looking to me for answers. I did something similar to this blog post. I posted messages to the distribution project pages and urged users to post to [META] Problems with Project Release Packaging.

When those users started posting comments like https://drupal.org/comment/8245767#comment-8245767...

"I just don't understand why more people haven't been working on this to solve it faster. It seems to be a universal problem that impacts the entire Drupal community."

Instead helping users understand what they could do to help, @jthorson just told them they weren't helpful.

Frustration is a form of passion. IMHO, the users who don't post anything and just sit on the sidelines waiting for someone else to both identify the problems and provide the solutions are far worse than someone who takes the time to post their frustration.

Ideally the community would have the bandwidth and tools to channel that into productive contributions, but we clearly don't.

Submitted by kreynen (not verified) on Thu, 12/12/2013 - 8:34am

Nice. I'll see your Men Without Hats and raise you a Billy Joel...

"We didn't start the fire,
It was always burning
Since the world's been turning.
We didn't start the fire,
No we didn't light it
But we tried to fight it."