This is a follow-up on last week's article Spare a thought for contrib and the comments it attracted. The article invited the reader to reflect on the huge body of D7 Contrib modules that will have to be ported to a new core architecture in order to give D8 the same or even richer functionality as D7.
Here are, in summary, some of the comments the Drupal community made to that article:
o "…someone needs to orchestrate the whole process…" (yakoub)
o "…start now, learn and grow with the changes, document, create and follow examples, use Coder tool -- when it's ready" (Gabor Hojtsy)
o "…There will be pain, but it will be necessary to create forward compatibility." (Larry Garfield)
o "… not enough D8 work is funded … " (catch)
o "… the amount of contrib work is overstated…" (Simon)
o "… NO it isn't! " (just-passin-thru)
o "… D6 support needs to continue beyond the D8 release, while at the same time there needs to be a big push to upgrade D7 with lots of tutorials and support." (juliangb)
o "… and then there's the sites that have developed their own custom modules and themes that need to be ported too." (Jeremy French)
So while most agree that we have a problem and that someone needs to do something, there seems to be no organised drive to make this happen. "She'll be right..." is what we seem to say collectively, "… Contrib will grow itself organically, magically…all we have to do is wait…".
It's the old bystander effect isn't it? That diffusion of responsibility. When someone falls in the water in front of one person, they'll be saved. When there's 100 people standing on the river bank, they'll all watch as the victim drowns.
Well Contrib is in the river… Are we all looking on?
What are you going to do about it?
If you are a wannabe Drupal contributor, will you pick a D7 project (module or theme), port it to D8 and send it to the maintainers to help them get over that initial hurdle? Or start a chip-in or other form of crowd-funding?
If you are good at documentation, will you be helping out Gabor and friends?
If you are the Drupal Association, or Acquia or another company making profits from this wonderful free website and business framework named Drupal, what will you do?
NB: for the record, our main Drupal contributor, RdeBoer, has recently ported to D8 some of his own modules (Session Cache API, Views Contextual Range Filter), as well as other author's modules (Geofield, Leaflet, Leaflet MarkerCluster). He says that it's been a drag and that he's over doing free D8 ports when he could have spent that time on a holiday instead ...
Comments
"He says that it's been a
"He says that it's been a drag and that he's over doing free D8 ports..."
and he's a professional developer! This is what scares the heebeejeebee's outta me. I'm not a professional developer, I have a day job that doesn't allow for this kind of time, and no one is likely to assist. what's a regular old module maintainer supposed to do?
And for the record, I'm on board with the whole 'the drop is always moving' and no backward compatibility thing. But d6 to d7 was a struggle enough... d7 to d8 is likely to be a nightmare for non professionals. i can't seem to shake the feeling that drupal is flippin the bird at non-professional contributors.
Re: assistance and flippin' the bird...
Actually... lots of resources are under development
So, um, wow. This post (and the original post) makes a lot of assumptions and inferences.
The change notices are definitely not "All you get." I think you've completely misinterpreted the comment on that handbook page. What it means is "We are not going to try to create a single monster document that has all of the changes in Drupal on one single unusable page."
People (including me!) are actively working on documentation and tutorials for upgrading modules to D8. It's still three months before code freeze, so a lot is still in flux.
webchick's pants.module is a work in progress (and did you see the video by the way? http://sydney2013.drupal.org/program/core-conversations.) webchick spent hours building this tutorial, so brushing it off as a "nice joke" and implying that no one cares is pretty unfortunate IMO. In fact, the fact that such a tutorial already exists with everything it has so far ahead of when Drupal 8 will be released is pretty incredible. (And, for the record, it alread includes plugins--see the blocks branch.)
effulgentsia will be doing an improved module upgrade session at DrupalCon Portland (http://portland2013.drupal.org/session/upgrading-your-modules-drupal-8), and there's lots of other sessions there on Drupal 8 concepts, including the plugin system (http://portland2013.drupal.org/session/drupal-8-plugin-system-deep-dive). If you can't make it to Portland, all the sessions will also be recorded and available online for free after the conference, as always, thanks to the DA.
tim.plunkett, Xano, and I will also soon be building a website at http://d8cx.org/ with lots of resources for contributors who want to work on module ugprades. (See Tim's BADCamp session for more information: http://2012.badcamp.net/program/sessions/d8cx-reckoning)
In short, have a little faith in people, and watch for more resources soon! :)
Handbook pages
Some of what you stated wasn
Some of what you stated wasn't factual, though. That was my concern. ;)
This, in particular, is what
This, in particular, is what I found to be the biggest concern:
"So while most agree that we have a problem and that someone needs to do something, there seems to be no organised drive to make this happen. 'She'll be right...' is what we seem to say collectively, '… Contrib will grow itself organically, magically…all we have to do is wait…'."
A number of Drupal 8 core developers are actively working on doing what they can to make things better for porting contrib--from writing change notices and documentation, to working to simplify D8's DX, and so on. I guess your previous post gave me the impression that you wanted to suggest that D8 core developers just don't care about contrib, which is really not the case since lots of us are contrib authors ourselves. Sorry if I took it the wrong way. :)
And yeah, I'd love to discuss it further with you! Maybe you can help us out on the D8CX site, once we get things rolling in the next couple weeks.
Yes that IS the biggest concern -- but does not refer to core
financials
So in short Kev did not have any financial support to make his modules or maintain them. The only reason the Drupal 8 upgrade seems like an insurmountable task is because Kev looks at all the modules, and doing them all at once is such a big job. Right. Some options:
- many module maintainers already post chip-in or pay-for-feature/support schemes on their project pages
- all modules have issue queues with other people contributing; if Kev's modules are popular, it is likely not all of the features have even been developed by Kev; upgrades can be done by others through the issue queue (I've seen some of my modules got this for Drupal 7)
- if Kev's modules are so popular, there are likely businesses relying on it, who would have the finances to support the upgrade either in-house or pay Kev
So there are various choices including breaking up the work to smaller chunks to resemble the initial contribution bursts from Kev, publicising the need of outside help to get issue queue or financial contributions, etc.
As open source contributors, I think its easy to get into this trap thinking that we have duties towards the community, but reality is that just like the core devs, documentation writers, upgrade facilitators, etc. do a best effort work, contrib developers do a best effort work too. Kev does not have an obligation towards the community. He should definitely take his personal preference and take the family holiday over porting modules, if that is what he wishes to do. If at the same time the community is informed about the stated need of support for upgrading this module (in whatever form), if the module is truly popular, help will emerge. Yes, passing the module on is a great example, if Kev does not have the time anymore to work on it, he will definitely enjoy using the free fruit of the work from others, when his module is upgraded without a minute of work from him :)
financials, indeed
Chasm of forward compatibility
Drupal being the one-size-fits-all tool it keeps growing into, contrib will simply never keep up (save for the precious few downloads that tens to hundreds of sites use). On the train that is Drupal, the drop is the locomotive and the contrib modules/themes are the carriages behind.
The contrib and bespoke space moves because there is business in it, e.g. site building, Platform as a service (PaaS), local Drupal user groups. Take that out (i.e. rely on contrib to upgrade itself) and it's a losing formula, because the only people committed to doing good things in contrib are those supported by crowdfunding (again, precious few) and new contributors wanting to garner attention (who simply will not keep doing it for too long). The only meaningful resolution I can think of to close the "chasm of forward compatibility" is to start by making it easier for maintainers to advertise and hand over projects where funding has shifted. Community engagement will be higher if only because development will be more active.
The upcoming "graduation" of Drupal 8 into the wider PHP community will have a significant impact as well. Drupal projects can then essentially share a much wider resource pool to build great websites. Drupal 8 will be easier integrate with popular third-party services, hence much of the complex legacy stuff can just be migrated to those services such that a site will not have much complexity to upgrade. Drupal modules will be much more based on universal (non-Drupal) design patterns and wildly popular projects merged in core (Twig and Guzzle come to mind). Future "upgrade" projects will be considerably less complex. The net result for the average Drupal developer is that we're actually upgrading our skill sets into a more stable, standard, best practice oriented ecosystem, which is infinitely more sustainable than cowboy programming.
The goal is not just to narrow the "chasm of forward compatibility", but to shorten it as well. Imagine how great it will be when we no longer have to learn new PHPTemplate quirks but just stick to Twig for many Drupal versions to come.
Re: that "graduation" of Drupal 8 ...
Re: Chasm of forward compatibility
"Drupal being the one-size-fits-all tool it keeps growing into, contrib will simply never keep up"
They will always be at least a little behind by nature. If drupal core is growing, contrib has to grow with it, and just as drupal core takes time, contrib has to take time too. You have to wait for drupal 8 before you have modules that are fully compatible with drupal 8 (although if you keep your module up to date with drual 8 dev as it moves along then you will need much less time investment come drupal 8 launch - that is at the expense of the time you spend keeping it up to date with dev, so you have to work out what is best for you as a contributor/maintainer of contrib modules).
Also, that fact that more and more of the larger building blocks, like fields & views, getting into core, plus we have some really flexible contrib modules like rules (which can replace the need for a bunch of little modules that do one small thing each), less and less contrib modules are actually having to be ported (I admit though there are still a lot that could/should be ported).
"The contrib and bespoke space moves because there is business in it, e.g. site building, Platform as a service (PaaS), local Drupal user groups"
Doesn't that cover everything? The "site building" part of that says (to me at least) that the contrib space moves because of site building, which is pretty much the only use of drupal contrib modules, so this site building item covers all modules. If there are modules not used in site building, why do they need to be ported?
The drupal user base is not made up of entirely non-developer types, which means that when developer types have a client that has a site that needs upgrading they include in the price of that upgrade the cost of porting the required modules (unless they can find a new or alternative module that can achieve the same results, in which case that module can replace the need for porting the other one). I understand that a lot of clients will not want to pay for such things, but you can bet that there are companies that are willing to pay for it that are using most of the more commonly used drupal modules.
(A lot of modules are also updated by loving contributors who aren't being paid, or drupal shops who choose to allocate some of their work hours to port modules that they maintain)
Also, it doesn't even have to be paying for a full module port. Maybe there is already a half done port or an almost complete port, and you just have to spend a little time to submit patches to fix those issues.
I would generally be submitting such patches quite regularly just as I go through my day to day work (which is being paid for by clients) and I'm sure a lot of other developers are the same.
If you are a non-developer and cannot find an alternative module to replace a non-upgraded one, and you really can't afford to pay someone to upgrade it for you, then unfortunately that's tough luck and you might have to wait to upgrade until someone else ports the modules you need. That is just the way of open source software (you could always use something proprietary in place of drupal, which will give you better backwards compatibility between releases, but it will be at the expense of something else, so you have to weigh that up).
There is also the fact that as much as we might want to, we cannot cater for completely non-technical users being able to do everything relating to their drupal site without help (paid or otherwise).
If you want to be able to use drupal completely for free and to not have to wait for others to port modules for you, fix bugs, add new features, etc. you have to learn drupal so you can do it yourself.
Just like if you want to service your own car so you don't have to go through a mechanic when you want bug fixed and new features added, you have to learn car mechanics (at least parts that are relevant to what you want to do - and their underlying components).
If you want it done fast, without having to learn, pay someone to do it
The same is true for pretty much anything in life, not just drupal and car mechanics.
(non-drupal-specific note from my inner cynic: I think maybe the world's population's sense of entitlement gets a little out of hand at times :))
Of course another option is also to review whether or not that module is even needed anymore. Often if you are upgrading an old site you might want to take that opportunity to make a few changes to the site for the better, which can mean removing un-needed or unused features or replacing them with slightly different features that are available in a different module that does exist for drupal 8.
Sometimes a little creative thinking can get you around the need to port an old module.
"because the only people committed to doing good things in contrib are those supported by crowdfunding (again, precious few) and new contributors wanting to garner attention (who simply will not keep doing it for too long)"
I don't really think this is true, and some contributors might also consider it a little insulting. - It is surprising how often you see insults thrown at people who are voluntarily allocating their own spare time to help others (often it is also helping themselves, but that is not the point), regardless of how much time that may be.
I will say though that the amount of time that these developers can allocate is possibly often less than the amount of time funded contributors can allocate.
"The only meaningful resolution I can think of to close the "chasm of forward compatibility" is to start by making it easier for maintainers to advertise and hand over projects where funding has shifted."
This is not a bad idea, although there are already ways to do this, like the "Development status" and "Maintenance status" fields on the project pages, and also the ability for maintainers to use the main text of the project page to advertise such things. Some maintainers already do this.
The problem is that probably any new system for this would still have the same weak link as the current system - It is up to the maintainer to set this information.
Another problem is that often people are willing to make a patch to fix something or port a module, but they don't actually want to be a long term maintainer of that module for potentially a number of reasons. Finding long term maintainers that have enough time to keep on top of everything is hard, we need more contrib developers.
"The upcoming "graduation" of Drupal 8 into the wider PHP community will have a significant impact as well. Drupal projects can then essentially share a much wider resource pool to build great websites. "
This is true. One thing to note though is that there is still a lot of drupal specific code too, so we will have a 3 basic groups of developers (I know I could split these up more): Those that are good at drupal, those that are good at non-drupal like OO PHP, Symphony, etc., and those that are good at both. The first 2 groups will both need to invest some time to get up to speed.
"The goal is not just to narrow the "chasm of forward compatibility", but to shorten it as well. Imagine how great it will be when we no longer have to learn new PHPTemplate quirks but just stick to Twig for many Drupal versions to come."
Is that really the goal? I thought the goal was to keep adding new things (and fixing old things) to make drupal better and keep it at the forefront of CMS solutions. We don't throw out everything every new release, for example the form api is relatively unchanged for a long time, so there are always features that remain with us for a few releases. It is just that new things come and old things go, which will always happen.
Drupal 8 is a massive change that brings great new things like twig that I think will be with us for a while, however I assume drupal 9, 10 and beyond will continue to bring more new things that will affect compatibility (maybe to a different extent), and those new things will be required for drupal to continue to evolve and remain such a great CMS.
I'm glad you've taken the
I'm glad you've taken the effort to read through my comment.
"You can bet that there are companies that are willing to pay for it that are using most of the more commonly used drupal modules."
Granted, but I've seen too many sites living in legacy Drupal platforms that simply will not be upgraded or rebuilt because of the terrifying quotes that Drupal consultancies inevitably give (usually precisely because of things that need to be reviewed and upgraded as a mandatory step in the upgrade process). I've frequently wondered how Drupal might learn from enterprise Linux business models, i.e. transparent upgrade with data migration/cleanup as a separate step. Where many Linux packages do not need to be modified to be incorporated into newer versions, Drupal core upgrades by default are exclusively backwards incompatible.
"We don't throw out everything every new release."
When I say some things should be left to experts in their fields who often pioneer certain projects as part of their core business, e.g. Symfony, I absolutely do not mean that Drupal core should stagnate. Instead, Drupal can just delegate those functionalities to other projects and focus on Drupal's core strength, i.e. as a CMS/CMF (including stuff like the Form API that makes Drupal development exciting). Ultimately, then, Drupal can focus on adding more exciting stuff for Drupal, e.g. WSCCI, SPARK, Mobile, etc., that no one else is quite doing in earnest. Core maintainers are already overloaded as it is with the amount of stuff to track in core. The scale of the Drupal community should be applied to great effect, just not in re-inventing the wheel.
"It is surprising how often you see insults thrown at people who are voluntarily allocating their own spare time to help others."
As for volunteering time to help others, about 25% of my Drupal contrib work is done in my own time without pay, so I sympathize, though I suspect this is true of a lot of contributors. I'm happy that people are willing to contribute to Drupal in this fashion, but I'm also quite despaired that this is a viable means as opposed to a more mature business model with better maintenance planning than "your Drupal software will be obsolete in 2 years if you don't spend money to keep upgrading".
Examples
Here's a handy list of bite-size D7 modules that you could port to D8, in order to whet your appetite.
It's the Examples project, where each module only demonstrates one thing. So not only do you get to learn how to do things the D8 way, you help others who will be reading your code and comments.
http://drupal.org/node/1880976
Best selling books and courses in 2014
To quote the previous blog
To quote the previous blog post:
"Drupal is NOT backward compatible. If one module important to you or your client has not been ported to D8, you're stuck on D7 for your entire web site."
Yes. Drupal is not backward compatible and I don't think that will ever change, and I think people have mostly accepted that. However, that doesn't mean you're stuck on D7. Build time into your projects to port Drupal 7 modules to Drupal 8. Be a contributor and don't just wait for "somebody" to come around and do the work for you. I'd guess that a lot of the useful Drupal modules that you use every day were developed (at least originally) as a component of a project done for a client. I know I've written a couple of modules that were originally part of a client's spec.
I would like to look it from
I would like to look it from a different perspective. By porting modules to 8 before majority of people, you have got such a big competitive advantage! You can project flink as the first web company to adopt drupal 8. As commented earlier, you can even write books on how to port module to drupal 8 and sell it on site like leanpub .
You have already done the hard work . Its time to benefit from it. To make the best out of it. But, your rants are just spoiling your efforts.
Contrib Module "Etiquette"
Moving forward into D8, what I would "love" to see is an emphasis on contrib "etiquette" and what I mean by this is that many maintainers, have no foreseeable end goal. I'm not talking about D7, or D6... I've seen it all through D5 -> D7 where maintainers start a project, it blows up with popularity, then stays in dev for a year and a half with the ONLY time a stable seeing the light is from a security flaw.
Yes, I realize MANY of the contrib's aren't crowd-funded, or they are started and maintained for a small use-case scenario by a guy who works out of his house or whatever. I also realize many contribs are waiting on other contribs to go stable (see a problem?) or fixes in core.
I would expect a lackadaisical approach from someone who is doing it all himself or a small firm working for a client but even the powerhouse projects seem to go through the same release cycle mentality: "Well release a stable at some point... in the meantime, just use the dev." I don't know of ANY Enterprise businesses who's IT departments keep them on the "bleeding edge" 100% of the time...
What I'm getting at is that I would love to see an initiative for contrib maintainers to set end goals for their projects. To stop being so aggressive in adding a bazillion features between stables and instead work towards maybe getting one or two features added and a few bug fixes. Sit on it and see if the stable produces any major show stoppers. If so, fix them right away and promptly release a stable update. I've run across multiple modules over the years that have a stable release which is completely non-working but quickly fixed in the dev. You would think the maintainer would issue an updated stable with those fixes but they instead leave the broken stable out there for "newbies" who think stable really does mean stable, only to become frustrated that the contrib isn't working.
Of the "powerhouse" modules out there, the one single module that comes to mind of a maintainer who IMHO "gets it", is Nathan Haug (quicksketch) the maintainer for Webform. I have seen him work countless times towards an end goal with webform since D5 and regularly produce a stable (working) version. (Yes, I realize he is funded for this project...)
I'm not saying we need a stable version every few weeks, or even every few months, but a year and a half on a dev version?? Come-on folks...
(Disclaimer: Sorry if I've offended anyone, that's not my intention. My hope is to see a culture of "rules" or "common practices" created within the contrib community to push towards more milestones of stable releases. Unfortunately, I don't see this happening unless it is encouraged from the top.)
Looking back over the last 15 years of working with computers, I've never seen a large software project that "lives in dev" like Drupal does. Maybe somebody needs to come out with a Drupal shirt that says "Bleed the Edge or Go Home!" ;-)
I agree that the practice of
I agree that the practice of "-dev is the only stable release we have" is very problematic, even outside the D8 discussion.
This is especially true for distribution development. I've spent a nice chunk of Commerce Kickstart development creating "please create a release" issues in various issue queues.
It is also not a matter of resources, just making sure the stable release is at least as stable as what was already committed and is a part of -dev.
So this is something that deserves a blog post of its own.
commercial imperative
Yes 'be a contributor' as the previous post says. But... there is a balance between giving to the community and making a living. Most core developers make a living from clients who can afford high-grade professional devs. That is influencing the way Drupal is going, big time. If those developers depended for their income on people with a few hundred dollars to spend on a site, and almost no maintenance budget, their choices would be different. This is not to criticise the core devs, for whom I have the greatest respect.
As it is, the relatively low budget and low grade users have pretty much been abandoned (which is not wrong, they can always go to Wordpress, but it has been a fork in the road for Drupal). I cannot honestly sell a Drupal site without warning the client about the possible cost of a major version upgrade, which is usually enough to put them off. I also meet clients for whom selecting Drupal has proved a painful mistake.
Put shortly, core developers make a living from in the main from larger clients, and it is no criticism, but a reality to be faced, that their voluntary as well as their paid work serves the needs of that market magnificently, with inevitable downsides which make Drupal a poor choice for low-budget players. This trend looks set to continue.
Sorry, but this is simply not true
You said:
"Most core developers make a living from clients who can afford high-grade professional devs"
and:
"Put shortly, core developers make a living from in the main from larger clients"
One can't make this generalization. Core developers are an extremely diverse group. Myself, until a month ago, I worked in a part-time job for a small university, barely made a living wage, and simply contributed back to Drupal in my free time because it gave me the challenge and community I didn't get in my day job. Many core developers work for small shops, or don't work for Drupal shops at all. Some are stay-at-home moms or dads. Some are musicians, thespians, or students. Most of us started in our spare time, because of personal passion or interest.
"One can't make this
"One can't make this generalization. "
Good to be corrected. Even if "most" core developers do not work on enterprise-level projects (and surely the balance do?), this is the focus of Acquia, which employs some particularly influential figures. So, even with this correction, it may still be that commercial pressures and motivations (as much as anything else) are making Drupal decreasingly adapted to the needs of the small site builder, and increasingly oriented to enterprise and large community web systems.
This is not meant as a criticism but to bring clarity. Every day I answer several support requests in the forums (a community contribution which brings little recognition, BTW), for amateur site builder struggling with their first major version upgrade and a host of other things, and I go away thinking it would be better for Drupal and for its end users if there were more clarity about what it is good for and what it is not.
The contrib lag which this post is about is another massive issue for these low-end users whom I have contact with. Whatever the rights and wrongs in the discussion, reality is that contrib lag caused and still causes the types of users I am helping a lot of pain in D7, and it does not look like being better in D8. From where I stand the discussion about whether the core devs have done enough to help contrib devs is secondary: the reality of the situation hurt low-end consumers of contrib in D7 and will hurt them more in D8. Users (to whom Drupal has been 'marketed') who read the page on major version upgrade, and start following the instructions, without any real understanding of the magnitude and types of problems they may be getting into. No one is to blame, apart possibly from those users who have not researched beyond the surface and worked out that the Drupal is increasingly the wrong tool owing to maintenance and hosting demands, major version upgrade woes even where there is an upgrade path, and contrib lag where an upgrade path comes late or never.
Nattering nitwits
Now hear this: The true believer mentality that anyone owes Drupal anything (because no one in fact does), or the mentality that expecting businesses/developers to completely rewrite all of their customizations for every single Drupal upgrade is somehow ok - these are mentalities thought of by true believer pin-headed people with little pin-headed brains. And they are hardwired pin-headed little brains. No one will ever convince a pin-head of anything that is not already hardwired, and all a pin-head will show you is their contempt if you try (as can be seen from the comment thread above).
I know that the comment above
I know that the comment above is a troll, but since it sometimes pops up as an actual opinion, I'll bite.
People should remember that Drupal doesn't owe them anything either.
Use D8, or use D7, or use Wordpress, or go have ice cream. It's all good.