My fellow writer placed the blame for this squarely on Adobe. They purchased all of the proper assets to make real change (FrameMaker, PDF, Quark, Illustrator, Photoshop), but left most of it to languish. FrameMaker has not substantially changed in the 10 years I've been doing documentation. In fact, the only real change from 1995 that I can see is that FrameMaker is now a Windows only product. PDF has also stagnated.
All of these tools allow you to improve the look of the output by messing around with the CSS files or adding your own processing logic to them. For example, there is no reason I couldn't make the DITA OT generate a fully collapsable floating TOC that is rendered using CSS beyond my skill with XSLT and my time. Therein lies the rub. Most people working in technical documentation today do not have the skills or the time to do major customization work beyond what the tools provide natively. Some tools make it hard, some just don't provide any help. Even when the customization methods are easyish and well documented there is little time.
While the tool vendors share some of the blame for the stagnation, it is the professional writers and the users of documentation that share most of the blame. We have not demanded better interfaces to work with documentation. We have accepted that it must live with the subpar UIs we've suffered with for years. Don't we really just want print of HTML served up Google style? Or is it that we have been conned into believing that is the best we can get?
Lately the world of electronic books has been showing us that we can expect more. iBooks, the Kindle apps, and a slew of other eReading platforms have recently appeared on the market that have shown that reading electronic content does not need to be boring or painful. I just read a beautiful version of Alice in Wonderland on my iPad. The typography was excellent, the page layout was top notch, the interface was intuitive, I could look up words in the dictionary without too much disruption. It was a dream. I've seen comic books on the iPad as well and they too work very well. In most cases they are just trying to replicate the feel of a printed publication, but some of the elements go beyond that. New eBook platforms allow for embedded audio and visual content as well as text.
When it comes to help systems, a different UI paradigm is needed from the book paradigm. The current bland HTML page with a generic search feature cannot be the best we can do. One thing we can do is break the TOC model. It provides a false sense of "bookishness" to a help system that is essentially non-linear. The search features can use some bulking up as well. Maybe adding features that allow users to save searches. How about letting user's build their own TOC based on their use patterns or save breadcrumb trails. More connection between the help system and the UI it documents would be good as well. Some help systems already have the ability to open dialogs and trigger actions in the associated products, but typically this feature is hard for a content developer to trigger.
Don't we owe it to our users to demand more and better usage paradigms. Ones that make the task of finding and consuming information easier and are more than just functional?