Introduction

Does the concept of “learning objects” translate easily to today’s business environment? Learning objects are the core underlying principle on which learning content management systems (LCMS) are founded. We’ve been monitoring the proliferation of enterprise learning constructed from a wide variety of source material, such as legacy documents, graphics, streaming media, PowerPoint, etc., added to a central database and then widely distributed as either structured e-learning or as a free-form, company-wide knowledge base used for just-in-time learning – or a combination of both. We’ve seen some dramatic success stories from companies like Cisco, IBM, and Toyota University, who understand and appropriately apply learning content management strategies to maximize their e-learning development processes. We’ve seen organizations large and small use LCMS technologies to successfully engage and leverage subject matter experts and novice developers as primary contributors to large-scale projects designed to collect and disseminate knowledge throughout the organization. We’ve watched companies that have completely transformed their costly in-house development practice into a well-organized, e-learning publishing machine.

However, we’ve simultaneously witnessed some dismal failures where learning content management technology has been completely misused or misunderstood; where technology designed to make modular learning content easily reusable has created a source of mass information overload, in which learning objects (topics) are linked together in sequences that don’t make sense or provide misinformation when presented out of context, serving to confuse rather than educate.

What’s the difference between the success stories and the failures? What role does the technology play in creating the problems? In some ways, the technology has outpaced our ability to use it wisely. LCMS vendors continue to add innovative new feature sets, when most of us are still grappling with some of the basic concepts of learning object design.

We believe that when LCMS technology is appropriately applied and matched to an orchestrated e-learning strategy, with a complete instructional design plan for designing and using learning objects, great efficiencies can and will be achieved, such as (1) rapid and productive content development efforts, (2) seamless collaboration among subject matter experts and course designers, (3) the ability to make instantaneous, company-wide changes to critical learning content, (4) the ability to create multiple, derivative versions of content applicable to different audiences from senior management to line-level workers, (5) access to find and reuse learning content just-in-time and just enough, and (6) ultimate reusability of content by making it available through a wide array of output types such as structured e-learning courses, CD-ROM courses, learning material available from a Palm device or PocketPC, print-based learning for use in classroom settings, etc.

In this light, we examine some of the best enterprise-level applications on the market that use learning object models for the purpose of creating, storing, reusing, managing and delivering learning content from a central object repository to learners throughout the organization. Indeed, the purpose of this research is to assist you in understanding the potential of learning content management systems and to help you identify commercially available systems that match your specific business needs, technological infrastructure, and most importantly, your instructional needs.

Research Methodology

The LCMS market continues to mature since our previous studies of learning object technologies. Much of the research contained in this KnowledgeBase is an extension of our previous research with an eye toward broadening the categories through further evaluation of end-user needs by assessing many requests for proposal (RFP) from customers looking to use learning object technologies.

For this research project, we initially created a comprehensive assessment questionnaire, searched out and selected candidate systems to study, collected information, performed live evaluations of each system, and summarized and assessed the collected data.

The first phase of the process was to develop an extensive tool to evaluate the products, even prior to assembling a list of systems for inclusion in the report. We gleaned information from dozens of RFPs, talked to companies already using LCMS technology, spent significant amounts of time talking to prospective customers of LCMS products, and participated in panel discussions at major trade shows with members of LCMS companies and potential end users. We also examined several needs analysis documents prepared by companies that were considering building their own learning content management systems; these particular companies were unable to identify commercially available products that met their needs. By studying these resources, we created categories of common needs and documented many unfulfilled needs.

As a result of this initial research, we took the information and converted it into our assessment tool, a 32-page questionnaire. In total, we originally identified over 70 LCMS products, and we sent research invitations to 38 companies for full reviews. Decisions about which systems to cover were made based on the categories contained in our assessment tool (questionnaire).

Here is a list of the characteristics we used to identify candidate systems:

• Content assembly and/or rapid content development tools
• Ability to accept content from a wide variety of sources (Word, PowerPoint, Flash, Dreamweaver, PDF, raw media files, other forms of legacy content, etc.)
• A well-defined schema for storing, retrieving, managing and clustering learning objects (most often database-centric)
• Maximum reusability of learning content in a variety of ways, including (1) finding individual media objects for use in other courses, (2) the ability to reuse entire topics in other courses, (3) the ability to use clusters of topics (modules, units) in other courses, (4) the ability to reuse an entire course, make changes and only store the differences, (5) the ability to swap out language elements without duplicating content, and (6) the ability to reuse source material for a wide variety of output types including e-learning, CD-ROM, print-based learning, Palm/Pocket PC, EPSS, to created knowledge bases, etc.
• Interoperability with LMS products
• Dynamic pre-testing capabilities and ability to create adaptive learning based on pretest performance
• Utilities for managing the workflow process of e-learning development, such as role-based systems, task assignments, project management tools, collaboration tools for content developers, built-in bug tracking, etc.
• Ability to do advanced archiving, file management, and revision controls
• Automatically wrap navigational controls, styles, look-and-feel, etc., around learning object-based content. (Note: we tried to completely stay away from systems where content developers have to manually create these things.)
• Strictly follow SCORM (Sharable Content Object Reference Model) specifications for creating and organizing content

A majority of those invited to participate submitted information, while a few decided not to participate for various reasons.

We developed a consistent method and process for testing and evaluating. Then, over a period of several weeks, we spent time with many of the systems, validating the information they sent. More importantly, we performed a complete walk-through with key systems using a pre-created set of legacy content objects to see how each system would handle standard content (content that was mentioned most often by consumers and potential users of LCMS technology).

Our content bundle included the following items:

Equipped with these resource materials, we approached each test by performing the steps as follows:

• Log-in to the system and create a new content developer role, setting appropriate access using on-board security tools. Spend time validating and reviewing the unique workflow tools available in the system.
• Create a new course and provide metadata tags for the course.
• Add legacy content directly into the content repository and tag content for later searchability and reusability.
• Create a page, search for our Flash title animation and add to page. Use on-board tools to edit, resize and position content.
• Bring in Word content and check options for converting content on-the-fly or through other manual processes.
• Bring in PowerPoint, checking for content conversion options.
• Add Dreamweaver to the content repository (if possible). Otherwise, try other methods for attaching this complex, multi-file interaction.
• Use on-board authoring tools to create similar content from scratch.
• Create a series of test questions using on-board assessment wizards and/or editors (using our source material).
• Create a dynamic pretest associating specific questions with specific pages of content to test the adaptive learning capabilities of the system.
• Reuse content from another course in a new course. Observe options given to us by the system; specifically, does the system allow us to link, copy, or (more importantly) both.
• Check for methods for storing complete documents such as PDF files for use as reference material or as part of a just-in-time learning system.
• Publish content as a structured e-learning course.
• Try to publish in different formats, specifically print-based output, just-in-time help, Palm/PocketPC, and/or offline format.
• Explore learner output by assessing extent of additional functionality, such as built-in collaborative tools, note-taking, bookmarking, skins, etc.
• Change look and feel and styles within course and re-publish. Assess ease and extensibility of changing the learner interface.
• Explore any unique features found in the system

We have tried very hard to keep the comparative information contained in this report as objective as possible. We do not have a specific view of the exact technology and set of features that should be found in every LCMS. Rather, we acknowledge and appreciate the uniqueness of each of these systems. Our primary objective is to provide people with research data we have collected and analyzed to aid them in their own selection process.

It is important to note that we do not feel the market has reached a point of commercialization – where every product is virtually the same. In fact, as we reviewed the products we found examples of unique innovation in nearly every LCMS. Our hope is that the comparative information and product reviews will help expand your vision of innovative learning object technologies.

How to Use the KnowledgeBase

You should consider this KnowledgeBase as a toolkit with modular components that can be used in your evaluation and selection process. For example, if you are brand new to the area of “learning objects” and/or learning content management, you will most likely want to spend some time reading Sections I through IV. They will help you get up to speed on the benefits of using an LCMS and also give you checklists of things to look for when assessing systems. The section entitled “Key Definitions: Compare and Contrast” will be most helpful if you are getting bogged down in the complex terminology associated with this new field.

Here is a high-level description of what each section in the KnowledgeBase contains:

Section I – Introduction

Section II – Why Use an LCMS?

LCMS technology isn’t for everyone. For simpler e-learning strategies, it would be considered overkill. In this section, you’ll learn why LCMS products were created in the first place, what problems they solve, and the conditions that make LCMS a technology to consider. We’ve included some tips we learned from reading many RFPs from companies large and small who have considered acquiring an LCMS. You’ll learn the top ten features sought by these companies and also be provided with a brief summary of recent trends observed in the adoption of these systems.

Section III – Key Definitions: Compare and Contrast

As you read LCMS advertising literature and listen to the vendors, you’ll often hear terminology used interchangeably. For example, some of the vendors call themselves “content management systems” rather than “learning content management systems.” However, there is already a well-established group of content management system providers that have very little to do with e-learning applications. Tools such as Documentum and Vignette, which are from the content management space, provide organizational infrastructure for all types of content and digital assets which can be found in a company. It is helpful to know the difference in philosophy and functionality between traditional content management systems and learning content management systems. Still other vendors refer to their systems as enterprise-level authoring tools, which leads to a confusing comparison between an LCMS and tools like Authorware and ToolBook. This section will provide a definition for each of the common terms often confused with (or sometimes associated with) learning content management systems.

Section IV – Comparing the Data: Meta-Analysis of the LCMS Industry

Assessing the features of an LCMS can be a bit complex at times. These enterprise-level applications literally have hundreds of features to consider. This section includes information on how to streamline the selection process by focusing on and prioritizing only critical functionality. It shows you how to create a short list based on your critical needs. Then, you can continue your search based on your short list. We’ve tried to make this short list process easier for you by including an “At-A-Glance” chart at the beginning of each product review. You can also compare a short list of tools or to use the LCMS Selection Tool, allowing the database to filter out systems that meet your specific needs.

Section IV is also a breakdown of the information these summary At-A-Glance charts contain. This meta-analysis data can show you averages, highs/lows, and other helpful information to give you added context about how LCMS products compare overall. This section will prepare you to quickly rule out systems that won’t meet your needs.

Section V – The LCMS Profiles

Section V contains detailed profiles on each system we included in our evaluation. These profiles are approximately 20 pages in length and include information on the specific functionality available in each system, in areas such as: content authoring capabilities, assessment, back-office interoperability, object reusability, pre-testing, adaptive learning, skill-gap analysis, workflow management, archiving, learner interface, multilingual capabilities, standards compliance, output formats, implementation times, systems requirements, and comparative pricing information for different implementation scenarios.

If you already know which systems you’d like to evaluate, we recommend going directly to this section and exploring the profiles; comparing candidates that you may consider for further due diligence.

Section VI – LCMS Comparison Tool

Section VI is where you will find the LCMS Comparison Tool. You simply select the systems you’d like to compare, and the KnowledgeBase will dynamically create a comparative grid, showing a side-by-side functional comparison of each selected LCMS.

Section VII – LCMS Selection Tool

Section VII provides an LCMS Selection Tool that will even further assist you in narrowing down a short list of candidates based on your requirements. For example, say you want to find an LCMS with (1) built-in pre-testing tools to prescribe specific learning content, (2) interoperability with Plateau (a commercially available LMS), (3) with multi-byte support to serve up content in Chinese, and (4) a system that has already passed SCORM certification. The LCMS Selection Tool will find the system for you. All you need to do is select the criteria, and the tool will scan the database and filter out only the systems that meet your specifications. The list will also link you directly to the 20-page system profile for further review.

Many of our readers with existing knowledge of LCMS functionality like to start in this section and work backwards – systematically adding critical functionality until the LCMS Selection Tool narrows down the list. Then, they significantly reduce the time spent performing due diligence on the systems by eliminating systems that are missing key components.

Please note that marking dozens of requirement filters simultaneously may result in a list with zero entries. We recommend starting with your most critical needs and layering in new requirements slowly so that you can see which systems drop off the list as new specifications are added.

Appendix

The Appendix provides a glossary of terms that will help you cut through online learning jargon and also contains a summary list of contact information for each product listed in the research. This will help you find the right contact without having to search for the information in each individual profile.