Brandon Hall Research

    Sign up for "Brandon Hall News" and get free research downloads. Your privacy is assured.
    Your Email:

    Home > Publications > LMS KnowledgeBase 2010 > Methodology

LMS KnowledgeBase 2005: In-Depth Profiles of 52 Learning Management Systems

 

Why this Service?

Since Brandon Hall Research's first "Learning Management Systems" report, published in 1997, things have changed dramatically. Over 65 percent of organizations now have a learning management system (LMS) in place, with many of them utilizing more than one LMS. In fact, this year's study represents 17,738 total LMS implementations (end-user companies that have implemented an LMS), with a collective grand total of 83.8 million registered e-learners — across all systems covered in this study — suggesting that things definitely have changed!

Learning management systems are now regularly considered mission-critical to the long-term success of most organizations, extending beyond the learning domain to include the convergence of learning, performance management, staffing, certification, providing just-in-time education to customers, providing instant access to learning content through 24/7 learning portals, and ultimately measuring the effects of all types of learning to help meet organizational goals and objectives.

With all that said, the fundamental question still exists: "So, what is the best LMS?" And, which one should we use in our organization? It's a question that we've continually been asked since we began evaluating LMS solutions. In our earlier reports, we made attempts to "short list" some of the best LMS products based on the maturity and range of functionality offered in each system. While these lists did assist somewhat in showing which systems were ahead of the LMS evolutionary curve, the lists themselves offered only limited help for the purpose of choosing a system that most closely matches an organization's specific critical needs and business requirements.

The purpose of this online LMS service is to provide an answer to this question for you. Just don't expect to see a pre-defined list of "The Best LMS Solutions" here. The selection and benchmarking process really does hinge on what you are trying to accomplish: your specific needs; your organization's limitations (e.g., infrastructure requirements, requirements for back-office integration, etc.); to some degree, your own subjectivity; even more esoteric issues such as cost vs. functionality. We understand that you face a multitude of budgetary and business requirement issues. This Web site is designed to put you in control of your own search, assisting you in determining your needs, then using the extensive database of LMS information to pinpoint your search.

The information is also valuable for those who are studying the dynamics of the learning management system market. In addition to extensive research data on LMS core and advanced functionality, the LMS research database also includes additional information about the types of users that choose each system, the typical size of end-user companies, pricing averages for different scenarios, which vertical markets are most likely to use an LMS, etc. This will help you benchmark your own needs against existing LMS users.

We welcome you to the largest LMS research project ever published, and we sincerely hope you enjoy your experience.


How We Selected Systems for Inclusion in this Study

Unlike some companies providing buying information about learning management systems, Brandon Hall Research does not charge vendors to be included in our research. (We never have. We never will.) Consequently, our LMS research covers the largest number of systems (63 and counting) and provides you with independent, objective information you can trust.

Any LMS vendor can participate in this research. To be included in this or any other Brandon Hall Research KnowledgeBase, please contact us.


How We Collected Baseline Information on Target Systems

The Brandon Hall Research research team began the process by evaluating dozens of request for proposals (RFP) from recent LMS selection projects. We dissected each RFP to assess the most frequently requested features and functionality. This information was used to create a massive RFI (Request For Information) with 372 questions. This questionnaire was designed to collect information that could be used to provide an apples-to-apples comparison across multiple LMS systems. You have full access to this data through this LMS service, and you can view the data in multiple views, depending on your needs.

The information collected was fed directly into our master database. The core of the LMS online service is based on this collected data, forming the foundation for our reviews, the individual product profiles, the section on benchmarking of the LMS industry, the comparative grid tool, and the LMS selection tool. The data was designed to help you quickly create and narrow down a list of systems that will meet your needs.


How We Reviewed Each System

In addition to collecting comparative information from vendors, the Brandon Hall Research team spent time with each system, performing a set of common LMS user tasks and then assessing how the system compares to other systems. The written review from the Brandon Hall Research team can be found in each LMS profile; it's called the "critique." As part of the critique, the reviewer wrote his or her "best use recommendation" for the system. For example, one system might fit the bill for an organization looking for a low-cost system with built-in LCMS functionality; yet another system would work much better for a sophisticated interoperability and data migration project.


How we went about providing benchmarking data for all LMS systems

Using the core data items in the LMS research database, we were able to look across all 50 systems simultaneously to observe trends and assess the direction of the LMS space, as well as provide information about which functionalities you would expect to find in a typical LMS. This information is included in the "Benchmarking" section of this service.

Through the benchmarking process, we discovered:

  • What types of organizations are using LMS products
  • Where LMS users are geographically located
  • What vertical industries have the largest number of LMS users
  • From the LMS vendor point-of-view, which systems do they run up against (as competition) most often
  • Average size and staffing of LMS vendor organizations
  • Total number of LMS implementations (worldwide)
  • Total number of registered e-learners (worldwide)
  • Largest hosted implementations
  • Largest locally installed, behind-the-firewall implementations
  • Amount of money spent on LMS research and development (by vendor)
  • Servers and databases most typically supported
  • Most frequent ERP and other back-office integrations
  • Depth of ERP integration in typical LMS implementations (these were fascinating results)
  • Third-party content that is most often interoperable with LMS solutions
  • Authoring tools that are most often interoperable with LMS solutions
  • LCMS solutions that are most often interoperable with LMS solutions
  • CMS solutions that are most often interoperable with LMS solutions
  • Virtual classroom/live e-learning applications that are most often interoperable with LMS solutions
  • Percent of systems that have the following functionalities:
    • Classroom management
    • Competency and performance management
    • Built-in content development tools
    • Built-in content management tools
    • Average number of report templates
    • Analytics (calculate ROI for training)
    • Multiple domain support
    • AICC certified/compliant
    • SCORM certified/conformant
    • Section 508-compliant
    • Built-in collaboration tools
    • Languages supported
    • Hosted vs. locally installed solutions
    • Built-in e-commerce
  • Average implementation time for an LMS (also the range)
  • Average pricing for both hosted and locally installed solutions for the following scenarios:
    • 500 learners
    • 10,000 learners
    • 25,000 learners
    • 100,000 learners

Summary and Purpose of the Methodology

Our purpose in creating this service is not intended to endorse any specific LMS solution; rather, to provide you with a toolkit and resources to identify systems that will meet your needs and to give you a "jump start" in your due diligence process. We do not accept money from vendors to be included in the research. Vendors are selected based on end-user feedback.

The ultimate purpose of this online service is to help you answer questions for yourself: "Which LMS is best?" And, ultimately, "Which one should we use in our organization?

 
© 2010 Brandon Hall Research