Project History

A high-level timeline of the project showing that 3 or the 5 phases are complete. Phase 1 Formation was completed in Fall 2020, Phase 2 Evaluation in Spring 2021, and Phase 3 Selection in Summer 2021. We are currently in Phase 4 Implementation which will last from Fall 2021 and Spring 2022. The final launch phase will take place in Summer and Fall 2022.

In choosing an LMS, the committee prioritized several key pathways of gathering feedback. User testing was at the heart of this process, providing students, faculty, and staff multiple opportunities to evaluate the LMS platforms. User data-collection took place over two main stages:

  1. Interviews and Surveys: Determined faculty, staff, and students’ satisfaction with the current LMS and desired features for a future product through in-depth interviews and a widely distributed survey.
  2. Evaluating LMSs: Faculty, staff, and students evaluated potential LMSs and their features. Users tried the LMS test environments hands-on and watched vendor-created videos of the most used features.

Teaching and Learning Centers promoted these opportunities to engage in data collection through customized communication plans. For example, they used mass emails, sent targeted invitations, promoted at division events, presented to departments, coordinated messages from division leadership, hosted live demos and office hours, orchestrated test cases, and more.

Technical Evaluation

The process also included a technical evaluation of the systems. IT@JH and Student Disability Services completed the foundational evaluations to ensure that the vendors could comply with the University’s policies and basic needs. The teaching and learning center staff from each division evaluated the possible LMS platforms. Using their technical knowledge and experience, they judged division-specific requirements as well as rated each vendor on the Request for Proposal requirements. They attended vendor demonstrations, completed in-depth, hands-on testing in the LMSs, and facilitated user testing.

Conclusion

Our current LMS, self-hosted Blackboard Learn, will no longer be supported by the vendor. This system retirement allowed us to explore modern, cloud-based LMSs that could better meet the University’s needs for an excellent learning platform. After a robust evaluation process, faculty, staff, students, and the Teaching and Learning Centers rated Instructure Canvas as the best system.

Evaluation Process Details

TaskTimeDescription
LMS Requirements DevelopmentFall 2018 – Sumer 2019As the end of our current Blackboard Learn contract neared, the Instructional Technology Advisory Council (ITAC) summarized and prioritized fifty requirements provided by the divisions for an LMS. With the rise of the COVID-19 pandemic, the project was put on hold.
Blackboard Product Retirement AnnouncedJuly 2020Blackboard announced the end of support for self-hosted Blackboard Learn. They would require migration to their cloud-based services, Blackboard Software as a Service (SaaS), and ultimately a move to Blackboard Ultra, their new product.
Release of Request for ProposalDecember 2020After evaluation of business needs, JHU released an RFP.
Vendors’ Proposals SubmittedJanuary 2021Five vendors responded with proposals.
Round 1 Vendor ReviewJanuary 2021Vendor proposals were evaluated by teaching and learning center staff and IT@JH to see if they met the RFP requirements. Three of the five vendors were selected for testing and further evaluation.
LMS Steering Committee FormedFebruary 2021Sponsored by Stephen Gange, the steering committee includes faculty, students, IT@JH, representatives from each division’s teaching and learning center, and other leadership.
Vendor PresentationsFebruary – April 2021The LMS vendors offered a series of presentations demoing their product and particular features of interest to divisions. They also hosted office hours for teaching and learning staff questions.
Stage 1 of User EvaluationFebruary – March 2021Thousands of faculty, staff, and students completed surveys and in-depth interviews, providing data about their satisfaction with the current LMS and what they would want out of a future one.
IT@JH and Student Disability Services TestingMarch – May 2021IT@JH completed testing of the LMSs to see if the product would meet the University’s needs. The evaluation included security, data, system administration, and more. Student Disability Services also reviewed the LMSs for accessibility considering students with disabilities.
Stage 2 User and TLC (Teaching and Learning Center) EvaluationApril – May 2021Faculty, staff, and students provided feedback after completing hands-on testing of the potential LMSs and watching demos. Teaching and learning centers completed in-depth testing of the LMSs, rating them on the RFP requirements and divisional needs.
LMS Committee RecommendationJune 2021The committee reviewed the aggregated data from faculty, staff, students, teaching and learning centers, IT@JH, SDS. The evaluation overwhelmingly favored adopting Instructure Canvas.
LMS Evaluation Process Timeline