ComputerWorld Honors Award Case Study

Summary

Long Summary

As educational institutions establish an online presence, initial successes are often due to individual faculty members ("early adopters" of this new technology), working long hours to develop material more or less single-handedly. Frequently, they are leaving behind scattered projects, which are of intrinsic value, but of little use for the institution and far less for the larger academic community. “Late adopters” of technology in education might altogether refuse to venture into creating new online educational resources, since the task of creating comprehensive material appears overwhelming in isolation. To address these problems, a team of faculty and staff from Michigan State University (MSU) are creating an infrastructure to provide a course management system (CMS) and resource sharing: the LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA.)

The roots of this system go back ten years, when a team of faculty members and professionals at Michigan State University developed a sophisticated online homework and assessment system, with a strong focus on the sciences and mathematics. Soon other universities adopted the system, and it was not long before an informal culture of inter-institutional sharing of such resources developed. To formalize and thus further these efforts, the team added digital library and learning content management capabilities, and the ability for instructors to assemble these resources.

Today (Fall 2002), LON-CAPA has become a distributed Learning Content Management and Assessment System. LON-CAPA and its predecessor systems are serving a total of 12,000 students per semester at MSU alone, and well over 23,000 students per semester system-wide, ranging from middle school to graduate level courses. Disciplines include astronomy, biology, business, chemistry, civil engineering, computer science, family and child ecology, geology, human food and nutrition, human medicine, mathematics, medical technology, physics, and psychology.

Grants from the Sloan Foundation and from the Mellon Foundation, as the well as the National Science Foundation, and strong support from Michigan State University have encouraged us to pursue the development of this enabling technology. The current leadership team is truly cross-disciplinary: Dr. Wolfgang Bauer and Dr. Edwin Kashy (physics), Dr. Cheryl Speier (business), Dr. Deborah Kashy (psychology), as well as educational psychology graduate student Helen Keefe as project manager, computer scientist Guy Albertelli as technical director, and Dr. Gerd Kortemeyer (science and mathematics education) as project director.

Over the coming three years, with continued support by the National Science Foundation Information Technology Research program (NSF-ITR #0085921), we plan to transform this system further beyond the boundaries of MSU's campus into a dynamic online collaborative community of faculty authors, commercial publishers, and learners. The LON-CAPA project currently has 18 pilot user institutions, and involvement from four major publishing companies and RedHat software. Efforts to couple LON-CAPA with the NSF National Science Digital Library (NSDL) project are under way.

We believe that quickly scaling up this effort while accommodating a diverse user community is crucial to reach a critical mass of educational content, which could transform this network into a nationally used pool of online instructional resources. LON-CAPA provides the infrastructure so that researchers can collect data about online teaching and learning, as well as market dynamics in an online educational economy. We plan to eventually develop this network into the independent “LON-CAPA Academic Alliance,” which remains driven by faculty and is part of the academic community, yet at the same time involves commercial partners and contributors.

Benefits

Right from its very beginning ten years ago, the CAPA system (“Computer-Assisted Personalized Approach”) provided personalized problems: while the core of a given problem remains the same, different students get different numbers, graphs, options, etc. The idea was that students can and should work together, but not simply exchange the answers. “LON-CAPA also allows me to assign a greater variety of problem types, including interactive problems, or problems with a visual component (video, etc.),” says Dr. Lars Jensen, who teaches physics at Truckee Meadows Community College in Reno, Nevada, “- all individualized, so that no two students get the same problem. These features help me challenge the students in ways not possible with traditional homework problems.”

“LON-CAPA comes with a proven most effective and powerful system for readily creating, delivering and grading automated, individualized, interactive, homework and learning resources of many types via the internet,” says Dr. Ray Batchelor, who teaches chemistry at Simon Fraser University in Vancouver, British Columbia. “With LON-CAPA,” continues Dr. Batchelor, “in a couple of hours, I can assemble all the homework resources for an entire course, ready for immediate student use. This includes: appropriately enrolling all students and instructors, assigning the dates and times at which resources are available and homework due, and invoking the necessary marking/grading schemes, as well as many other optionally adjustable parameters.”

“For the instructor, LON-CAPA is a time saver. It grades my homework assignments that I normally spend hours on each week,” says Dr. Jensen, “[and] the students benefit from LON-CAPA by getting immediate feedback on their homework. This allows them to go back and re-work the set and ultimately get a better understanding of the topic, as well as a chance to improve their score.”

But not only the students reap benefits from the immediate feedback, as Dr. Batchelor confirms: “LON-CAPA has a wide and ever expanding range of tools for monitoring and analyzing learner progress, and for interacting (asynchronously) with them both individually and collectively.”

Dr. Dennis Houk from Westshore Community College in Scottsville, Michigan, uses LON-CAPA in his classroom: “LON-CAPA gives me the ability to assign 10 problems during a regular physics class session and I can then spend time with students that have trouble solving problems.”

Time saved in routine grading of student work, as well as better insights into the learning progress, both foster communication between learners and educators – delegating work that a computer does best to a computer allows humans to interact on a higher level, even in rather unexpected ways, as Dr. Cornelius Bennhold from George Washington University observes: "The addition of CAPA to the instructor-student relationship removes the adversarial relationship between me and my students with regard to turning in late homework, partial credit and immediate feedback. Rather, the student and I are on the same side, struggling valiantly with the immovable CAPA beast that seemingly comes up with these tough homework problems out of nowhere."

Writing this kind of quality highly interactive educational content is time consuming, often beyond the capacity of a single faculty author. “This project is most likely to build effectiveness due to the sheer intensity of the collaborative effort,” says Thomas Russell, author of 1999 paper "The No-Significant Difference Phenomenon,” which summarizes 355 research reports and papers in which no significant differences in student learning outcomes were shown when comparing technology-assisted instruction with standard classroom instruction. With LON-CAPA, material can be shared across the whole network of servers, improving instructional outcomes over traditional course development where faculty members work in isolation on their course materials.

Over the years, the project has sponsored annual conferences and workshops. At the 2002 LON-CAPA user conference, hosted by Florida State University, 56 faculty members -- mostly science researchers – from 22 institutions attended. LON-CAPA provides a catalyst for them to discuss educational issues, for which they would not usually have a forum.

Importance

“We are at a time when commercial and private use of information technology threatens to both undermine and overwhelm the educational process,” says Dr. Batchelor, “It is of vital importance for educators to be able to effectively use IT to deliver course content and resources, in a controlled way, to their students. LON-CAPA brings powerful IT features and resources to the fingertips of the average user, without compromising their intellectual independence.”

While LON-CAPA was never intended as a competitor to commercial course management systems, we expect that today open-source systems, including LON-CAPA, will drive – in fact, force – the market for learning content and course management systems toward greater innovation and openness, much as Linux had a significant impact on the operating system market. “LON-CAPA is an outstanding, full-featured course management system for campus courses and distance education courses alike,” says Dr. Jensen, “I know of no other systems of this type that is as versatile as LON-CAPA. I have used it for 3 semesters and I am enthusiastic about it. As an instructor, it meets my needs and it challenges me to think about better ways to deliver my instruction.”

LON-CAPA offers an inherently academic approach to online teaching and learning. Its sharing of resources is formed around the principle of peer-review (through peer-adoption of resources and giving credit to the author), and will soon further formalize this concept for explicitly peer-reviewed subsets of the resource pool. Through its ability to be molded and modified, as well as its extensive logging of learner interaction, it allows instructors to conduct educational experiments on a large scale, and to track the educational impact of targeted interventions.

The LON-CAPA project was also recognized by IEEE and ASEE with three best papers awards at the annual Frontiers in Education Conference (1997, 1998, 2000), the Ben Dasher Award (1998), and the Wickenden Award (1999).

Originality

Cross-institutional resource sharing is at the heart of LON-CAPA’s system design: “The unique resource sharing component of LON-CAPA is extremely valuable for instructors,” says Dr. Jenson: “It allows instructors to easily find the best available resources for the topic they're teaching, and to share their own materials for others to use.” Dr. Burks Oakley II, Associate Vive President for Academic Affairs at the University of Illinois, agrees: “[You] really need to emphasize the LON-CAPA advantage: Identical servers on multiple campuses, content in same format, good tools to assemble a course out of learning objects.”

Learning objects could be simple paragraphs of text, movies, applets, homework problems, etc. In addition to providing a distributed digital library with mechanisms to store and catalog these resources, LON-CAPA enables faculty to combine and sequence these resources at several levels: An instructor from Community College A can combine a text paragraph from University B with a movie from College C and an online homework problem from Publisher D, to form one page. Another instructor from High School E can take that page from Community College A and combine it with other pages into a module, unit or chapter. Those in turn can be combined into whole course packs. Faculty can design their own curricula from existing and newly created resources instead of having to buy into a complete off-the-shelf product.

The actual path through and the presentation of the learning resources is determined by instructor-specified combinations of learner choices and system-generated adaptations (for example, if the learner does not pass a test, additional resources may be included). Each learner can have an individualized curriculum according to preferences, capabilities and skills.

LON-CAPA tracks resource usage: every time a resource gets incorporated into a learning object of larger granularity, every time it gets deployed in a course, and every time a learner accesses it, the fact is recorded and added to the metadata for the resource. The former two events constitute a form of peer-review, with the approval being the adoption of a resource. In addition, data such as degree of discrimination, degree of difficulty, and average number of attempts until mastery of each resource are recorded. Instructors can use this data to make informed decisions when selecting resources for their courses and students.

LON-CAPA is a distributed network of servers that are linked through the internet. The system is scalable by adding additional servers, allows for load-balancing between the servers, and has a sophisticated content replication mechanism to avoid bottlenecks with highly utilized learning resources.

LON-CAPA is unique amongst course and learning content management systems in that it is developed under the open-source GNU General Public License. We do not expect that the Michigan State University team of developers alone will be able to sustain the code-base of LON-CAPA (currently around 60,000 lines of code) without input from a broader development community. In contrast to proprietary commercial software, the cooperative open-source community has a fast turn-around on debugging and adaptation to new technologies (as demonstrated by the success of the Linux operating system). We expect this model to be successful for a system that operates in the academic community. Already today, the LON-CAPA system has a code base comprised of contributions from four universities, and incorporates a vast number of available open-source tools and libraries. In addition, “because of its GPL licensing, LON-CAPA is easily adapted to the local administrative structure,” remarks Dr. Jensen.

Success

The ultimate measure of success for our system is whether it indeed proves to be an effective tool in increasing educational outcomes for students. A number of systematic studies, conducted primarily within undergraduate physics courses, suggest that LON- CAPA can have a pronounced impact on student learning. One study followed an introductory calculus-based physics course from the years before system implementation until late into its deployment. In the years before using LON-CAPA, the final grade distribution exhibited the traditional bell shape around a grade of 2.5, with relatively few students receiving grades of 3.5 or 4.0. After the move to LON-CAPA, the proportion of students earning a grade higher than 3.0 increased dramatically. Notably, independent evaluators judged that the examinations used in the course after deployment were more challenging than those used in earlier years, and so the positive change in educational outcomes can not be attributed to a lowering of standards for the class.

Two other studies suggest that LON-CAPA may increase the participation and success of women in the sciences. One study focused on a yearlong physics course for non-science-majors, in which the system was used only during the second semester. Final grades from the second semester indicated that women were especially likely to benefit from the system. The second study indicated that women, who began the course significantly less well prepared than men, improved their performance relative to men across the semester until there were no gender differences by the final exam.

“A majority of students, typically 80%, consider that LON-CAPA helps them learn and understand the course material,” says Deborah Kashy. This is surprising, since the time students spend working on assignments and other course requirements has increased by nearly a factor of two.

Difficulty

LON-CAPA still has a small software development team, and when it comes to making interface design decisions, the development of the learner interface takes priority. “I feel the software was great,” says David Muir, an undergraduate student at MSU who just finished one semester of ‘virtual physics’ using LON-CAPA, “ The first thing I look for with new technology, especially with computers, is if that technology is user friendly. I was able to navigate through the material without any difficulty, which to me is very important.” Providing the same ease of use for instructors has been a challenge. Dr. Batchelor explains, “LON-CAPA is and will continue to be under extensive development. LON-CAPA is presently easily accessible to users of low or average IT experience provided they have some computing systems support. Students appear to assimilate the main features of the LON-CAPA interface easily. For instructors and administrators the standard interface presents a more significant learning curve, because of the many new features and interesting capabilities LON-CAPA provides.” Dr. Janet Batzli, who teaches biology at the University of Wisconsin, underlines this point: “Although [LON-CAPA’s] potential benefits as a distributed-network resource indexing system far outweigh what could be achieved by an off the shelf course management system (e.g., Blackboard, WebCT) it still needs considerable interface development and user testing.”

Increasing LON-CAPA’s user community is crucial to both the immediate success as well as the long-term sustainability of the project. Started as a purely academic effort, eventually the system will need to exhibit financial sustainability without grant support. Making this transition without loosing the unique characteristics of the system and without turning an academic department with research faculty into a purely financial interest-driven service entity is a challenge.

Unanticipated Challenges