FINDING THE RIGHT WAVELENGTH: SCALABLE ADVANCED LEARNING ECOSYSTEMS

Robert S. Kadel *

Center for 21^st Century Universities, Georgia Institute of Technology, Atlanta, Georgia 30332-0765, USA

* Address all correspondence to: Robert S. Kadel, Center for 21st Century Universities, Georgia Institute of Technology, Atlanta, GA 30332–0765; Tel.: (+1) 404-894-6863; E-mail: rob.kadel@gatech.edu.

1. INTRODUCTION

The Georgia Institute of Technology has made headlines recently for the publication of a report titled “Deliberate Innovation, Lifetime Education” (Office of the Provost, 2018). This report, the culmination of over two years of research and collaboration, details Georgia Tech’s plans for meeting the challenges of higher education through the year 2040. It is ambitious and, to a certain degree, open-ended, because the Institute recognizes that while we have some ideas about the direction of higher education, there are still a number of unknowns.

Nevertheless, Georgia Tech is working toward redefining higher education given a rapidly shifting landscape. One such shift that is inescapable to Georgia Tech and to many in higher education is that the market for higher education is changing (Selingo, 2013). Postsecondary education is now a requirement for entry into the middle class, yet accessible and affordable educational opportunities are out of reach for many of America’s high school graduates and those workers looking to upskill or develop new skills. This has happened for a number of reasons, the first of which is that the cost of higher education has risen dramatically while productivity has not increased (Kimball and Luke, 2018; Baumol and Bowen, 1965). It is important that colleges and universities compete to keep talented educators from scrapping their teaching roles in favor of taking higher paying jobs elsewhere (Baumol, 1967). Secondly, costs spiral upward because colleges and universities, the vast majority of which are nonprofit institutions, must reinvest all revenue back into their institutions, continuously seeking to build their reputations (Bowen, 1980). They must spend all that they have, and once they do that, they will need more, which they will spend, and then request more, in a seemingly never-ending cycle to demonstrate to students, parents, lawmakers, and philanthropists that they should invest their money in those institutions (Martin and Gillen, 2009). These funding needs have created a perfect storm in which average four-year-institution tuition costs have had a 28-fold increase between 1963–64 and 2016–17 (National Center for Education Statistics, 2018), while consumer prices have had a 7.8-fold increase over the same period (U.S. Bureau of Labor Statistics, 2018).

Georgia Tech’s goal is to change this equation through scalable online programs. A scalable online program is one in which additional students can be served by a marginally smaller cost for adding those students. Fixed costs must factor in, e.g., the technical infrastructure, the cost of the professor’s knowledge and teaching, an instructional designer to design the course and develop effective assessments, and video production for recording lecture videos and demonstrations. Once those fixed costs have been determined, the variable costs include course maintenance and hiring enough teaching assistants (TAs) to oversee the work of the students. Marginal costs are measured as the change in the total cost when adding additional students. If a course can enroll an additional 50 students for the cost of only one additional TA, the average cost for each additional student is relatively small. Meanwhile, the revenue of adding an additional 50 students is quite large.

A very simple example is shown in Table 1. This table omits the costs of technical infrastructure and administration and focuses solely on the costs for building and teaching a single online course using very rough measures of pay. ^† Note that even when the student-to-staff ratio increases from 30:1 up to 50:1, the revenue increases dramatically because of the number of students that can be served. This revenue could be invested in more TAs, lowering the student-to-staff ratio, or into other programs at the institution.

Georgia Tech educators have debated the merits of such scalable online courses for several years (DeMillo, 2015). The guiding mission has always been that the Institute would not offer such programs if we could not be certain that their quality would be at least as high as our on-campus programs. The evidence has demonstrated that not only do students in the online programs have the same assessment and project scores as students in the traditional, face-to-face programs, they sometimes do even better (Goel and Joyner, 2017; Joyner et al., 2016). Further, there are tangential benefits. Faculty have found that the resources created for scalable online courses such as massive open online courses (MOOCs) — including videos, online quizzes, and discussion forums — can be repurposed for use in blended learning environments that benefit students in traditional on-campus programs (Madden et al., 2019).

2. FLEXIBILITY IN SCALABLE ONLINE CLASSES

One key factor in delivering scalable online classes is having the technology to deliver them, and in this area there is no shortage of available choices. There are perhaps a dozen or so major learning management systems (LMSs) available on the market. Some are geared toward massive course enrollments but have fewer customizable options for the instructor. Others are more customizable but have some limitations in delivering, for example, streaming video with interactive transcripts. Additional technology choices are required when considering the literally hundreds of plug-ins powered by Learning Tools Interoperability (LTI; IMS Global, 2018a) and other similar standards. (LTI is designed to allow students to move seamlessly from one learning tool to another within a learning management system or other application.) Simulations, tutorials, wikis, group collaboration tools, online textbooks, encyclopedias, and video sharing tools make for a smorgasbord of choices when considering how to build a scalable online course or program.

That was the challenge Georgia Tech faced when putting together such programs. The Institute hosts some of the most innovative scalable degree programs (McKenzie, 2018) with the online Master of Science (OMS) degrees in computer science (OMSCS, 2018), analytics (Georgia Tech Professional Education, 2018a), and a forthcoming program in cybersecurity (Georgia Tech Professional Education, 2018b) in 2019. These unique programs combine the flexibility, scalability, and affordability of MOOCs with the rigor and quality of traditional Georgia Tech degree programs. Courses are taught by top faculty and supported by multiple TAs such that there is still a high degree of personalized, hands-on support. These programs are designed, developed, and delivered by the Georgia Tech colleges that host the residential programs, Georgia Tech Professional Education (2018c) and the Center for 21^st Century Universities (2018).

3. THE CHALLENGES OF MULTIPLE TOOLS AND PLATFORMS

As one might expect, the successes in building and delivering these programs have not been without challenges and setbacks, and this article highlights one in particular: interoperability. This is meant both in the technical sense of systems that can interface with each other as well as the more colloquial sense of how professors, TAs, instructional designers, administrators, researchers, and, of course, students use the multiple systems, such as the LMSs and LTI tools described above, that are employed to make these programs possible.

As an example, the OMS Analytics program’s course content is delivered on the edX (EdX, 2018) platform serving three different populations of learners that Georgia Tech defines as audit learners, verified learners, and Georgia Tech enrolled students. Audit learners take the free, open MOOC and are not enrolled at Georgia Tech. Verified learners are not enrolled at Georgia Tech but do pay a small fee to edX to earn a certificate if they complete a course. ^‡ Georgia Tech enrolled students are enrolled formally at Georgia Tech in the OMS Analytics degree program and are on their way to completing their master’s degree.

All learners, regardless of enrollment type, receive videos and certain supplemental handouts through edX. Verified learners and Georgia Tech students take quizzes through edX, but two different proctoring systems are used: edX verified learners use Verificient, which is built into edX, and Georgia Tech students use ProctorTrack. Grades for Georgia Tech students are kept on Georgia Tech’s LMS, Canvas. For many, but not all professors, course discussions take place on Piazza. (Piazza is a customizable, wiki-style Q&A tool that many Georgia Tech professors use for discussion forums.) In courses that require students to practice writing computer code, most professors use the Vocareum platform, which allows students to enter their code into a window that compiles it and checks it for errors, also allowing the instructor to grade each submission as an assignment.

Indeed, in one MOOC-based undergraduate course, Introduction to Computing Using Python, CS 1301, the instructor uses seven different systems (i.e., Canvas, edX, Vocareum, ProctorTrack, Piazza, Slack, and an instructor-created Smartbook made in collaboration with McGraw-Hill Education specifically to teach this subject). There is no one-size-fits-all platform that can provide all the tools that the instructor of CS 1301 feels he needs so that students will be successful in a scalable, personalized course. In the OMS Analytics courses, such combinations of tools are also quite common.

That same instructor also worked with another professor in the design of a course in the OMSCS degree program, Knowledge-Based Artificial Intelligence. This course was the first of its kind to employ an AI-based tutor, “Jill” Watson, so-named because “she” was powered by IBM’s Watson (IBM, 2018) technology (Korn, 2016; Leopold, 2017). The course combined the Udacity (Udacity, 2018) MOOC platform with Georgia Tech’s Sakai LMS along with proprietary tools for peer feedback and for autograding the source code of computer programming projects. Again, the goal was to find that special combination of tools that the instructors believed would give students the best learning experience and therefore the best chance at success in the course.

However, these systems, with all their different angles on how best to enable learning, are not without some difficulties. Students have to navigate not just one or two different learning systems but as many as seven in a single course. Those systems could also vary from course to course, making the user experience challenging. In short, our courses, professors, and students are simply not always on the same wavelength.

4. WHEN ALL THE WAVELENGTHS BECOME CHALLENGING

As with any complex system, we must understand its indirect effects. The tools described previously for CS 1301 provide the experience that the instructor wanted his students to have. But even minor changes to one or more tools can be challenging. The instructor recently gave this example:

[The platform] has made some changes to its interface that we weren’t ready for that have changed the student experience. For example, my new most common question is, “I can’t get the green checkmark to appear!” to which my answer was originally, “What green checkmark?”, and is now, “Yeah, ignore that, it’s glitchy.”

This demonstrates that even the most minor changes to a platform can have simple yet unintended consequences that can reverberate through the course. Changes can take up valuable time on the part of the students and the instructor. When working with multiple platforms, that wasted time increases geometrically.

Tracking students can also challenge time constraints. For example, a professor in the OMS Analytics program recently recounted the difficulty he experienced when the time came in the semester for the standard verification of enrollment process at Georgia Tech (i.e., reporting to the registrar whether students have participated in order to verify eligibility for federal financial aid). As this professor was delivering content on edX, administering assignments on Canvas, and hosting discussions on Piazza, he had to go to three different sources of data in order to determine who had in fact participated in his course up to that time. In short, when there is no centralization in what tools are used, how they are employed, or how to use their data, educators face the difficulties of managing multiple user experiences (including their own).

There are policy necessities that further challenge instructors and students. Due to Family Educational Rights and Privacy Act requirements, learners who are enrolled through edX but are not enrolled through Georgia Tech cannot be in the same discussion forums on Piazza as Georgia Tech enrollees, requiring the use of separate forums and thus separate monitoring by the professor and TAs.

Interestingly, in some courses, students have taken such matters into their own hands. For example, in the OMSCS program, students created their own Google Plus community where they have been sharing common questions and answers as new students enter the program. The enrollment in OMSCS is now approaching 6,500 students since its launch in 2014. The Google Plus community exceeds 9,000 highly interactive members. In the OMS Analytics program, students make use of Slack to discuss course materials. There appears to be a mix of Georgia Tech students, verified students, and even some audit students in the Slack discussion area.

Lastly, there is the straightforward problem of technical interoperability. Some of these systems simply fail in accepting LTI transmissions from other systems at times. It is useful that these courses cater largely to computer science students or those with some coding background, so they can sometimes help the professors and TAs to debug an issue. However, such support is not a scalable solution, nor does it apply to other disciplines seeking to scale their online programs.

5. BRINGING THE WAVELENGTHS TOGETHER

We need to bring the wavelengths, or challenges, together in convergence. A prism is a triangular glass device that disperses (white) light into its divergent wavelengths, resulting in seven colors — red, orange, yellow, green, blue, indigo, and violet. What needs to happen in order for scalable online learning to be effective is the opposite — converging all the different wavelengths back into a single beam of light. Newton (1704) demonstrated that this is possible when done in precision. Consistent with the wavelength metaphor, the work of scalable online learning must be done very precisely to bring forth the necessary convergence (see Figure 1).

FIG. 1: The reverse prism of scalable advanced learning ecosystems

What we propose at Georgia Tech are termed scalable advanced learning ecosystems (SALEs), which combine the following components:

• Personalized learning systems—Artificial intelligence and machine learning are used to establish customized educational plans for each student.
• Intelligent tutoring systems—The presentation and sequence of materials are modified in response to student performance.
• Data mining and learning analytics—Useful and actionable information is extracted from large datasets to understand student uses of learning resources and outcomes.
• Scalable online environments—LMSs are designed to deliver high-quality, high-engagement learning experiences for large groups of students.
• Immersive learning environments—A physical presence in real or imagined worlds is simulated, and content is overlaid on those worlds such that real and virtual worlds are merged to produce new learning opportunities.

Figure 1 demonstrates how a SALE is the reverse of a prism, bringing the various wavelengths of light back together to form a single beam of light. This Figure shows how the five different components of SALEs described earlier converge through a prism (the triangle) in the center to form SALEs on the right. As noted in the prism in Figure 1, there are many tools and strategies that can, and will, be employed to the benefit of SALEs. These include the new standards in Learning Tools Interoperability now under the heading of LTI Advantage (IMS Global, 2018a), which goes beyond previous LTI standards to also allow for “provisioning of user names and roles so a tool can intelligently address the learner on launch, and the sharing of assignments from a platform to an assessment tool and the subsequent scores back to a central gradebook” (IMS Global, 2018, para. 2). Caliper Analytics (IMS Global, 2018b) standards, which structure disparate data and allow it to be exchanged between applications, will allow for the sharing of data between learning resources so that we can better personalize the learning experience. Compound learning objects will allow us to create, remix, and share content in ways that meet learners in their space and in augmented and virtual worlds. Content management systems, learner management systems, and LMSs can be used to allow people and resources to be organized with a more holistic vision.

This is by no means an exhaustive list of strategies or solutions; it is not designed to be granular enough to speak to the practical issues described above. However, we feel that addressing issues of interoperability at the top level — the level of vision — will have a cascading effect through the rest of the program.

In Georgia Tech’s early forays into scalable online learning, the computer science and analytics courses described above were not designed with a whole ecosystems approach in mind. Rather, each instructor was presented with an opportunity to teach in an innovative online course or degree program and worked with administrators and instructional designers to choose tools that would get the job done efficiently. Consequently, we ended up with professors on several different wavelengths, as described. Using the SALE approach and leveraging some technologies that are only just now coming to the market, we might have made different decisions. Perhaps each OMS Analytics course would have its own Smartbook for personalized learning, similar to the one the CS 1301 instructor developed with McGraw-Hill Education for Introduction to Computing Using Python. Perhaps each would use the same LMS for all populations of learners. Perhaps a centralized data warehouse would make verification of enrollment a single-step process, rather than one that takes three steps through three different databases.

Some of these decisions seem obvious, while others are less so. For example, what will be the role of augmented and virtual reality (AR/VR) in this to-be-determined ecosystem? Such technologies, while promising and exciting, are still at the fringes of learning and content delivery. Yet, perhaps the idea of teaching through video lectures will give way to immersive learning environments where students will interact with their content in a virtual world.

Meanwhile, the aforementioned professor’s use of the “Jill” Watson AI-based tutor holds promise and is currently under expansion. However, that AI-based tutor is only as good as the data used to program it. It is programmed based on the interactions that human TAs have had in similar situations in the past. In order to have more relevant data, we must continue to have more human interaction that is both nuanced on the one hand and recorded and analyzed on the other hand in order to realize the potential of intelligent tutoring systems. Similar to AR/VR, the promise exists, but there is still much work to do. When it comes to fruition, perhaps such an intelligent tutor will be able to enter our virtual world and teach us the interactive content. The point is that right now, we cannot reach a vast untapped market of learners hungry for postsecondary education with today’s content, let alone tomorrow’s, unless we consider the potential of scalable online learning.

In November 2018, Georgia Tech will host a summit on SALEs funded by a grant from the National Science Foundation (NSF) where we will bring together teachers, researchers, and administrators from around the country — institutions both large and small that are facing similar questions and challenges — so that we can attempt to answer some of these questions. The event promises to be an engaging couple of days, and we expect it will produce not only a white paper for the NSF but also several articles on the topic. It is this author’s intention to write a follow-up to this piece where I will report on what we have learned and where we hope to go next. We do not presume to have all the answers, but I hope the reader will agree that we have posed some interesting questions that require, if not converging wavelengths, at least an agreement on how best to view them.

REFERENCES

Baumol, W.J. (1967), Macroeconomics of Unbalanced Growth: The Anatomy of Urban Crisis, American Economic Rev., 57(3), pp. 415–426, Available at http://www.jstor.org/stable/1812111.

Baumol, W.J. and Bowen, W.G. (1965), On the Performing Arts: The Anatomy of their Economic Problems, American Economic Rev., 55(1/2), pp. 495–502.

Bowen, H.R. (1980), Costs of Higher Education: How Much Do Colleges and Universities Spend per Student and How Much Should They Spend?, Hoboken, NJ: John Wiley and Sons, Inc. (Jossey-Bass, Inc.).

Center for 21^st Century Universities (2018), Retrieved November, 19, from https://c21u.gatech.edu.

DeMillo, R.A. (2015), Revolution in Higher Education: How a Small Band of Innovators Will Make College Accessible and Affordable, Cambridge, MA: MIT Press.

EdX (2018), Retrieved November, 19, from https://edx.org.

Georgia Tech Professional Education (2018a), Retrieved November, 19, from https://pe.gatech.edu/degrees/analytics.

Georgia Tech Professional Education (2018b), Retrieved November, 19, from https://pe.gatech.edu/degrees/cybersecurity.

Georgia Tech Professional Education (2018c), Retrieved November, 19, from https://pe.gatech.edu.

Goel, A. and Joyner, D. (2017), Using AI to Teach AI: Lessons from an Online AI Class, AI Mag., Summer 2017, pp. 48–58.

IBM (2018), Retrieved November, 19, from https://www.ibm.com/watson/.

IMS Global (2018a), LTI v1.3 and LTI Advantage, Retrieved November, 19, from https://www.imsglobal.org/activity/learning-tools-interoperability.

IMS Global (2018b), Caliper Analytics, Retrieved November, 19, from https://www.imsglobal.org/activity/caliper.

Joyner, D., Goel, A., and Isbell, C. (2016), The Unexpected Pedagogical Benefits of Making Higher Education Accessible, Proc. of 3rd ACM Conf. on Learning @ Scale, pp. 117–120, New York: ACM.

Kimball, B. and Luke, J. (2018), Historical Dimensions of the “Cost Disease” in US Higher Education, 1870s–2010s, Social Science History, 42(1), pp. 29–55.

Korn, M. (2016), Imagine Discovering That Your Teaching Assistant Really is a Robot, Wall St. J., May 6, 2016, Retrieved from https://www.wsj.com/articles/if-your-teacher-sounds-like-a-robot-you-might-be-on-to-something-1462546621.

Leopold, T. (2017), A Professor Built an AI Teaching Assistant for His Courses – and It Could Shape the Future of Education, Business Insider, March 22, 2017, Retrieved from https://www.businessinsider.com/a-professor-built-an-ai-teaching-assistant-for-his-courses-and-it-could-shape-the-future-of-education-2017-3.

Madden, A., Margulieux, L., Kadel, R.S., and Goel, A. (2019), Blended Learning: A Guide for Researchers and Practitioners, Cambridge, MA: The MIT Press.

Martin, R. and Gillen, A. (2009), Breaking the Cost Spiral, Inside Higher Ed, August 7, 2009, Retrieved from https://www.insidehighered.com/views/2009/08/07/breaking-cost-spiral.

McKenzie, L. (2018), Online, Cheap – and Elite, Inside Higher Ed, Retrieved March 20, 2018, from https://www.insidehighered.com/digital-learning/article/2018/03/20/analysis-shows-georgia-techs-online-masters-computer-science.

National Center for Education Statistics (2018), Table 330.10: Average Undergraduate Tuition and Fees and Room and Board Rates Charged for Full-Time Students in Degree-Granting Postsecondary Institutions, by Level and Control of Institution: Selected Years, 1963–64 and 2016–17, Retrieved from https://nces.ed.gov/programs/digest/d17/tables/dt17_330.10.asp?current=yes.

Newton, I. (1704), Opticks: Or, a Treatise of the Reflections, Refractions, Inflexions, and Colors of Light, Also Two Treatises of the Species and Magnitude of Curvilinear Figures, London, UK, Retrieved September, 28, 2018, from http://www.newtonproject.ox.ac.uk/view/texts/normalized/NATP00034.

Office of the Provost, Georgia Institute of Technology (2018), Retrieved November, 19, from http://www.provost.gatech.edu/cne-home.

OMSCS, Georgia Institute of Technology, (2018), Retrieved November, 19, from http://www.omscs.gatech.edu.

Selingo, J. (2013), College (Un)Bound: The Future of Higher Education and What It Means for Students, Las Vegas, NV: Amazon Publishing.

Udacity (2018), Retrieved November, 19, from https://udacity.com.

U.S. Bureau of Labor Statistics (2018), Consumer Price Index Data from 1913 to 2018, Available at https://www.usinflationcalculator.com/inflation/consumer-price-index-and-annual-percent-changes-from-1913-to-2008/.

NOTES:

† These are not meant to be completely factual numbers but merely a thought exercise showing that as enrollments increase, revenues increase and the marginal cost to teach additional students is small.

‡ Such learners can also pay a fee that is higher than the certificates but less than full tuition to try the Georgia Tech/edX MicroMasters program in which they take three courses. If they successfully complete and earn the certificates in those courses, and if they are admitted to the OMS Analytics program, the three courses count toward their degree. It is considered to be a lower-cost/low-risk way to try an analytics graduate program without committing to the full degree.