LIBRARY SUBSCRIPTION: Guest
Abstract
A comprehensive STEM education for students is meant to develop critical thinking, problem-solving, creativity, collaboration, and self-directed learning. These are the main objectives of the BLOSSOMS Initiative (Blended Open-Source Science Or Math Studies), an MIT-sponsored program founded in 2008. MIT BLOSSOMS has an OER online repository of interactive, inquiry-based STEM video lessons at the high school level, created by university and high school educators in ten countries, including the United States. The program puts forward a novel approach to STEM teaching and learning in order to achieve the above-stated objectives, as well as to inspire students and enhance retention in the STEM pipeline. However, systemic barriers and the lack of empirical evidence have impeded its widespread adoption in high school education. In this paper, we will present MIT BLOSSOMS and examine major implementation challenges with an eye towards designing a Project-Based Learning (PBL) BLOSSOMS assessment procedure.
KEY WORDS: technology-enabled education, assessment methodologies, project-based learning, MIT BLOSSOMS
As the world becomes increasingly technological, ensuring STEM (Science, Technology, Engineering, and Mathematics) literacy among younger generations is critical to enable them to understand the world around them and succeed in their professional and personal lives. This reality is emphasized by Larson (2014), who points out that in this data-informed, technology-intensive 21st century, all the population needs to become STEM literate. In addition to factual scientific learning, comprehensive STEM education is also meant to develop critical thinking, problem-solving, creativity, collaboration, and self-directed learning (DeCoito, 2014).
Promoting STEM education and STEM career paths is critical to secure the ability of the United States to compete in the global economy. Recent research by Xue and Larson (2015) has shown signs of a potential STEM crisis in the US, including both a shortage of STEM-oriented workers and a partial misallocation of the current STEM workforce. Overall, workers' proficiency in STEM fields is considered vital to the health of the economy (Atkinson and Mayo, 2010). This is the reason why federal and state policymakers have called for national efforts to develop strong STEM pathways from high school to college that will eventually expand the STEM-capable workforce nationwide.
It is commonly believed that efforts should be concentrated at the K–12 level, when young people receive first exposure to STEM thinking and topics. Student inputs to universities, “freshmen,” are the outputs of the K–12 system; their STEM skills at age 18 provide a system boundary condition for future STEM skills to be acquired in higher education. Data for the nation's elementary and high school students reveal that achievement gaps in mathematics continue to persist for students from disadvantaged backgrounds, and international assessments reveal that the United States lags behind other developed countries in average mathematics and science literacy scores. In recent years, educators have joined an initiative to develop common national K–12 mathematics and science standards, as well as assessments and indicators for monitoring student progress and science teaching and learning (National Science Board, 2016).
A 2013 report by the National Research Council (NRC), entitled Monitoring Progress Towards Successful K–12 STEM Education, addresses the need for research and data that can be used to track the progress in the K–12 education system and for making informed choices about improving it. The NRC defined a set of 14 progress indicators related to students' access to quality learning, educators' capacity, and policy and funding initiatives in STEM. Among these, indicator 4 emphasizes the pressing need for science-related learning opportunities throughout K–12 using high-quality instructional materials (National Research Council et al., 2013).
It is very important to have a clear understanding of what teaching STEM really means. In our view, it is the process of using materials and methods to (i) enable students to develop basic STEM skills and achieve mastery in STEM concepts as well as (ii) to be able to “go deep” in framing, formulating, and solving new problems they have not seen before, and (iii) to inspire passion and excitement for STEM subjects that will lead students to look for further STEM opportunities throughout their academic career. STEM teaching is too often limited to its first objective, lecturing contents meant for the students to learn basic more formulaic skills that will later be tested through problem sets and exams. Leaving out the objectives 2 and 3 is a dangerous mistake in that they may feel that all STEM problems only require “plug-and-chug” formulaic answers and that there is no role here for the student herself to contribute to STEM solutions of important new problems. Student STEM dropout is often an outcome.
There have been concerns that the traditional academic approaches (including, for example, the one-size-fits-all lecture format), which emphasize rote memorization or the application of simple algorithms, do not properly address the challenges posed by STEM teaching (Bransford, 1999). Recent mathematics teaching research has even revealed that a large majority of students find mathematics “boring, mostly irrelevant and unrewarding”. Most of them view mathematical thinking as the mechanical application of ready-made recipes to numerous small variations of the same problems (Sinay, 2016). As a result, we support the design of a STEM education system in which novel, innovative ways of teaching STEM go hand in hand with traditional learning structures, while making the most of innovative education technologies and truly engaging the students through a more interactive and research-based approach to knowledge and discovery.
In this paper, we focus on MIT BLOSSOMS 1 (Blended Learning Open Source Science Or Math Studies). Since 2008, the BLOSSOMS team has worked with educators from the USA and other countries to create and use an online library of free (Open Source) STEM interactive video lessons for high school teachers and their students. BLOSSOMS also offers online resources for both teachers and students, such as supplementary materials and animated simulations. By getting into the live classroom with a novel pedagogical model, requiring new skills from teachers and encouraging full engagement of students, BLOSSOMS attempts to make several systems changes at once in the secondary school educational system (Larson and Murray, 2008). Usually designed by passionate teachers, professors, and students, BLOSSOMS videos are high-quality challenging educational lessons that require several months in production (Kalantari, 2015).
The typical BLOSSOMS lesson has four to six short video segments, each followed by a live active-learning segment in the classroom, guided by the in-class teacher while the students are sitting in their regular classroom seats. Before class, the teacher is advised to download the entire BLOSSOMS video lesson onto her/his laptop computer. The class starts with segment 1 of the BLOSSOMS video. Towards the end of the segment, a challenge is given to the class. As the video fades to black, the in-class teacher takes the teaching baton and guides the students in an active learning exercise. This baton-passing process involving the video and the in-class teachers is a type of blended learning referred to as the Teaching Duet (Larson and Murray, 2017).
The video will expose the class to a fun, nontrivial, real-world scientific problem, such as: “How do mosquitoes fly in the rain?” or “Why do boomerangs come back to the thrower?” The major goals of these videos are to enhance students' critical thinking skills and have them act as scientists and engineers; to get them excited about STEM subjects and careers; to show relevance of math and science to everyday lives; and to enhance cross-cultural awareness, sensitivity, and appreciation (Larson, 2015). These objectives are achieved through a phenomenon-based approach to science and engineering education. Although natural phenomena are the context for all scientific investigation, they have traditionally been a missing piece in science and math education. However, learning from phenomena is believed to be deeper and more sustainable, since the focus of learning shifts from learning facts about a topic to figuring out why or how something happens (Achieve, 2016). Several BLOSSOMS lessons have accompanying online animated simulations to engage the student in deeper learning following the in-class experience.
MIT BLOSSOMS began in 2008, and operates in the USA as well as in nine other countries, in order of appearance: Jordan, Pakistan, Saudi Arabia, Lebanon, Brazil, Malaysia, China, Japan and Mexico. Partnering with a foreign country involves collaborating with local educational institutions (universities, high schools) as well as national authorities such as ministries of education. While these countries recognize the high benefits that BLOSSOMS could bring to their educational system, the actual use of BLOSSOMS videos in class faces implementation issues and stakeholder skepticism, for two main reasons.
The first implementation issue relates to teacher training. Since BLOSSOMS promotes a very innovative pedagogical approach, some partner countries have experienced difficulty with engaging teachers in the BLOSSOMS project. Due to lack of training, some teachers have failed to deliver quality BLOSSOMS lessons, while many others have been reluctant to move away from existing scripted-lecture practices that rely on textbooks and require less preparation time and effort (Abdullah and Shukor, 2017).
The second, but nonetheless critical, impediment to the adoption of a BLOSSOMS-like curriculum is the strong resistance of students, parents, and school administrators to innovative teaching methodologies whose horizons go beyond standardized testing. Students and their parents are, in many cases, mostly concerned by the marks obtained in public examinations because they are the prime determinant for getting into good engineering and medical schools. As a result, teaching to the test and rote learning are often preferred to BLOSSOMS (Malik, 2017). More generally, Herold (2015) emphasizes that the current test-based accountability system jeopardizes the transition to more innovative forms of instruction leveraging technology and active learning strategies.
Past BLOSSOMS experiences have revealed the difficulty of implementing BLOSSOMS-based teaching approaches in many countries. In addition to cultural barriers, persuading key stakeholders to support the use of BLOSSOMS in the current curricula proves challenging, especially as there is still little empirical evidence of the project's effectiveness other than students' positive excitement when interviewed after they experience a BLOSSOMS lesson. One of the priorities of MIT BLOSSOMS thus includes the design of assessment procedures, meant to demonstrate the program's efficacy on student learning and retention in STEM pathways. In the era of data-driven policy, designing such rigorous procedures is essential to (i) validate the program and demonstrate its utility as well as (ii) encourage its widespread adoption and diffusion in K–12 institutions.
In essence, BLOSSOMS lessons are not from textbooks, they do not involve rote learning, they are not lectures, and they require students to think through novel problem situations in science, mathematics, and engineering. BLOSSOMS is intended to complement, not replace, lecture-style teaching, as it does not teach the same skills! Indeed, BLOSSOMS will not teach students how to compute triple integrals or solve complex differential equations. Instead, it is meant to expose students to the inquiry-based nature of science, and foster interaction, communication, free thinking, and research skills, among many others. We believe that the ideal configuration would be an average of one BLOSSOMS lesson per month in a STEM course, to get the students to begin to think as scientists and engineers, to see that the subject they are studying is alive with challenge and excitement.
For this reason, a BLOSSOMS lesson does not include a test. Too many programs make the mistake of systematically testing and ranking the students. While these procedures create young people who excel at test taking with its rote learning, they do not encourage inventiveness, risk tasking, or innovative thinking (Larson, 2014). Assessing BLOSSOMS using standardized tests would thus not only be inappropriate, but also counterproductive. Other forms of assessment are needed in this case, that capture the deeply interactive, inquiry-based nature of BLOSSOMS. The main objective of this research is to develop such a novel and rigorous form of assessment to evaluate the impact of MIT BLOSSOMS on student learning and their retention in a STEM pathway. The end product should be simple assessment procedures and materials that could be used by local teachers to assess their students.
Prior to developing assessment procedures for MIT BLOSSOMS, we briefly visit traditional methods of education assessment. Assessing the impact of any teaching method on student learning is a causal question, often addressed in the literature using randomized controlled trials (RCTs). Students randomly assigned to the treatment cohort would be taught STEM subjects using BLOSSOMS resources, while students assigned to the control cohort would receive a more fact-focused, lecture-based, traditional education. Numerical measures of learning outcomes would then be used in a statistical study in order to determine the quantitative causal impact of BLOSSOMS on these outcomes. Such outcomes can include performance in standardized tests, on-time high school graduation, college enrollment, college retention, among many others. These indicators are computed in aggregate as well as separated according to gender, race, or socioeconomic status.
RCTs are difficult to justify in the case of MIT BLOSSOMS, due in part to ethical concerns. One could expect parents to complain that their children are being assigned to one or the other cohort, depending on what they believe to be the best educational option. The solution in this case is to rely on quasi-randomized experiments, leveraging the natural randomness in the curriculum taught at various institutions. The idea would be to compare student outcomes across schools that have adopted BLOSSOMS and schools that have kept the traditional curriculum. While there are many more schools that have not implemented BLOSSOMS-based teaching than schools that have, this bias can be accounted for with matching procedures, for example, using propensity scores.
This assessment design has been chosen to evaluate innovative educational approaches such as the deeper learning initiative. Deeper learning refers to a form of education that promotes mastery of core content knowledge as well as of the skills that help students communicate their ideas effectively, think creatively, work collaboratively, and manage their own learning (Yang et al., 2016). In a recent study funded by the William and Flora Hewlett Foundation, researchers examined the effect of attending a deeper learning network high school on students' college enrollment based on updated post-secondary data. Statistical data analysis revealed that relative to students from control high schools, students who attended deeper learning high schools demonstrated higher levels of soft skills, scored higher on standardized tests, and were more likely to enroll in four-year and selective colleges (Zeiser et al., 2014).
Results from the deeper learning study naturally carry on to BLOSSOMS, which is, in fact, one dimension of the broader spectrum of deeper learning tools and methods. As such, we expect BLOSSOMS to have a positive impact on student learning, soft skills, on-time graduation, STEM college admission and retention, and even STEM retention throughout the student's academic and professional career. However, we believe that looking at p-values from statistical analyses is not the best way to assess the impact of BLOSSOMS on student learning, both on a long-term, global scale and a shorter-term, local scale.
Our first reservations are specific to the long-term BLOSSOMS' impact on student learning. Randomized experiments are not a reasonable option in this case, due to their heavy financial costs and ethical barriers. There are no simple natural experiments that could be leveraged to assess its impact on student learning, because the schools which implement MIT BLOSSOMS are more likely to implement many other aspects of deeper learning, thus introducing many confounders that jeopardize causal analyses.
We also express reservations for the short term, local evaluation of BLOSSOMS. One could argue that in order to pinpoint the specific impact of BLOSSOMS on student learning, local teachers could test students after they have participated in a BLOSSOMS lesson and compare results with similar students who have not attended the session. However, due to the very inquiry-based nature of BLOSSOMS, testing students will likely interfere with and hinder their learning experience, through increased level of stress, leading to a lower impact on learning.
More generally, assessing holistic educational tools such as BLOSSOMS through statistical analysis on observational data does not seem the most relevant option, because many of the desired outcome variables are complex and not necessarily measurable quantitatively (Maxwell, 2004). Learning, for example, is a difficult notion to quantify, since it encompasses much more than what can be inferred from a multiple-choice, standardized test. While numerical indicators can be crafted to summarize qualitative data on learning, collaboration, communication, problem solving or critical thinking, these indicators are often quite design sensitive.
After seeing the flaws and limitations of statistical analyses, and reviewing the literature on novel education methods and deeper learning, we decided to leverage the most obvious, yet underused resource to solve our assessment challenge: the teacher herself! Building on her educational experience, and her subtle understanding of individual students in the class, she is best qualified to evaluate the impact of BLOSSOMS using soft, noninvasive methods. Project work appears to be the ideal fit in this situation, because it requires and develops many of the skills that BLOSSOMS intends to teach. After they have actively participated in a BLOSSOMS session, students team up and work on a short-term project (two to three weeks) building on the content of the session. While project-based learning (PBL) is a rather well-studied field, the idea of using a project-based approach in order to evaluate an education program has, to the best of our knowledge, never been examined in the literature.
Traditional PBL aims to promote more student-centered and experiential approaches to education through the interactive exploration of real-world challenges. PBL involves completing complex tasks resulting in a product, event, or presentation to an audience. Thomas (2000) identifies five key components of effective PBL: It should be (i) central to the curriculum, (ii) organized around major concepts, (iii) focused on a constructive investigation, (iv) student driven, and (v) focusing on problems that occur in the real world. PBL approaches often yield gains in factual learning that are equivalent or superior to those of traditional forms of instruction, while developing additional skills such as the ability to use knowledge more proficiently in performance situations (Barron and Darling-Hammond, 2008).
An insightful understanding of STEM PBL is provided in Capraro and Slough (2013). The authors define it as “an ill-defined task within a well-defined outcome situated with a contextually rich task requiring students to solve several problems which when considered in their entirety showcase student mastery of several concepts of various STEM subjects.” In this multidimensional definition, the well-defined outcome refers to precise skills, listed ex ante in an assessment rubric, that students should master by the end of their project. The ill-defined task emphasizes the freedom that students have as they work on their project, both in terms of methods and self-defined objectives. We find that this definition captures well the essence of our approach to STEM PBL.
Project-based learning approaches in education involve a number of advantages and drawbacks that have been discussed in the literature. A growing body of research has shown that PBL has a positive impact on student learning and leadership (Barron and Darling-Hammond, 2008). Active learning practices have a more significant impact on student performances than any other features, including student background and prior achievement. Students are more successful when they are taught how to learn as well as what to learn. PBL thus results in enhanced student performance, increased student motivation, improved teacher/student interaction, and increased development of soft skills such as creativity, critical thinking, collaboration, and communication (Speziale, 2017). Students learn more deeply when they can apply classroom-gathered knowledge to real-world problems, and when they can take part in projects that require sustained engagement and collaboration.
These observations have led Speziale (2017) to wonder, “If the results are so consistently good, why aren't more schools and teachers employing this teaching strategy?” It turns out that PBL also faces implementation challenges as well as more essential criticisms. These challenges include teachers' philosophical beliefs about effective teaching methods, deficient training, and support from school administrators, and the lack of appropriate policies (at the school, state, and federal levels) providing enough time and incentives to experiment with PBL approaches (Herold, 2015). Challenges also exist on the students' side, making PBL even harder to implement for the inexperienced teacher. In the following paragraphs, we provide the reader with more details about these challenges as well as potential solutions to them.
As mentioned above, one major implementation challenge for PBL comes from the teachers themselves, who may consider PBL a waste of classroom time and see the open-endedness of questions and the freedom granted to the students as an invitation to slack and spend less time studying. Provided that they approve of PBL, another challenge is development of the skills and knowledge of the teachers engaging in this innovative form of teaching and learning. When teachers do not fully understand the subtleties of project-based learning, they may fail to provide the necessary scaffolding, assessment, and redirection as projects are being conducted (Barron and Darling-Hammond, 2008). Teachers need to participate in relevant professional development sessions to help them migrate from traditional teaching techniques to more active, inquiry-based approaches. Among others, they should learn how to handle new skills such as making class time to implement PBL, designing meaningful projects and performance rubrics, and developing new classroom management techniques.
Many teachers acknowledge the difficulty of designing and implementing PBL strategies with their classes while making sure that students master the standards. Due to lack of experience and scaffolding, teachers may design ill-posed projects that will lead students to failure, not because they do not work enough but because this project was not appropriate for conducting further investigations in the form of a team project. For example, an eighth-grade teacher relates in the education blog “The Synapse” how he failed at making his “Linear Equations” math block fully project-based (Spencer, 2018). After he designed a “Linear Equations in Real Life” project, he realized that “…students could not find enough real examples, much less, experts they could interview. The project felt forced and the end goal was something they did not find relevant. To make matters worse, students struggled to determine linear functions while looking at a graph or to solve a linear equation using an algorithm.” He had tried to shoehorn PBL into these mathematics standards which he understood later were not a good fit for the project-based approach.
Even when project-based approaches seem well adapted to teach the standards and inspire the students about a topic, lack of experience leads teachers to make crucial mistakes in the conduct of the PBL unit. Spear (2015) relates the story of a failed “Self-Study Project” unit, in which the teacher wanted students to examine their background, self, and ambitions in the light of a study of different types of self-portraits. The end product would have been a self-portrait of their choice. According to the teacher, she made the following three mistakes that undermined the success of the project: (i) she failed to consult the district calendar and the project was broken up by vacation and testing; (ii) she did not give herself sufficient planning and collaboration time; and (iii) she did not consult students when choosing a topic and guiding question. Just as the one before, and many other testimonies that one may read online, this story informs the way one should understand and implement PBL. In this specific case, the teacher emphasizes the importance of scheduling, time management, topic selection, and collaboration in the design of PBL strategies.
On the students' side, PBL poses implementation challenges because students often find it difficult to collaborate, achieve efficient time management, and address complex issues. They may struggle and lose motivation when their research leads them to disappointing findings, or when they fail to obtain relevant data. Small group learning also poses the issue of equitable workload distribution. Even if she regularly meets with the different teams, the teacher has little control over who in each team is actually doing the work, and it is possible for some team members to slack and let the others do the work. This may lead to unfair assessment of some students by the teacher. Hence efficient cooperative learning poses the challenge of developing norms and structures within groups that enable students to work together and prevent free-riding (Barron and Darling-Hammond, 2008).
We anticipate that using project-based approaches to assess the impact of MIT BLOSSOMS on student learning will give rise to a number of criticisms. As mentioned above, one of these criticisms could be that PBL approaches are difficult to implement successfully because teachers are not properly trained to master inquiry-based teaching. While we recognize this reality, we recall that there is no silver bullet, i.e., no individual initiative will radically improve the education system. Rather, it is a collection of such initiatives, PBL and MIT BLOSSOMS being two of them. The fact that teachers are not sufficiently trained for innovative pedagogy should motivate national institutions to improve teacher training rather than to oppose PBL approaches.
Another criticism could be that a project-based assessment method for MIT BLOSSOMS is too qualitative to produce rigorous and unbiased performance measures, as opposed to standardized tests which are best suited for large-scale, objective evaluation of students. Here, we recall that it is very commonly accepted that student evaluations in humanities and social sciences are based on teachers reading and assessing papers. This is also true for architecture, and the arts. Later in life, adults in all fields are assessed based on their resumé which is also a form of portfolio evaluation. Why is it, then, that portfolio evaluation is not an option for STEM assessment in high school? We believe that this is partly due to inertia of the current system, as well as on a form of intellectual laziness. It is indeed easier and cheaper to have a machine scan and record the marks of standardized test pages than to have the teacher read and evaluate a set of ten-page reports.
Portfolio evaluation, however, is the only way to assess the student's ability to apply the factual knowledge learned in class. Learning by doing is critical in STEM, especially as science in the 21st century is an art as much as a technique: being good at differentiating complicated mathematical functions does not necessarily make you a better scientist, especially when these tasks are easily performed by computers. Contrary to common beliefs, building one's portfolio through project experiences is not a waste of in-class time; in fact, it is essential to college admission. Extracurricular STEM projects can be the main differentiating factor between applicants, while standardized testing only acts as a first cutoff. In addition, STEM projects provide an opportunity for students to work more closely with a teacher or professor, leading to more personal (hence stronger) reference letters for college applications. This is the reason why, as part of the ambitious system change to which Larson and Murray (2017) aspire, we believe that standardized testing should coexist with portfolio evaluation in some proportion. Implementing BLOSSOMS with a PBL follow-up experience could be an ideal way to get an edge into this new paradigm.
Our key idea is this: First, with their teacher as guide and mentor, the students experience in their classroom one BLOSSOMS lesson suitable for follow-on PBL activities. Second, the in-class BLOSSOMS lesson is followed by a three-week PBL assignment to the class. The BLOSSOMS lesson would comprise the “anchor component” of the three-week shared learning experience. Throughout the PBL after-class process, students will also be encouraged to leverage the diversity of online resources (academic and nonacademic literature, blogs, animated simulations) available on the BLOSSOMS course website. Evaluation of the students' follow-on PBL activities would now become the core method for assessing the impact and effectiveness of BLOSSOMS.
In this section, we take a concrete example in order to illustrate potential project-based assessment strategies for MIT BLOSSOMS. We believe that using a specific example to develop a methodological procedure will be more valuable than delving into abstract and theoretical considerations. We choose the BLOSSOMS video “Tragedy of the Commons” to build up our methodology. As explained in the teacher's guide to the video (available on the MIT BLOSSOMS website 2), the purpose of this lesson is to introduce students to the idea of the tragedy of commons, an extended metaphor for problems of shared environmental or man-made resources that can become overused and eventually depleted. Using some basic mathematics, students are inspired to think about this problem and contemplate potential solutions. The lesson also includes an animated simulation accessible online 3, for students to test the concepts both in class and at home. In the following sections, we will sequentially explore the major steps required for the design of a PBL-type assessment of “Tragedy of the Commons.” These steps can be generalized to many other BLOSSOMS lessons.
We first encourage teachers to think deeply about the lesson's objectives. What are the main concepts and skills that students should take away from it? This critical step will later inform the design of the project rubric. In our specific example, we identify the following objectives:
Acquisition of these three essential components should be central to the design of the projects, in addition to traditional objectives of PBL approaches including inquiry-based learning, critical thinking, collaboration, and communication.
The next step in project design involves choosing what type of projects are best suited to the objectives, in addition to delivery options. Note that several types of projects and deliveries could be appropriate. Project types include (but are not restricted to):
Similarly, project delivery methods include (but are not restricted to):
Our specific examples (below) of the tragedy of the commons are wide problems, involving numerous stakeholders in very complex environments. Acting as the teacher, we decide to require that students conduct a phenomenon case study and interview real-life players as part of their projects. We decide that this case study will lead to an oral presentation in front of the class, where each group will be subject to the critical assessment of the teacher as well as of their peers. Note that other delivery methods could also be possible. Once again, while our design choices are relevant to this specific problem, they are also somewhat arbitrary; we emphasize that the teacher will know what kind of design is best suited for her specific audience.
We recall that BLOSSOMS lessons are meant to inspire students with passion and excitement for STEM subjects. As a result, incentives should be at the heart of procedures to design project-based approaches to follow up on BLOSSOMS sessions. Incentivizing students can be achieved through several strategies:
While the third option may be less feasible in our case, as Tragedy of the Commons issues are highly complex, the teacher may leverage the two first strategies. She will collect ideas and then ask students to select the best propositions in order to generate project directions. Note that the teacher can ask teams of students to work on different projects depending on individual interests, or to work on one or several aspects of the same problems as a way for students to see how other teams performed at the same task.
Developing hackathon-type competitions requires more involvement from the teacher, as well as potential funding for prizes. But we emphasize that these approaches can be highly motivating for students. For example, students will defend their projects and solutions to overfishing in front of a jury composed of the teacher herself, fishing professionals, and parents, as well as other students. This jury will be in charge of critically assessing the group's solution and award a prize for the best team. These types of approaches have been successfully implemented for many years, for example, in the form of STEM hackathons (Gumina, 2017) or scientific Olympiads (Sahin et al., 2014).
In the following subsections, we demonstrate that the process of choosing exciting projects that are related to the student's daily lives, while requiring elementary preparatory research by the teacher, is fairly easy and natural. In particular, we create fictional cases for our Tragedy of the Commons example. We decided to spin a world globe and selected three locations at random, for which we will provide examples of follow-up projects for the students: Georgetown (Guyana), Melbourne (Australia), and Nairobi (Kenya).
Located on the Atlantic coast of South America, Guyana is a country with abundant mineral wealth, including significant gold, bauxite, diamonds, and manganese resources. Extractive industries are a major driving component of the country's economy, with a majority of highly dispersed artisanal, small, and medium-scale (ASM) miners. Mining poses several inherent challenges due to its negative impact on the environment and strong barriers to the efficient enforcement of existing regulations (Pasha et al., 2017). Indeed, ASM miners use simple extraction technology with poor levels of gold recovery. As a result, more material must be processed, implying more deforestation, more soil disruption, and worse water quality in rivers.
The teacher may assign projects investigating the sector of ASM mining in Guyana, its impact on the country's environmental commons, and potential mitigation policies. This is a particularly interesting and complex problem in the case of Guyana, for two reasons. First, although the mining sector is harming the environment, it is a major national economic strength, hence dismantling it is not an option, but rather implementing policies to promote more sustainable practices. Second, Guyana's commons include part of the Amazon forest, which is also a world's common that helps lower CO2 atmospheric concentration. Policy issues in the case of mining in Guyana are thus a matter of global governance, bringing in more complexity for the students to explore.
Students in Melbourne, Australia, will probably be familiar with the Murray-Darling basin, one of the most significant agricultural areas in Australia, including the state of Victoria. Its name is derived from its two major rivers, the Murray River and the Darling River. Over the years there has been an over-allocation of water entitlements in many areas of the basin, contributing to repeated droughts since the 2000's. Water resources have not been able to meet the water needed for environmental flows and human requirements. This is mainly a consequence of water entitlement policies that lacked consistency with the ways that water that flows across land can be intercepted (Young and McColl, 2009).
In this context, the teacher may assign projects related to the causes (farm dams development, plantation forestry, salinity interception schemes) and institutional failures (entitlement scheme) leading to water misallocation in the Murray-Darling basin, its consequence on the environment (including adverse climate change and bush fire) and human activities, and potential ways to mitigate the situation. Students will have to investigate a complex scientific as well as policy landscape, starting with the 2012 Murray-Darling Basin Plan to secure the long-term ecological health of the Murray-Darling Basin. Students wishing to work on other Australian-specific tragedy of the commons issues may look further into issues around the Great Barrier Reef and the Hunter Valley.
Traffic congestion is a key concern in Nairobi, Kenya, where it impacts economic activity and health with a related annual economic cost of USD 20 million (Gachanja, 2015). It is very likely that students in Nairobi would be aware of this situation, which is bound to escalate given a high population growth in the city and a continuous increase in the registration rate of new motor vehicles. Traffic congestion is a classic example of the tragedy of the commons: while individual agents try to minimize their travel time by taking the car instead of using public transportation, they create massive congestion leading to increased travel times as well as many adverse health effects.
Students will examine the traffic congestion issue from a tragedy of the commons perspective, identifying the causes for such individual behaviors, and their consequences on travel time and health. They will then be asked to review the existing literature on mitigating policies and to think deeply to come up with their own policies. The teacher could ask the students to discuss this situation with their parents or elder siblings. This will raise their awareness of the complexity of the problem, since penalizing car use may require upgrading of public transportation and may pose certain enforcement issues.
As an illustrative example in this Kenyan case, we provide a possible outline of a fictional student team's follow-up PBL paper. We expect teams of three to four students to work on their PBL paper during three weeks, and report progress to the teacher on a regular basis (once a week, for example). The output should be a ten-page report on the chosen topic, then leading to an oral presentation in class. The report should present and analyze the chosen instance of the tragedy of the commons, while building clear analogies with the BLOSSOMS video. Throughout the report, students are encouraged to be scientific and rigorous, citing references and building clear comments analyses. Simple mathematical models, data tables, graphs, pictures, and any other support materials are welcome in this exercise. While each individual teacher will know best what constitutes a good work in her specific context, we believe that a good report in this case will include:
In addition to the aforementioned examples, many instances of the tragedy of the commons can be found both at a local and worldwide scale, with relevance to many communities around the globe. The point here is that it will be fairly easy for the local teacher to build a project-based evaluation of BLOSSOMS that addresses issues that are relevant to the students' daily lives and pose interesting scientific questions. Some of these other instances include:
A rubric should be created before the project starts and provide clear criteria that define the extent to which a team meets expectations in terms of learning, cooperation, collaboration, presentation, content completeness, language, visual appeal, and marketing. In our application, a rubric is well described by this Merriam-Webster definition: “a guide listing specific criteria for grading or scoring academic papers, projects, or tests.” 4 A PBL rubric is essential to the teacher (for grading) and to the students, as it will guide their free exploration of the topic under investigation. The rubric can be codeveloped by the students, and the assessor can be the team itself (self-assessment), peers, the teacher, an administrator, or an external stakeholder. Rubric creation is among the most complex steps of the PBL approach. While common components can be shared across rubrics, a rubric should still be relatively specific to the BLOSSOMS session at hand. Table 1 outlines a potential starting chart for creating a rubric.
TABLE 1: Rubric Canvas for PBL Assessment
| Skill | Fail | Fair | Good | Assessment |
| Factual Learning | Key mistakes | Some understanding | Good understanding | Written report, presentation, peer questions |
| Topic Relevance | Irrelevant | Somewhat relevant | Relevant | Written report, presentation |
| Team Work | Clear imbalance | Siloed work | Active collaboration | Project monitoring, oral presentation |
| Presentation | Distracts audience | Supports content and purposes | Use of technology and multimedia | Oral presentation |
| Communication | Poor speech and body language | Poor speech or body language | Good speech and body language | Oral presentation |
| Organization | No planning | Some planning but not optimal | Well-planned | Oral presentation |
Project assessment is another critical aspect of the application of PBL to MIT BLOSSOMS. Depending on the type of project, assessment will take different shapes and involve different stakeholders (this is especially the case in hackathon-type projects, where the jury should involve various relevant players). In a more traditional classroom setting, assessment relies on the predefined rubric and involves as many as four components:
Student project assessment is the first step in a two-step procedure to assess MIT BLOSSOMS as a program. We intend to poll local teachers to determine whether they think BLOSSOMS enabled a deeper and better learning compared to the more traditional teaching style. Teachers and students' feedback will be the main resources used by BLOSSOMS staff to assess the program. The output may be an assessment report involving the qualitative evaluation of BLOSSOMS from individual comments and BLOSSOMS staff's observations, as well as basic statistics such as, for example, the fraction of projects showing a deeper understanding of class material than usually expected with traditional a teaching style.
We propose that BLOSSOMS staff interview teachers from specific BLOSSOMS pilot programs (which could be both in US and international schools) and ask them to complete questionnaires indicating the impact of BLOSSOMS on their students. These teachers will also be requested to communicate with BLOSSOMS any truly outstanding student projects, which would indicate a high efficacy of BLOSSOMS in the students' learning experience. Questionnaires will also be given to the students (or interviews, when this is feasible), with the clear message that these are not tests but rather surveys that will be used to better understand and improve their educational experience. Student responses, while probably less informative than teachers' comments, will provide additional insights on student learning as well as the strengths and weaknesses of the program.
We now get closer to an operational assessment procedure by providing examples of questionnaires to be sent to teachers and students, in the specific context of tragedy of the commons.
Teachers Questionnaire – BLOSSOMS Tragedy of the Commons
Teaching – What was your opinion on MIT BLOSSOMS and active-, phenomena-based learning before implementing this BLOSSOMS session? Did you consider it positively or negatively? Has this experience changed your mind, and for what reasons? – What are the main challenges that you encountered (these may include project design, class management, project direction, among others)? – Were the resources available on the BLOSSOMS platform enough to overcome these challenges? If yes, which BLOSSOMS resources did you use? If not, which other resources did you resort to? – Will you continue working with MIT BLOSSOMS and share it with your colleagues? Would you have any advice or recommendation to improve MIT BLOSSOMS? Factual Learning – Were the follow-up projects well connected to the tragedy of the commons? What fraction of students went off track and were not able to correctly understand what the tragedy of the commons was, or to relate the metaphor presented in the video to actual real-world issues? – Did students use their skills and knowledge from class to shed light on the issue they were addressing? For example, did they build a simple mathematical interpretation of the commons problem under investigation? – Overall, did students demonstrate through their projects a good command of the tragedy of the commons and related topics (such as mitigating policies)? Did they demonstrate a better knowledge compared to students taught using the standard lecture format? Teamwork and Organization – Were students able to work together efficiently? How did you create an atmosphere that is conducive to collaboration? Did collaboration play a significant role in student learning? – What are the main challenges that students encountered in collaboration? Was free-riding a problem? How were you able to control for this type of behavior? – Did students plan ahead and meet schedule expectations? If not, what were the main reasons for them not respecting the schedule? How did they react to potential re-directions as the project unfolded? Final Production and Communication – Was the final report correct, clear, and concise? Did it respect the rules set at the beginning of the project? Did students use or build relevant visualizations? Did students properly cite references? – Was the final presentation correct, clear, and concise? Did it respect the rules set at the beginning of the project? Did it use technology and multimedia? Was speech evenly distributed among team members? Were students able to answer their peers' questions? – Did the students achieve a good communication strategy? Were they able to successfully present their projects in written and oral form? Did they demonstrate proper body language? Miscellaneous – How do you rate your overall teaching experience? Do you consider it pleasant and effective? Do you prefer this teaching style to the more traditional lecture teaching style? – Were there any outstanding student projects that you would like to report to BLOSSOMS? Do you have any further comments or advice for BLOSSOMS? |
Students will be given a similar questionnaire, where questions will be directly targeting them and their experience rather than the teacher's appreciation of their experience. We do not provide a sample for the students' questionnaire in this paper for concision purposes.
We are hopeful that project-based assessment of BLOSSOMS will not only help evaluate the program's impact on student learning; it should also contribute to reshape the way BLOSSOMS has been designed and operated for ten years. In the current state, we estimate that about 15% of existing BLOSSOMS video lessons are compatible with the PBL approach. This percentage is not homogeneous among all disciplines and trying to shoehorn PBL into lessons that are not appropriate is not a feasible option. PBL-compatibility will thus constitute an important criterion for the creation of future BLOSSOMS contents. Specifically, content creators will need to design their lesson so that high school students are able to delve further into the concepts through follow-up projects. These new contents should be conducive to active-, inquiry-based learning so as to broaden the program's scope and make it a broader tool for deeper learning.
This novel PBL-compatibility requirement raises the issue of defining a list of attributes for BLOSSOMS contents that make them good candidates for follow-up PBL. Note that these attributes come in addition to the attributes that are already required for the creation of high-quality BLOSSOMS contents. Authors and creators submitting new ideas to the platform will need to argue that these ideas include such or similar attributes. A tentative list of these attributes is proposed below:
Uploading exemplary projects will become a requirement for BLOSSOMS contents generators. As it has previously been mentioned, each BLOSSOMS lesson comes in with supplementary resources, including simulations, teacher's guidelines, and external references. A new “Sample Projects” category will display a few (three to five) fully crafted projects related to the concepts at hand. Each project will include the actual project statement and a written report, as well as a video with an oral presentation of the project. It will also come with a teacher's guide, providing the teacher with assistance in terms of class management and pointing out potential challenges during the course of the projects.
While these exemplary projects will be fully provided, we emphasize that teachers should not use these projects off the shelf, but simply use them as examples to build their own project statements based on the local specificities of the place where they teach. For example, we could imagine providing as exemplary projects “Mining in Guyana,” “Water Usage in the Murray-Darling Basin, Australia,” and “Traffic Congestion in Nairobi.” While these make sense in their respective locations, it would not be optimal to have students in Japan work on such themes, while these students might be more interested in studying tuna over-fishing in the Pacific. It is critical that students feel connected to the issues they will address, or else BLOSSOMS might lose its beneficial impact on learning. Our view is that teachers should carefully review the exemplary project materials, then use their structure to build their own projects and inspire their students. Teachers will be encouraged to notify BLOSSOMS in the case that they would like to publish online a remarkable project of their invention.
Our presentation has not emphasized online education in the traditional sense. But the students' and teachers' use of the Internet in PBL activities, anchored by BLOSSOMS lessons, can be significant. First, the BLOSSOMS repository of lessons is Internet based, an OER project that requires at least the teacher to access the lesson via the Internet and to download it for in-class presentation. Second, several BLOSSOMS lessons have accompanying animated simulations that students explore using their own computers. We expect most future PBL-compatible BLOSSOMS lessons to have such animated simulations. Third, each BLOSSOMS lesson has a rich set of freely available online supplementary resources, online knowledge assets that can be very valuable in any three-week PBL assignment. Students can thus connect to BLOSSOMS from home and investigate with simulations, experience the supplementary materials, and even view the video lesson again if they have missed something! Fourth and perhaps most exciting, we have near-term plans to add to the BLOSSOMS website student collaborative learning capabilities, so students, say, in the US could simultaneously explore tragedy of-the commons issues with students in India, Nigeria, Malaysia, and other countries. Online learning can take many forms with MIT BLOSSOMS!
This MIT-based research was funded in part by NTT Com Asia Ltd., Trumptech (Hong Kong) Ltd. Limited, and the Hong Kong Innovation and Technology Fund (ITS/212/16FP) under the HKUST-MIT Research Alliance Consortium. It was also funded in part by the Open Education Resources Foundation Inc. of Florida, Linc Miller – Chairman. Findings and opinions expressed herein are those of the authors and do not necessarily represent those of the sponsors.
Abdullah, Z. and Shukor, N. (2017). Challenges in Integrating BLOSSOMS in Malaysia's STEM Education System. Syst. Res. Behav. Science, 34(3): 304–306.
Achieve, Next Gen Science Storylines, S.T.T. (2016). Using Phenomena in NGSS-Designed Lessons and Units. Technical report.
Atkinson, R. and Mayo, M. (2010). Refueling the US innovation economy: Fresh approaches to science, technology, engineering and mathematics (STEM) education. The Information Technology & Innovation Foundation (to be published).
Barron, B. and Darling-Hammond, L. (2008). Teaching for Meaningful Learning: A Review of Research on Inquiry-Based and Cooperative Learning. Book Excerpt. San Rafael, CA: George Lucas Educational Foundation.
Bransford, J.D., B. A. C. R. (1999). How People Learn: Brain, Mind, Experience and School. 27
Capraro, R.M. and Slough, S.W. (2013). Why PBL? Why STEM? Why now? An Introduction to STEM Project-based Learning. In STEM Project-Based Learning, pp. 1–5. Berlin: Springer.
DeCoito, I. (2014). Focusing on Science, Technology, Engineering, and Mathematics (STEM) in the 21st Century. Ont. Prof. Surv., 57(1): 34–36.
Gachanja, J. (2015). Mitigating Road Traffic Congestion in the Nairobi Metropolitan Region. The Kenya Institute for Public Policy Research and Analysis (KIPPRA) Policy Brief, (2).
Gumina, M.J. (2017). Using Hackathons as a Tool in STEM Education. PhD Thesis – California State University, East Bay.
Herold, B. (2015). Why Ed Tech Is Not Transforming How Teachers Teach. Retrieved July 5, 2018, from https://www.edweek.org.
Kalantari, M. (2015). My Experience Making a BLOSSOMS Video. OR/MS Today, 42(5).
Larson, R.C. (2014). Dr Larson: "STEM is for everyone". Retrieved July 5, 2018, from http://www.wise-qatar.org/richard-larson-stem-mit-blossoms.
Larson, R.C. (2015). MIT BLOSSOMS – Five Years Later. OR/MS Today, 42(2).
Larson, R. and Murray, E. (2008). The MIT BLOSSOMS Initiative: Employing a Blended Learning Approach with Appropriate Technologies to Encourage OER Usage and Creation in Developing Countries. COSL Center for Open Sustainable Learning. Open Education, pp. 24–26.
Larson, R.C. and Murray, M.E. (2017). STEM Education: Inferring Promising Systems Changes from Experiences with MIT BLOSSOMS. Systems Research and Behavioral Science, 34(3): 289–303.
Malik, N.A. (2017). Challenges to High School STEM Education in Pakistan. Syst. Res. Behav. Science, 34(3): 307–309.
Maxwell, J. A. (2004). Causal Explanation, Qualitative Research, and Scientific Inquiry in Education. Educ. Res., 33(2): 3–11.
National Research Council et al. (2013). Monitoring Progress toward Successful K–12 STEM Education: A Nation Advancing? Washington, DC: National Academies Press.
National Science Board (2016). Science and Engineering Indicators 2016. Report.
Pasha, S., Wenner, M.D., and Clarke, D. (2017). Toward the Greening of the Gold Mining Sector of Guyana: Transition Issues and Challenges. Technical Report, Inter-American Development Bank.
Sahin, A., Ayar, M.C., and Adiguzel, T. (2014). STEM Related After-School Program Activities and Associated Outcomes on Student Learning. Educ. Sci.: Theory Pract., 14(1): 309–322.
Sinay, E.N.A. (2016). Teaching and Learning Mathematics Research Series I: Effective Instructional Strategies. Technical Report, Toronto, Ontario, Canada: Toronto District School Board.
Spear, K. (2015). My PBL Failure: 4 Tips for Planning Successful PBL. Retrieved July 5, 2018, from https://www.edutopia.org/.
Spencer, J. (2018). How Do You Teach to the Standards When Doing Project-Based Learning? Retrieved July 5, 2018, from https://medium.com/synapse.
Speziale, K. (2017). Study Confirms Project-Based Learning Has a Positive Impact on How Students Learn Science and Math. Retrieved July 5, 2018, from https://blog.definedstem.com/project-based-learning-research/.
Thomas, J.W. (2000). A Review of Research on Project-Based Learning. Novato, CA: Buck Institute.
Xue, Y. and Larson, R.C. (2015). STEM Crisis or STEM Surplus: Yes and Yes. Mon. Lab. Rev., 138: 1.
Yang, R., Zeiser, K.L. and Siman, N. (2016). Deeper Learning and College Enrollment: What Happens After High School? Technical Report. Washington, DC: American Institutes for Research.
Young, M.D. and McColl, J.C. (2009). Double Trouble: The Importance of Accounting for and Defining Water Entitlements Consistent with Hydrological Realities. Aust. J. Agric. Resour. Econ., 53(1): 19–35.
Zeiser, K.L., Taylor, J., Rickles, J., Garet, M.S., and Segeritz, M. (2014). Evidence of Deeper Learning Outcomes. Findings from the Study of Deeper Learning Opportunities and Outcomes: Report 3. Washington, DC: American Institutes for Research.
Bakhmat L, Babakina O, Belmaz Ya, Assessing online education during the COVID-19 pandemic: a survey of lecturers in Ukraine, Journal of Physics: Conference Series, 1840, 1, 2021 Crossref
Change Subscription
Information
Server Error