1 University of Hawaiʻi at Mānoa, Honolulu, Hawaii, USA
2 Leeward Community College, Pearl City, Hawaii, USA
* Address all correspondence to: Rich Gazan, University of Hawaiʻi at Mānoa, 2550 McCarthy Mall Hamilton Library 002, Honolulu, HI, USA; Phone: (+1) 808-956-7321, Fax: (+1) 808-956-5835, E-mail: email@example.com
How are nontraditional, participative resources such as social media content being integrated into online STEM education, and how can libraries best support these fast-changing practices? Using conceptual content analysis of information literacy standards, a survey of undergraduate students' use of supplemental learning resources, and participant observation of librarian-led information literacy instruction sessions, we investigate the extent to which these less-visible actors enable, resist, and extend the incorporation of participative resources into STEM education. Our preliminary findings suggest that information literacy standards are evolving to encompass participative resources and practices, that students actively incorporate formal and informal resources into their learning, and that participative dimensions are incorporated into information literacy instruction sessions, though they may not be formally documented. We conclude with a discussion of the implications of the apparently uneven translation and inscription of information literacy standards into practice and how actors outside the classroom impact STEM education in a social media environment.
KEY WORDS: information literacy, collaborative learning
Turnitin, an antiplagiarism service, estimates that of over 40 million student papers it analyzed in a 2011 study, 33% contained references to social media and similar user-generated content sites (Turnitin, 2011). While undergraduate instructors and librarians might view these statistics with a mixture of recognition and resignation, social media resources can sometimes support and even surpass traditionally authoritative resources in terms of quality metrics such as currency, community vetting (or its algorithmic equivalent), and their positionality as primary source material (Coleman, 2013). In this paper, we investigate the extent to which students are deriving tangible educational value from the nontraditional, participative resources they use and how this use is enabled or constrained by actors outside the classroom. We hope to contribute a method and analytical lens through which the integration of nontraditional resources into STEM education can be better understood.
Information literacy defies a simple, comprehensive definition, but the American Library Association (1989) offers a useful distillation: information-literate individuals can “recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information” (American Library Association, 1989). All three aspects—location, evaluation, and application—are central to science, technology, engineering, and mathematics (STEM) fields and are inherently participative and multivocal. Multiple sources, methods, analyses, findings, and interpretations are debated and tested until an always-tentative consensus is reached.
As with all forms of infrastructure, information literacy standards exist socially and technologically, and they both enable and restrict certain interpretations and actions, usually invisibly (Star and Griesemer, 1989). Information literacy standards and frameworks are built atop multiple installed bases such as the scholarly publishing infrastructure, traditional notions of expertise and authority, the student as an individual learner, and assessment culture, to name a few. Digital and social media resources challenge these and other aspects of infrastructure, yielding areas of resistance for multiple actors. Instructors who incorporate nontraditional, participative resources and practices risk not adhering to professional standards, not meeting departmental learning outcomes, and not meeting codified expectations of accrediting bodies. Students who defy instructor expectations risk poor grades. We are therefore interested in the role of both human and nonhuman actors outside the classroom—in this case, information literacy standards, learning resources outside the syllabus used by students, and librarian-led instruction sessions—in incorporating participative social media resources and practices into STEM education. Juxtaposing top-down standards with bottom-up practices can reveal assumptions and disconnects in the middle layers of education, such as syllabi, curricula, and student learning outcomes, and help provide a rationale from both directions to incorporate change.
As a learning theory, connectivism is focused on access pathways and relationships more than on individuals as repositories of transferred knowledge. Connectivism builds on some concepts related to constructivism and actor-network theory—for example the equivalent roles of human, nonhuman, and collaborative (nonindividual) actors and how their interactions reflect institutional social patterns (Latour, 1996; Law, 1992). From a connectivist perspective, resources are understood to be dynamic—not just when the resources themselves change, but when people and situations accessing them change as well (Siemens, 2005).
Within and beyond STEM fields, the idealized model of solo researchers searching library databases and using only published, traditionally authoritative works to ground their scholarship fails to reflect the innate messiness of actual science practice (Hagund and Solsson, 2008; Nicholars et al., 2017). A study of the search and current awareness practices of early career researchers (doctoral students and recent PhDs) by Ince et al. (2018) found that they employ a “pastiche” of tools and workflows to locate and discover research-related materials, with scholarly databases being only one component of a much broader information environment, including social media, personal networks, forum and Q&A sites, social recommendation and annotation tools, and even pirate sites, allowing researchers to access publications unavailable through their home institutions.
Particularly relevant here is the social aspect of scholarly practice. The early career researchers studied by Ince et al. report that their inquiries often began with a targeted or diffuse request on social media for recommendations about people or resources to consult, and the resulting personal connections and informal conversations helped them access unpublished or in-process works, sometimes resulting in collaboration opportunities that would not have happened otherwise. More indirectly, they use social recommendations such as those built into ranking algorithms like Google Scholar citations. The authors conclude that their subjects' research practices are socially embedded, are process-driven rather than resource-driven, and “tend to emphasize tools that are not traditional library databases… Scholars were able to circumvent the library if it proved inconvenient” (p. 247).
Some researchers identified the social nature of information literacy well before the social web became the force it is today. For example, Tuominen et al. (2005) adopted a constructionist (Hyslop-Margison and Strobel, 2007), collaborative approach to knowledge creation based on the concept of epistemic communities (Knorr Cetina, 1999) and conceptualized information literacy not as a metric of how well individuals can find and evaluate germane facts but as a sociotechnical practice where facts and technologies are in constant interplay and coconstruction. While their approach is not as relationship-centric as connectivism, The main point of Tuominen et al. is that information literacy standards should reflect this embodied knowledge of practice (Gee, 2003).
McLoughlin and Lee (2007) reviewed the pedagogical affordances of user-generated content systems and identified both collaborative content creation and vetting of resources inside and outside the classroom as critical elements. During the “Web 2.0” period of the mid- to late 2000s, the pedagogical affordances of digital and social media yielded refinements of the learner-teacher-content triad, and constructs such as formal and informal groups, policies and standards, and interactivity itself were identified and proposed as objects of study enabled by digital and social media (Anderson, 2008).
This is not to say that participative media have always been embraced when implemented in the classroom—for example, some students tasked with posting on a Wiki reported extreme frustration when their work was changed or removed by others (Wheeler et al., 2008). Friesen and Lowe (2012) question the usefulness of social media resources in education, claiming that their silo architecture results in a failure to “foster the capacity for debate and disagreement.”
Still, researchers such as Yu et al. (2010) suggest that online social networking sites are so deeply socially embedded that their use in university coursework supports a range of benefits such as social learning, student engagement and adapting to university culture, all of which support improved learning outcomes.
STEM engagement and education can occur in informal, nontraditional environments such as makerspaces, game jams, and hackathons (Briscoe and Mulligan, 2014). The common factor among these and similar environments—and what distinguishes them from lecture-based classroom environments—is their interactive nature, where participants create, not just consume.
Conversely, the social component of STEM engagement tends to be underemphasized, if not actively discouraged, within traditional educational environments where individual mastery is inscribed in student learning outcomes and teaching practices. For example, Asselin et al. (2011) suggest that YouTube videos can be important supplemental learning resources in formal and informal environments, extending traditional notions of information literacy. They conducted a preliminary discourse analysis of user-generated YouTube content and student participation via comments and report that students “engage with and make meaning from user-generated video to support their learning” (p. 640). The authors also point out that while students regularly consult YouTube as part of their practice, some schools and libraries actively restrict or block access to it.
Participative resources and practices have been associated with the success, retention, and persistence of postsecondary students in STEM education, including cocurricular activities, peer support, and mentoring through learning communities (CIRTL, 2018; Detlor et al., 2010; Estrada, 2014; Olson and Riordan, 2012; Ramnarine-Rieks, 2013; Schultz et al., 2011). Learning communities have been associated with positive STEM outcomes (Packard, 2012; Pfund et al., 2012). Freeman et al. (2008) explored the role of motivation and collaboration in STEM learning and concluded that learning communities positively influence college students' attitudes, motivation, and learning experiences in STEM-related courses. Learning communities, which cocurricular programs sometimes provide, can become gathering places that “enable students to work with and learn from each other” (Graham et al., 2013, p. 1126).
Kumar et al. (2018) studied learner-participants on Reddit and propose a coding schema designed to capture “the social, conversational, and collaborative elements (all defining features) of informal, online learning environments” (p. 1941). Their results suggest that when compared with communities in politics, academia, and history, the most STEM-focused community (askScience) had the highest percentage of contributions coded as “explanation with neutral presentation” and the lowest percentage of those coded as social interactions, though the authors emphasize that community rules and norms on the askScience subreddit may be a factor.
There is also evidence that social peer support through learning communities is of particular benefit to underrepresented student populations. Sithole et al. (2017), for example, investigated students' attraction, persistence, and retention within STEM fields and suggest that learning communities may lessen the dropout rates of women and minorities. Fayon et al. (2010) integrated learning communities into science-related courses to address the “STEM achievement gap” (p. 15) in the English language learner (ELL) population and found beneficial differences in attitude and beliefs for ELL students and an increased level of academic engagement. Their findings were measured using the CLASS instrument (Adams et al., 2004), which was initially designed to measure and assess student attitudes toward learning in physics but has since been adapted to many STEM fields.
Strong social interest in increasing both the number and diversity of information-literate students, particularly in STEM fields, requires that students gain a baseline of understanding of how to access and evaluate information resources and the processes of scientific inquiry and knowledge creation. Information literacy education has been explicitly identified as a critical foundational component of STEM education (Berendt, 2012; Jang, 2016).
Information literacy standards emerge from a combination of enacted best practices and desired outcomes, are shared via research and professional literature, and are integrated and distilled into standards by professional organizations such as the Association of College and Research Libraries (ACRL). Standards are shared back with the community, then adapted into learning outcomes and implemented in specific educational environments. Data from the resulting local implementations yields population-based best practices and literature as input to the next iteration of the standards.
At the K-12 level, the Next Generation Science Standards: For States, By States (NGSS, 2013) are primarily focused on the skills students need to demonstrate within the boundaries of established fields of science. The standards include “crosscutting concepts” such as pattern recognition that occur in multiple fields, but they make no specific mention of information literacy as a baseline of understanding. The point here is not to critique the standards for failing to incorporate information literacy, but to point out areas where the language and/or the spirit of the standards are sufficiently flexible that they might encompass the kinds of sources and practices from the social web that people actually employ. Though these interpretations may be unstated within the standards, they harmonize well with the high-level goals of the standards. For example, the eight practices of science and engineering that the framework identifies as essential are (p. 382):
This list of practices was created to evaluate individual students, but even with that perspective, the social aspects of science are either missing or highly implicit. As current literature suggests, most of these aims for K-12 science education fall squarely within science researchers' actual practices—in other words, they are accurate—but as Tuominen et al. (2005) suggest, these practices do not only take place within the confines of published literature. Asking questions, defining problems, planning and carrying out investigations, constructing explanations, engaging in argument from evidence, and communicating results are all now practiced to a lesser or greater extent within a social media environment. In addition, combining the practice of designing engineering solutions with the environment of the social web results in peer production, such as open-source communities and citizen science.
Understandably, the indicators and metrics of successful outcomes of information literacy instruction have tended to favor the traditional resources and services offered by academic libraries. For example, Detlor et al. (2010) operationalized information literacy success in terms of variables such as increased self-efficacy using online library resources, improved perceptions of librarians as being more helpful and valuable, and improved perceptions of the library. While these are important and valid indicators, it does raise the question of whether resources discovered outside the library, without the assistance of librarians, might indicate a different dimension of information literacy.
Pfund et al. (2012) define learning communities as an educational approach of conjoining groups of individuals “for shared learning and the discovery and generation of knowledge,” and learning communities may occur in both formal and informal spaces, such as online. Gruoping Ma [as cited in Domínguez-Flores and Wang (2011)] states that “an online learning community is a group of people who meet online and communicate via communication networks, sharing common interests and goals, engaging in knowledge-related transactions, and supporting each other in their learning agendas.”
Learning communities encourage the transformation of the learning process in STEM education. For example, Freeman et al. (2008) explored the role of motivation and collaboration in STEM learning and concluded that learning communities positively influence students' attitudes, motivation, and learning experiences in STEM-related courses at the collegiate level. Online learning communities are becoming more popular with the convenience of technology. Graham et al. (2013) suggest that it is critical to provide “STEM learning communities,” which can be virtual or physical in structure. This community can become a gathering place that “enables students to work with and learn from each other” and can improve the academic culture for all students.
Online learning communities promote web-based learning interaction and provide the opportunity for online peer feedback, which can benefit both providers and receivers (Van Popta et al., 2017). The results of their research indicated that the online peer feedback provider develops higher-level learning skills, such as those identified by Bloom's Taxonomy (revised): analyzing, evaluating, and creating (Chuang, 2015). They identified a need for further research on the effects of the feedback provider, specifically “since providing online peer feedback is part of a collaborative process between at least two people within a larger group of learners, a social perspective would be relevant for the learning benefits and conditions for learning.”
Online learning communities can result in more positive outcomes in terms of student attitude, engagement, and academic achievement in STEM. Domínguez-Flores and Wang (2011) found a direct connection between the incorporation of an online learning community and the enhancement of information literacy skills acquisition, and they report that students were very satisfied with their experience.
Mackey and Jacobson (2011) describe a need to redefine and reframe information literacy as metaliteracy in order to unify multiple literacy types that have emerged as a result of evolving technologies, social media environments, and collaborative online communities. They argue that “producing and sharing information are significant activities for lifelong learning in social media environments and online communities.” Burgoyne and Chuppa-Cornell (2015) developed an embedded librarian model which evolved into an integrated learning community. The community college English course was paired with an online information literacy course and resulted in higher grades and other metrics of student success, indicating the positive impacts of embedded librarianship under the learning community model.
This brief review suggests that both STEM education and information literacy standards and practices are beginning to reflect more participative, dynamic approaches. Nontraditional actors such as standards-making bodies, librarians providing information literacy instruction to students in diverse disciplines, and unknown others authoring user-generated web content that students access and interact with all have a role to play in STEM learning and are the focus of the present study.
Our overall project is conducted as an iterative participatory design, where instructors, librarians, and students contribute their views and practices about information literacy and participative content to help create an evolving set of data collection instruments and analytical frames (Selvin et al., 2010). In this paper, we take a three-stage approach to data collection and analysis:
To address the extent to which information literacy standards and practices might be interpreted to support participative resources and practices, we begin with a conceptual content analysis of the information literacy standards specific to STEM, summarize preliminary findings of a survey of undergraduate students' use of participative learning resources, and conclude with a report on our observations of two information literacy instruction sessions.
The ACRL offers a range of information literacy standards and frameworks focused on specific fields, including journalism, nursing, and anthropology. These standards serve as one of the primary benchmarks for information literacy instruction in academic libraries. Relevant here are the ACRL Information Literacy Standards for Science and Engineering/Technology (2006), which at this writing remains the most recent version related to STEM. While these standards were published over a decade ago and are based on previous standards dated as early as 2001, and they understandably reflect the largely pre–social media era in which they were developed, in our previous work we found that of 42 educational institutions studied, just 13 explicitly aligned their information literacy instruction with newer, more explicitly participative standards and frameworks.
Standard One: The information-literate student determines the nature and extent of the information needed.
This wording gives students agency to determine for themselves what they need. For example, if a student determines that the nature of the information they need at a particular point in their search process consists of information exchanges via social networks or informal contacts, they are addressing the standard.
Standard Two: The information-literate student acquires needed information effectively and efficiently.
While it is usually true that five minutes spent with a reference librarian or a web tutorial can save hours of effort and frustration, it is equally true that information literacy instruction, and library use more generally, is designed more for effectiveness than efficiency. It is always quicker and easier to use one's phone than travel physically to a library, or to use a downloadable paper that is good enough instead of waiting for the perfect print volume to arrive via interlibrary loan. The rationale of the library reference interview is built on the idea that two-way communication is essential to appropriately represent a person's information need; students using interactive, n-way communication channels such as social media posts or direct messages to connected individuals can often propagate a query and return responses very effectively and efficiently.
Standard Three: The information-literate student critically evaluates the procured information and its sources, and as a result, decides whether or not to modify the initial query and/or seek additional sources and whether to develop a new research process.
Students working for possibly the first time with resources provided by an instructor or university library might reasonably conclude that an evaluative step has already been done for them, and understanding how and why particular results appear on the screen as a result of a search is a mystery even to professionals, due to the proprietary nature of most relevance-ranking algorithms. Students may critically evaluate informally procured information by relying on aggregate indicators of quality and trustworthiness that are often less opaque than those in professional search engines.
Standard Four: The information-literate student understands the economic, ethical, legal, and social issues surrounding the use of information and its technologies and, either as an individual or as a member of a group, uses information effectively, ethically, and legally to accomplish a specific purpose.
Before they ever enter an academic library, students may have already developed a personal understanding of issues related to information use—for example, deciding whether to download and reuse a copyrighted image, to rip or otherwise acquire a song without paying for it, or to use file sharing services to view pirated movies. For example, a student who holds the opinion that the academic publishing model is itself unethical might demonstrate the understanding called for in Standard Four by consciously bypassing subscriptions and paywalls to access resources, perhaps via open access or direct exchange at the edge of legality and/or via unauthorized repositories that lie clearly beyond it.
Standard Five: The information-literate student understands that information literacy is an ongoing process and an important component of lifelong learning and recognizes the need to keep current regarding new developments in his or her field.
Actively adopting new tools and workflows, such as integrating social media platforms into the research process, meets this standard. In addition, the latency period between conducting research, data analysis, write-up, editorial review, and journal publication has long been an obstacle to timely scholarly communication. Keeping current with new developments prior to publication via social networks, informal resources, and preprint servers directly addresses this standard.
The results of this phase of our analysis suggest that information literacy standards, even those authored near the beginning of the social media era, can be interpreted to encompass nontraditional, participative resources.
The preliminary results of the survey of introductory computer science students indicate that over 40% of students studied report using social and participative resources outside the syllabus to supplement their learning. Students were asked:
In addition to the instructor/TAs and the assigned resources on the syllabus, which of the following alternative resources have been most helpful to you in learning course material? (select all that apply):
Additionally, 37.5% reported using an alternative resource in a graded assignment. Students emphasized both social/participative and traditional forms of authority signaling when asked: Why did you find these alternative resources trustworthy?
Responses in the Other category included STEM-relevant assessments such as triangulation—for example, “sounded like something we talk about in class, close to my notes,” and “usually cross referenced with other pages.”
While traditionally authoritative resources and quality indicators were commonly reported here as well, the preliminary results suggest that these students access, apply, and can discern quality indicators of social and participative resources as a supplement to assigned course learning resources. These results generally support the resource use distribution suggested by the Turnitin (2011) study, but our survey added the condition that students assess which alternative resources they found helpful in learning course material, where the Turnitin data analyzed references and passages without regard to the student's perception of their helpfulness. The results also suggest that students are most likely to search on their own for supplemental content but will commonly engage with online participative resources, apart from instructors, teaching assistants, and librarians.
While academic librarians might find the low percentage of students who report accessing library services as more than a little troubling, the fact that they value nonsocial web resources most highly may suggest an opportunity for librarians to more tightly integrate the portals and LibGuides and information literacy instruction they already create into course syllabi, the latter of which is the focus of the third phase of our study.
One of the two-year institutions coded in our previous work as taking a more traditional, nonparticipative approach to information literacy based on their publicly available documentation is Leeward Community College, part of the University of Hawaii system and a partner in this project. We conducted a participant observation of two information literacy instruction sessions at Leeward (n = 30 students across the two sessions) and discovered that a much higher level of participative resources and practices was integrated into the sessions than the public documentation indicated.
Information literacy is a general learning outcome all students at Leeward Community College must demonstrate, in part by passing an information literacy exam while taking the English 100 course. Library sessions are offered to each section of an introductory English course and range from brief library overviews to one- or two-session research challenge or jigsaw activities (Aronson et al., 1978). Longer library sessions are led by a librarian and include time for applied, hands-on research.
The research challenge consists of an overview of library resources, followed by demos utilizing traditional library resources such as the catalog and research databases, but also less traditional sources of information such as websites and interviews. The thread that ties all of the resources together is a modified CRAAP (currency, relevance, authority, accuracy, purpose) test method of evaluation (Blakeslee, 2004), and students actively rate the resources as they relate to their research challenge topic. Traditional and nontraditional information sources alike can pass the CRAAP test. For example, local sources of information from newspapers and websites are presented alongside scholarly articles, and students actively evaluate the perspective and purpose of each. Incorporating a range of sources is recommended for exploring a topic holistically, and the final product of the research challenge is an annotated bibliography that challenges students to add their own summary and reflection to each piece of information.
At Leeward, the instruction sessions are interactive and engaging. A librarian leads students through collaborative research exercises that explore a range of resources. In the last year, jigsaw activities, which feature the same types of resources and stress evaluation of diverse sources of information, have gained popularity among faculty and students. Jigsaw activities involve creating small groups and allowing time for each group to become experts in one area of research, guided by worksheets and a librarian. Next, groups are mixed so that there are experts from each area represented, then the student experts teach each other how to access resources and find and evaluate resources. This is a more social and active approach to library instruction and demonstrates an integration of participative standards, resources, and practices that we could not observe in documentation alone.
Students were challenged throughout these sessions to consider more refined senses of relevance. For example, when information on local job market conditions was sought in the session, a discussion board of job seekers and recent graduates in the area was identified as a more pertinent resource than a government site offering regional employment statistics. Though many of these students will not pursue a STEM degree path, evaluating and applying participative resources and practices alongside more traditionally authoritative resources both meets the spirit of STEM learning standards and reflects actual researcher practices.
While the STEM-specific ACRL standards have not been updated since 2006, the broader ACRL information literacy framework for higher education was updated more recently (ACRL, 2015) and includes statements such as:
Learners who are developing their information-literate abilities:
The 2015 ACRL standards framework is targeted primarily at undergraduate students, but the foundation for a shift toward participative resources and practices is being constructed even earlier in the educational process. Within the American Association for School Libraries Standards Framework for Learners (AASL, 2018), two of the four “Domains and Competencies” are to share and to create, and one of the three crosscutting “Shared Foundations and Key Commitments” is to collaborate.
Our findings suggest that in the conceptual network of social patterns comprising STEM education, information literacy standards need not serve as barriers to the integration of participative social media resources into the learning environment. Our findings also suggest that students are more than willing to engage with social resources in their coursework, and previous studies indicate that they derive social and educational benefits from doing so. Information resources, technologies, and networks themselves present no barrier, at least for those able and willing to engage. Their addictive design, generally short learning curves, and sheer ubiquity enable near-constant interaction.
What seems to remain are barriers of custom and capacity. It is enough of a challenge for instructors to select resources and design lesson plans around static, traditional resources—to expect them to do so while meaningfully integrating social media content that may change several times per minute is a significant capacity barrier. Similarly, students asked to critically evaluate and contribute overmuch to multiple, ever-changing social media resources may find themselves longing for a quiet textbook. Significant line items in library budgets include resource selection and acquisition, licensing and purchasing, as well as the cost of librarians with specialized graduate degrees to organize and provide access to it all. A shift away from traditionally authoritative resources in support of research and education might understandably make librarians fear innovating themselves out of existence.
In general terms, organizations that meet education standards get accredited, and instructors who can meet learning standards effectively stay employed. A content analysis and reinterpretation of relevant standards can be an effective way to defuse resistance to new or unfamiliar resources and practices, and reframe them in terms of current goals and values. To be sure, not every administrator or instructor agrees with or is even aware of the potentially relevant standards designed to guide their pedagogical practices. Some may be more motivated by innovations arising from classroom practice, such as those we observed in this study.
As new forms of digital and social media emerge, learning resources and practices are being continuously renegotiated. We also suggest through our work here that an analytical approach based on the dynamic nature of actor networks can yield a more refined identification and understanding of the roles and incentives of the diverse actors around the negotiating table. Our future work will build on this method and apply it to more specific questions of how participative platforms and dynamic content such as web forums and Q&A sites might have a role in STEM learning.
When complex institutions such as STEM education change, it tends to be slow, uneven, and nonlinear. At the grass roots level, where students learn foundational skills not just for STEM but for citizenship in our information society, we see evidence that participative perspectives and practices are taking hold.
This project was made possible in part by the Institute of Museum and Library Services, Grant No. LG-81-16-0025.
AASL: American Association of School Libraries (2018), AASL Standards Framework for Learners, retrieved May 9, 2018, from http://standards.aasl.org/wp-content/uploads/2017/11/AASL-Standards-Framework-for-Learners-pamphlet.pdf.
ACRL: Association of College and Research Libraries (2006), ACRL Framework for Information Literacy for Higher Education, retrieved May 9, 2018, from http://www.ala.org/acrl/standards/ilframework.
ACRL: Association of College and Research Libraries (2015), ACRL Information Literacy Standards for Science and Engineering/Technology, retrieved May 9, 2018, from http://www.ala.org/acrl/standards/infolitscitech.
Adams, W.K., Dubson, M., Finkelstein, N.D., and Wieman, C.E. (2004), The Design and Validation of the Colorado Learning Attitudes About Science Survey, AIP Conf. Proc., Sacramento, CA, vol. 790, pp. 45–48.
American Library Association (1989), Presidential Committee on Information Literacy, Final Report, Chicago: American Library Association, retrieved June 4, 2018, from http://www.ala.org/acrl/publications/whitepapers/presidential.
Anderson, T. (2008), The Theory and Practice of Online Learning, Edmonton, Alberta, Canada: AU Press.
Aronson, E., Stephan, C., Sikes, J., Blaney, N., and Snapp, M. (1978), The Jigsaw Classroom, Beverly Hills, CA: Sage Publications.
Asselin, M., Dobson, T., Meyers, E.M., Teixiera, C., and Ham, L. (2011), Learning from YouTube: An Analysis of Information Literacy in User Discourse, Proc. 2011 iConference, ACM, Seattle, WA, pp. 640–642.
Berendt, B. (2012), Data Mining for Information Literacy, in Data Mining: Foundations and Intelligent Paradigms, D. Holmes and L. Jain, Eds., vol. 25, Intelligent Systems Reference Library, Berlin, Heidelberg: Springer.
Blakeslee, S. (2004), The CRAAP Test, LOEX Q., vol. 31, no. 3, p. 4.
Briscoe, G. and Mulligan, C. (2014), Digital Innovation: The Hackathon Phenomenon, Creativeworks London Working Paper No. 6.
Burgoyne, M.B. and Chuppa-Cornell, K. (2015), Beyond Embedded: Creating an Online-Learning Community Integrating Information Literacy and Composition Courses, J. Acad. Libr., vol. 41, no. 4, pp. 416–421.
Center for the Integration of Research, Teaching and Learning (2017), CIRTL Network, University of Wisconsin, retrieved May 30, 2018, from https://www.cirtl.net/.
Chuang, Y.T. (2015), SSCLS: A Smartphone-Supported Collaborative Learning System, Telematics Informatics, vol. 32, no. 3, pp. 463–474.
Coleman, V. (2013), Social Media as a Primary Source: A Coming of Age, Educause Rev., vol. 48, no. 6, retrieved June 6, 2018, from https://er.educause.edu/articles/2013/12/social-media-as-a-primary-source-a-coming-of-age.
Detlor, B., Julien, H., Serenko, A., and Booker, L. (2010), Factors Affecting Student Learning Outcomes of Information Literacy Instruction, Proc. 2010 ASIS&T Annual Meeting, Pittsburgh, PA, Oct. 22–27.
Domínguez-Flores, N. and Wang, L. (2011), Online Learning Communities: Enhancing Undergraduate Students' Acquisition of Information Skills, J. Acad. Lib., vol. 37, no. 6, pp. 495–503.
Estrada, M. (2014), Ingredients for Improving the Culture of STEM Degree Attainment with Co-Curricular Supports for Underrepresented Minority Students, National Academy of Sciences White Paper, retrieved June 8, 2018, from https://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088832.pdf.
Fayon, A.K., Goff, E., and Duranczyk, I.M. (2010), Impacting Attitudes of ELL Students: Integrated Learning Communities in Introductory Science Courses, Learn. Assist. Rev., vol. 15, no. 2, pp. 7–19.
Freeman, K.E., Alston, S.T., and Winborne, D.G. (2008), Do Learning Communities Enhance the Quality of Students' Learning Motivation in STEM?, J. Negro Ed., vol. 77, no. 3, pp. 227–240.
Friesen, N. and Lowe, S. (2012), The Questionable Promise of Social Media for Education: Connective Learning and the Commercial Imperative, J. Comput. Assist. Learn., vol. 28, no. 3, pp. 183–194.
Gee, J.P. (2003), What Video Games Have to Teach Us About Learning and Literacy, New York: Palgrave Macmillan.
Graham, M.J., Frederick, J., Byars-Winston, A., Hunter, A.B., and Handelsman, J. (2013), Increasing Persistence of College Students in STEM, Science, vol. 341, no. 6153, pp. 1455–1456.
Hagund, L. and Solsson, P. (2008), The Impact on University Libraries of Changes in Information Behavior Among Academic Researchers: A Multiple Case Study, J. Acad. Librar., vol. 34, no. 1, pp. 52–59.
Hyslop-Margison, E.J. and Strobel, J. (2007), Constructivism and Education: Misunderstandings and Pedagogical Implications, Teach. Ed., vol. 43, no. 1, pp. 72–86.
Ince, S.F., Hoadley, C., and Kirschner, P.A. (2018), A Study of Search Practices in Doctoral Student Scholarly Workflows, Proc. 2018 Conf. on Human Information Interaction & Retrieval, New Brunswick, NJ, pp. 245–248.
Jang, H. (2016), Identifying 21st Century STEM Competencies Using Workplace Data, J. Sci. Ed. Tech., vol. 25, pp. 284–301.
Knorr Cetina, K. (1999), Epistemic Cultures: How the Sciences Make Knowledge, Cambridge, MA: Harvard University Press.
Kumar, P., Gruzd, A., Haythornthwaite, C., Gilbert, S., Esteve del Valle, M., and Paulin, D. (2018), Learning in the Wild: Coding Reddit for Learning and Practice, Proc. 51st Hawaii Int. Conf. on System Sciences, pp. 1933–1942.
Latour, B. (1996), Actor Network Theory: A Few Clarifications, Soz. Welt, vol. 47, pp. 369–381.
Law, J. (1992), Notes on the Theory of the Actor-network: Ordering, Strategy, and Heterogeneity, Syst. Pract., vol. 5, no. 4, pp. 379–393.
Mackey, T.P. and Jacobson, T.E. (2011), Reframing Information Literacy as a Metaliteracy, Coll. Res. Libr., vol. 72, no. 1, pp. 62–78.
McLoughlin, C. and Lee, M. (2007), Social Software and Participatory Learning: Pedagogical Choices with Technology Affordances in the Web 2.0 Era, Proc. of ASCILITE - Australian Soc. for Computers in Learning in Tertiary Education Ann. Conf., pp. 664–675.
McTavish, D.G. and Pirro, E.B. (1990), Contextual Content Analysis, Qual. Quant., vol. 24, pp. 245–265.
NGSS (2013), Next Generation Science Standards: For States, By States, Washington DC: National Academies Press, retrieved June 6, 2018, from https://www.nap.edu/read/18290/.
Nicholars, D., Boukacem-Zeghmouri, C., Rodriguez-Bravo, B., Xu, J., Watkinson, A., Abrizah, A., and Swigon, M. (2017), Where and How Early Career Researchers Find Scholarly Information, Learned Publ., vol. 30, no. 1, pp. 19–29.
Olson, S. and Riordan, D.G. (2012), Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics, Report to the President, Executive Office of the President, Washington, DC, retrieved June 6, 2018, from https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_2-25-12.pdf.
Packard, B.W.L. (2012), Effective Outreach, Recruitment, and Mentoring into STEM Pathways: Strengthening Partnerships with Community Colleges, Community Colleges in the Evolving STEM Education Landscape: Summary of a Summit, retrieved June 6, 2018, from http://nas-sites.org/communitycollegessummit/files/2011/12/NAS_Packard_Mentoring_toupload-2.pdf.
Pfund, C., Mathieu, R., Austin, A., Connolly, M., Manske, B., and Moore, K. (2012), Advancing STEM Undergraduate Learning: Preparing the Nation's Future Faculty, Change, vol. 44, no. 6, pp. 64–72.
Ramnarine-Rieks, A. (2013), Learning by Game Design: Exploring Its Potential in Undergraduate Information Literacy Instruction, Proc. 2013 ASIS&T Annual Meeting, Montreal, Canada, Nov. 1–5.
Schultz, P.W., Hernandez, P.R., Woodcock, A., Estrada, M., Chance, R.C., Aguilar, M., and Serpe, R.T. (2011), Patching the Pipeline: Reducing Educational Disparities in the Sciences Through Minority Training Programs, Ed. Eval. Policy Anal., vol. 33, no. 1, pp. 95–114.
Selvin, A.M., Buckingham Shum, S.J., and Aakhus, M. (2010), The Practice Level in Participatory Design Rationale: Studying Practitioner Moves and Choices, Human Tech., vol. 6, no. 1, pp. 71–105.
Siemens, G. (2005), Connectivism: A Learning Theory for the Digital Age, Int. J. Instruct. Tech. Distance Learn., vol. 2, no. 1, pp. 3–10.
Sithole, A., Chiyaka, E.T., McCarthy, P., Mupinga, D.M., Bucklein, B.K., and Kibirige, J. (2017), Student Attraction, Persistence and Retention in STEM Programs: Successes and Continuing Challenges, Higher Ed. Stud., vol. 7, no. 1, pp. 46–59.
Star, S.L., and Griesemer, J.R. (1989), Institutional Ecology, ‘Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39, Soc. Stud. Sci., vol. 19, no. 3, pp. 387–420.
Tuominen, K., Savolainen, R., and Talja, S. (2005), Information Literacy as a Sociotechnical Practice, Libr. Q., vol. 75, no. 3, pp. 329–345.
Turnitin (2011), Plagiarism on the Web: Myths and Realities—An Analytical Study on Where Students Find Unoriginal Content on the Internet, retrieved from http://go.turnitin.com/paper/plagiarism-web-myths-realities.
Van Popta, E., Kral, M., Camp, G., Martens, R.L., and Simons, P.R.J. (2017), Exploring the Value of Peer Feedback in Online Learning for the Provider, Ed. Res. Rev., vol. 20, pp. 24–34.
Wheeler, S., Yeomans, P., and Wheeler, D. (2008), The Good, the Bad and the Wiki: Evaluating Student-Generated Content for Collaborative Learning, Br. J. Ed. Tech., vol. 39, no. 6, pp. 987–995.
Yu, A.Y., Tian, S.W., Vogel, D., and Kwok, R.C.W. (2010), Can Learning Be Virtually Boosted? An Investigation of Online Social Networking Impacts, Comput. Ed., vol. 55, no. 4, pp. 1494–1503.
Keebler Daniel W, Huffman Jessica, Effective eLearning and Transformative Pedagogical Strategies, International Journal of Online Pedagogy and Course Design, 10, 2, 2020 Crossref