class: center, middle, inverse, title-slide # The Role of the Twitter Hashtag #NGSSchat in the Context of Science Education Reform Efforts ### Joshua M Rosenberg, Joshua W Reid, Matthew J Koehler, Christian Fischer, and Thomas J. McKenna ### 2019-01-04 --- background-image: url("/aste-19/nsta-ss.png") background-position: center background-size: contain --- background-image: url("/aste-19/chat-s.png") background-position: center background-size: contain --- # The Next Generation Science Standards - NGSS (or NGSS-inspired) standards in 39 states (National Science Teachers Association, 2018) - Catalyzed by a series of reports (National Research Council, 2007, 2012) - While reform efforts in science education are *not* new (DeBoer, 2014), the new reforms have some cause to be thought of as different --- # Is This Time Different? - More **discussion** between practitioners and researchers than before (i.e., RPPs; Coburn & Penuel, 2016) - Including in science education (Loucks-Horsely, Stiles, Mundry, Love, & Hewson, 2010; Luft & Hewson, 2014) - Increased prominence (ubiquity?) of digital **technologies** in teacher professional development (e.g., Reiser et al., 2017) - Greater recognition of **informal learning** through networks (i.e., PLNs; Trust, 2012) --- # Teacher Learning - Teacher learning can occur anywhere (Desimone & Garet, 2015) - Involvement is often limited by barriers especially geographic ones - One way to break out of these boundarie: conceptualize professional learning as *participation in online and communities and networks* (i.e., Siemens, 2014) - Can lead to more heterogeneous learning communities (Luft & Hewson, 2014) ??? Should align with current science standards, practices, and content that teachers will encounter (i.e., coherency and content focus); afford teachers opportunities to actively and collaboratively construct appropriate requisite knowledge (i.e., active engagement and collaboration); and foster lifelong learning mindsets within teachers Effective teacher professional development and learning is facilitated through interactions with other stakeholders that provide opportunities to foster discussions about research and practice (Borko, Jacobs, & Koellner, 2010; Darling-Hammond, Hyler, & Gardner, 2017; Luft & Hewson, 2014) --- # Teacher Learning Through Twitter (Part 1) - Some teachers' PLNs include Twitter communities (Lord & Lomicka, 2014; Rehm & Notten, 2016) - Twitter has been used (in education) for many differerent purposes (Greenhalgh & Koehler, 2017; Krutka, Asino, & Haselwood, 2018; Romero-Hall, 2017) - Has affordances and some constraints for professional learning (Torphy & Frank, in preparation) - Its informal, *just for me* approach to learning is in contrast to common *one-size fits all* perspective of traditional professional development --- class: center, middle background-image: url("/aste-19/tweet-picture.png") background-position: center background-size: contain --- # Teacher Learning Through Twitter (Part 2) - Some past research has focused on a convention designed to make the *firehose* of information more manageable, a focus on hashtags - A hashtag is a convention on Twitter that is used to organize conversations, and, allows for regularly occurring (or synchronous) chats at pre-specified times --- # What is the role of #NGSSchat Regarding Teacher Learning Through Twitter? - Twitter and #NGSSchat are *potentially* valuable to science educators - Chats organized through hashtags are possibly somewhat distinct (Rosenberg, Akcaoglu, Staudt Willet, Greenhalgh, & Koehler, 2017) - May be one of the largest, and perhaps important, networks for science educators - Can be a place for teachers to interact with other teachers and with other science education stakeholders - Important because doing so can shape how teachers engage in sensemaking (Coburn, 2001) --- # Need For Study - Unlike many formal opportunities and PLNs, this informal (yet growing and inclusive of a myriad of educators and researchers) platform has not yet been the focus of any studies - One excellent *NSTA Reports* article by Shelton and Ende (2015) - Repeated calls for more empirical research on the effectiveness of online teacher education opportunities (Borko, Jacobs, & Koellner, 2010; Fishman, Konstantopoulos, Kubitskey, Vath, Park, Johnson, & Edelson; Moon, Passmore, Reiser, & Michaels, 2014) ??? Articulates learning as a process enacted through networks and communities, and these communities are enhanced through technological mediums (Siemens, 2005) such as #NGSSchat --- # Framing and Research Questions - Within web-based communities like #NGSSchat, participants may be a part of a dynamic, cultural community (Gutiérrez and Rogoff) - We used connectivism (Siemens, 2013) and social network theory (Carolan, 2014) to frame the study 1. Which groups of individuals are the most central? 2. Which groups of individuals have received and sent interactions? 3. What explains who individuals have chosen to interact with? --- # Research Approach - Social Network Analysis (SNA) - Conventionally offline *and also* online and using digital traces (Spillane, Kim, & Frank, 2012) - Augmented offline with information from digital traces (e.g., McFarland, Lewis, & Goldberg, 2015) - In this latter tradition, we employed SNA to describe and examine #NGSSchat --- class: center, middle <table> <thead> <tr> <th style="text-align:left;"> Measure </th> <th style="text-align:left;"> Description </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Actors </td> <td style="text-align:left;"> Represents the social unit of analysis in a social network (i.e., person); serves as a measure as a count for how many individuals are involved in the #NGSSchat network </td> </tr> <tr> <td style="text-align:left;"> Edges </td> <td style="text-align:left;"> Represents the relationship between two actors; measures the number of interactions within the network </td> </tr> <tr> <td style="text-align:left;"> Density </td> <td style="text-align:left;"> The proportion of the number of edges in a network to the possible number of edges and indicates a more </td> </tr> <tr> <td style="text-align:left;"> Endorsement </td> <td style="text-align:left;"> A type of edge in the #NGSSchat network: A Twitter interaction that is either a favorite, retweet, or quote tweet </td> </tr> <tr> <td style="text-align:left;"> Conversation </td> <td style="text-align:left;"> A type of an edge in the #NGSSchat network: A Twitter interaction which is either a mention or a reply </td> </tr> <tr> <td style="text-align:left;"> In-degree centrality </td> <td style="text-align:left;"> The number of edges received by an individual (being favorited, retweeted, quoted, replied to, or mentioned) </td> </tr> <tr> <td style="text-align:left;"> Out-degree centrality </td> <td style="text-align:left;"> The number of edges sent by an individual (favoriting, retweeting, quoting, replying to, or mentioning someone else) </td> </tr> <tr> <td style="text-align:left;"> Group membership </td> <td style="text-align:left;"> The category assigned to each individual, based upon qualitative coding of individual users’ Twitter profile information </td> </tr> <tr> <td style="text-align:left;"> Number of tweets </td> <td style="text-align:left;"> The number of tweets an individual sent </td> </tr> </tbody> </table> --- # Data Sources - Novel data source: Tweets that were archived in the #NGSSchat community between 2012 and 2017 on *Storify* - Tweets come from "chats," the synchronous periods of time during which #NGSSchat users arranged to meet to discuss pre-arranged topics - In total, collected data from 103 chats --- background-image: url("/aste-19/topics.png") background-position: center background-size: contain --- ## Participants - Edge list included information about who sent and received the interaction - From 787 individuals - Focused on *n* = 517 unique Twitter user profiles who posted more than one original tweet - On average, tsent 14.88 (SD = 50.47) original tweets - Qualitatively coded for participants' professional affiliation/group --- class: center, middle <table> <thead> <tr> <th style="text-align:left;"> Code </th> <th style="text-align:left;"> Group </th> <th style="text-align:right;"> % </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Teacher </td> <td style="text-align:left;"> Teacher </td> <td style="text-align:right;"> 44.5 </td> </tr> <tr> <td style="text-align:left;"> Administrator </td> <td style="text-align:left;"> Support </td> <td style="text-align:right;"> 6.2 </td> </tr> <tr> <td style="text-align:left;"> Instructional Support </td> <td style="text-align:left;"> Support </td> <td style="text-align:right;"> 12.8 </td> </tr> <tr> <td style="text-align:left;"> Educational Researcher or University Faculty </td> <td style="text-align:left;"> Research </td> <td style="text-align:right;"> 6.8 </td> </tr> <tr> <td style="text-align:left;"> Educational Institution </td> <td style="text-align:left;"> Research </td> <td style="text-align:right;"> 0.8 </td> </tr> <tr> <td style="text-align:left;"> Educational Organization </td> <td style="text-align:left;"> Research </td> <td style="text-align:right;"> 6.4 </td> </tr> <tr> <td style="text-align:left;"> Education-Connected </td> <td style="text-align:left;"> Other </td> <td style="text-align:right;"> 10.1 </td> </tr> <tr> <td style="text-align:left;"> Media </td> <td style="text-align:left;"> Other </td> <td style="text-align:right;"> 1.0 </td> </tr> <tr> <td style="text-align:left;"> Hashtag / Chat Accounts </td> <td style="text-align:left;"> Other </td> <td style="text-align:right;"> 2.3 </td> </tr> <tr> <td style="text-align:left;"> Not Clear </td> <td style="text-align:left;"> Other </td> <td style="text-align:right;"> 9.3 </td> </tr> </tbody> </table> --- ## Interactions - 10,658 original tweets from Storify - Data processed into an an edge list using the *igraph* and *ggraph* packages from **R** and coded as representing one of two types of interactions for two networks - 34,668 favorites (*endorsing* network) - 13,498 replies (*conversing* network) - 10,382 mentions (*conversing* network) - 8,912 retweets (*endorsing* network) - 1,899 quotes (*endorsing* network) - Filtered the edge list only to contain participants who sent more than one original tweet - Final edge list consisted of 55,807 interactions - Visualized the data in two sociograms for the endorsing and conversing networks --- background-image: url("/aste-19/endorsing-ngsschat-soc.png") background-position: center background-size: contain --- background-image: url("/aste-19/conversing-ngsschat-soc.png") background-position: center background-size: contain --- # Data Analysis - Analysis for RQ1 - Use of centrality measures by their professional affiliation/group - Analysis for RQ2 - Predict interactions received (in-degree) and sent (out-degree) for both conversing and endorsing - Analysis for RQ3 - Predict formation of ties for both conversing and endorsing using a selection model --- class: center, middle ## Centrality: Endorsing Network (Results for RQ1) <table> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Betweenness </th> <th style="text-align:left;"> Centrality </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Administrator </td> <td style="text-align:left;"> 1217.82 (6134.22) </td> <td style="text-align:left;"> 0.06 (0.14) </td> </tr> <tr> <td style="text-align:left;"> Researcher </td> <td style="text-align:left;"> 625.81 (2277.89) </td> <td style="text-align:left;"> 0.05 (0.12) </td> </tr> <tr> <td style="text-align:left;"> Teacher </td> <td style="text-align:left;"> 510.48 (3723.25) </td> <td style="text-align:left;"> 0.03 (0.09) </td> </tr> <tr> <td style="text-align:left;"> Other </td> <td style="text-align:left;"> 124.63 (395.26) </td> <td style="text-align:left;"> 0.02 (0.04) </td> </tr> </tbody> </table> --- class: center, middle ## Centrality: Conversing Network (Results for RQ1) <table> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Betweenness </th> <th style="text-align:left;"> Centrality </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Administrator </td> <td style="text-align:left;"> 1391.2 (6560.08) </td> <td style="text-align:left;"> 0.07 (0.15) </td> </tr> <tr> <td style="text-align:left;"> Researcher </td> <td style="text-align:left;"> 732.88 (2453.52) </td> <td style="text-align:left;"> 0.05 (0.13) </td> </tr> <tr> <td style="text-align:left;"> Teacher </td> <td style="text-align:left;"> 531.62 (3805.09) </td> <td style="text-align:left;"> 0.04 (0.09) </td> </tr> <tr> <td style="text-align:left;"> Other </td> <td style="text-align:left;"> 142.33 (431.96) </td> <td style="text-align:left;"> 0.02 (0.04) </td> </tr> </tbody> </table> --- class: center, middle ## Predicting Interactions (Results for RQ2) <table> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Conversing In-degree </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Intercept </td> <td style="text-align:left;"> 2.18 (0.14)*** </td> </tr> <tr> <td style="text-align:left;"> Administrator </td> <td style="text-align:left;"> 0.17 (0.19) </td> </tr> <tr> <td style="text-align:left;"> Researcher </td> <td style="text-align:left;"> 0.43 (0.19)* </td> </tr> <tr> <td style="text-align:left;"> Teacher </td> <td style="text-align:left;"> 0.28 (0.16) </td> </tr> <tr> <td style="text-align:left;"> Original Tweets </td> <td style="text-align:left;"> 0.29 (0.01)*** </td> </tr> </tbody> </table> --- class: center, middle ## Predicting Interactions (Results for RQ2) <table> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Conversing In-degree </th> <th style="text-align:left;"> Conversing Out-Degree </th> <th style="text-align:left;"> Endorsing In-degree </th> <th style="text-align:left;"> Endorsing Out-degree </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Intercept </td> <td style="text-align:left;"> 2.18 (0.14)*** </td> <td style="text-align:left;"> 2.09 (0.13)*** </td> <td style="text-align:left;"> 2.78 (0.12)*** </td> <td style="text-align:left;"> 2.88 (0.13)*** </td> </tr> <tr> <td style="text-align:left;"> Administrator </td> <td style="text-align:left;"> 0.17 (0.19) </td> <td style="text-align:left;"> 0.29 (0.17) </td> <td style="text-align:left;"> 0.25 (0.17) </td> <td style="text-align:left;"> 0.24 (0.17) </td> </tr> <tr> <td style="text-align:left;"> Researcher </td> <td style="text-align:left;"> 0.43 (0.19)* </td> <td style="text-align:left;"> 0.34 (0.18) </td> <td style="text-align:left;"> 0.37 (0.17)* </td> <td style="text-align:left;"> 0.17 (0.19) </td> </tr> <tr> <td style="text-align:left;"> Teacher </td> <td style="text-align:left;"> 0.28 (0.16) </td> <td style="text-align:left;"> 0.29 (0.15)* </td> <td style="text-align:left;"> 0.28 (0.14) </td> <td style="text-align:left;"> 0.24 (0.15) </td> </tr> <tr> <td style="text-align:left;"> Original Tweets </td> <td style="text-align:left;"> 0.29 (0.01)*** </td> <td style="text-align:left;"> 0.26 (0.01)*** </td> <td style="text-align:left;"> 0.24 (0.01)*** </td> <td style="text-align:left;"> 0.22 (0.02)*** </td> </tr> </tbody> </table> --- class: center, middle ## Selection Models: Endorsing Network (Results for RQ3) <table> <thead> <tr> <th style="text-align:left;"> Coefficient </th> <th style="text-align:left;"> Estimate </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Intercept </td> <td style="text-align:left;"> -5.18 (0.25) *** </td> </tr> <tr> <td style="text-align:left;"> Same Group </td> <td style="text-align:left;"> 0.08 (0.03) * </td> </tr> <tr> <td style="text-align:left;"> Sender - Researcher </td> <td style="text-align:left;"> 0.27 (0.28) </td> </tr> <tr> <td style="text-align:left;"> Sender - Administrator </td> <td style="text-align:left;"> .44 (.25) </td> </tr> <tr> <td style="text-align:left;"> Sender - Teacher </td> <td style="text-align:left;"> .37 (.22) </td> </tr> <tr> <td style="text-align:left;"> Receiver - Researcher </td> <td style="text-align:left;"> .30 (.26) </td> </tr> <tr> <td style="text-align:left;"> Receiver - Administrator </td> <td style="text-align:left;"> .49 (.24)* </td> </tr> <tr> <td style="text-align:left;"> Receiver - Teacher </td> <td style="text-align:left;"> .20 (.21) </td> </tr> <tr> <td style="text-align:left;"> Coefficient </td> <td style="text-align:left;"> ICC </td> </tr> <tr> <td style="text-align:left;"> Sender </td> <td style="text-align:left;"> 0.34 </td> </tr> <tr> <td style="text-align:left;"> Receiver </td> <td style="text-align:left;"> 0.29 </td> </tr> </tbody> </table> --- # Some Key Findings - Administrators are central in terms of connecting others and potentially having influence - Researchers are popular in terms of *being conversed with* and *endorsed* - Teachers, on the other hand, were more likely to *start conversations* - Individuals who participate in the hashtag can expect to interact with others - Individuals appear to be endorsing others from the same group --- # Limitations: Studying Social Media - Use of a large data set from a social media source - *Positive*: Data from over five years, with more than 500 individuals and 60,000 observed interactions - But, this is also, in other ways, a very limited data set - Net methods that can be termed *computational social science* (Salganik, 2017; Lazer et al., 2009) - There are also other ways we could use the data we have (DeLaat & Schreurs, 2013) ??? - While we have rich information about the interactions we observed, these were four specific interactions made possible through Twitter - and what we can say about relations is necessarily limited. - Such methods have many strengths, such as being able to study groups of individuals who may not otherwise be studied or to do so in ways that would not be feasible, such as examining interactions from such a diverse group of individuals—from beginning teachers to superintendents—as in the present study - Nevertheless, questions such as how representative our sample was, and how much we can know about participants beyond their profession/role, limit the present study. They also suggest future directions that involve collecting survey data from participants (separately or perhaps in conjunction with data from social media). Another limitation of this study comes out of the assumptions that are made when using social network analysis (Wasserman & Faust, 1994) - Described three ways of studying learning in networks: surveying about relations (i.e., social network analysis), content analysis of conversations that are occurring, and understanding the reasons behind the specific behaviors and conversations observed in the network (i.e., contextual analysis) --- # Limitations: Representativeness - How much we can know about participants - Value of other data and additional insight from participants - Diversity of participants involved in #NGSSchat not studied here - Reports of the value of NGSSchat may differ from those evidenced in this data --- # Limitations: Impact - Value in these research findings, but need to go further - Share findings with participants - Talk with participants - Consider the role of other science education hashtags - Consider #NGSSchat to be a site for potential future research - Find out impacts on classroom (or policymaking) practice - Explore other settings based upon where stakeholders already are --- # Implications for Practice - Less recognized platform aligns with recommendations for teacher professional development (Desimone & Garet, 2001) - Supporting collaboration - Focus on content - Alignment with new standards - Use of an educational technology for teacher learning and professional development in science education - Usefulness of a technology (some/many) stakeholders already use - Has some impacts for science teacher preparation and education (e.g., Carpenter, 2018) - May connect teachers and others who may not otherwise (Gunawardena, Hermans, Sanchez, Richmond, Bohley, & Tuttle, 2009) ??? , such as, aligning to national and state standards (NGSS), focusing on content (science), and supporting a collaborative learning environment --- # Thank you Joshua M Rosenberg, Joshua W Reid, Matthew J Koehler, Christian Fischer, and Thomas J. McKenna [https://joshuamrosenberg.com](https://joshuamrosenberg.com) [@jrosenberg6432](https://twitter.com/jrosenberg6432) [https://osf.io/9ex7k/](https://osf.io/9ex7k/) *We welcome your questions and feedback on this work!*