Last summer, I wrote a proposal to the NSF CAREER program, which provides grant funding to early career researchers. The proposal was submitted in July and I’m still awaiting the results of the review and award process.
Related, this semester, I am teaching a one-credit class for Ph.D. students; part of the class focuses on creating a really early record of one’s ideas - a brief “blurb” to pitch the core idea of the proposal. This got me thinking about the origins of the idea in my CAREER proposal—the ideas go far back! The most concrete starting point was a proposal I submitted to the ISLS conference. I’ll start there, working forward to the July, 2022 submission date.
December, 2019
The International Society of the Learning Sciences (ISLS) conference has an early career workshop. The proposals for it were brief (two pages) and due literally on New Year’s Eve - at 11:59 pm on December 31, 2019. Here’s the description for the one I applied to. They asked for the following: “A 2-page summary of your research including tables, figures and references using the template by the ICLS 2020 general submission (Please follow the template for a 2-page poster submission).” I submitted this, an overview of my research I titled More Confidently Uncertain? Teaching Learners to Apply Bayesian Methods to Make Sense of Scientific Phenomena. What I remember was trying to pitch the weirdest, most ambitious form of my ideas. I’ll share the abstract here:
While analyzing data is important learning in science domains, existing methods and tools for those learning to work with data have key limitations, particularly concerning scientific modeling. This early-stage research is intended to begin a line of work on students’ data analysis that is not yet widely used in K-12 learning environments, Bayesian statistical methods, with implications for how learners use evidence in science education.
I think what the abstract captures is the intersection of two things I had cared about: 1) students engaging in the doing of science and 2) using advanced statistical methods (myself), especially multi-level models and their Bayesian version/extension. What I tried to do in this proposal was merge them.
After a few months, I learned the proposal was accepted. I was paired with a mentor and I also received feedback from other scholars. One scholar, Michelle Wilkerson, was positive about the idea. This early feedback was highly encouraging to me.
The simple idea was (and I think still is) a cool one. Bayesian approaches can at once be more intuitive and empowering (and powerful) than traditional ones. They may be especially valuable to science teachers and learners because they incorporate prior ideas and beliefs about the thing that is being investigated or explored. That core idea motivated all of the subsequent work (at least my part) on this topic.
I didn’t have super concrete ideas about how to move forward, until …
February, 2020
Marcus Kubsch, a friend I first met when he was a post-doc visiting Michigan State University (where I was a doc student) read a pre-print of the above-mentioned paper, writing:
I just read your ICLS paper and it sounds really cool! I actually have been thinking about something similar because students struggle a lot with uncertainty in learning about evolution and nuclear fission - both of which usually embrace a frequentist concept of probability.
This sparked a collaboration that eventually led to a significant expansion of what I had been thinking; Marcus deepened and broadened the literature I was aware of and helped me see several contexts in which using Bayesian methods was pertinent. We presented a paper at the next ISLS conference (in 2021), Considering K-12 Learners’ Use of Bayesian Methods.
January, 2021
I saw the short book Bayesian Thinking for Toddlers and almost immediately emailed the author, E.J. Wagenmakers. E.J. responded and he, Marcus, and I setup a meeting to discuss strategies for teaching Bayesian methods. We started a paper together in response to a special issue of the journal Science & Education that was focused on trust and science. As we were working on it …
July, 2021
I came across a podcast interview with Mine Dogucu entitled How to Teach and Learn Bayesian Stats. I emailed Mine and we setup a meeting to discuss our shared interest in teaching Bayesian methods. After we talked, I think Mine and I saw how several of our interests were really closely related, and so we invited the other to work on several ongoing projects; on my end, the Science & Education article with E.J. and Marcus. We worked on this for several months, eventually seeing the article published (open access) here. This work represented a lot of thinking and collaboration, as well as some development work that Marcus led to build an interactive web application to make Bayesian methods more accessible. Mine and I continued to work on several projects, including implementing an app she developed with her colleague Sibel Kazak that shared several elements with (but was also distinct from) the app Marcus developed.
At this point, I felt like I was in a very different place from December 2019. At the same time, several important issues lingered: the work of E.J., Mine, and Sibel was oriented toward audiences very different from that of science education and there remained the need to contextualize these methods for science teachers and learners (and to see how they aligned with the focus on organizing learning around learners figuring out how scientific phenomena work). Also, Marcus had not yet implemented the app we wrote about in our article. More broadly, there was the need to hear what teachers think; to align what Bayesian methods can do with what science teachers care about. The same need exists for learners—students—and to see what it helps learners to do. And there is the need to integrate the apps that Marcus and Mine and Sibel developed with some kind of data analysis tool.
May, 2022
I started to pitch an NSF CAREER grant proposal to colleagues around this time, starting with the instructions for a “blurb” that Teya Rutherford laid out here. The pitch involved integrating many of the arguments use in the Science & Education paper as well as the ideas for making Bayesian data analysis more practical spearheaded by Marcus and Mine and Sibel (by trying out versions of them in DataClassroom. It also drew on a recent paper I worked on with Aaron at DataClassroom, Elizabeth Schultheis and Melissa Kjelvik at DataNuggets, and Omiya Sultana on which data sources and tools science teachers use with their students in their classrooms. It also involved a more personal interest—one in biology (the subject I taught as a high school teacher in a long ago career!) and one in the outdoors more generally. Specifically, it drew on my love of spending time in the most biodiverse national park in the United States… and a place that has become special to me, my wife, and my kiddo, the Great Smoky Mountains. I reached out to the Great Smoky Mountains Institute at Tremont, a nationally-recognized outdoors education center in the Smokies, through a connection made by Amanda Garner. Amanda pushed me to think about place-based education, her area of expertise. She also introduced me to tools to combine place-based education and data analysis and data science, including iNaturalist. I asked if they were interested in developing a professional development opportunity for teachers that combined data, Bayesian methods, and outdoors and place-based education.
Back to the blurb.
An apropos fact about the blurb—the same file I used to write the blurb had nearly the same contents as the file I submitted. The file with the blurb is here. I think this is noteworthy because it shows how I modified what was originally a few sentences into a coherent description of what I was proposing. The contributions and work of Marcus, Mine, Sibel, and E.J. are clear here; those collaborations were nothing less than invaluable to the work I pitched. At the same time, I am happy to still be collaborating with each of these people on ongoing projects; the collaborations fed more individually-driven work as well as other, shared projects.
At one point (May 24), the document was in what I thought was a good place. Here’s the doc at that point. I shared it with colleagues, including Teya as well as Victor Lee, Frances Harper, Mehmet Aydeniz, Christina Krist, Teya Rutherford’s daughter (actually, I think Teya was logged into the wrong account at that time), and Hollie Raynor. They tore it to shreds (in a constructive and collegial way!). Here are example comments:
This hasn’t been introduced or motivated at all
… something feels disconnected to contemporary science ed discourses in this blurb
There are way too many constructs (IMO) for a one pager here.
This was the best kind of feedback I could receive—really.
June, 2022
I took their feedback into account, revised the blurb, and then emailed several program officers at the NSF with whom I had previously corresponded (or knew). One responded they weren’t part of the program I planned to submit to at this time, but the other responded that they’d be willing to speak further with me about the idea. I sent the blurb and we setup a time and talked about it. I remember the program officer being encouraging and seeing value in the core idea, but also suggesting I needed to do some serious work to make the idea “CAREER-worthy”. As pitched, the project seemed like a one or two year endeavor—not a five-year project with multiple elements. I took their feedback into account and worked my tail off in June to simultaneously secure letters of support, write and revise the document, and eventually navigate the submission process (all with lots of help and encouragement along the way, especially from Lynn Hodge, Aaron Reedy at DataClassroom, and Annie Roth and John Di’Diego at the Great Smoky Mountains Institute at Tremont).
July, 2022
At the end of the month, the proposal was submitted.
Here’s the final version of the overview:
There have been prominent and widespread calls for high school science students to work with data in more complex ways that better align with and support the work of professional scientists and engineers (Lee & Wilkerson, 2018; National Academies of Sciences, Engineering, and Medicine, 2019). This is part of the broader practice shift in science education research and standards and more data-intensive areas of study and work—and explorations of the roles of data science at the K-12 levels (Jiang et al., 2022; Wilkerson & Polman, 2020). Although teachers express the desire to do more with data in their science classrooms (Banilower et al., 2018; Rosenberg et al., 2022b), science teachers do not presently have a way to connect the work their students do with data to the science ideas they are working to help their students understand. The result is that students’ work with data can be isolated from the sense-making students are doing about science. There is a need and an opportunity to provide science teachers with practical tools that are grounded in the framework of Bayesian data analytic methods that explicitly connect and weigh between initial ideas and empirical evidence. The specific opportunity for using Bayesian methods involves recent advances in informal statistical inference. This type of statistical inference considers how students not only interpret but also how they model data and make probabilistic generalizations from data. This opportunity is bolstered by current emphases on students’ investigations of phenomena as a locus of activity in science classrooms. This project advances middle and high school students’ data modeling in ecological contexts by taking a Bayesian approach that is supported and studied as an informal statistical inference framework. It is instantiated through a unit, digital tool, and teacher professional development program for 15 middle grades and 15 high school educators that will be conducted in a way that emphasizes students’ data analyses with phenomena of interest that relate to local context for teachers and students. Accordingly, the professional development program will begin with a two-day weekend program at the Great Smoky Mountains Institute at Tremont. A field experiment will compare the students’ informal statistical inference capabilities by using qualitative analyses of written embedded assessments of students’ probabilistic generalization from empirical data, and changes in the teachers’ confidence engaging their students will be assessed using validated surveys.
After I saw someone I admire do the same, I shared the proposal as a pre-print so others can simply see an example of a process. The first page includes an overview which represents something like the final form of the blurb. I think posting this has already helped me as an easy place to point to when I hear of others considering applying who are looking for examples of proposals; I hope others see it, too.
Stepping back from the grant proposal and the blurb, I am struck by how social the process was for me. This process was almost the opposite of someone sitting down and deeply thinking about an idea, writing a blurb and proposal from doing those things. I am proud that the original idea for using Bayesian methods was the result of my experiences (as a teacher and a data analyst). But, the maturation and development of these ideas was the result of nearly endless feedback, collaboration, and efforts at trying out different things and seeing how they might work. And now I wait and see what reviewers think and a program officer decides.
I know this was a long post; I hope that parts of it are edifying for others, but in any case it was an enjoyable and gratitude-supporting thing for me to do. Thanks to everyone named and many not named in this post for being my collaborator and source of support.