instagram

Impact

Source: McClanahan Associates published an evaluation of our student- and teacher-level outcomes, with analysis of math outcomes, teacher experiences, strengths and pain points.

Publications

Student Impact

Making Hay from Wheats: A Classsourcing Method to Identify Misconceptions ::
Koli Calling International Conference on Computing Education Research, 2022
“This paper conjectures that novices’ incorrect examples are a rich source of insight and presents a classsourcing method for identifying misconceptions.”

Towards a Notional Machine for Runtime Stacks and Scope: When Stacks Don’t Stack Up ::
ACM Technical Symposium on Computer Science Education (SIGCSE), 2022
“What impact does coursework around an interpreter-based approach to programming languages education have on their understanding? What is the value of the tooling we create for students to express notional machine states?”

Assessing Bootstrap:Algebra Students on Scaffolded and Unscaffolded Word Problems ::
ACM Technical Symposium on Computer Science Education (SIGCSE), 2018
“This paper reports on a more detailed study with (a) data from more than 3x as many students, (b) analysis of performance changes in incorrect answers, (c) problems in which the Bootstrap scaffolds have been removed, and (d) an IRT analysis... Our results confirm that students improve on algebraic word problems after completing the module, even on unscaffolded problems.”

Transferring Skills at Solving Word Problems from Computing to Algebra Through Bootstrap ::
ACM Technical Symposium on Computer Science Education (SIGCSE), 2015
“Many educators have tried to leverage computing or programming to help improve students' achievement in mathematics. However, several hopes of performance gains — particularly in algebra — have come up short.”

Creativity, Customization, and Ownership: Game Design in Bootstrap:Algebra ::
ACM Technical Symposium on Computer Science Education (SIGCSE), 2018
“Our study shows that despite very limited personalization, students still feel a strong sense of ownership, originality, and pride in their creations. We also find that females find videogame creation just as satisfying as males, which contradicts some prior research but may also reflect the nature of games created in this curriculum and the opportunities it offers for self-expression.”

Automated, Targeted Testing of Property-Based Testing Predicates ::
The Art, Science, and Engineering of Programming, 2022
“Effective teaching requires a clear understanding of student knowledge and progress. Our methods enable a rich and automated analysis of student performance on PBT that yields insight into their understanding and can capture misconceptions.”

Structural versus Pipeline Composition of Higher-Order Functions (Experience Report) ::
ACM SIGPLAN International Conference on Functional Programming, 2022
“In teaching students to program with compositions of higher-order functions, we have encountered a sharp distinction in the difficulty of problems as perceived by students...We describe the phenomenon, provide some preliminary evidence of the difference in difficulty, and suggest consequences for functional programming pedagogy.”

Plan Composition Using Higher-Order Functions ::
SIGCSE International Computing Education Research Conference, 2022
“Explicit planning atop a vocabulary of primitive operations has not received much attention in computing; our work suggests that HOFs provide a useful initial set of planning primitives.”

Developing Behavioral Concepts of Higher-Order Functions ::
SIGCSE International Computing Education Research Conference, 2021
“Higher-order functions deserve far more attention than they have been paid in the literature on programming education. Their increasing use in several important domains makes this need critical. Our proposed methods for conducting such research are another contribution of this work. Our findings and methods should also be relevant for exploring how students understand libraries and APIs.”

Early Post-Secondary Student Performance of Adversarial Thinking ::
SIGCSE International Computing Education Research Conference, 2021
“'Adversarial thinking' (AT) is viewed as a central idea in cybersecurity. We believe a similar idea carries over into other critical areas as well, such as understanding the perils of social networks and machine learning.”

The Influence of Teaching Programming on Learning Mathematics ::
Society for Information Technology & Teacher Education International Conference (SITE), 2013
“A pre and post mathematics inventory was used to measure the impact teaching this course (which used a video game development programming pedagogy) had on student understanding of mathematics. The findings from the study suggest that student understanding of functions and variables did increase after participating in the course.”

Using Design Alternatives to Learn About Data Organizations ::
SIGCSE International Computing Education Research Conference, 2020
“We consider a technique for getting students to engage with [data organization], grounded in theories of examples and differences.”

Using Relational Problems to Teach Property-Based Testing ::
The Art, Science, and Engineering of Programming, 2021
“We find that the use of 'relational' problems --- those for which an input may admit multiple valid outputs --- easily motivates the use of [Property-Based Testing]. ”

Our Curricular Philosophy

Integrated Data Science for Secondary Schools: Design and Assessment of a Curriculum ::
ACM Technical Symposium on Computer Science Education (SIGCSE), 2022
“We propose that secondary-school data-science curricula should be based on four key ingredients: two are technical (programming and statistics, with visualization sitting at their intersection), while two are human-facing (meaningful domains, and civic responsibility). We describe their relationship and argue for their importance.”

Evolving a K-12 Curriculum for Integrating Computer Science into Mathematics ::
ACM Technical Symposium on Computer Science Education, 2021
“We discuss the initial design and the various modifications we have made over the years to better support math instruction, leading to our goal of using integrated curricula to cover standards in both math and CS.”

Data-Centricity: A Challenge and Opportunity for Computing Education ::
Communications of the ACM, 2020
“Rethinking the content of introductory computing around a data-centric approach to better engage and support a diversity of students.”

Data Science as a Route to AI for Middle- and High-School Students ::
AAAI 2019 Fall Symposium: Teaching AI in K-12, 2019
“This paper briefly describes [Bootstrap:Data Science]'s design, content, and outcomes, and explains its value on the road to AI curricula.”

What Does It Mean for a Curriculum to Succeed? ::
Communications of the ACM, 2019
“Examining the expansion, proliferation, and integration of computing education everywhere across academic disciplines.”

Programming Paradigms and Beyond ::
The Cambridge Handbook of Computing Education Research, 2019
“This chapter suggests the use of notional machines, i.e., human-accessible operational semantics, as a central organizing concept for understanding languages.”

Modernizing Plan-Composition Studies ::
ACM Technical Symposium on Computer Science Education, 2016
“Plan composition is an important but under-studied topic in programming education...This paper presents rationale and details for a modernized study of plan composition that accommodates a broader range of programming languages and problem features.”

In-Flow Peer Review ::
Conference on Innovation and Technology in Computer Science Education, 2014
“Peer-review is a valuable tool that helps both the reviewee, who receives feedback about his work, and the reviewer, who sees different potential solutions and improves her ability to critique work...”

Bootstrap: Going Beyond Programming in After-School Computer Science ::
SPLASH Education Symposium (SPLASH-E), 2013
“Adding computer science to already-packed middle- and high-school curricula can be difficult; after-school programs offer an enticing alternative to broadening student exposure to computing.”

Why Computer Science Doesn't Matter ::
Communications of the ACM (CACM), 2009
“Aligning computer science with high school mathematics can help turn it into an essential subject for all students. ”

Error Messages Are Classifiers: A Process to Design and Evaluate Error Messages ::
SPLASH Onward!, 2017
“We take the perspective that error reports are really classifiers of program information. They should therefore be subjected to the same measures as other classifiers (e.g., precision and recall).”

Our Technology Edge

Tools to Support Data Analysis and Data Science in K-12 Education ::
National Academies, September 2022
“Scripting languages (e.g., Python, R, and Julia) are commonly used by professional statisticians and data scientists to conduct statistical analyses. They are increasingly being used in education contexts and some languages (e.g., Pyret within Bootstrap, 2022) have been specifically developed for use in education...”

Adapting Student IDEs for Blind Programmers ::
Koli Calling International Conference on Computing Education Research, 2020
“Even in the absence of legal demands, our discipline has a moral obligation to accommodate the needs of all students...”

Will Students Write Tests Early Without Coercion? ::
Koli Calling International Conference on Computing Education Research, 2020
“In this work, we augment the student’s programming environment to encourage examples-first development, and design a novel measure to assess students’ adherence to this methodology...”

Accessible AST-Based Programming for Visually-Impaired Programmers ::
ACM Technical Symposium on Computer Science Education, 2019
“Users had improved accuracy when completing tasks, were significantly better able to orient when reading code, and felt better about completing the tasks when using the tool.”

Event Loops as First-Class Values: A Case Study in Pedagogic Language Design ::
The Art, Science, and Engineering of Programming, 2019
“The new design enables programmatic control over event-driven programs. This simplifies reflecting on program behavior, and eliminates many unnecessary curricular dependencies imposed by the old design.”

Executable Examples for Programming Problem Comprehension ::
SIGCSE International Computing Education Research Conference, 2019
“We propose a mechanism that provides students with instant feedback on their examples, independent of their implementation progress. We assess the impact of such an interface on an introductory programming course...”

Putting in All the Stops: Execution Control for JavaScript ::
ACM SIGPLAN Conference on Programming Language Design and Implementation, 2018
“This paper presents Stopify, a source-to-source compiler that extends JavaScript with debugging abstractions and blocking operations, and easily integrates with existing compilers.”

Evaluating the Tracing of Recursion in the Substitution Notional Machine ::
ACM Technical Symposium on Computer Science Education, 2018
Unlike most K-12 CS courses, all of Bootstrap's modules use functional programming, which allows students to leverage the substitution model that's also learned in standard algebra. This study demonstrates the benefits of this approach for teaching sophisticated programming concepts.

Blocks Versus Text: Ongoing Lessons from Bootstrap ::
Blocks and Beyond Workshop, 2015
“One important viewpoint that we do not believe has been explored in the literature is the effect on teachers during professional development workshops....”

Measuring the Effectiveness of Error Messages Designed for Novice Programmers ::
ACM Technical Symposium on Computer Science Education, 2011
“Unlike existing work in this area, we study messages at a finegrained level by analyzing the edits students make in response to various classes of errors.”

Mind Your Language: On Novices’ Interactions with Error Messages ::
SPLASH Onward!, 2011
“ This paper reports on a series of studies that explore beginning students’ interactions with the vocabulary and source-expression highlighting...”

Whalesong: Running Racket in the Browser ::
Dynamic Languages Symposium (DLS), 2013
“Implementing this model in a browser's JavaScript is not trivial. The main obstacles lie in asynchronicity and event-driven event-loops.”

WeScheme: The Browser is Your Programming Environment ::
Conference on Innovation and Technology in Computer Science Education (ITiCSE), 2011
“WeScheme is being used successfully by students, and is especially valuable in schools that have prohibitions on installing new software or lack the computational demands of more intensive programming environments.”

A Functional I/O System ::
ACM SIGPLAN International Conference on Functional Programming (ICFP), 2009
“Once middle school students are hooked on this form of programming, our curriculum provides a smooth path for them from pre-algebra to freshman courses in college on object-oriented design and theorem proving.”

Prior work from "Program by Design"

[Bootstrap is grateful for the tremendous research done by the Program by Design team.]

DrScheme: A Programming Environment for Scheme ::
Journal of Functional Programming, 2002
“The environment is especially useful for students, because it has a tower of syntactically restricted variants of Scheme that are designed to catch typical student mistakes and explain them in terms the students understand.”

DrScheme: A Programming Environment for Scheme ::
Journal of Functional Programming, 2002
“The environment is especially useful for students, because it has a tower of syntactically restricted variants of Scheme that are designed to catch typical student mistakes and explain them in terms the students understand.”

The Structure and Interpretation of the Computer Science Curriculum ::
Journal of Functional Programming, 2004
“In this pearl, we propose an alternative role for functional programming in the first-year curriculum.”

The TeachScheme! Project: Computing and Programming for Every Student ::
Computer Science Education, 2004
“The TeachScheme! Project aims to reform three aspects of introductory programming courses in secondary schools.”