A multidisciplinary team of faculty and graduate students were recently honored with a Best Paper Award by the American Educational Research Association (AERA) for a study that uncovered unexpected results.

The team, comprised of Albert Ritzhaupt, associate professor of Educational Technology, Zhen Xu, doctoral student in Educational Technology, Fengchun Tian, former School of Teaching and Learning Visiting Scholar, and Karthik Umapathy, University of North Florida associate professor in the School of Computing, were selected to receive the award in the special interest group category of “Computer and Internet Applications in Education.”

Their collaborative article, titled “Block-based versus text-based programming environments on novice student learning outcomes: a meta-analysis study” compares the overall effectiveness of block-based and text-based programming environments in introducing new students to computer programming concepts. In text-based environments, students engage with text to enter computer commands, while block-based environments offer drag-and-drop interactions to move blocks of computer commands.  

Block-based environments have been considered more beneficial in introducing students to computer programming and in producing better prepared computer programmers, but there is very little evidence to support or deny these claims.

“Interest with adopting block-based programming is on the rise, particularly for introducing programming to novice programmers,” Umapathy said. “However, advanced programming courses and working professionals use text-based programming environments.”

The question became how to prove which environment was in fact more helpful for novice students. The answer: Conducting a meta-analysis study.
The team sorted through hundreds of research articles seeking those that explicitly compared block-based and text-based programming environment. Only 13 articles met their criteria.

By the end of the study, rather than concluding with an answer, they instead discovered new questions.

“I think that’s a good sign of the study” Xu said.

Currently, there is not enough evidence in the field to support the use of one programming environment over the other. While the outcomes showed block-based programming had a narrow lead, the difference was statistically insignificant. In fact, their results highlight that publication bias may be present in the data.“The most interesting thing is, is that we can’t make the claims that are being made right now,” Ritzhaupt said.

“The team hopes that their research will help to shed light on the true differences between text-based and block-based programming environments. There were many things that came to light through this literature that we were able to say in our conclusions and discussion, that are things that need to be addressed in the future,” Ritzhaupt said, “and I think that’s what probably sold the article.”

Ritzhaupt and Xu accepted the award at the 2019 AERA Conference on behalf of the team.