How Cisco data scientists helped CommonLit innovate teacher feedback for better learning

The Transformational Tech series highlights Cisco’s nonprofit grant recipients that use technology to help transform the lives of individuals and communities.

Student reading and math comprehension is in decline. The U.S. “Nations Scorecard,” based on long term scores, recorded by the National Association of Education Progress (NAEP) and analysis by the National Bureau of Economic Research (NEBR), shows the largest average score decline in reading since 1990, and the first ever score decline in mathematics. This decline comes from analysis of testing data from over two million students in 10,000 schools in 49 states.

More can and will be done to address the implications of widening achievement for all students, and in particular students from underserved districts. To stimulate an academic recovery, we need innovative classroom solutions like CommonLit to support teachers and their students in today’s classrooms.

For years, Cisco nonprofit grantee CommonLit has focused on their mission to help students learn how to be better readers and writers. They’ve been successful in their approach: by giving students online access to reading materials, assignments, and tests, and through giving teachers resources, like dashboards that show where kids may be struggling with certain skills.

“CommonLit offers programs that are fully interactive and have everything teachers and students need—much like a reading program in a box.” Agnes Malatinszky, Chief Operating Officer at CommonLit explains.

Their highly engaging Annotation Tool, launched in July 2019, enables teachers to give relevant and real-time feedback to students. But research shows that receiving timely feedback leads to better student outcomes. So, the team at CommonLit wanted to find ways to make their Annotation Tool more effective for teachers to use.

So two years ago, with support from the Cisco Foundation, CommonLit came to Cisco’s data scientists, who volunteer with AI for Good, to help them review Annotation Tool usage information and determine ways to optimize the Annotation Tool through machine learning (ML) to help teachers and students better connect.

Partnering up and giving back

At Cisco, we have a proven track record of supporting nonprofits through our strategic social impact grants along with a strong culture of giving back. Cisco’s AI for Good program brings these values together by connecting Cisco data science talent to nonprofits, like CommonLit, that do not have the resources to use AI/ML to meet their goals.

This CommonLit and AI for Good partnered project was led by data scientist Kirtee Yadav, who also served as cause champion–which means she led the project from start to finish to ensure the project’s success. Other members of the project included Technical Lead Sampann Nigam and Team members, William Bickelmann, Bob Lapcevic, Aakriti Saxena, Sree Yadavalli, and Tana Franko.

“This CommonLit project was a good opportunity for Cisco’s data scientists to help for a good cause by using their unique skills,” stated Kirtee Yadav, a customer experience product manager at Cisco. “I jumped on this opportunity because it offered me the chance to learn new skills while I make an impact on this nonprofit.”

Finding the gaps through data science

Multiple studies have shown that the more feedback that kids receive, and the faster they receive it after completing an assignment, the more they will interact and learn from content. Yet many teachers often don’t have the time to provide detailed personalized feedback.

CommonLit challenged AI for Good data scientists to find out how that issue can be improved or resolved. The first thing the AI for Good team did was look at how Cisco’s machine learning models could modify, streamline, or improve the Annotation Tool so teachers could more efficiently give feedback to students. Through results from pulled data, they found that teachers could only provide feedback to an average of two percent of student’s comments.

“Based on our analysis of the [CommonLit] data,” Sampann Nigam, data science leader at Cisco and tech lead on the AI for Good team, pointed out, “we found that feedback from teachers pushes engagement up. So, we created an AI model to recommend feedback options to teachers.”

The data science-built solution

After months of research and hard work, The AI for Good team built a natural language processing (NLP) solution to help teachers with limited bandwidth give feedback to more students. Through NLP, the improved Annotation application will generate three suggested feedback phrases with the added option of freeform feedback.

“The feedback is built to look like teacher’s direct feedback, but instead it’s a tool that provides feedback options, which teachers can pick and send to students with just a click,” Kirtee reasoned. “In the end, AI for Good helped CommonLit improve their student teacher feedback loop.”

Sampann described the technical process to us: The AI for Good team built the phrase prediction solution using the BERT (Bi-directional Encoder Representations from Transformers) model and free-form feedback prediction, using a T5 model. Data scientists trained (fine-tuned) the BERT model by using the provided data set of annotated texts and student notes as the feature set and the feedback phrases as labels.

“Besides generating a set of phrases as suggested feedback, we decided to provide freeform feedback,” Sampann said. “These models use transfer learning, a specialized machine learning process.”

Proof of success

Over one million teachers use CommonLit in more than 80,000 schools. By fall semester of 2019, just months after launch, over 603,000 students had used the Annotation Tool, creating 3,210,156 Annotations and 5,029,973 highlights. So it comes as no surprise that the Annotation Tool, with its newly improved natural language processing, has proven very useful within its first year – close to 2.5 million annotations were logged.

“An education technology tool like CommonLit could never replace a classroom teacher, but we can make their time more effective.” Agnes said, “The way we think about machine learning and natural language processing tools, like what’s been developed for the Annotation Tool, is to make the jobs of teachers as easy as possible. We can make their jobs easier. We can nudge them towards best practices.”

This data science focused collaboration included a team of ten AI for Good data science volunteers and over 200 hours of their time spent on research, analysis, and problem-solving to successfully build a natural language-informed Annotation Tool.

The AI for Good team is working with CommonLit on options for releasing their Feedback recommendation AI model as open source, so that the education community can benefit from it.

“Cisco has been a partner for us for a couple of years now. They have supported some of our most innovative work around the technology,” Agnes explained. “And working with the Cisco AI for Good team was a unique experience that moved our organization’s most cutting-edge work forward.” 

Learn more about our partnership with CommonLit.