"The possibilities of harm are tremendous if teachers are naïve enough to think these scores mean anything for the vast majority of students," Professor Dede says.
CourseSmart says the data it is collecting now is a beginning. "We'll ultimately show how the student traverses the book," Devine says. "There's a correlation and causality between engagement and success."
There is also correlation, the students are learning, between perception and success.
Hillary Torres, a senior, is a good student with a low engagement index, probably because she is taking notes into a computer file not being tracked.
This could be a problem; she is a member of the Society for Human Resource Management, whose local chapter is advised by Mr Guardia.
"If he looks and sees, 'Hillary is not really reading as much as I thought', does that give him a negative image of me?" she said. "His opinion really matters. Maybe I need to change my study habits."
After two months of using the system, Guardia is coming to some conclusions of his own. His students generally are scoring well on tests and assignments. In the old days, that might have reassured him but their engagement indexes are low.
"Maybe the course is too easy and I need to challenge them a bit more," Guardia says. "Or maybe the textbooks are not as good as I thought."
Sign up for Computerworld eNewsletters.