education's digital future

Feb 5, week 5: what happened in class EDF forum - college credentials in the digital future

Today's class was a lively forum on accreditation in education's digital future. After a brief introduction from Mitchell, NYU Education and Sociology professor Richard Arum took the stage to elaborate his views on the multiple meanings of credentials. As in Arum's 2011 book with Josipa Roksa, "Academically Adrift: Limited Learning on College Campuses", he offered persuasive evidence that the credential of a college degree does not represent what we might expect it to. He began by observing that American college students study on average only about 12 hours/week, which is literally half of the amount of study time officially expected of them by the accepted definition of a credit-hour and a typical load of 12 credits. Colleges and universities participate in an elaborate ritual of certification while somehow overlooking this fact. Should we call this fraud committed with government money? Clearly, accreditation standards need revision. At the level of individual signaling, credentialing is relatively simple. Specialized benchmark and performance assessments exist for many domains, and individuals can in theory seek these out alongside or instead of college degrees (as discussed by Emily Goligoski later). But at the level of systemic evaluation, we need a more standardized unit of comparison between institutions in their quality of educational services. Individual performance assessments aren't effective for this, because they mostly measure capacities obtained prior to entering college. In his view, higher education ultimately needs some form of value-added assessment to demonstrate value to accrediting bodies and potential students.

Therese Cannon, former executive associate director at WASC, began her presentation by listing some of the challenges facing higher education of which 403x students are so familiar - huge national and global demand for skilled labor, low American degree attainment, and an American college system with a strong reputation but questionable learning outcomes, packed campuses, and skyrocketing prices. Accreditation agencies operate in a patchwork over this system, offering varying loosely-defined certifications that are often incompatible with major universities. Cannon emphasized the need for accrediting bodies themselves to maintain higher standards of quality control and share objective criteria for award of credit.

Emily Goligoski of Mozilla's Open Badge Initiative began by answering the question "What does a non-profit known for its web browser have to do with education?" Mozilla is in fact actively courting the next generation of web-crafters and informal programmers, who typically demonstrate their competence through their contributions to social sites like GitHub and Stack Overflow. Mozilla aims to improve the ability of learners to signal discrete skills by developing a secure, transferable web framework for making and sharing skill certification badges. Anyone can create a badge, and set conditions for its acquisition, without Mozilla's involvement. Badges contain metadata, they are verifiable, and they can accomodate formal and informal learning pathways. In turn, learners can share their badges through social networks and traditional resumes in order to secure jobs and other opportunities. Goligoski was optimistic about the contributions of a more pluralistic "badge ecosystem" for accrediting granular and unique skills.

Finally, John Katzman (of 2Tor, Noodle, and Princeton Review) introduced himself as "the ghost of unintended consequences", alluding to Princeton Review's success at gaming the SAT and thus rendering it less informative as an assessment. He noted that textbook publishers and MOOCs are both working towards similar goals - building a body of asynchronous curated content. And to the degree that college courses can be supported by this content, providers have an economic incentive to forget about discussion sections and "race to the bottom". Katzman pointed out that in this competitive environment, the temptation is to accredit institutions based on their ability to contribute to short-term outcomes. But once these outcomes become the focus, they incentivize providers to game them and destroy their predictive power for the long-term outcomes that we ultimately care more about. As an example, he noted that business schools are disincentivized from producing entrepreneurs, because their public rankings depend on graduating students who immediately earn high salaries. Institutions of learning need to articulate measurable long-term goals, measure them rigorously, and then make that data available for more meaningful accreditation and student choice.

Mary Huber posed a question to the panel: How do these issues play out in other countries? Richard and Therese pointed out that many countries have some kind of top-down government accrediting agency, and with it, more thorough data collection.

Greg Bybee asked about the degree to which Richard Arum's calls for a more universal standard of credit conversion were compatible with John's views about institutional goal-setting. Richard replied that yes, institutions can provide their own goals, but that this is not adequate for system-level comparison across universities. Even student reports of things like time spent studying are a step in the right direction. In addition, long-distance goals are worthwhile, but it's very hard to track the influences of schools that far ahead. John reiterated that once you attach stakes to these qualification standards you will begin to see gaming behavior.

Mitchell pushed back on Richard's calls for a uniform value-added standard - Emily offers a more plural world of certifications, so why not encourage plurality? Richard's response: because the government is spending huge amounts of money on education outcomes! If it wants to allocate that money wisely, then it needs consistent evaluation standards. Therese also noted that the current system is already something of a plurality of evaluations of varying quality!

Jeremy Jimenez asked about possible boundaries of badge topics - should people create and distribute badges for "soft skills" or traits like empathy? The panel unanimously agreed that every institution can set its own mission and therefore make its own badges. In fact, WASC just added a requirement of public-good service to its accreditation standards. And with a few final words from Mitchell about the intellectual energy of the discussion, the event was over.