The end of the essay?

"Robots Playing Chess" (CC BY-NC-SA 2.0) by Joe Shlabotnik

Will artificial intelligence kill off the essay as an assessment form? That seems very likely after reading an article from the University of Sydney, Assessment and integrity in the age of essay-writing artificial intelligence. It describes how you can generate a plausible and well-argued essay on just about any topic using AI algorithms without any risk of falling foul of anti-plagiarism software. The text will be original and offer a variety of perspectives gathered from analysis of countless related texts on the net. In many cases these essays are far better than many students could ever write themselves and the technology is developing rapidly. Similarly the use of AI in teaching is progressing rapidly and we face the prospect of robots teaching and assessing robots.

The most remarkable aspect of this article is that it was written by an AI application called GPT-3 with some minor human edits. It analyses the arguments for and against AI in education and gives examples of how AI generated texts could be used in a positive manner: for example, letting AI generate a text on an introductory paragraph and asking students to compare the AI text with the original. It is remarkably insightful on its own limitations and how academics need to rethink traditional practices to counter the threat of AI.

Because artificial intelligence is trained on a huge corpus of text and has access to the entire internet, it excels at writing and responding to textual prompts. This includes topics that would otherwise be perceived as meeting criteria for authentic assessment. This presents a challenge for higher education academics because we are so accustomed to using exams and other assessments that focus on student knowledge. If artificial intelligence can write essays and answer exam questions, higher education academics need to radically rethink learning, teaching, and assessment in the post-machine era.

But the question remains of whether the essay is a valid form of academic assessment and what new methods we should turn to. The gut reaction to the problem could be doubling down on traditional proctored exam hall tests with no access to digital devices or textbooks but I hope we can think further than this. Learning to write a well argued essay or article is a fundamental skill in all forms of science and it is hard to imagine higher education without this crucial element. But if we can instantly generate an acceptable imitation the exercise becomes somewhat futile. Problem and project-based learning as well as a greater focus on interviews and live seminar discussions would seem to be more relevant both in terms of assessment and as training for professional practice. Writing and critical thinking skills must be learned and practiced but somehow we need new ways to use them. There are already examples of AI-generated texts getting accepted for journal publication and as conference submissions. The foundations of academia are under threat and we need to develop new strategies and methods. 

Jon Dron writes about this in a post, So, this is a thing… and sees the answer in a refocus on people and genuine interaction.

This is a wake-up call. Soon, if not already, most of the training data for the AIs will be generated by AIs. Unchecked, the result is going to be a set of ever-worse copies of copies, that become what the next generation consumes and learns from, in a vicious spiral that leaves us at best stagnant, at worst something akin to the Eloi in H.G. Wells’s Time Machine. If we don’t want this to happen then it is time for educators to reclaim, to celebrate, and (perhaps a little) to reinvent our humanity. We need, more and more, to think of education as a process of learning to be, not of learning to do, except insofar as the doing contributes to our being. It’s about people, learning to be people, in the presence of and through interaction with other people. It’s about creativity, compassion, and meaning, not the achievement of outcomes a machine could replicate with ease. I think it should always have been this way.

Another article, Will artificial intelligence be able to write my college essay? by Eamon Costello and Mark Brown, Dublin City University, raises the need to rethink how we assess learning rather than finding ways to defend the traditional essay. 

Do we try to tame AI to protect old ways of learning or should we embrace its potential and reimagine our assessment practices to reflect the modern reality of living in the 21st century? One creative educator had his students purposefully use and evaluate AI essay writers as part of their assignment.

Finally, this perspective is echoed by the AI-generated article itself, showing a surprising level of insight on its own limitations.

In particular, there is a need to focus on developing higher-order thinking skills such as creativity, critical thinking, and problem-solving, which are not easily replicated by machines. Additionally, it will be important to create opportunities for students to interact with each other and with their instructors on a regular basis, in order to promote the social and emotional skills that are essential for success in the workplace.

 

AI-generated essays - time to rethink written assignments

Students will employ AI to write assignments. Teachers will use AI to assess them. Nobody learns, nobody gains. If ever there were a time to rethink assessment, it’s now.

This is a quote from an article by Mike Sharples, London School of Economics, New AI tools that can write student essays require educators to rethink teaching and assessment. There are now tools that use artificial intelligence (AI) to generate highly plausible academic essays, complete with references. The article takes an example of a short essay about the problems around the popular concept of learning styles generated by a Transformer AI program, GPT-3. The user simply entered the first sentence and AI completed the essay. The result is not particularly insightful but good enough to pass and it won't show up in any plagiarism control since the text is completely original. If the essay is fed back into the tool by the teacher it can write a similarly plausible comment on the essay. The whole assignment can therefore be performed by AI, prompting the quote above.

Even if some of the essays generated this way may still have weaknesses (the example in the article has false citations) the whole point of AI is that it is constantly learning and improving. The phenomenon is not new, it has been possible for many years to pay someone else to write your essays for you via essay mills, but now the human element has finally been removed. Does this mean the end of the written assignment as an examination form? 

The author suggests a few ways of using the AI transformer in a constructive way, for example by getting students to generate AI texts and then find faults in them and improve them. This reminds me of how some teachers tackled plagiarism by writing a sample essay/article that included several form of plagiarism as well as poor citation practice and asked the students to find the problems and correct them. 

But the main point here is that we need to move on to new ways of assessing students and avoid examination methods that ask questions that can be automatically generated or copied from the internet. 

Finally, as educators, if we are setting students assignments that can be answered by AI Transformers, are we really helping students learn? There are many better ways to assess for learning: constructive feedback, peer assessment, teachback. If Transformer AI systems have a lasting influence on education, maybe that will come from educators and policy makers having to rethink how to assess students, away from setting assignments that machines can answer, towards assessment for learning.

Live assessment activities like interviews, presentations, debates or round table discussions can be run either on-site or online and are almost impossible to cheat in. But that brings us to the eternal question of how to move the focus in education from extrinsic motivation (exams, credentials) to intrinsic motivation (satisfaction, self-confidence, pride). Focus on competition, rewards and results encourages cheating among some, whereas activities that focus on community, learning for pleasure and intangible rewards are generally free of cheating. If learning is in the forefront there is simply no point in cheating. Hopefully AI-generated essays will remain a mere curiosity.

The power of grades - student resistance to collaborative learning

"Ellis Hall Active Learning Classroom" (CC BY-NC-ND 2.0) by queensu

The concept of the flipped classroom, where lecture input is recorded and classroom meetings focus on discussion and problem-solving, has become extremely popular, not least during the pandemic as a way of making online classes more participatory. However, like all attempts to diverge from traditional practices, there is often opposition, especially from the students who may see new methods as a threat. The key issue seems to be the perceived withdrawal of the teacher from the centre stage as the students are given the freedom to take control of their learning and learn through collaboration. This can be seen as an unwelcome imposition by many students and leads to insecurity since traditional roles and models have been disturbed.

This sense of threat is clearly described in an article in the journal Teaching in Higher Education, What does it mean to do teaching? A qualitative study of resistance to Flipped Learning in a higher education context (sadly not open access). It describes the implementation of a flipped learning approach in a teacher training programme at Macquarie University in Australia. The main benefit of flipped learning is that by switching the delivery of content to recorded lectures and demonstrations this will enable teachers to devote contact time to providing timely feedback, guiding, challenging and encouraging. The problem is that students have high expectations of the teacher as performer and see the flipped model as an abdication of responsibility. The stand-up lecture is seen as the definition of teaching - that's what teachers do. When that is withdrawn, the students feel cheated.

As indicated in the title of this paper, the Flipped Learning trial was met with resistance – in fact vociferous resistance. As I analysed data from the study and also reflected on my engagement with students during the delivery of the flipped course, a ubiquitous refrain emerged. Students drew persistent attention to an absence. In absenting from the traditional stand and deliver tutorial presence, I had denied my student’s access to the sage on the stage. My student’s indicated fear of missing out on critical information, discomfort of engaging in core content in asynchronous environments and resentment of what they understood as my absence in the tutorial (i.e. I was no longer sage on the stage but a guide to the side).

The students saw the method as at best interesting, but most commonly useless, time-consuming and confusing. The traditional lecture-based approach is seen as a way of guaranteeing that the necessary course material has been "covered" and that the students know what they have to learn for the exam.

The need to get the best exam grades to enhance future job prospects is the hard currency of higher education and pedagogic innovation sometimes clashes with these very pragmatic needs. This clash is demonstrated in another interesting article: Mediating Students’ Fixation with Grades in an Inquiry-Based Undergraduate Biology Course. Here students on a year one course in biology at Borealis University were studied as they worked through a redesigned course based on inquiry-based learning. Instead of the traditional lecture-based format, the students were expected to collaborate to investigate the main topics and reflect on the process.

... the curriculum required students to work cooperatively in groups to design and execute experiments that are authentically novel, gather and analyze data, and present their findings orally and in writing. Evaluative criteria included rubrics for transferred skills, and summative assessments for acquisition and application. Scientific concepts of uncertainty, peer review, and self-reflection were explicit parts of the design.

This approach has been shown to lead to deeper learning as well as learning vital skills of search skills, source criticism, collaboration and self-reflection, but the results show a serious mismatch between the teachers' well planned approach and the students' perception of teaching. As in the first example, the students were vociferous in their criticism of the course methods and were especially critical that the experiment seriously threatened their chances of getting the top grades they need to progress in their career path. The teacher expected them to work things out for themselves and that was seen as an abdication of responsibility - the teacher is expected to go through the syllabus and make it clear to students exactly what they need to know to get a top grade.

Instructors who did not give explicit directions or answer questions directly were perceived as obstructionist. One student framed her professor’s response to her repeated requests for directions as adversarial, saying, “[The instructor] said, ‘You’re not gonna have a step by step. You need to figure it out.’ And I said, ‘Well, I’m sorry, but I’m not paying tuition to figure it out myself.’”

The article concludes that the extrinsic motivation of grades has a negative effect on the intrinsic motivation needed for an inquiry-based approach to succeed. The inquiry-based approach was designed to enhance student autonomy but the effect was the opposite - students felt cast adrift and powerless whereas in the traditional setting they felt more autonomous because it was clear what they had to do to achieve desired results. It's hard to be creative in a world governed by grades.

The key takeaway here is that any changes in course design have to be clearly communicated to the students and they need to know why the changes have been made and how it will benefit them. In this case the student expectations were completely at odds with the teachers' methodology. Maybe if you want to implement more collaborative methods you should introduce them in stages though the first year, slowly working towards collaborative learning rather than throwing the students in at the deep end. This means that all teachers in a programme have agreed on this progress and that each course is part of a progression towards greater autonomy and active learning.

References

Wilson, K. (2020) What does it mean to do teaching? A qualitative study of resistance to Flipped Learning in a higher education context, Teaching in Higher Education, DOI: 10.1080/13562517.2020.1822312

DeFeo, D.J., Tran, T.C. & Gerken, S. Mediating Students’ Fixation with Grades in an Inquiry-Based Undergraduate Biology Course. Sci & Educ 30, 81–102 (2021). https://doi.org/10.1007/s11191-020-00161-3

Escaping from the exam hall

"Exam time at Goldsmiths" (CC BY-NC 2.0) by Rain Rabbit

Traditions are so hard to break and one of the most entrenched is the proctored written examination. Writing for hours in a silent hall with only your memory to help you has been a rite of passage for generations of students all over the world; my own student years were dominated by these stressful events. The exam halls have been empty for over a year now but the problem of how to examine large groups of students online has lead to much stress and debate. Many have questioned the value of high-stakes summative examination; what valuable career skills are we actually testing in the exam hall? The problem is finding an alternative that will work with large student cohorts and can minimise the risk of cheating.

Interest is growing in more authentic assessment methods such as projects, case studies and portfolios, but these all involve major shifts in course design and teaching practice. Teachers who have always used proctored exams need training and time to reorganise their courses around alternative assessment methods and students will also need to be convinced of the benefits. In a recent article, a teacher here in Sweden described how he carries out one-to-one interviews with all students as an examination method, even with classes of over 200 (read article in Swedish). Each interview takes 15-20 minutes and if that seems like a lot he claimed that correcting written papers took roughly the same time per student. An interview is more personal and makes cheating extremely difficult and an experienced teacher gets a good picture of the student's ability in that time. 

This however is an exceptional option and most teachers have chosen solutions that try to emulate the exam hall. A common makeshift solution is where all students log into Zoom and are then monitored via their webcams and microphones. There are plenty of digital examination tools that will lock down the student's computer for the duration of the exam but the most controversial solution has been to use the remote proctoring solutions offered by companies like Proctorio, ProctorU and ExamSoft. These solutions involve remote monitoring of students, AI analysis of facial expressions, keystrokes, mouse clicks and registration of any suspicious activities in the student's room (someone entering the room). The integrity and privacy issues of this kind of surveillance and the question of what the companies do with all that data has been well covered in the past year.

However, it is tempting for a university to opt into remote proctoring since it fits in so well with traditional examination. Remote proctoring allows teachers to continue using their summative examinations and the alternatives all seem to demand extra work and retraining, something most already over-worked teachers will object to. A good example of an institution that was brave enough to avoid remote proctoring and help teachers make the transition to authentic examination is described in  an article by a group from the University of Michigan–Dearborn, What Happens When You Close the Door on Remote Proctoring? Moving Toward Authentic Assessments with a People-Centered Approach. Despite pressure to implement remote proctoring they decided to focus on changing their approach to assessment and examination by supporting teachers in their transition. The article explains their rationale for this move and describes the support process that included weekly practical workshops to work through new methods. To lessen the teachers' workload they hired a numbered of specially trained graders to help with the more time-consuming assessment of the new examination methods. The initiative is not complete and they have not completed a transition to authentic assessment but the new approach is gaining acceptance. In conclusion they offer the following advice.

In our experience, leaders such as educational developers, instructional designers, academic technologists, department chairs, and deans will need to be prepared to extensively support faculty during the transition. Many instructors have been using proctored exams for their entire careers and see them as essential to the teaching and learning process. Even as teaching and learning professionals may be eager to see instructors drop high-stakes exams all together, it is important to recognize that changing a long-held teaching practice can feel destabilizing and will likely bring some unforeseen challenges and resistance from instructors (Smith, 2020). Monetary resources saved from the refusal of remote proctoring services can be redirected to invest in ongoing faculty development on assessment and other teaching-related topics. We believe our investment in people, rather than remote proctoring services, will be more robust to the inevitable changes in learning formats and technology that are to come.

You are being watched - the rise of educational surveillance

"Big Brother is Watching" (CC BY 2.0) by abracapocus_pocuscadabra

The use of artificial intelligence in education seems increasingly to be more about surveillance and control than fostering learning. During the past year there has been considerable concern about the ethics of remote proctoring where students were subject to constant surveillance by webcam, microphone and analysis of keystrokes and mouse clicks during online exams, see for example an article in Inside E-learning, Learning or surveillance? What students say about edtech and covid-19. Any unexpected activity in the student's behaviour (eg "abnormal eye movement") or movement in the room is detected and may lead to disqualification. Not only are such methods highly intrusive it is unclear what happens to all the data collected by the responsible companies. There must be better ways of assessing a student's ability than this.

If remote proctoring raises concerns then the next example sends shivers down the spine. An article on the site Rest of the world, China is home to a growing market for dubious “emotion recognition” technology, describes how classrooms are equipped with face recognition technology and each student's facial reactions are constantly monitored. 

Every second, the surveillance cameras installed in each classroom at Niulanshan First Secondary School in Beijing snap a photo. The images are then fed into the Classroom Care System, an “emotion recognition” program developed by Hanwang Technology. It identifies each student’s face and analyzes their behavior: a student rifling through their desk might be labeled “distracted,” while another looking at the board would be labeled “focused.” Other behavioral categories include answering questions, interacting with other students, writing, and sleeping. Teachers and parents receive a weekly report through a mobile app, which can be unsparing: In one, a student who had answered just a single question in his English class was called out for low participation — despite the app recording him as “focused” 94% of the time.

Imagine then coupling this with analysis of every mouse click and keystroke and we have total control, or rather the illusion of control since the conclusions made by AI may be based on in-built biases (like the student called for low participation in the example above). The idea that algorithms can accurately assess a student's emotions by analysing facial expression is ridiculous. We all know how hard it is to read another person's face. But once you get AI making decisions it becomes almost impossible to question them since we tend to see computers as impartial and infallible. No matter how questionable such technologies may be there's big money to be made. The article claims that the emotional recognition market may be worth more than $33 billion by 2023. Money talks. For further serious concerns about facial recognition, see an article in Mashable, 9 scary revelations from 40 years of facial recognition research.

There are, of course, positive applications of AI in education but the key question in all cases is who owns the data, on what terms and do the students have the right to be forgotten? We all need to be very cautious about letting these technologies into the classroom.

Online exam cheating - a matter of trust and support

One of the most discussed aspects of the last few months has been the issue of online examination. The sudden transition to online delivery worked reasonably well for teaching and collaboration, but teachers whose courses rely on traditional written examinations had difficulties and there are plenty of news stories on the increase in cheating in such settings. Many exams relied on makeshift invigilation via Zoom/Teams/Skype since there was no time to find a more secure option. Online proctoring systems were purchased by many institutions, often hastily, and have resulted in some privacy issues as illustrated in an article in Sydney Morning Herald, 'You’re being watched and recorded, every breath': Students unsettled by exam software.

It’s not fun, knowing you’re being watched and recorded, every breath and movement. It’s even more invasive than a normal exam. Someone is staring at just you, but you have no idea who it is.

In many cases it was not possible to change the whole assessment process so quickly since such changes in a curriculum must be announced to students before commencing the course. 

There's a good discussion on online cheating in an article in Inside Higher Ed, Best Way to Stop Cheating in Online Courses? ‘Teach Better’. It's based on a webcast, The Academic Integrity Braintrust, featuring three experts in the field, Tricia Bertram Gallant (University of California San Diego), Douglas Harrison (University of Maryland) and David Rettinger, University of Mary Washington. One important point to remember when judging spring term 2020 is that it was a global crisis when all university education went online with virtually no preparation time. Students were forced to study from home, in less than perfect learning spaces and with often considerable pressure and stress. The temptation to cheat is very strong and if the opportunity arises to cut a corner, many of us will. 

An important factor that leads to cheating is when students feel isolated and out of touch with teachers and colleagues. Harrison picks up this theme:

... when students don’t feel connected and a sense of belonging to the learning community, whether it's online or face-to-face, they are more likely to detach from any sense of collective community responsibility or ethics and substitute for that a pure ethic of mercenary self-interest.

 Maybe increased focus on team-building activities and collaborative learning could create a climate where cheating becomes irrelevant, especially if the course builds on group assignments and the demonstration of skills. Ramping up surveillance and prevention only leads to a cat and mouse mentality. Harrison again:

We end up focusing on the worst possible negative outcomes that the most malicious and malintended student would engage in, rather than starting with, 'What’s the best teaching and learning experience I can construct and deliver for the vast majority of students who are there to learn authentically and who want to succeed?

As a way forward, David Rettinger offers a list of small scaleable changes for improving academic integrity, based largely on community building and the redesign of assignments and assessments. Establishing a sense of community takes time and requires careful planning, but if the class can agree on common ground rules for how they want to work this can lead to a sense of group loyalty that makes cheating extremely difficult to consider. Academic integrity issues need to be taught and discussed and institutional policies made clear to all. Basically, if the students are consulted and involved in how they learn they will be much less likely to cheat. In terms of examination forms, using numerous low-stake tests or assignments reduces stress and makes cheating less tempting.

Rettinger said he had replaced exams in some courses with lots of low-stakes quizzes -- with "stakes so low that it’s not worth cheating." When his courses do include final exams, they are open book, and all of the questions on them are drawn from those students have shared in discussion boards over the course of the term, so that "gives them a sense of control over the assignment."

Maybe this experience will see a drop in traditional 3-4 hour exam hall testing once our campuses open up again. Maybe we can move to more project-based assessment and assignments that reflect the skills and knowledge used in professional situations. Maybe ...

Indian MOOCs going mainstream

There are plenty of examples of universities offering credits for MOOCs and the key to awarding credentials is a formal proctored examination at the end of the course. In some countries this can be rather expensive but in India it seems that MOOCs are being integrated into the state higher education system. According to an article in Class Central, In India, MOOCs Are Now Part of the Education System, the state sponsored MOOC platform SWAYAM offers learners the chance to sit digital examinations at over 1000 regional centres all over the country and the successful candidates get valid credits that can count towards their degrees. This national coverage means that taking an exam becomes more accessible and although candidates pay around $15 to sit an exam the money is refunded if they pass.

Another interesting feature of SWAYAM's strategy is that teachers are given the financial incentive of up to $150 per hour of online teaching (recorded video) and other rates for material development, according to a remuneration model issued by India’s Ministry of Human Resource Development. Universities can now allow students to take up to 20% of their degree programme as MOOCs thus offering greater flexibility and also recognising online education as an integral part of higher education in India. Universities can also offer other institutions' MOOCs and thereby widening their offer and making credit transfer more common.

Twice a year, institutions pick SWAYAM courses they’ll grant credit for in the upcoming term. Note that they may pick courses offered by other institutions, allowing them to tap into the strengths of schools nationwide to build richer curricula. For instance, they may leverage SWAYAM to offer high-demand courses for which they lack qualified instructors on campus.

The Indian government hopes that this scheme will help to widen participation in higher education by allowing new students to try a course, get credits and then hopefully move on from there into a full programme at a university. Maybe local centres can offer practical help to new student groups to help them learn the skills of online learning, for example in face-to-face introduction meetings to help them get started. None of this is particularly new since open universities have been working in this way for many years but it's refreshing to see that MOOC platforms can be integrated into the higher education system and offer alternative paths with tangible rewards.

Is edtech simply automating tradition?

CC BY-SA Some rights reserved by ISAPUT on Wikimedia Commons

For a few years the internet was an exciting new space where we thought we could build a better world through innovation, borderless collaboration and a culture of sharing. Today big business reigns supreme and technology is being used for control, surveillance and commercial gain. After several years of speculation and experimentation we are now beginning to see how analytics and artificial intelligence are going to be used in education. As governments become increasingly concerned with accountability, profitability and competition, technology is feeding this drive with statistics.

A post by Mike Crowley, Google, ISTE, and the Death of EdTech, reflects on the recent ISTE conference and sees worrying tendencies that today's edtech industry is increasingly about control rather than creativity. Technology is being used to make traditional practice more efficient rather than fostering real learning.

All week, the ISTE Expo Hall gushed with predictions that Artificial Intelligence will automate the traditional classroom. Auto-grading is now a thing. Marketeers touted emerging tools to help students study smarter, by enabling them to memorise more information faster. Ultimately, though, the Big News, like Google Classroom, is the proliferation of software that allows teachers to monitor and keep students on-task. The lofty ambitions for education were summed up thus. Make tradition more efficient. Stop bored students from cheating on mindless, low-level assessments. Deliver content like Windows 98 is the next big thing. New tools. Old thinking. Systems, not learners.

The tyranny of tradition is extremely hard to escape from. The traditional educational paradigm of learning facts that can be easily tested and categorised is a perfect fit for the smart systems of today. We can use analytics and AI to provide the statistical illusion that students have achieved the learning objectives we set them. But technology can and should be able to do so much more and that's why Crowley states:

Make no mistake about it. EdTech as we currently know it is dead, it’s over. We should retire the phrase right now. If education is to be the target of an industry that has grown increasingly obsessed with standardization, control, automation, and delivery efficiencies, then we must opt out.
Related to this is the use of automated grading of assignments that has been under development for several years now. This involves computer analysis of a written assignment to check for coherence, argumentation, linguistic style, grammar and fluency. An article on NPR, More States Opting To 'Robo-Grade' Student Essays By Computer, examines recent experience of automated grading at several US universities. Many educators are impressed by the accuracy of these tools, often giving similar grades to human examiners and if the computer is unsure it flags for teacher assessment. The potential savings are obvious and very attractive to cash-strapped institutions.

Several states including Utah and Ohio already use automated grading on their standardized tests. Cyndee Carter, assessment development coordinator for the Utah State Board of Education, says the state began very cautiously, at first making sure every machine-graded essay was also read by a real person. But she says the computer scoring has proven "spot-on" and Utah now lets machines be the sole judge of the vast majority of essays. In about 20 percent of cases, she says, when the computer detects something unusual, or is on the fence between two scores, it flags an essay for human review. But all in all, she says the automated scoring system has been a boon for the state, not only for the cost savings, but also because it enables teachers to get test results back in minutes rather than months.

Impressive indeed, but there will always be ways to hack the system and already there are examples of text generators that produce nonsensical texts that satisfy the grading system's preferences. This type of cat-and-mouse game has nothing to do with learning. When the only purpose of writing an assignment is to get a grade then people will simply do whatever it takes to get that grade as easily as possible. If the assignment has real meaning to more people than simply an examiner (or a robot) then the game element disappears. We need to develop methods of assessing student ability based on the impact of their projects on real people. Maybe traditional for-teacher's-eyes-only assignments are the problem.

One clear use for automated grading systems is to help the students improve their writing skills. The system can provide repeated formative assessment opportunities, helping students to improve the coherence and style of their assignments. They can submit several times during the writing process, something that no teacher could ever have time to deal with, and then submit the final version to the teacher for grading. However a more collaborative approach already used by many teachers is to encourage peer assessment with fellow students providing feedback during the writing process. This method benefits everyone and the students can learn a lot from being actively involved in the assessment process.

The real purpose of technology should be to facilitate human collaboration and learning. I love using digital tools to facilitate creative and collaborative activities and that's where educational technology should be focused. The use of big data in education, however is worrying and we need to be very careful not to be tempted into handing over control of our students to big business.

Invisible skills

"Invisible" (CC BY-SA 2.0) by bugie.de

A recurring theme in the media is the perception that universities are not equipping students with the skills demanded by employers and that there is a serious gap between theoretical knowledge and practical work skills. One common complaint is that university degree certificates generally focus on describing mastery of content rather than the development of soft skills such as critical thinking, communication, collaboration, leadership, digital literacy etc. Of course, matters have improved over the last 20 years with learning outcomes often including practical skills and many programmes that include work experience and close cooperation with industry. However, there still seems to be a problem in reliably assessing soft skills and including this in credentials.

A new report, Skills, Competencies and Credentials, by Alan Harrison (Higher Education Quality Council of Ontario) investigates this issue and argues that university credentials still fail to provide the information that employers require when hiring staff.

This state of affairs does not serve most undergraduate students well: the graduating student’s credential and the associated transcript indicate the extent of the student’s knowledge of content, but neither directly conveys any information to employers about the level of the student’s skills. As a result, employers, in respect of most undergraduate degrees, must infer the level of skills from information about content knowledge.

The report describes an awareness gap; the problem that students themselves are often not aware of the skills they have developed at university because those skills were not explicitly part of the assessment. Developing critical thinking, for example, is integrated in all courses but is often done so in a manner invisible to the students and not explicitly tested. Many projects and assignment aim to develop creativity, teamwork, communication and organisational skills but that aim is not always clear to the students nor are the skills explicitly developed or assessed and recognised. 

Harrison then looks at several attempts to address these issues such as competency-based degrees and e-portfolios and comes to the conclusion that greater cooperation is required between universities and employers in determining key skills and agreeing on how to assess these.

Universities must come to terms with two facts: first, their undergraduate programs are where general skills are developed and second, it is these skills that make the graduates of these programs employable. Universities need to work with each other and with input from employer groups to the point where they agree on both what these skills are and how they are most effectively assessed. Once this is resolved, the next step will be to embed these skills into the curriculum and include the outcome of the assessment of these skills in a concise student record that quickly and effectively tells employers what the graduating student knows and can do. In short, the universities need to do all they can to help students make the match with employers.

One promising element missing in the report is the growing use of badges by universities to provide evidence of soft skills. Badges are awarded to students who meet set skills criteria during their course work and can be a very useful supplement to the more content-based formal credentials. However as long as badges are not tied to the hard currency of credits they may be seen as merely optional extras by students rather than as essential elements of their degrees. I wonder if badges are then the real answer. The answer lies probably in providing more comprehensive credentials that describe both the knowledge and skills acquired. 

Reference: Harrison, A. (2017) Skills, Competencies and Credentials. Toronto: Higher Education Quality Council of Ontario.

In your own words ...

"computer words wallpaper" (CC BY-NC 2.0) by tarale

The cat and mouse game of plagiarism detection is becoming increasingly difficult for schools and universities as the technology that assists cheating becomes increasingly sophisticated and hard to detect. Simple straight plagiarism generally gets caught by the widely used detection tools like Turnitin but these tools are relatively powerless against more advanced forms of cheating. The most effective method is a human solution like essay mills where students can earn money writing other students' essays and guaranteeing "originality". This is almost impossible to detect unless there are genuine examples of the students' writing to compare with. However a new grey area of plagiarism has come to light with the rise of automatic paraphrasing tools that can rephrase a text so that it will not be flagged as plagiarism in automatic checks.

Paraphrasing is one of the most important skills that students must learn. Summarising a text to capture the essence in your own words and of course providing a reference to the source is a skill that takes years to perfect. However many people are unsure of where the line is drawn between paraphrasing and copying; simply changing a few words to synonyms is not enough and in all cases the original text must be cited. A new study by Ann M. Rogerson and Grace McCarthy, Using Internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism?, examines how these paraphrasing tools, also known as essay spinning, can be used to evade plagiarism detection and the implications for both students and teachers.

There are many free paraphrasing/essay spinning tools, for example Paraphrasing Tool and GoParaphrase are both tested in the study, and all you do is paste in the text you want summarised and then click for the new text. The results are not impressive, often with many unusual synonyms and awkward turns of phrase but in many cases the text is sufficiently far from the original to avoid being detected in a plagiarism check. If you are willing to pay there are more sophisticated tools that no doubt produce more polished paraphrasing. The study showed that the resultant texts were not detected as plagiarism by standard anti-plagiarism tools. So how can we detect this practice and more importantly how can we prevent people from being tempted to use such tools in the first place?

One aspect that interests me is when paraphrasing is used by non-native English speakers who may be weak at paraphrasing on their own due to limited vocabulary and lack of linguistic fluency. The unusual vocabulary that the paraphrasing tools dig up could be seen by the teacher grading the essay as simply linguistic inexperience rather than signs of automatic paraphrasing. For native speakers this would hopefully start alarm bells ringing but it's not so easy for non-native speakers, most of whom will make similar errors when genuinely paraphrasing themselves.

Where a student is considered to lack the necessary linguistic skills, the errors or inaccuracies may be interpreted by assessors as a student having a poor understanding of academic writing conventions rather than recognising that a student may not have written the work themselves. Where an academic is working in an additional language, they may find the detection of the errors or inaccuracies more difficult to identify.

Another aspect is when the paraphrased text includes references. One amusing side effect of these tools is that even the references get paraphrased and the titles of cited articles get changed beyond recognition, as well as finding synonyms for the authors' surnames. However if you paraphrase a text and use the citations without acknowledging that you did not find these references yourself then that is also dishonest. Finding your own references to support your arguments is an integral part of academic writing.

Furthermore, students using an online paraphrasing system fail to demonstrate their understanding of the assessment task and hence fail to provide evidence of achieving learning outcomes. If they do not acknowledge the source of the text which they have put through the paraphrasing tool, they are also guilty of academic misconduct. On both counts, they would not merit a pass in the subject for which they submit such material.

What can educators do to prevent students from using these tools? The most obvious strategy is to discuss the issue regularly in class, showing that you are aware of these tools and pointing out the dangers. I suspect that many students are simply unaware that automatic paraphrasing is wrong. More support should be offered developing paraphrasing skills and why it they are so vital. Furthermore, the wider use of oral testing is recommended in the article since it is much harder for a student to take short cuts and the teacher can quickly gauge the student's understanding of the subject. Finally educators need to learn how to spot signs of machine translation and automatic paraphrasing and realise that their own professional judgement is still the most important element in assessing student work, even when anti-plagiarism tools are in place.

Of course this post is itself an example of paraphrasing to a certain extent. I hope I pass the test!

Reference
Rogerson AM, McCarthy G.(2017) Using Internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism? International Journal for Educational Integrity

Cat and mouse - cheating in education

"Cat and Mouse" (CC BY 2.0) by katie_mccolgan

Every week there are reports of cheating in school and university exams and assignments and it's easy for some to inperpret this as a purely modern phenomenon enabled primarily by digital technology. Is cheating really on the rise or has technology enabled us to detect it more easily? What makes people go to such lengths and expenses to cheat the system? A recent revelation here in Sweden showed that some people were paying well over €10,000 to cheat on their university entrance exams. If you have the cash you can get someone else to write your essay for you and they will also guarantee that it will pass any plagiarism detection system. University entrance and qualifications are hard currency and whenever there are clear rewards on the table the temptation to cheat will always be there. It's certainly prevalent in business, politics and sport so it's naive to think that education should be exempt.

Cheating and counter-cheating is becoming a frantic arms race as each side tries to out-trick the other. Online examination proctoring services are flourishing such as Proctor U and Software Secure. These solutions offer secure online examination by locking down the students' computer and using multi-factor identification, screen, webcam and microphone surveillance and keystroke analysis. In the traditional campus exam hall students can now use their laptops but in most cases they are offline, locked into the exam administration system and monitored. However for every new solution to increase exam security there are counter moves from the cheaters. A classic case of cat and mouse.

This cat and mouse game is discussed in a new post by Donald Clark, Lecture, essay, cheat, repeat… plagiarism, why it's endemic and 10 ways to avoid. He sees the traditional reliance on summative assessment as the main problem. Predictable exam questions can easily be memorized and it's easy to get ready-made answers to many common titles, especially if you're willing to pay for it. He suggests several alternatives to summative assessment methods.

Essays are sometimes appropriate assignments if one wants long-form critical thought. But in many subjects shorter, more targeted assignments and testing are far better. There’s a lot of formative assessment techniques out there and essays are just one of them. Short answer questions, open-response, formative testing, adaptive testing. I’d argue that student blogs are often better than essays as one can see progress and it’s not something that’s easy to plagiarise. Truth be told, HE wants it easy, and essays are easy to set. They also have to accept that they are also easy to cheat.
One simple trick Clark suggests is googling your essay question and see if the answers are already out there. We simply have to set assignments that aren't so easy to cheat. If you do set an essay then provide formative assessment by reviewing the drafts to see how the work is progressing. Interviews, face-to-face or video, can give an excellent idea of the student's ability and are very difficult to cheat in. The investment from the teacher's part is often less than that of marking written papers. despite this very few dare to move away from the examination methods that are so clearly easy to cheat. Clark blames institutional inertia and tradition for the situation:

This has reached crisis point. Everyone knows it but there’s a conspiracy of silence. Universities are scared to admit the scale of the problem, as they trade on reputation. We’ve created this monster but institutional inertia is incapable of solving the problem, as they refuses to change.

Clearly no-one benefits from playing cat and mouse and although we'll never eliminate cheating from education there must be better ways of assessing students' ability. The essay or scientific article has for so long been the prime medium of academic communication but maybe we should widen our scope and include other forms of expression for assessment, moving the focus to a richer assessment mix of formative and summative methods as well as using peer assessment and assessment of practical work experience. By widening the focus like this we can at least make cheating extremely difficult.

MOOCs going mainstream? Reflections from eMOOCs 2016

University of Graz

Social learning, secure online examination, new pathways to credentials, unbundling and the application of learning analytics. These are some of the main themes of lat week's eMOOCs 2016 conference in Graz, Austria, that I attended together with MOOC practitioners mostly from Europe but some from further afield. Earlier conferences attracted many policy makers and managers but now the focus was very much on sharing experience and moving MOOCs to a new level. The overall tone of the conference was that MOOCs are beginning to mature and the earlier criticisms of instructivist pedagogy, low completion rates and hyped glossiness are now being addressed. Many MOOCs now offer virtual proctoring to enable MOOCs for credits, micro-credentials though badges are improving learner motivation and the focus is now on social learning rather than self-study. The business models of the main consortia are built around a freemium model with only access to course material available for free and a host of layered value-added options available at a price.

The conference featured two inspirational keynote speakers, Anant Agarwal (Professor at MIT and CEO of EdX) and Pierre Dillenbourg (Professor at EPFL Lausanne), and the rest of the time was devoted to practitioner experience, research findings and workshops. Agarwal gave an infectiously enthusiastic presentation of the EdX consortium's progress and future direction, stressing the need for MOOCs to break the sonic barrier of credits. This demands of course increased accountability. MOOCs should be as good if not better than traditional courses and this is achieved by innovative solutions, virtual proctored exams, social learning, project teams, tutoring and teacher grading in combination with peer assessment and guided studies that have become a hallmark of the genre. Verified certificates have increased course completion rates to around 60% whilst more advanced verification and course packages have given even higher rates.

Conference begins

Offering MOOCs as a stepping stone to campus studies is already in place, notably the Global Freshman Academy initiative between EdX and Arizona State University. This involves replacing the freshman year with MOOCs with open admissions. When you sit the course exams you only pay for credit if you pass. Admission to campus studies from year two is available for those who pass their year one courses and pay for the credits, what Agarwal referred to as inverted admissions. The big question here is whether more universities will accept this concept and give learners access to year two of a degree programme on the basis of credits gained from other universities' MOOCs, even with proctored examination. A similar concept is that of micromasters, bundles of MOOCs with tutoring, project work and proctored examination, which are being developed in cooperation with major companies (see also for example Coursera's specializations concept and  project-based courses). These micromasters can be counted as half of a regular masters degree thus allowing learners to complete their masters degree at half the normal price. The partner companies are on board but it remains to be seen whether this approach will gain acceptance throughout the higher education sector and widespread employer recognition.

The frequently voiced concerns about the credibility and security of online examinations may well be addressed by the practice of virtual proctoring through companies such as Software secure and Proctor U. These companies have centres where virtual proctors monitor thousands of online examinations using webcam, microphone and screen surveillance. Student computers are locked down and only allow the examination application and the webcam will register any activities like checking a mobile under the table. The software registers all exceptional behaviour and movements both on screen and in the room and these can later be investigated if there is any suspicion of cheating. This is already in practice in MOOCs for credit as well as in regular for-credit online courses though as far as I could gather had not really been tested extensively in Europe. I can imagine that European universities will be unwilling to risk such extensive surveillance data being stored outside the EU but I'm sure there will be a solution. The claim is that this technology will make online examinations at least as secure as traditional on-site exams (ie. not completely cheat-proof but probably good enough).

The conference closed with a thought-provoking speech by Pierre Dillenbourg who called on European universities to agree on how to recognize each others' MOOCs and open the way for real virtual mobility through a kind of virtual Erasmus programme. He argued that we already have the key ingredients in place; we have a common academic currency in terms of the Bologna agreement and ECTS, we have publicly funded universities giving around 4000 potential testing centres and we have cultural diversity so why can’t we offer credits for MOOCs? He urged universities and university associations to start discussing this and there were several representatives in the hall who immediately agreed to investigate further. The conference Twitter feed immediately informed us that there is already a cooperation in place between a few universities for exactly this sort of virtual mobility called Opening universities for virtual mobility.

My one concern with all this is that that the potential of MOOCs to truly open up education is being tamed and adapted to fit nicely into the traditional model. The focus now seems to be on the value-added layers and the promise of examination and credits. The wider potential of open networked learning I believe will be the task of non-traditional educational organisations rather than the higher education sector.

There were of course many other extremely interesting presentations and discussions and I may well return to them in a later post as I may do about my own contribution.

Download the full proceedings of the conference.

Can open badges become educational hard currency?

  by Dennis Wong

Can open badges become accepted as credible credentials in education or will they remain as an optional extra with very little impact on the labour market? The concept of open badges is to provide a digital certificate for skills and achievements with links to skills required, awarding body, assessment criteria and date of award. You can be awarded badges by your employer, college, association, training provider or even peer groups and they can give proof of proficiency in soft skills and competences that regular certificates seldom acknowledge. The idea is that badges as well as formal qualifications can all be included in your e-portfolio giving future employers a more detailed and fair overview of your skills and ability from a wide range of perspectives rather than the limited skills assessed in traditional qualifications. Badges can therefore complement rather than replace traditional credentials.

Open badges have made considerable progress in the USA where they have the backing of major organisations like the MacArthur Foundation and several elite universities like Carnegie Mellon and Purdue have already started testing the concept. However mainstream uptake is slow and an article by Bernard Bull, 10 reasons why people are not using open badges, presents some good reasons for this. Being new there is an understandable suspicion that they may be easily faked or copied. Teachers are naturally wary of a concept that is still not completely user-friendly and badge design still demands technical skills. The badges movement is still in the pioneer phase and although there are communities and help sites it needs to be more standardised and streamlined before most teachers will start showing interest.

For those who have heard about digital badges, most still have limited understanding of their affordances and limitations. There are not many resources that explain different usage scenarios in a quick and easy to understand format. We have the cases from the Digital Media and Learning Competition a few years ago, but beyond that, there are not many places to go and look through examples of how badges are being used.

Bernard also points to the valid objection that badges are seen by many as the digital equivalent of the gold stars pasted into our old school exercise books, simple rewards for making an effort but academically empty (see his post Beware of badges as biscuits). There is a risk of organisations awarding badges in a trivial manner but the transparency of badges means that all award criteria and information about the badge must be available in the metadata and therefore any badges awarded for simply trying hard or being there will be revealed by the lack of solid evidence behind the award. Traditional certificates actually show very little detailed information about learning outcomes, skills and requirements unless you contact the awarding institution whereas badges can reveal full details at a click.

Digital badges are a new currency for credentialing in a world of connected learning. They are more than glorified digital biscuits for good behavior. While they may have motivational elements to them, their greatest potential is in revolutionizing how we think about credentials in the digital age.

But maybe the main problem at present is the term badge which for many people trivialises the concept by association with scout badges and gaming. This is not surprising since that's where much of the inspiration behind open badges has come from but maybe it's time to change in order to gain more credibility. A recent post by Valar Afshar in the Huffington Post, A Solution To The Massively Disengaged Workforce, offers one solution:

One way to position badges away from games and marketing is to give the concept a different name. At UC Davis, for example, the achievements are called "skill qualifications" (SQs) to give them more career relevance and to set them apart from game-oriented achievements.

This article also cites a recent survey of employers' attitudes to badges and the main reasons for the slow uptake are given as: better industry and market recognition and acceptance of specific badges (67%), standardized requirements of criteria for similar achievements (55%), and lower cost systems to implement badges (37%).

The open badges movement is in that difficult transition period from pioneer phase to mainstream acceptance and is caught in the situation where uptake is slow because it's still relatively unknown and lacks mainstream credibility but it can't prove itself unless some major organisations implement and evaluate it seriously. Maybe that name change could be the catalyst.

Credentials for experience

 by Oberazzi

University has always been a rite of passage; a period of your life devoted to study and network building before entering the world of employment. If you attend for three years you get a batchelor's degree and if you stay a year or two longer you get a master's degree. The focus is on the time spent on campus and the intangible assets of "student life" that can only be gained by being there. This time-based institution has therefore great difficulty accepting that some people can learn the same things without physically uprooting themselves and living on or around campus for the allotted time. No matter how well online students perform there's always suspicion that they have not gained the full benefit of higher education and that off-campus learning can never match "the real thing" (see my last post for more on that topic).

However, as university fees soar in many countries there are more and more new paths to higher education (MOOCs, for-credit online courses etc). The rite of passage aspect is also less relevant, especially for the growing ranks of lifelong learners who need small and regular injections of higher education but have no interest in actually attending a brick and mortar institution.

An article in the New York Times by Anya Kamenetz, Are You Competent? Prove It.
Degrees Based on What You Can Do, Not How Long You Went, discusses the demand for credits for experience and the concept of fast-tracking a degree for people with extensive experience in the subject area. Some colleges are offering students the chance to complete courses as fast as they want and get due credit for validated practical workplace experience.

College leaders say that by focusing on what people learn, not how or when they learn it, and by taking advantage of the latest technology, they can save students time and lower costs. There are 37 million Americans with some college but no degree, and political leaders at the local, state and national levels are heralding new competency-based programs as the best way to get them marketable diplomas.

We need to develop credible systems for giving talented people recognition for work experience and informal learning without having to return to campus of take on crippling loans. This is a potentially massive new market for universities but is viewed with extreme suspicion since it means offering credit to students who haven't attended the college in the traditional sense. One institution that is working on this is the University of Wisconsin who are offering flexible courses that can be completed at the student's own pace. According to Kevin P. Reilly who's in charge of the university's flexible initiative:

“It’s scary for faculty,” Dr. Reilly says. “There’s a continuing sense that students can and do draw on so many sources of information that are now available at their fingertips. They don’t need to come to the monastery for four years and sit at the feet of the monks.”

This week saw the long awaited official launch of  OER University which offers a serious alternative to the hyped xMOOC consortia. OER University are a partnership of 37 universities and organisations under the auspices of UNESCO and Commonwealth of Learning that offer truly open online courses using open educational resources and learners can get their learning assessed for real credits at a fraction of the cost of attending campus. Students need to pay a fee to get their learning and experience validated and credits awarded but it's a fraction of the cost of the full campus experience.

The rite of passage campus experience is not going to die out. It's still attractive for many but is only possible when you're between 18 and 25 and if you can afford it. For everyone else we need to offer other paths that may not give you that all-round learning experience of campus but are more practical and give recognition for the skills and knowledge that people gain at work and in leisure time. Will universities dare to award credentials to someone else's students? The members of OER University say they are willing and let's see who joins in.

MOOCs for credit - few takers so far

 by Ian Wilson

When MOOCs first appeared there were many who criticized them for not offering university credits to successful students. The main providers like Coursera, Udacity, EdX etc only provided certificates of completion that were not educational hard currency and so the search began to find ways of offering MOOCs for credits. One of the earliest initiatives on this front was Colorado State University who offered successful MOOC students from Udacity the opportunity to complement their studies by paying to sit a supervised examination with the reward of university credits. The fees for this were a fraction of the price of taking the equivalent cost on campus and many thought that this would open up a new business model for higher education with major universities producing course content and structure and other universities offering validation and examination opportunities.

The Chronicle of Higher Education writes this week that the scheme at Colorado State has been a total flop so far with no students applying for MOOC validation in the past year: see article A University's Offer of Credit for a MOOC Gets No Takers. Although this sounds like a serious failure I think we should reserve judgement until we get statistics from other similar schemes and when the MOOCs for credit model is more mature. The question at the moment is whether current MOOC participants are even interested in credits; the Udacity students who had the option of going to Colorado State for examination were clearly not interested. A lot of stats I've seen indicate that most are already graduates and are studying out of interest and that MOOCs are mostly a part of lifeline learning rather than seriously competing against traditional higher education. Those of us involved in e-learning and education in general may feel that MOOCs have been hyped to death but out in "the real world" I'm not sure that so many people even know that MOOCs exist. The concept has not reached the people MOOCs are attempting to reach; namely those who have no access to higher education.

So what about MOOCs for credit? If we award credits we need learning outcomes, curriculum, assessment, examination, identity control, administrative requirements etc and once all that's in place isn't this really a regular online university course? I believe that there will be a massive growth in informal learning, encompassing MOOCs and many other forms, and that the demand for validation will increase accordingly but at the moment the demand for crossover (from MOOCs to formal credentials) is simply not there. Evolution takes time and the MOOC wave will develop into new structures and new opportunities but right now we need patience to let organisations try out different solutions.

This is a sentiment that is developed in an article from Inside Higher Ed, Beyond MOOC Hype, where leaders in higher education call on everyone to take a step back from the hype and examine the MOOC phenomenon more critically. It's too soon to make categorical judgements one way or the other but it is time for all parties to seriously examine the opportunities available.

A MOOC on cheating

  by Mr_Stein

Cheating in online courses always attracts a good deal of media attention and is often accompanied by cries for tougher measures to counter cheating as well as criticism of online learning in general. So if we are going to deal with this issue how about organising a course on the subject? This is exactly what is now offered by Bernard Bull of Concordia University Wisconsin - a new MOOC called Understanding Cheating in Online Courses. It's a course for educators to investigate different types of academic cheating, how to spot them and especially how to design courses that make cheating either very difficult or irrelevant. Not surprisingly the course is already full (1000 students) but is most likely to be repeated given the current interest.

"Participants in this eight-week open course will examine philosophical and psychological perspectives on cheating; consider instructor, institutional, and student perspectives on cheating; learn about specific strategies and practices used by students to cheat in online courses; and develop a plan for cultivating a culture of honesty, integrity, and accountability in online courses. The end goal of the course is for participants to gain a deeper understanding of cheating in online courses."

As with most MOOCs there are no credits available for the course (not worth cheating basically) but there are a number of badges available for successful completion of the modules. However it may inspire other universities to include for-credit courses in this field since we need to investigate and understand more fully what we really mean by cheating and whether we even invite cheating by choosing certain types of instruction methods. 

In a networked world where we borrow, remix and reuse other's work in increasingly sophisticated and creative manners we really need to revise our ideas on "cheating". Copying and imitating others is a vital part of learning, as long as we clearly show that we are doing just that. When you don't know how to approach a problem at work you consult your network to see if someone has the answers or at least some assistance. At work this shows initiative whilst if you did that with an examination task it would be cheating.

Read a good overview of the course in an article in The Chronicle of Higher Education, MOOC Teaches How to Cheat in Online Courses, With Eye to Prevention.

Test for success?

  by jackhynes

There are many paradoxes in education today. For example, on the one hand there is an increased focus on standardised testing and educational league tables that compare schools and universities based solely on test scores and on the other hand the need to foster entrepreneurship and creativity in schools. There is considerable evidence that success in tests does not correspond to creativity and innovation. However, the more national authorities focus on success in, for example, PISA the more teachers will need to teach the test and that in turn leads to students focusing not on learning but on cramming facts to pass the tests.

There is of course intense competition between nations as markets become more global and every country tries to stay ahead of the competition by being more innovative. But maybe standardised testing in schools and colleges is counter-productive. This is the essence of an article by Katrina Schwartz, In an Era of Global Competition, What Exactly Are We Testing For? which discusses the research Yong Zhao, professor of education at University of Oregon. Zhao has compared students from countries with high test scores in mathematics, like Korea and Japan, with countries with lower average scores and discovered a negative correlation between high math scores and confidence. The "successful" students are not particularly interested in the subject they are supposedly excellent at and have low confidence in their ability. They have focused on memorising the facts needed to pass the test rather than applying them and thinking creatively.  In schools where students can choose more freely and have more control over what they learn and how they learn there is greater commitment and higher levels of creativity, even if they may not perform so well in standardised tests.

What is the focus for educational institutions then? Should education start with a curriculum of facts that every student must know or does it focus on the individual and build on her/him? It would seem to be a choice between conformity and creativity. We need to learn how to learn rather than learn to pass tests.

“The new education needs to start with the child. Not with the prescribed content,” Zhao said. “We start with individual differences; we start with their cultural strengths.” Beginning with the individual and building upwards from there allows each person to become uniquely great at something. And when students are passionate about anything, they can then be creative and entrepreneurial. For Zhao, the new model has to be about creating a new middle class based on creativity.

To do that, he suggests giving students more autonomy over their learning and emphasizing the importance of making authentic products that solve problems. He also emphasizes a global learning community that can collaborate to fill the gaps that each country, school or teacher experiences.

The problem is that tests focus on facts and acquired knowledge but do not test the skills that today's employers value most: creativity, entrepreneurship and collaboration. If we really want to promote these skills we need to find new methods of assessing real skills and practical competence and let students follow their natural talents and learn in the manner that suits them best.

Verification as MOOC add-on

CC BY-NC-SA by Viking KARWUR

Not surprisingly I start off 2013 with another development in the MOOC saga. As the search to find viable business cases for MOOCs intensifies Coursera have just announced a new revenue avenue - namely identity verification. According to an article by Paul Fain in Inside Higher Ed, Paying for proof, Coursera  students will be able to buy in to a "Signature Track" with photo and keystroke identification to prove that they have done the course work themselves. Student will then have a verified certificate with a detailed course performance record stored by Coursera which will then give added credibility to the student's work if it is to be exchanged at another institution for credits. These institutions will naturally be wary of student portfolios from MOOCs unless there is convincing proof that the student actually wrote the material.

This fits nicely into the layered model of education I mentioned a few weeks ago: A layered model for education. The foundation layer is the open content available to all and free. Above that in layer 2 comes various packaging solutions where institutions and educators can package the resources into courses that can be offered for free or not. Layer 3 includes the add-ons that are probably going to cost money and in this layer we find Coursera's signature track solution as well as services like tutoring and support. Layer 4 could include testing, examination, quality assurance and so on. The principle is that access to education is open and free for all whereas the higher layers will most likely be offered at a price, though free alternatives may be offered.

What we're seeing here is that universities have realised that you can't sell content and that the business case for MOOCs is selling credentials, verification and examination. The article mentions that  the other MOOC players, edX and Udacity, are also selling proctored testing of students at local test centres. Learning is free and open but if you want credentials and other add-ons then you will have to pay. However if the total costs for this path is lower than the full campus experience then we have succeeded in opening up education to more people. That's the benefit in my opinion.