We can radically redesign the 11 plus exam to make it fairer, so what is stopping us?

AI assessment systems could provide a fairer eleven plus selection, it could also start to address the vexed question of assessing potential rather than just current ability. We know that well designed AI systems that assess learning, are accurate in their assessment. AI assessment can tackle more than subject specific knowledge and reasoning, it can also evaluate skills such as planning and knowing what we know. AI assessment would also provide a fairer assessment system that would evaluate students across a longer period of time and from an evidence-based, value added perspective. We also know how to prevent people from gaming AI assessments, in addition to which AI Assessment systems would also offer tutoring for everyone and support and formative feedback to help students learn and improve. If there is to be a revamp of the grammar school system then we must explore these possibilities.

Theresa May’s plans for new or expanded grammar schools in England have brought a torrent of comment, debate, criticism and rhetoric since these plans were inadvertently revealed last week. Most of the discussions seem to have focused on whether or not grammar schools are the right mechanism to aid social mobility. This is an extremely important issue, but let’s put the rights and wrongs of selection and grammar schools to one side for a moment and look at the eleven-plus examination itself.

grammar-school-doc

The eleven-plus examination is the key to the door of one of the 164 grammar schools in England, or one of the 69 grammar schools in Northern Ireland. The examination is sat by children in their last year of primary school and it varies depending upon where in the country it is taken. In fact, the situation is very complicated with a wide range of approaches even within the same county.  For example, in Yorkshire there are three Local Authorities with Grammar Schools: Calderdale has 2, Kirklees has 1 and North Yorkshire has 3. The 2 grammar schools in Calderdale use Verbal Reasoning tests, and Maths and English examinations using GL Assessment, University of Edinburgh and the school themselves as their examiners. However, the 1 school in Kirklees uses tests in Verbal Reasoning and Non-Verbal Reasoning, plus an English examination and a Numerical Reasoning test. These are all examined by University of Durham. The situation in North Yorkshire is different yet again, with 2 schools using Verbal Reasoning and Non-verbal Reasoning tests examined by NFER and the 1 remaining school administering and examining its own selection tests.

unknown

The complexity in the selection process is not helpful to poorer parents, who do not have the time, and possibly not the capability, to navigate the process. In addition to which the examination approach is traditional and outdated. The need to look deeper than the selection process to the eleven plus examination itself was highlighted in an interesting discussion on the Radio 4 Today programme last week. The discussion was between Laura McInerney, the editor of Schools Week, and Sean Worth, from Policy Exchange. Sean pointed out that the current mechanism for selecting children for grammar schools can be gamed and that we therefore need to change the examination if we are to ensure that the poorest children are not disadvantaged. Laura McInerney also pointed out the major problem for poorer children accessing grammar schools is that we “put a test in the way”, especially divisive when the parents of poorer children can’t pay for tutoring to get their offspring through the eleven plus examination.

3823AF1F00000578-3782523-image-a-54_1473459484984.jpg

The Guardian published a depressing article on the problems inherent in the eleven plus test ‘‘Tutor-proof’ 11-plus professor admits grammar school test doesn’t work’. The article reports the failure of a ‘coaching resistant’ test developed by CEM at the University of Durham for use in Buckinghamshire. CEM has now withdrawn the claim that the test could assess “natural” ability. Prof Coe director of CEM is reported as saying: “Whatever system you use it is imprecise, there are false positives and negatives and probably more of those than people realise.” He goes on to reflect that whilst he does not agree with creating if we are to have more then we need to try and make the system fairer. I couldn’t agree more – and the need for a radical rethink is echoed in what the IOE’s Tina Isaacs says about the problems of coming up with any test that can assess future potential.

5751d9d292baf522d9cc8e1877682767

So, let’s take the test away and develop a radically different, socially equal eleven plus. We are lucky enough to be in a very different situation today from that which existed when the original eleven plus was introduced in 1944. There is now a realistic and economically attractive alternative at our fingertips. We have the Artificial Intelligence (AI) technology to build a superior assessment system should the proposed reforms become a reality. AI provides a powerful tool to open up the ‘black box of learning,’ to provide a deep, fine-grained understanding of when and how learning actually happens. Intelligent algorithms can process information about each learner and reach a view about their progress, knowledge and understanding of a subject or skill over a ‘period of time’. Unlike the eleven plus examination, this ‘period of time’ could be a whole school semester, a year, several years and beyond.

Of course there are serious ethical questions around AI being used in education and these must be explored. But the over-riding and uncontested fact in this debate is that education is the key to changing people’s lives. We trust AI with our personal, medical and financial data without a thought, so let’s trust it with the assessment of our children’s knowledge and understanding. Let’s open our minds and explore the challenges to build a new generation of eleven plus assessment that genuinely irons out the inequalities and gives all children a chance to shine.

[3] Hill, P. & Barber, M. (2014). Preparing for a renaissance in assessment. London:  Pearson., DiCerbo, K. E. & Behrens, J. T. (2014). Impacts of the digital  ocean  on  education.  London: Pearson.

To appear on the IOE blog

https://ioelondonblog.wordpress.com

Calling education: wake up and smell the coffAI, don’t miss a great opportunity to drive prosperity for all

A recent article in the THES got me thinking. David Matthews reported under the title: The robots are coming for the professionals, and asked if universities need to rethink what they do and how they do it now that artificial intelligence is beginning to take over graduate-level roles? This motivated me to write a blog post for THES that was published on 9 August: Four ways that artificial intelligence can benefit universities, in which I suggested that HE needs to embrace the positives of AI, not just look at the negatives.

 

images-2robot400

These issues are not limited to HE, in fact this is a wake up call for all of Education. We must engage with these technologies and those who are developing them NOW in order to ensure that the AI that we end up with in classrooms, homes and the workplace is informed by what we know about learning and NOT what we know about what the technology can do.

7fba0586036d8b0f2cdf47df1d037557There is a huge and growing interest among those who invest in new technology ventures, specifically Artificial Intelligence (AI) techniques and methods. For example, between 2011 and 16 May 2016 Sentient Technologies received over 143 million USD in funding (Data from CB Insights) Much of the excitement about AI has focused on general purpose AI i.e. intelligence that is applicable across a variety of industries and activities. This is being promoted for technology businesses as a force for good. For example, Antoine Blondeau, the CEO of Sentient, has stated that: “From healthcare to finance to e-commerce, we’re focused on changing people’s lives.” Sentient is reported to be working on financial platforms and on an AI nurse to diagnose patients with sepsis.It is a business that like many who are adopting AI methods has no problem in attracting funding.

However, the same is not yet true of organisations who are adopting AI for education. Yes, there are things like Udacity, that claims it will change HE, and Knewton whose CEO Jose Ferreira, really does believe that his technology will replace human teachers. Such an outcome would make ‘driverless classrooms’ into a science reality. These commercial AI in Education ventures are well funded. BUT it is hard to find mass investment in the application of AI to education, despite the fact that the Educational Technology sector is predicted to grow from £45bn to £129bn by 2020. And to my mind much more significantly, despite the fact that education is the real key to changing people’s lives.maxresdefault.jpg

We need to take a fresh look at education if we are to ensure that the global population is able to reap the potential of the AI revolution that is sweeping across the workplace. AI is both a cause of the radical changes to the workplace that prompted David Matthews to write his piece in the THES and a provider of an answer to the problem of how we make the most of the workplace automation that AI is enabling. The purpose, methods and outcomes of education need re-thinking and AI can help us to tackle the challenge of this re-thinking if we invest in its development and build on the thirty years plus of research in AI for Education.

The importance of the social and economic significance of the developments in autonomous systems and AI was reflected at the annual meeting of the World Economic Forum 2016 in Davos, where the focus was on ‘The Fourth Industrial Revolution. This revolution “is characterized by a range of new technologies that are fusing the physical, digital and biological worlds, impacting all disciplines, economies and industries, and even challenging ideas about what it means to be human.” These radical changes do not however seem to manifest themselves in a concerted effort to use AI to revolutionize education. This oversight is shortsighted to say the least. The few exceptions that one can find where AI is being applied to education at some scale have a very narrow perspective and are a long way from changing people’s lives in the positive way that we want and need. For example: Knewton, is just one a a host of companies who believe that Subject Knowledge is the key to unlocking education for all. Through, for example, making artificially knowledgeable adaptive tutors who can personalize their content to meet an individual learner’s needs. This is all very well, but there is so much more to education than subject knowledge and so much more to AI than adaptive educational content

So what are the key attributes of AI for Education that will enable it to start attracting the sort of investment that Horizons Ventures and Tata Communications have made in Sentient Technologies? What are the attributes of AI that will persuade research funders that AI for education is a subject they must prioritize and that it must be a truly interdisciplinary enterprise that is not driven purely by technologist’s dreams. For a change let’s focus on disadvantaged learners’ dreams and see if we can work with technology to turn these dreams int o reality.

One key attribute of AI for Education is the ability that Educationally driven AI techniques and algorithms bring to the analysis of the vast amounts of data about learners that is routinely harvested by the increasing amount of technologies in the world around us from CCTV, to smartphones, wearable technologies and online courses, such as MOOCs. For example, we can

  • Conduct fine-grained analysis of learners’ skills and capabilities so that their development can be tracked at the student/employee, workplace, school, area, and country level;
  • Enable the collation of a dynamic catalogue of the best training and teaching practices across a range of environments and as a result enable us to educate and train the future workforce in an economically productive manner.

A second key attribute of Educationally driven AI is that it can help us to tackle the toughest educational challenges, including learner achievement gaps, teacher skill shortages, continuous professional development for educators. If we think about the business of education for a moment, imagine the AI teaching assistant that can be used to stretch the brightest pupils, while the human teacher devotes their expertise to giving the less able learners the sensitive human support that they need in order to progress. The teacher would train their personal assistant to work in the way that the teacher and their students need and would demand that the AI assistant explain the decisions it has made about students and the educational opportunities the assistant has provided.

But perhaps what we need to focus on first is using AI systems that go beyond the machine learning and neural network techniques that dominate the work of the main AI protagonists within and beyond education, from Knewton to Google DeepMind. The types shutterstock_260422808.jpgof AI we need within education is the AI that enables the technology it powers to explain its reasoning, to justify its decisions and to negotiate with its users. This is the sort of AI technology that could help us address one of the toughest challenges within the current workplace:  The lack of understanding about how humans can best work with AI systems so that the result is AI augmented human intelligence that is greater than the sum of its parts. We need workers who understand how to make the best use of the power that AI automation can bring to industry and commerce. Workers who understand enough about AI to know where and how human intelligence can work with AI to achieve a blended intelligence that can increase productivity. And what is beautiful about all this is that the appropriate type of AI can help us educate and train people to understand enough about their AI colleagues to work alongside them effectively.

 

 

We have the technology to eradicate exams, tests and stress forever, so why aren’t we using it?

The recent leaking of SAT papers and the growing body of evidence on the stress and anxiety experienced by students who have to sit a battery of tests and exams highlight an area of serious concern. It is all particularly frustrating because it does not have to be like this.

Artificial Intelligence (AI) could wipe out all this pain and change schools forever: it could do away with the need for exams.

This is not to suggest that we should do away with Assessment. It is essential that we know how students are progressing in their knowledge, understanding and skills, and how teaching practices and educational systems are or are not successful. However, assessment does not have to mean tests and exams.

exam_stressArtificial Intelligence is difficult to define because it is constantly shifting and interdisciplinary. However,  AI systems can be described as computer systems that have been designed to interact with the world through capabilities (for example, visual perception and speech recognition) and intelligent behaviours (for example, assessing the available information and then taking the most sensible action to achieve a stated goal) that we would think of as essentially human.[1]

AI has been in the news recently with the AlphaGo programme beating a human champion Go player for the first time and the prospect that Google’s driverless car will soon be available for us to try (). On the negative side there are concerns about the impact of increasingly sophisticated AI on our economy and in particular the jobs market.

 

However, the sort of AI I am talking about here is specific to education and has the catchy acronym AIEd. It has been the subject of academic research for more than 30 years and promotes the development of adaptive learning environments and other tools that are flexible, inclusive, personalised, engaging, and effective. At the heart of AIEd is the scientific goal to “make computationally precise and explicit forms of educational, psychological and social knowledge which are often left implicit.”[2] In other words, in addition to being the engine behind much “smart” EdTech, AIEd is also a powerful tool to open up what is sometimes called the ‘black box of learning,’ giving us deeper and more fine-grained understandings of how learning actually happens.

images

Artificial Intelligence tools and techniques could do away with the need for stop and test assessments and all the stress and anxiety that goes with them. There would be no more need for marking and re-marking, no appeals about results, none of the machinery of exam sitting that dominates the summer term in secondary schools with its “Silence, exam in progress” signs and the commandeering of sports facilities for use as exam halls. There would be more time for teaching, more time for sport and more time for curriculum enrichment.

 

AIEd provides the technology to conduct fine-grained analysis of learners’ skills and capabilities as they learn so that their developmimages-1ent can be tracked continuously and appropriate support provided. Instead of traditional assessments that rely upon evaluating small samples of what a student has been taught, AIEd-driven assessments could be built into meaningful learning activities, perhaps a game or a collaborative project, and will assess all of the learning (and teaching) that takes place, as it happens[3]. AIEd also offer the capability to track the 21st Century Skills that the modern workplace requires and that traditional assessment miss. Skills such as critical thinking, collaboration and initiative.

There is of course a considerable commercial ecosystem surrounding the current assessment system and this may cause some hesitation about adopting the AIEd continuous assessment and support approach. There are also significant ethical issues that need to be considered, such as who has access to the data-stream about student performance and can it be edited or commented on by parents, teachers or the student. The adoption of an AI driven assessment system would be a huge cultural change and not everyone would understand it or feel comfortable with it. Many innovations do not meet with immediate popularity: electric vehicles for example, but over time they are accepted, their benefits are appreciated and their popularity grows.

happy-teacher

 

Unfortunately, there is a hesitation in the UK to exploit either the social and economic potential of AIEd or its commercial benefits. Funding is poorly targeted and as a consequence the UK is at risk of losing its internationally leading research base and its competitive edge. We need to move from the cottage industry of existing UK AIEd research, to a rich ecosystem of disciplined innovation. And we need to move from siloed and short term funding to a funding landscape that reflects AIEd’s enormous potential.

 

But, most importantly of all we need to engage teachers and learners, employers and workers, in the design of the AIEd systems that are developed to provide both the assessment and the learning benefits that this technology has to offer.

 

This blog post can also be found on the UCL IOE blog. It draws on the following publication, where readers can find out more about AIEd: https://www.pearson.com/content/dam/corporate/global/pearson-dot-com/files/innovation/Intelligence-Unleashed-Publication.pdf

 

[1] ODE: The Oxford Dictionary of English (Oxford Dictionaries online). Oxford University Press, Oxford (2005) AND Russell, S.J., Norvig, P., Davis, E.: Artificial intelligence: a modern approach. Prentice Hall, Upper Saddle River (1995).

[2] Self, J.: The defining characteristics of intelligent tutoring systems research: ITSs care, precisely. International Journal of Artificial Intelligence in Education (IJAIEd). 10, 350–364 (1999).

[3] Hill, P. and M. Barber (2014) Preparing for a Renaissance in Assessment, London: Pearson.; DiCerbo, K. (2014). Why an Assessment Renaissance Means Fewer Tests. http://researchnetwork.pearson.com/digital-data-analytics-and-adaptive-learning/assessment-renaissance-means-fewer-tests

What the Research Says about How AI Benefits Education

Thursday 24th March at 1pm
SIGN UP Here for FREE, only a few tickets remaining

https://www.eventbrite.com/e/where-is-the-evidence-that-artificial-intelligence-can-benefit-education-tickets-22502360165

Through a mix of presentations and discussion we will explore the evidence about if and how Artificial Intelligence (AI) can support teaching and learning. For those who would like to know more about AI and Education please see the report published earlier this month.

https://www.pearson.com/content/dam/corporate/global/pearson-dot-com/files/innovation/Intelligence-Unleashed-Publication.pdf

Speakers include:
Prof Benedict du Boulay – University of Sussex
Dr Wayne Holmes – Zondle
Dr Kaska Porayska-Pomsta – UCL Knowledge Lab
Prof Gautam Biswas – Vanderbilt University, USA
Dr Manolis Mavrikis – UCL Knowledge Lab

Junaid Mubeen – Whizz Education

images-2

WhenThursday, March 24, 2016 from 1:00 PM to 4:00 PM (GMT) Add to Calendar WhereUCL Knowledge Lab – UCL Institute of Education. 23- 29 Emerald Street . London, London WC1N 3QS GB – View Map

When
Thursday, March 24, 2016 from 1:00 PM to 4:00 PM (GMT) Add to Calendar
Where
UCL Knowledge Lab – UCL Institute of Education 23- 29 Emerald Street , London WC1N 3QS, United Kingdom – View Map

Here is what ‘smart’ looks like in an AI tutor

 

So, what would a piece of education technology driven by AIEd look like? Here is a simplified picture of a typical model-based adaptive tutor.

ModelBasedTutor

 

It is based on the three core models as described above: the learner model (knowledge of the individual learner), the pedagogy model (knowledge of teaching), and the domain model (knowledge of the subject being learned and the relationships between the different parts of that subject matter). AIEd algorithms (implemented in the system’s computer code) process that knowledge to select the most appropriate content to be delivered to the learner, according to their individual capabilities and needs.

While this content (which might take the form of text, sound, activity, video, or animation) is being delivered to the learner, continuous analysis of the learner’s interactions (for example, their current actions and answers, their past achievements, and their current affective state) informs the delivery of feedback (for example, hints and guidance), to help them progress through the content they are learning. Deep analysis of the student’s interactions is also used to update the learner model; more accurate estimates of the student’s current state (their understanding and motivation, for example) ensures that each student’s learning experience is tailored to their capabilities and needs, and effectively supports their learning.

Some systems include so-called Open Learner Models, which present the outcomes of the analysis back to the learners and teachers. These outcomes might include valuable information about the learner’s achievements, their affective state, or any misconceptions that they held. This can help teachers understand their students’ approach to learning, and allows them to shape future learning experiences appropriately. For the learners, Open Learner Models can help motivate them by enabling them to track their own progress, and can also encourage them to reflect on their learning.

One of the advantages of adaptive AIEd systems is that they typically gather large amounts of data, which, in a virtuous circle, can then be computed to dynamically improve the pedagogy and domain models. This process helps inform new ways to provide more efficient, personalised, and contextualised support, while also testing and refining our understanding of the processes of teaching and learning.

In addition to the learner, pedagogical, and domain models, AIEd researchers have also developed models that represent the social, emotional, and meta-cognitive aspects of learning. This allows AIEd systems to accommodate the full range of factors that influence learning. Taken together, this set of increasingly rich AIEd models might become the field’s greatest contribution to learning.

fd6bc6f23d6f9824c0033625442e95cf_Social_Emotional_Development-578-288-c

 

This post is an adapted extract from Intelligence Unleashed published by Pearson.

Annotation may be old-fashioned, but it is effective for learning and can be supported with technology

5510a4270443ecab4f9b41f11c9e8152When I was at school and university I spent a lot of time taking notes. These notes were sometimes much better than others, but they were always invaluable for revision – even if only to identify the gaps in my knowledge.

I still take notes, in meetings and at talks, for example. Increasingly these notes are technology based and much more structured than those of my youth. In fact, part of the reason I blog is to help myself remember key findings and issues as well as to flag them up for others. Another of the 19 learning acts that I have been discussing is:

Annotation. This can be described as an interaction with existing knowledge material (particularly text) such that the learner is able to construct a personal elaboration of that material. This might take the form of elaborative commentary closely attached to the original (“marginal notes”) or it might be a form of personal précis or reflection (“summarising” etc.). Knowledge is thereby elaborated or personalised.

ae1e261bce501152284696fff8dc597b
This practice suits situations where a body of well-formed material is available for study. Its cognitive benefit arises from the effort of actively re-casting material and selectively linking it with existing knowledge. Clearly annotation is something that can be done more or less skilfully and demands practice and support.

big-genius
Technology can structure annotation as well as providing an infrastructure for composition and storage. There are many tools for writing and filing the results of annotation and a whole range of practices for organisation.

Educational AI can know about teaching, learners, and subjects like math or history.

I said in AI in Education provides smart knowledge modeling tools that AIEd systems build computational models of the process of teaching, the subject matter being studied and of the learner as they progress. The table below provides some examples of the sorts of information that can be stored in the Pedagogical model, the Domain model and the Learner model.

AIEd Models.jpg

To delve deeper into just one of these examples, learner models are ways of representing the interactions that happen between the computer and the learner. The interactions represented in the model (such as the student’s current activities, previous achievements, emotional state, and whether or not they followed feedback) can then be used by the domain and pedagogy components of an AIEd programme to infer the success of the learner (and teacher). The domain and pedagogy models also use this information to determine the next most appropriate interaction (learning materials or learning activities). Importantly, the learner’s activities are continually fed back into the learner model, making the model richer and more complete, and the system ‘smarter’.

images-2

This post is an adapted extract from Intelligence Unleashed published by Pearson.