AI Assessment

Assessment in the age of artificial intelligence

Assessment in the age of artificial intelligence

An interesting, clear and well-structured article. It deals with the application of AI in evaluation. The authors identify 5 key problems with the standard evaluation paradigm:

  1. the design of the evaluation is complex,
  2. they provide a discrete snapshot,
  3. it assigns the same tasks to all students,
  4. it does not take prior knowledge into account,
  5. assesses outdated skills.

The authors then outline how AI-based assessment addresses each of these problems, creating a compelling value proposition for them.

The authors also express some concerns:

  1. marginalisation of expertise and opacity of accountability,
  2. limitation of the pedagogical role of evaluation,
  3. the use of surveillance technology and
  4. assessment centred on traditional Western values.
COVID-19 Online Pedagogy

73% of students prefer fully online Post-Pandemic courses

A survey conducted by Bay View Analytics on behalf of several adult education and online training institutions in Canada and the US.

Original article: 73 Percent of Students Prefer Some Courses Be Fully Online Post-Pandemic

Summary table of key issues
Post-Pandemic EducationStudentsTeachers
Fully online courses73%53%
Hybrid courses: Combination of face-to-face and online courses68%57%
More technology, materials and digital resources75%75%
More details:
  • The survey shows that almost three-quarters of students – 73% – said they would prefer to take some of their courses entirely online after the pandemic. However, only half of the faculty (53%) felt the same way about teaching online.
  • Sixty-eight per cent of students also favoured a combination of face-to-face and online courses. Fifty-seven per cent of teachers would prefer to teach hybrid courses after the pandemic, slightly more than those who preferred fully online teaching.
  • The survey also measured preferences around the use of more technology, materials and digital resources for post-pandemic teaching and learning. Students and teachers agree: About two-thirds said they would like to use more technology and digital materials in the future.
  • Another aspect was the challenges faced by students. Both students, faculty and administrators cited the feeling of stress, the level of motivation and the availability of time to complete courses as the most pressing problems for students at the moment. However, students perceive the lack of support from their institution as a bigger problem than faculty and administrators, who see other areas (such as Internet connectivity) as a greater concern.
  • More than half of students said they felt more optimistic about online learning and the use of digital materials than before the pandemic. And almost half (48%) were more optimistic about courses that combine face-to-face and online learning.

Online Pedagogy

AI-based Instructional Design

Photo by Alex Knight on Pexels

Artificial intelligence is a concept in modern computer science that deals with machines and innovative software that can work and behave like humans. Machines can think and learn.
Typically, learning solution design processes are based on simple decision tree structures. Any checklists used for audience analysis, and the conclusions drawn from them, could be part of the AI of a Learning Solution Designer (LSD) system.
Researchers talk about AI-assisted learning solution design to produce personalised instruction.

But they could go further

An AI-assisted Learning Solution Designer could use human resource records to perform ongoing employee/student analysis and define learning solution algorithms based on that data and create and present instruction in the most appropriate format for each individual.
It means writing unique content for each individual based on skill level, prior experience and personal data and then presenting it with contextually appropriate images and multimedia content in the right format for each device.

The possibilities are limitless. But this scenario will not happen overnight.
What impact will AI have on instructional design?
Will it change instructional design forever?

Recall that a few years ago we had to manually code multiple-choice questions. Today it is possible to import a spreadsheet of questions. The future Learning Solutions Design system will review the training video and create automatic assessments based on a multitude of personnel, institutional, corporate and business data.

Let’s look at content production

Instructional designers have often been faced with the challenge of catering to a wide audience in terms of appropriate learning stages, content and assessment. Thus, while adaptive learning is useful and relevant, it is difficult to implement.

With the introduction of AI technologies, learning systems will be able to provide information about the individual needs of each learner. Instructional designers can focus on meaningful lessons and learning progression, rather than recalling basic facts, and can design content for optimal learning.

What about assessment?

Many students do not ask questions in a traditional classroom setting. There may be an overwhelming sense of embarrassment or shyness or lack of prior knowledge that prevents students from asking a question on a particular topic.

AI will enable functionalities such as virtual instructors with the ability to clarify topics or answer questions. The learner is assured that this is a one-to-one session with a virtual instructor, so they can ask questions and clarify freely, leading to a more complete learning experience. The virtual instructor can also pick up on the individual learning style and current knowledge level of the learner to provide personalised feedback with the correct depth of information.

This interaction will allow for more complex forms of assessment. Instructional designers will no longer be limited to multiple-choice quizzes in the e-learning environment. AI will enable assessment and feedback appropriate to the level of knowledge and individual learning style.

Feedback for instructional designers

The correct implementation of AI will eliminate many of the guessing games that still exist when creating learning experiences. Designers will be able to spend their time creating instruction, assessment and feedback for appropriate knowledge levels and learning styles without having to guess whether they are creating for the right audience.

AI technology will be able to take all content and apply the right level of depth of knowledge and assessment per learner. Instructional designers will be able to use this information in subsequent content creation. AI technology will be able to identify gaps in a course based on student performance and assessment results in that course.

AI could also be used to mitigate any negative impact of cognitive load on the learning experience, meaning that instructional designers can be better informed about the impact of their work on students.

Empowering the role of instructional designers

There is a natural fear that AI is destined to replace instructional designers, instructors and analytics specialists.

In general, this fear is unfounded.

Just as you can’t create financial solutions if you don’t understand complex financial products, you can’t create immersive learning experiences with AI if you don’t understand instructional design. The development of new technologies will always require experts from each field to work with the engineers who create the technology.

The threat of job replacement through automation and technology has been around since the industrial revolution.

What never changes is that all technology needs the support of human beings, not just engineers who understand how to build the software, but experts in the individual fields in which the technology is applied.

What will probably be needed is for instructional developers, teachers and instructors to become analysts of the learning process, facilitated by AI technology.

Of course, we are talking about the necessary digital competencies.

Online Pedagogy

Pay services and other herbs…

An interesting by Stephen Downes Knowledge, Learning, Community about the risks of payment services on the web and the commercial consequences. I prefer to keep exploring services as available on the web.

Online Pedagogy

Is the coronavirus a great opportunity for education?

We must acknowledge it: The virus is a great opportunity for education

Let's stop pretending that children will be harmed by having to learn at home during the virus and school closures

It's actually a great opportunity

The problems caused by the closure of schools are evidence of the digital deficiencies of education. Despite the large amount of free technology and resources available and the goodwill of teachers and students.

The same chaos that the virus is causing in various sectors of industry and the economy makes us focus on the possibilities of the online world.

Educational institutions are faced with the need to transform traditional (and rigid) teaching presence into distance education from one day to the next.

Let's relax our rigidity in education. Let's change the role of the teacher. Instead of repeating information (in the hope that it is enough to learn), we should suggest that a child produce something (anything) and help him or her when necessary and then provide some form of evaluation of the quality of what is produced. We need to encourage children to learn what they want to learn and to be motivated and attracted to learning.

Teachers must stop "teaching instructionally" and start helping children to achieve what they really want to achieve. Teachers can encourage children to work together as a team.

Imagine if school were fun and allowed you to learn what you've always wanted to learn. How many would decide to learn the quadratic formula, or how to balance a chemical equation, or memorize a physics formula?

We must recognize that a lot of learning happens outside of school (informal learning), with the help of parents (and friends). Teachers should stop being the ultimate authority and instead learn to mentor and provide support when needed (also online) and stop pushing test preparation.

According to The Chronicle of Higher Education, once colleges develop the ability to meet more of their students' needs remotely, there is not much reason to return to previous models.

Instead of online learning becoming a disaster because schools don't really understand how to educate, the virus could become the savior of education. Let's make school closure a positive thing.

In these times of physical isolation, we need and can strengthen human connections, even if they are not face-to-face. In general, we have the technology to maintain social relations at a distance. Technology helps us to stay close, especially in these moments of physical distance.

Online Pedagogy

Impact of data analytics on higher education

Leaders at higher education institutions understand that the use of analytics can significantly transform the way they work by enabling new ways to attract current and potential students, improve student retention and completion rates, and even boost faculty productivity and research. However, many of these leaders are still unsure how to incorporate analytics into their operations and achieve the results and improvements they envision.

If used effectively, the enormous amount of information generated by higher education can enable institutions:

  • to better understand the needs of students;
  • to improve the quality of teaching, learning and counselling;
  • to reduce costs; and
  • to predict and avoid risks

The analysis tools exist.
But if the conditions for their effective use do not exist, the enormous volume of data does not produce concrete results. In fact, while artificial intelligence and machine learning make headlines, most universities lack that level of analytical sophistication.

According to IBM (IBM, 2016), more data have been created in the last two years than in all of humankind’s previous history.
But, the raw data are of limited use. To extract value from these data we need to refine, integrate and analyze them for understanding.

The potential benefits are too great to ignore.

Universities have tried to re-evaluate and reconfigure their business models in the hope of better serving students, communities and economies. These efforts to improve student outcomes while reducing costs have focused primarily on the large-scale adoption of programs, practices and services designed to optimize learning outcomes, shorten time to degree, reduce excess credit and streamline credit transfer, all while improving teaching, learning and advising in a cost-effective manner.

These seem like difficult but not impossible tasks.

Data analysis is at the heart of collecting the evidence and knowledge needed to achieve the transformational changes required. In recent years, sound data analysis has proven to be a key ingredient for strategic innovation.

Even though:

  • the higher education community is showing signs of embracing the analytical revolution, and
  • that the data and analysis tools are abundant, the reality is that most institutions are not able to use them optimally, for several reasons:
  • insufficient or misaligned resources,
  • endless demands for information,
  • disjointed or rigid infrastructure,
  • limited skills and experience, and
  • lack of trained executives to manage data

Any of these challenges can undermine the development of an analytical culture.

Analytics is the visual representation of the evaluated data. To transform it into visual representations, such as tables, charts and graphs, it is necessary to apply human criteria. Analytics is an informative tool that does not replace reflection, evaluation and decision making.

How can universities overcome existing barriers to harnessing the power of data analysis?
Many institutions would benefit from a solid data base based on accuracy, timeliness, relevancy, integration and security.

As the volume of data available increases, so do the pressures to use it, so it is important to develop procedures to ensure that it is of quality and usable in a contextualised way. There are multiple steps in the acquisition, processing and analysis of data. These include data discovery, extraction, reformatting, uploading, normalizing, enriching, comparing, presenting, and integrating workflow.

Data and information need to be delivered in a timely and accessible manner, otherwise their usefulness may be lost regardless of their accuracy. This is especially true for colleges and universities seeking real-time solutions to the challenges facing students. The longer it takes to acquire, process and analyse data related to the student’s life cycle, the less likely it is that knowledge can be used to predict risks and prescribe solutions.

The aim is to translate accurate and timely data into programs or services that support students and decision makers, but this is rarely achieved. Somehow there is so much data that it becomes difficult to separate the good from the bad. With so much data, it becomes more important to identify the right analysis tools and infrastructure. Analysts must be prepared to offer knowledge, products and services that are important to the end user.

Decision makers want access to information in near-real time, which means that the steps of data acquisition, processing and analysis must be done quickly. A major obstacle to providing accurate, timely and relevant insight has been a lack of integration. Difficulties in connecting data from disparate sources create a host of challenges, including differences in storage, definition, structure (or lack thereof), and intended use. Unstructured data, which can be incredibly rich, accounts for 90% of institutional and corporate data. This makes effective integration an important step.

We must protect and use data ethically. Policies and best practices on data privacy and security, intellectual property and ethical practices deserve careful attention. Analytical functions must adhere to best practices to maintain privacy and security, and create ethical review boards to mitigate the risks associated with the analytical revolution, large data, and predictive analysis.

There are also other factors to consider:


  • The right infrastructure is needed to acquire, process and analyze data from various sources in a relevant and secure manner. In recent surveys on the main problems faced by the participating institutions
  • more than half (57%) of respondents chose data governance as their main problem.
  • The main problems were related to technology. These included data quality, data metadata and definitions, predictive analysis, data visualization, integration and self-service.


Investing in quality data, knowledge and infrastructure requires that higher education institutions reorient their cultures towards a collaborative model of data-based decision making. Without a culture of analysis, efforts to integrate analysis can lead to concerns about quality, the elimination of choice, the tracking of students, the cutting of programs and jobs, and the loss of institutional identity. Leadership must advocate the use of data and the linking of data analysis into a future vision focused on student success and institutional sustainability.

Improving student success

The main advantage of the analytical revolution is the success of the students. Some examples of predictive analytics results at various universities:

  • by using a predictive grade model to place students in the courses that offered them the greatest chance of success.
  • increasing graduation rates and reducing gaps in graduation rates for low-income, under-represented, first-generation students.
  • decrease in credit hours at the end of the course.

Do you want to go deeper?
Analytics in Higher Education – Universities UK
OU Analyse. Open University UK
The Analytics Revolution in Higher Education, Stanford University

Online Pedagogy

Is climate change an “elephant in the glassware” of international education?

Photo by Markus Spiske from Pexels

The Chronicle of Higher Education publishes an interesting article on the impact of academic air travel on global warming.

Millions of people around the world, many of them young people, have mobilized in recent weeks to raise awareness about climate change.

At the same time, university institutions receive and send millions of international students to study abroad.

A clear contradiction emerges

Experiencing other countries and cultures is fundamental to international education, but air transport is one of the main factors contributing to global warming.

“It’s the huge elephant in the room,” says Ailsa Lamont, from Australia’s international education sector, who founded the Climate Action Network for International Education, or CANIE (, a group that seeks to raise awareness of the environmental impact of international education – and to find solutions to mitigate it.
Lamont proposes some actions to make international educators more environmentally friendly:

  • Be a smarter traveler. Lamont believes in the value of cultural exchange, so it does not believe that students and educators should stop traveling. But they can be more intentional, by grouping visits or meeting with partners at conferences.
  • Compensation: New Zealand’s Massey University buys carbon credits to offset staff travel emissions, while overseas study provider API matches the students’ $15 offset contribution. Middlebury offers $500 scholarships to students traveling abroad for sustainability or research projects. The University of Gothenburg, Sweden, charges a fee for staff members’ air travel and uses the funds to support projects that reduce the institution’s environmental impact.
  • Use technology. Technology is not a substitute, but it helps to hold meetings at a distance by limiting travel. Online courses also help link students with classrooms abroad.
  • Increase visibility. Universities can open a broad debate on sustainability. CANIE ( hopes to put climate change on the agenda of international education conferences through roundtables, poster fairs and meetings.

Read the article at (requires free registration).

The Chronicle of Higher Education is a newspaper and website that features news, information and papers for faculty and student affairs professionals at colleges and universities.

Online Pedagogy

Nearly 3 million students are saving millions by using free textbooks from OpenStax

Online Pedagogy

Innovating Pedagogy 2019

This report explores new forms of teaching, learning and assessment to guide educators and policy-makers.

A full-text PDF version of this report is available to download from:

This seventh report proposes ten innovations, many of which are currently implemented but have not yet had a profound influence on education. To produce the report, a group of academics from The Open University collaborated with researchers from Norway’s Centre for the Science of Learning & Technology.

From a long list of new educational terms, theories and practices, this short list of ten innovations has the potential to bring about major changes in educational practice.

Online Pedagogy

What We Learned From Reading 1,000 Articles On Lifelong Learning

A very interesting article about Lifelong Learning.

The 20 conclusions:

  1. There is a growing interest in continuous and lifelong learning
  2. People are naturally curious and lifelong learners
  3. Younger people have a particularly strong focus on lifelong learning
  4. The knowledge economy drives continuous learning
  5. Change means continuous learning
  6. Change also means unlearning
  7. Organisations aim to recruit lifelong learners
  8. Organisations need to support the continuous learners they recruit
  9. CEOs must address lifelong learning
  10. Continuous learning platforms are the future
  11. Continuous learning will disrupt current learning approaches
  12. The future is open, continuous and embedded
  13. Shifting responsibilities for learning
  14. Learners have the key responsibility
  15. Continuous learning means doing not just reading or studying
  16. Continuous learning is essential for career success
  17. Job security is being employable
  18. Lifelong learning is about more than career success
  19. Continuous learning needs to be a habit
  20. Algorithms are also continuous learners

Read the complete post on