Data Analytics and Instructional Design: Can They Be Combined?

Contents:

• How do teachers get data about students?
• Do teachers need big data analytics?
• Is it possible to combine learning analytics and pedagogical design?

Data analytics provides methodologists and teachers with significant advantages. It can be used to predict student performance, enabling the proactive identification of problem areas and the implementation of corrective measures. Furthermore, analytics helps optimize teaching loads, ensuring the efficient allocation of resources and time. Improving course content and structure is also possible through data analysis, contributing to a higher-quality educational process. The use of data analytics in educational environments enables the creation of adaptive and personalized learning programs, thereby increasing the overall effectiveness of the educational process. In practice, the situation is often more complex. Elena Drugova, a research fellow at the Center for Sociology of Higher Education at the National Research University Higher School of Economics, focused on the gap between research and actual instructional design in her presentation at the "Transforming Education" forum organized by the Tomsk State University Institute of Education. In her report, "Learning Analytics for Instructional Design of Higher Education Courses: A Review of Existing Practices and Faculty Demand," she presented the views of HSE faculty on learning analytics and shared the results of a review of research papers on the use of data analysis in course development. This report highlights the importance of integrating learning analytics into course design, which can significantly improve the quality of higher education and meet student expectations.

How do faculty obtain student data?

Instructional design and learning analytics are two interrelated fields that are in the early stages of collaboration. They can significantly reinforce each other: analytics provides methodologists and faculty with valuable insights into the learning process, while instructional design helps define the context and set specific goals for data selection and analysis. This collaboration can lead to more effective educational strategies and improved learning.

A survey conducted by Elena and her colleagues among HSE faculty showed that most do not use big data analysis to improve their courses, but prefer to conduct independent surveys. This allows faculty to receive feedback on materials and assignments, as well as to understand student goals and needs. This approach helps them more accurately focus on the needs of students when developing courses.

Recently, Microsoft Teams was one of the popular tools for collecting data, but Microsoft stopped selling and supporting its services on Russian market. In response to this situation, HSE launched its internal online learning platform, Smart LMS. However, according to Elena Drugova, instructors have not yet fully mastered all the capabilities of this new system.

Student evaluation of teaching (SET) is a popular tool among faculty, but its subjectivity limits the ability to fully trust the data obtained. Furthermore, SET is only administered at the end of a course, making it difficult to make changes during the course. This means that faculty can only adapt the course for the next cohort of students rather than respond to current student needs. Therefore, it is important to use SET in conjunction with other assessment methods to gain a more complete understanding of teaching quality and improve the educational process.

Do Faculty Need Big Data Analytics?

Researchers surveyed faculty to find out how data analytics can improve their professional practice. The responses presented a variety of ideas regarding the use of analytical tools to improve the efficiency of the educational process and optimize learning activities.

• track the sequence of topics within similar courses, as well as the sequence of courses within a program - for logical and consistent teaching;
• monitor the teaching load - this would help coordinate the schedule of deadlines and make the workload of students more evenly distributed;
• track moments when students need help and support - in order to respond in a timely manner and reduce the risk of a student accumulating "debt" and dropping out;
• track student activity in the university LMS - how often they log in, whether they view materials, and so on;
• create student profiles - information about their past and current results, interests and achievements would help identify underperformers, recommend the most suitable topics for projects, term papers and theses.

Research has shown that faculty do not consider course improvement a priority. The surveys did not express a desire to analyze and address course weaknesses. Teachers' primary focus is on other issues, such as reducing workload and combating plagiarism. Elena Drugova and her colleagues conducted a study that found that most teachers struggle with understanding learning analytics. Many encounter problems with existing data presentation methods. Questions regarding the legal and ethical aspects of collecting and analyzing information in the educational environment also remain unresolved. This highlights the need to develop more accessible and understandable tools, as well as conduct educational activities for teachers to increase their awareness and confidence in using analytics to improve the quality of teaching.

Read also:

Digital personalization in education: does it promote effective learning or create an information bubble? In recent years, technology has played a key role in the educational process. Personalized learning allows programs and resources to be tailored to individual student needs, potentially increasing learning. However, there is concern that excessive personalization may limit access to diverse information and reinforce biases. It is important to consider how to find a balance between tailoring educational content and providing a wide range of knowledge. Effective digital personalization can be a powerful tool for improving educational outcomes if used wisely and with an awareness of the potential risks.

Can learning analytics and instructional design be combined?

To assess the spread of learning analytics in instructional design, the researchers conducted a literature review. They identified approximately 600 articles using keywords related to the research topic. The focus was on practical course improvements, which led to the exclusion of all theoretical works. The final list included 49 articles that demonstrate the real-world application of learning analytics in educational practice.

During their research, the researchers developed seven key questions, but Elena focused on only two of them in her presentation.

• How do the authors of the selected scientific papers view the integration of learning analytics and instructional design?
• What real improvements in instructional design are described in the selected articles?

In this text, we will discuss the results in detail, following the logical order of their presentation.

Research shows that a third of the 49 analyzed papers do not consider the integration of analytics with instructional design or focus on data-driven approaches. The authors of these studies select specific measurable indicators and formulate conclusions based on their analysis. The speaker notes that this approach is insufficiently mature. A more effective method is considered to be one in which the goals of data collection and analysis are determined based on pedagogical tasks.

The authors of most articles (33) explore methods of integrating analytics into the process of pedagogical design. Elena Drugova's team classified these studies according to the stages of the ADDIE model, which allowed them to obtain a structured result.

• Quite a few articles (14) describe the use of learning analytics at the course design stage (Design).
• Approximately the same number of works (13) cover the development and implementation stages of the course.
• The authors of six articles integrate analytics at the evaluation stage of the educational program (Evaluation). According to the speaker, this is an ineffective approach for improving the current course - after all, it is too late to change anything; this can only be done when working on the next iteration.

According to Drugova, most authors use analytics at the initial stages of course development. This allows them to identify problems and shortcomings, as well as make the necessary improvements. Analyzing data during the creation of educational materials helps improve the quality and effectiveness of the course, which in turn affects student satisfaction and their outcomes.

Rework the text, maintaining the main topic and avoiding unnecessary elements. Optimize it for SEO while expanding the content. Pay attention to important keywords to improve visibility in search engines. Eliminate emojis and unnecessary characters, and avoid structured sections like lists. The result should be clear, informative, and easy-to-read text.

Improving an Existing Course: A Real-World Case Study

Optimizing a course is key to making it more effective and engaging for learners. It's important to regularly review and update content to align with the latest trends and requirements in the educational environment. Let's look at how an existing course can be improved using a practical example.

The first step in improving a course is analyzing participant feedback. Evaluating their opinions and suggestions helps identify weaknesses and determine what exactly needs improvement. This could be both the content of the material and the format of information presentation.

The next step is updating the course materials. It's important to add relevant data, real-world examples, and new research to ensure the course remains relevant and useful. It's also worth considering the inclusion of multimedia elements such as videos, infographics, and interactive activities, which can significantly increase learner engagement.

It's equally important to keep up with changes in technology and teaching methods. The introduction of new educational platforms and tools can significantly improve the user experience. For example, using mobile apps to access course materials can make learning more flexible and accessible.

Don't forget to regularly update assessment methods. An effective knowledge assessment system not only assesses the level of material assimilation but also motivates students to delve deeper into the topic.

In conclusion, regularly improving a course is not only a necessity but also an opportunity to make it more engaging and effective for students. Analyzing feedback, updating materials, and implementing new technologies and assessment methods will help create a high-quality educational product that meets modern requirements.

To determine the extent to which analytics contributes to the improvement of educational courses, researchers analyzed 49 scientific papers, dividing them into three categories. This study provides a better understanding of the impact of analytics on educational processes and course effectiveness.

• The first group included 16 articles that only formally mentioned the use of learning analytics in instructional design. Elena gave a typical example: "Learning analytics can be used or adapted by faculty or departments to inform teaching practices." This is simply a vague statement with no specifics.
• Articles in the second group (18 papers) already identify specific potential improvements. The developments described have not yet been implemented, but the researchers already understand their potential uses. For example, they could be used to predict academic failure and inform faculty about what exactly needs to be changed in course content, where students will need scaffolding, at what stages feedback should be added, and so on.
• Finally, the third category includes 15 articles that describe real ways to implement analytics in instructional design. Based on data analysis, researchers tried adding new assignments, restructuring deadlines and adjusting the teaching load, implementing additional monitoring, and adapting the course for mobile learning. However, only three studies demonstrated the effectiveness of these changes.

Elena Drugova emphasizes that the current analysis revealed that data analysts and instructional designers have different approaches and terminology. In most cases, instructional design is mentioned only formally or is considered in the context of pedagogical theories that have no practical application. Successful integration of analytics and instructional design requires interdisciplinary dialogue. It is also important to conduct additional research to understand how data-driven decisions impact the achievement of educational goals.

Read also:

• How to build a course that will lead a novice student to professional mastery
• The improvement loop: how to improve an existing course
• What research to conduct before designing a course
• How to find and read research on education