Education

Improvement loop in online courses

Improvement loop in online courses

Course with employment: "Profession Methodologist from scratch to PRO"

Learn More

This article will introduce you to the main aspects of the topic, as well as provide helpful tips and recommendations for a deeper understanding. We will cover key points and provide information to help you better navigate this area. Read on to discover important details and practical tips.

  • What is a course improvement loop?
  • What influences student completion rates?
  • What is the difference between NPS and CSAT when assessing student satisfaction?
  • How to assess the level of a problem found in a course and what to do about it?
  • Examples of course failures and the lessons they can teach you.
  • Program Director of the Marketing department at Skillbox.
  • Developed seven Skillbox programs, including SMM from Scratch, Targeter from Scratch, Targeter PRO, and Internet Marketer from Scratch.
  • Participated in the launch of at least ten company courses.
  • Accredited educational partner of myTarget and VKontakte.
  • Taught at IQBuzz, Netology, 1day1step, and Delovaia Sreda.
  • An experimental practitioner and only then a manager.

The course is completed, and shortcomings arise, examples given by teachers become outdated, the interfaces of programs and applications discussed in the course. It is necessary to regularly return to the content, adapt it, make corrections and improvements. This requires constant updating of information to ensure relevance and high quality of training. Updating courses helps keep them in line with modern requirements and provide students with access to the most relevant knowledge and skills in the rapidly changing world of technology.

The inner perfectionist of every educational product creator strives for perfection from the very beginning. However, it is important to realize that mistakes are a natural part of the process, and they will be inevitable. Only those who do nothing make no mistakes. It is best to tune in to the fact that working on mistakes and improving them is a normal practice after the launch of the course. The key is proper planning. I will share my approach to this process.

What is an improvement loop

An improvement loop is the process of continuously analyzing a product and evaluating its customer perception, as well as improving and adapting it. This process effectively addresses two key objectives:

First, it ensures continuous monitoring of customer needs and preferences, enabling timely product adjustments. Second, it helps identify weaknesses and issues requiring attention, improving the overall user experience.

Therefore, implementing an improvement loop is an important step for companies seeking to improve the quality of their products and services and strengthen their market position.

Achieving learning goals. Consider a situation where our goal is to teach students how to make delicious cherry pies. In this case, the end result will be not only the finished pie, but also its flavor characteristics. Even if the recipe presented in the lesson contains a few typos, the student will still master the basic skills: kneading dough, making jam, and using the oven. Ultimately, they will be able to create their own pie. Don't worry about minor errors or rush to re-record the lesson—this is ineffective. The key is to focus on practical skills and learning outcomes.

Consider a situation where we were training students to use a specific online service, and suddenly its design changed: the horizontal menu became vertical, but all the functions remained the same. Although the information has become slightly outdated, the value of the acquired knowledge remains, and students will still be able to use our recommendations. This highlights the importance of an adaptive approach to training that allows skills to be effectively used despite interface changes. Training should focus not only on specific design elements but also on understanding the principles of how the services work, which promotes deeper assimilation of the material and improves students' overall competence.

If a course or webinar meets its stated objectives, then corrections should be made carefully and purposefully. However, it is worth considering that deficiencies can be significant. In the next section, we will consider how to identify such deficiencies and what steps to take to eliminate them.

Reducing refunds is an important task for any company, including educational institutions. Returns of goods and services result in financial losses and can indicate student satisfaction with the learning platform. For example, at Skillbox, the return rate is less than three percent, indicating a high level of student satisfaction, as students gain the knowledge and skills they enroll in the courses for. Maintaining such positive dynamics requires continuous analysis and improvement of the user experience. It is important not only to identify and correct your own mistakes but also to closely monitor student opinions about specific courses and the platform as a whole. Effective management of reviews and the quality of educational content will help reduce the number of returns and increase student loyalty, which in turn contributes to the growth of the reputation and success of the educational company.

Collecting Feedback

There are several key metrics that help us analyze audience perception of the course. Let's consider the main ones.

The NPS (Net Promoter Score) index is an important indicator of customer loyalty to the brand. It assesses customer satisfaction and their willingness to recommend a company to friends and acquaintances. The primary question customers answer in this study is, "Are you willing to recommend our company to your friends?" This index helps businesses identify strengths and weaknesses, improve service quality, and build long-term relationships with customers. A high NPS indicates strong loyalty, which can positively impact sales and a company's reputation. NPS, or Net Promoter Score, is a scale from one to ten, where one indicates a customer would not recommend a product or service, and ten indicates they would definitely recommend it. Typically, if a customer is willing to recommend, they give a score in the 8-9 range. NPS scores between 5 and 8 are considered average and even good across various industries. It's worth noting that a larger audience increases the likelihood of negative reviews, while a smaller audience provides more opportunities to increase loyalty and improve NPS scores.

In the Skillbox LMS, we offer a series of surveys that ask the question, "Would you recommend us?" Students can select the most appropriate answer, after which they are presented with three categories they would recommend most, as well as a field for writing reviews. This allows us to better understand the preferences of our students and improve the quality of education.

Screenshot from the Skillbox website. This is what an online student survey to determine the NPS level looks like.

I consider NPS (Net Promoter Score) to be one of the least effective metrics when improving products and services. A user's recommendation often depends not only on the quality of the content or its presentation, but also on their emotional state at a given moment. If a user is in a good mood, they may be willing to recommend a product, while if they are in a bad mood, they may refrain from doing so. This makes NPS a less reliable indicator of the actual level of customer satisfaction and loyalty.

The dynamics of the number of ratings and the NPS (Net Promoter Score) value may not be directly related to the content and its presentation, but they are closely linked to the perception of your brand. The higher the quality of service and the better the reputation of your brand, the more positive the metrics will be. However, you should not rely solely on this metric, as its effectiveness is relative. To achieve sustainable results, it is important to consider a range of factors, including product quality, service level, and customer interaction.

Course completion rate, also known as COR, is a key metric that reflects how students master course materials and how effectively we maintain their interest. This information helps us analyze the quality of the educational process and identify areas for improvement. A high COR indicates that the course is engaging and meets student expectations. On the other hand, a low COR may indicate a need to revise the content, teaching methods, or structure of the course to make it more engaging and effective.

Course creators often assume that adult learners make conscious decisions about their learning, invest in educational programs, and therefore are diligent and proactive. However, in practice, many are simply seeking to satisfy a need for self-improvement. They are often unwilling to put in the effort to turn their desires into action, which can negatively impact the learning process and its outcomes.

Students often lose interest in learning after viewing the first module, and in some cases, do not even begin. This can be due to various factors, such as lack of motivation, complexity of the material, or lack of support. It is important to identify the reasons why students are unable to maintain interest and develop strategies to engage them. Effective methods such as interactive assignments, instructor support, and building a community of like-minded individuals can help maintain motivation and increase student engagement.

The audience I refer to as "now or later" consists primarily of people over 45. They often face the need to learn new skills. For example, the only suitable job in their small town has closed, or their profession has become obsolete. These people may not know what "scroll" means, "Google Doc," or how to create a copy of that mysterious document. They must constantly turn to instructors for clarification on even the most basic questions. However, despite all the difficulties, they are willing to put in maximum effort to acquire the necessary knowledge and skills, which underscores their desire to learn and adapt to a changing world.

Various methods are used to improve student engagement. Let's look at several effective approaches. Helping students define their goals is an important stage of their learning. At Skillbox, for example, personal consultants interact with prospective students even before the course begins. They discuss key issues: why they are coming to study, how much time they are willing to devote to learning, and what support they can get from tutors, curators, and in group chats. This approach significantly increases student motivation, which, in turn, helps increase course completion rates.

Check out our materials:

Six Signs of a Student Who Will Successfully Complete a Course

Students who are committed to completing a course often exhibit certain qualities and habits. Here are six key signs that indicate a student is likely to complete their studies.

The first sign is determination. Students who clearly understand their goals and motivations are more likely to persistently work towards achieving them. The second sign is organization. Successful students know how to plan their time and prioritize, which helps them not only study but also manage other aspects of their lives.

The third sign is active participation. Students who actively participate in discussions, ask questions, and engage in the learning process are more likely to achieve success. The fourth sign is persistence. The ability to cope with difficulties and not give up when obstacles arise is an important quality for successful course completion.

The fifth sign is teamwork. Students who can effectively collaborate with others not only share knowledge but also receive support from their peers. Finally, the sixth sign is openness to new knowledge. Students who are eager to learn and grow are more likely to complete a course and master the material successfully.

These traits help identify students who are likely to successfully complete the course. If you recognize these qualities in yourself, you are well on your way to achieving your educational goals.

Reinforcing success is an important aspect of the learning process. It often seems that a student who successfully completes the first homework assignment will be able to continue to perform equally well on subsequent assignments. However, in reality, many students experience burnout and lose interest in learning. Therefore, it is important to maintain motivation and a positive outlook throughout the course. Skillbox offers special motivating lessons to help students stay engaged: they are held before and after the first module, at the midpoint of the course, and in the final module. These activities are aimed at supporting students and promoting their successful learning.

A still from a motivational video for Skillbox students

Several short 1.5-minute videos can significantly support students, helping them not to quit their studies and move forward. After reaching the halfway point of a course, many students leave positive feedback, noting the importance of such care and celebrating their success. Internal perception of the learning process and successful course completion are closely linked, emphasizing the importance of support at all stages of learning. Maintaining the pace of learning is an important aspect of the educational process. Microlearning, which involves dividing a single global task into competencies or modules, is currently becoming popular. Each module consists of short lessons lasting from 5 to 15 minutes. The pace of learning depends on both the course developers and the students themselves. A seemingly simple lesson at the beginning of the course can be an effective way to maintain student motivation. The course leader can state, "You have successfully completed the first module, now we continue at the set pace. You have recognized your capabilities and goals—write them down." This helps students feel like they are making progress and builds their self-confidence. When preparing a course or lesson, an expert is often tempted to share all the information about the subject. However, this can lead to problems. A huge amount of knowledge presented in a short period of time is not absorbed by students, which hinders the achievement of learning objectives. Effective learning requires focusing on key aspects of a topic to ensure comprehension and retention. It is important to select the most relevant and relevant materials to ensure maximum retention and practical application of knowledge.

In the past, the modules I created took up to four hours to complete, although the standard time for completing them is one to two hours. The process of mastering such modules dragged on for two weeks. Methodologists strongly recommended that I simplify and shorten the content. I heeded their advice, and as a result, the time to complete the modules was reduced to eight to nine days, which positively impacted the overall effectiveness of the training.

People forget the material they have learned within an hour, which emphasizes the importance of a clear learning structure and a system of supporting materials. It is recommended to use time codes for videos and detailed instructions to facilitate the process of memorization and assimilation of information. This will help increase the effectiveness of learning and improve the retention rate, which is especially important in the fast-paced modern world.

Reading is an important part of our lives and contributes to the development of thinking, broadening horizons and enriching the language. A good book can inspire, provide new knowledge and change the perception of the world. It is important to choose literature that matches your interests and goals. Discussing your reading with friends or participating in book clubs can deepen your understanding and make the reading process more engaging. Remember that reading is not only entertaining but also helps develop critical thinking and creativity. Choose a variety of genres and authors to enrich your experience and open new horizons.

Microlearning: Advantages and Disadvantages of Short Formats

Microlearning is a teaching method based on the use of short, targeted learning modules. This approach is becoming increasingly popular due to its effectiveness and accessibility.

One of the main advantages of microlearning is the ability to absorb information in a short time. Short formats allow learners to easily integrate learning into their daily schedule, making the learning process more flexible and convenient. Microlearning also promotes better retention of material, as the information is presented in a compact and easily digestible form.

However, microlearning also has its drawbacks. Short formats may not cover all the nuances of a topic, which can lead to a lack of understanding of the material. Furthermore, this approach requires a high degree of self-discipline from students, as they must independently manage their time and choose what to study.

In conclusion, microlearning is an effective method with both its pros and cons. It's an approach that can be applied in a variety of fields, but it's important to consider its limitations and choose the most appropriate formats based on learning goals.

CSAT (Customer Satisfaction Score) is an index that measures customer satisfaction. Unlike NPS, which evaluates the overall learning experience, CSAT focuses on specific aspects of the educational process and allows you to understand how they meet student expectations. This metric is one of the most effective, as it covers areas such as comprehension of the material, course content, and presentation style. Therefore, to get an accurate picture, I recommend collecting CSAT data, taking into account various parameters.

Content and presentation are key factors for successful audience engagement. High-quality content should not only be informative but also structured. It is important to consider the interests and needs of the target audience to ensure maximum engagement.

When creating content, pay attention to its uniqueness and relevance. SEO optimization also plays a vital role: using keywords, meta tags, and proper heading structure will help improve search engine visibility.

Furthermore, the presentation style should be simple and clear so that readers can easily digest the material. Using visual elements such as images and infographics can significantly enhance the content's appeal and improve information comprehension. Thus, the combination of high-quality content and effective presentation contributes to successful audience engagement and retention.

It is important to note that both content and presentation should be evaluated. At Skillbox, we offer students the opportunity to express their opinion by asking, "What specifically didn't you like: the course content or its presentation?"

Both parameters are important, as they determine the quality and effectiveness of information delivery. For example, you may have a talented speaker who inspires the audience, but the content of their lecture may be uninformative and superficial. On the other hand, an expert can provide valuable and rich information, but if their delivery is boring, it will diminish the audience's interest. Therefore, a combination of engaging delivery and meaningful content is essential to ensure maximum audience engagement and satisfaction.

This metric will be useful for course creators, especially if they manage to attract a talented speaker who can offer engaging and high-quality content.

When searching for a teacher, I consider several key parameters. It is important that the candidate meets three main criteria. This will help me select the most suitable specialist and ensure a high-quality learning process.

I have personally experienced how difficult it can be to create high-quality content. While recording my courses, I often made the cameraman laugh, and it was not always intentional. For example, we had to reshoot a segment about moderating creatives in advertising three times, and the cameraman couldn't hold back his laughter with each take. Although these moments could not be removed during editing, the final module was engaging and lively thanks to this spontaneous humor.

Instructors are evaluated not only on the content and quality of their lectures, but also on the level of assignments they assign for students' independent work. Effective homework helps students better understand the material and apply the acquired knowledge in practice. Instructor evaluation largely depends on how well the assignments contribute to the development of skills and understanding of the topic.

Paying attention to the clarity of assignments and feedback from the instructor is key to successful learning. It is important that students clearly understand the requirements and expectations. To achieve this goal, it is advisable to develop guidelines and recommendations for creating and grading homework in advance. This will help improve the learning process and increase the effectiveness of assignment completion.

If the content is high-quality, the instructor is professional, and the homework is appropriate, but CSAT scores are starting to decline, it is worth considering other factors. Perhaps the target audience for the course has changed, and new participants do not meet previous expectations. In this case, it's not necessary to completely rework existing material. First, it's necessary to conduct an analysis to understand who exactly has enrolled in the course and what their needs and expectations are. This will help adapt the content and teaching approach to meet the needs of new students.

Tests and related questions play an important role in the educational process. They help assess the level of material assimilation and identify knowledge gaps. Properly designed tests can be an effective tool for self-assessment and exam preparation. Test questions can vary in difficulty and type, including multiple choice, open-ended questions, and matching problems. Using tests promotes active learning and allows students to systematize their knowledge. It's also important to consider that regular testing helps reinforce material and increase students' confidence in their knowledge.

Tests play a key role in online education, allowing students to assess how well they have mastered the material and are ready to move on to the next stage of learning. The effectiveness of testing can be determined by analyzing how students perceive the tasks. If students consistently fail tests, this may indicate problems in the course design, such as questions that are not relevant to the material covered. Therefore, properly designing tests and aligning them with learning objectives is the foundation of successful online education.

Screenshot from the Skillbox website. This is what a Skillbox student test assessment looks like.

The problem often stems from differences in terminology used by teachers and students. It's important to remember that students should clearly understand the content of the course materials or questions. If the text or test contains terms that may be unfamiliar to the audience, a glossary should be included. This promotes better comprehension of the information and facilitates the learning process. A glossary helps remove communication barriers and provides a deeper understanding of the subject.

This metric has many interpretations, so it is important to take into account all feedback, including:

  • left in chats;
  • sent via email;
  • given to teachers;
  • left along with an assessment;
  • received from external experts as a result of an audit or training.

Receiving feedback is an important aspect of the educational process. Students should be able to leave their feedback, as it helps improve the quality of education. Analyzing negative feedback plays a key role in development, allowing you to identify weaknesses and promptly work on them. This prevents situations where you have to refund money, which can negatively affect the reputation of the educational institution and its financial condition.

Feedback can concern both specific technical problems, such as no sound or broken links, and the subjective opinions of viewers. Here are some examples of real reviews of a single video to demonstrate the diversity of perception.

  • "The speech speed is too fast, there are practically no pauses between topics, you don't immediately realize that we are talking about something else."
  • "This material is much better than it was in 2019! If you watch it at 1.25 speed, it's even better!"

Technical errors can be corrected in no time. However, subjective opinions do not always require immediate changes to the content or its presentation. As you know, everyone has their own point of view, and tastes can vary greatly. Nevertheless, these opinions should not be ignored, as they can provide valuable information for further content improvement.

Determining the level of the problem and the speed of its solution

I classify course problems based on their significance and the speed required to resolve them.

  • Stop-obstacle. In other words, some error, often technical, that prevents the student from completing the course. For example, the sound was initially fine, but for some reason it suddenly disappeared.
  • Refunds. Of course, we are talking about multiple refunds. If one student demands a refund, this is a reason to analyze the situation, but if there have already been three refunds, this indicates a system, meaning the students have encountered a common problem.
  • Content outdated. It happens that the tools we use in our work change—for example, the same interfaces of work applications used in the course. Over time, information can be updated, but it is important to understand whether the changes are truly critical and how they affect the achievement of the goal.
  • Low CSAT. This could indicate poor content or, for example, tests, or problems with the instructor. In any case, finding and resolving these issues will take time.
  • Completion Rate. The problem may lie both in the courses themselves and in student motivation. The second case is also the responsibility of course creators, but is not included in the course improvement loop.

It is important to immediately respond to obstacles, as they have a significant impact on the learning process. Quickly identifying and eliminating such obstacles can significantly improve the effectiveness of the educational process, promoting better assimilation of material and skill development.

All other changes can be planned for a period of one to three months. During this time, it is important to monitor the dynamics of changes in quantitative metrics and analyze user feedback.

Working on Mistakes

There are three main types of reactions to problems related to content, tests, homework, and interaction with students. These reactions help determine the approach to solving the issues that have arisen and improving the educational process. Correctly understanding these reactions can significantly increase the effectiveness of learning and interaction.

Examples of Failures with Conclusions

Instead of discussing "successful success," I will share several difficult situations that I have encountered in my professional work and tell you how we solved them. These examples will help you better understand how to cope with challenges in the workplace and find optimal solutions.

While recording the course, I noticed that the speaker suffered from a pronounced stutter. Recognizing that listening to his presentation for extended periods could be difficult, I decided to modify the recording process. I sat next to the speaker and began recording the content together: he spoke at some points, I followed his words at others, and we also switched to a dialogue. This approach was new for us, but given the speaker's speech characteristics, it was necessary to find a way to address this situation.

A conclusion is a summary that summarizes the main findings and conclusions reached during the analysis or discussion. It is important that the conclusion is clear and concise, emphasizing the key points covered. It should capture the main essence of the material, allowing the reader to quickly understand what conclusions can be drawn based on the data presented. An effective conclusion helps reinforce the information and may include recommendations for further action or research. Properly formatting and structuring the conclusion facilitates better comprehension of the information and increases its value.

During the course development stage, it is important to identify and eliminate potential problems, as correcting them later can be difficult and costly.

During one of my courses on targeted advertising, I explained how to correctly calculate a sales funnel. However, I made two mistakes: I added an extra zero in the presentation and misread what was written in the transcript. Most students dismissed this as a simple typo, but for some, it raised serious doubts: how can a teacher teach if they make such basic mathematical errors? I corrected the typo only two months later. Despite this, the error did not prevent the students from deepening their knowledge of the topic, although it did cause temporary dissatisfaction.

A conclusion is a key part of any analysis or research, summarizing the work done. It allows you to formulate the main results and make practical recommendations. It's important for the conclusion to be clear, concise, and contain the main information presented in the text. This will help the reader quickly understand the essence and significance of the research or work. A well-structured conclusion demonstrates a deep understanding of the topic and highlights key points worth paying attention to. It can also include suggestions for further research or practical application of the findings. A well-written conclusion improves SEO by containing keywords and phrases that can attract the attention of the target audience.

People invest significant resources in training and expect high-quality content. Any shortcomings can irritate them. However, the main goal—training new specialists—does not suffer from typos. It is important to provide a high level of educational material to meet student needs and enhance learning effectiveness.

An example of a successful course is a marketing course that emphasizes practical learning. I decided to change my approach and proposed: leave the theory aside; we will teach you how to manage Instagram in practice. This course focuses solely on real-world activities: you create a profile, fill out a bio, and launch targeted advertising. This learning method allows students to quickly master skills and apply them in real-world situations, making the marketing course as effective as possible.

The idea of ​​changing the educational market was ambitious, but I encountered significant challenges because I disrupted established patterns. It turned out that students really needed an introductory module that explains the basics of marketing, its potential, the terminology used, and the responsibilities of marketers. Without this theoretical foundation, the wow factor was lacking.

We updated the course, which significantly improved its quality and made the material more accessible and understandable for our students.

A conclusion is a key step in the process of data analysis, research, or design. It is the final summary of the results, allowing for conclusions and recommendations. It is important that the conclusions are clear and logical, based on the data presented, and consistent with the purpose of the work. A good conclusion helps the reader understand the main findings and their significance, as well as identify areas for further research or action. An effective conclusion can significantly enhance the value of the material presented by ensuring clarity and persuasiveness. Avoid unnecessary complexity and stick to simple, understandable principles, while taking into account student habits. Optimizing the learning process requires a careful approach to what students already know. Effective teaching methods are based on an understanding of their needs and preferences, which allows for a more productive learning environment. We conducted a course called "Web Analytics from Scratch" and identified many errors, starting with the title. We assumed that the course offered basic knowledge of web analytics, and therefore training began from scratch. However, we recognized that people typically enter this field with a certain level of knowledge in marketing. However, it turned out that many perceived the course title as an invitation to beginners with no experience in this field. For example, they might think: "I just made shoes, and now I can become a web analyst." As a result, participants who chose the course because of the "from scratch" phrasing often lacked even basic marketing knowledge. This led to a misperception of the course content and expectations. It's important to understand that web analytics requires prior preparation and an understanding of marketing fundamentals to successfully master the topic.

The second mistake was that we provided extensive instructions on creating a website on the Tilda platform in order to demonstrate the practical use of web analytics. However, the students were unprepared for this task, as website layout was not their area of ​​expertise. We had no right to expect them to meet such requirements.

The third mistake occurred unexpectedly. At some point, Tilda changed its rules, and the free builder no longer provided the necessary tool. Due to service limitations, completing homework became impossible. This, of course, was a cause for concern, as we needed to provide paid access to the service for thousands of students. Ultimately, we were able to negotiate promo codes with the platform, which helped resolve the issue.

The conclusion is a key step in the process of analyzing and evaluating information. It is a summary of the results based on the collected data and conducted research. A correct conclusion allows for informed conclusions and recommendations that can influence further actions. It is important that the conclusion is clear, concise, and reflects the essence of the analysis. An effective conclusion not only summarizes the results but also opens up new perspectives for further research and the practical application of the information obtained. Thus, the conclusion is an integral part of any scientific or analytical process, providing a link between theory and practice.

When developing training materials, it is important to consider the target audience, their level of competence, and the capabilities of technical resources in advance. This is especially relevant if the use of third-party services is planned for the training process. The success of the educational process directly depends on the content meeting the needs of the listeners and the availability of the technologies used.

To achieve effective results during the improvement process, it is important to strive for high-quality task execution from the very beginning. However, mistakes are inevitable, and we must learn to accept them. The right approach is to minimize the number of iterations in the improvement cycle, focusing on carefully elaborating each stage. This will reduce the time spent on revisions and improve overall efficiency.

Read also:

  • How to choose a course topic that will work well
  • How to create a high-quality online course: 6 conditions
  • How to evaluate the effectiveness of an online course
  • How to create innovations in education