Evolution of Samiksha

Samiksha is an OpenSource product designed to improve school quality control activities conducted by state education department officials in government schools. This typically includes mentoring to support teachers based on observations of their in-class teaching practices and monitoring to inspect the status of infrastructure and quality of school processes such as classroom teaching, availability of teaching learning materials, functional toilets, completion of civil works projects in the school and so on.

Samiksha consists of a mobile application, an application control console and a set of review dashboards. Since July 2019, Samiksha has been used by state education departments of two states (Haryana and Himachal Pradesh), nearly 30,000 government schools with over 14,000 active government users.

This post is about how we factored in user-centricity into the design of Samiksha.

#Inquiry as an art

Curiosity drives inquiry. Passion for curiosity and the method of inquiry can lead to discovery of insightful and actionable information. When a teacher sparks the curiosity of a student to understand addition of numbers using a bunch of straws, the teacher is unleashing the student on a path of inquiry to discover information about how addition works. Consider the alternative. If the teacher says, “Let’s do addition. Now write 5+5=10 in your notebooks and remember it”. In this case, students would mindlessly and often grudgingly copy paste information without inquiring further.

Our principle for designing Samiksha was to build a product that makes the process of inquiry into the quality of school processes as interesting for government officials as the inquiry into addition of numbers using a bunch of straws for students.

#School mentoring and monitoring as a process of inquiry

To start with, there are three types of government users who are responsible for the success of mentoring and monitoring during school visits. These stakeholders include:

1. Administrators (state level officials responsible for designing school quality control process)

2. Mentors & Monitors (District Education Officers, Block Education Officers, Cluster Education Officers and coordinators for different initiatives at the state level who are responsible for conducting these visits and collecting information)

3. Reviewers (State, District and Block leaders who are responsible for periodically reviewing information and taking decisions for improving quality of education in their respective state)

For each of these users, our governance consulting teams understood what users were expected to do and identified challenges they faced. On the basis of this information, our technology team created a user-centric process of inquiry for all the three users listed above.

#Designing school observations processes on Samiksha application control console

Administrators are state level officials who are responsible for designing the school observation process. Some of the aspects that administrators have to plan for, are:

• Designing school observation forms

  • What questions should be asked in the school observation forms?
  • How to update questions on an ongoing basis?
  • How to provide access to different school observation forms to different officials?

• Managing base data in the school observation application database

  • How to update user information in the database?
  • How to update school lists in the database in case of mergers or closures of schools?
  • How to update the total number of visits to be conducted by each official in a month?

In Haryana and Himachal Pradesh, the state education department administrators were facing two challenges with respect to their technology solutions for school quality monitoring. Challenges were:

1. High dependency on engineering teams from vendor agencies to update basic information such as data collection forms and user profile data in the application. This would lead to loss of time.

2. Limited feature extensibility of the technology product thereby making it difficult to implement new features such as designation-based access to school observation forms. This would lead to loss in user experience as the users would have to view all observation forms even if they were not relevant to the user.

Let’s say you have to write an essay using Microsoft Word and every time you need to change the font of the document you need to speak with a Microsoft engineer. Sounds tedious doesn’t it?

Our primary objective while building Samiksha was to ensure that administrators, who are typically non-engineers, can have the ability to manage basic aspects through the application control console independently. Today, while using Samiksha, administrators can:

• create their own data collection forms using excel sheets
• manage the base data of users and school lists
• define role-based access to school observation forms

These are examples of how transferring control of managing basic aspects of the technology to administrators can make them more independent in managing school quality control processes and more engaged in using technology to fulfill their responsibilities. Adding role-based access to observation forms was possible in Samiksha as it is built as a reusable, scalable, extensible and deployable product.

Screenshots from the Samiksha application control panel being used by administrators in the state education department in Himachal Pradesh and Haryana

#Conducting school observation visits

Mentors and monitors are responsible for visiting schools to collect information regarding the quality of school processes. Some of the aspects that mentors and monitors look at while conducting the visit are:

• Preparation required for the school observation visits

  • What to physically bring for the visit (printed formats, cell phone with internet)?
  • What is the past observation visit information for schools to be visited?

• Implementing the school observation process

  • What are the questions to be covered during the school observation visit such as classroom infrastructure, classroom teaching observation, teaching-learning material availability, head teacher discussion?
  • How should the observation visit be conducted in terms of what data needs to be collected first, second, and so on?

While designing the questions it is important to consider the user scenarios for mentors and monitors who conduct these observation visits. Our consulting teams in both Haryana and Himachal Pradesh had joint discussions with mentors and monitors to understand their perspective. Key challenges they faced with their existing observation mechanisms were:

1. Questions in data collection forms seemed like an endless checklist
2. Feedback collected was fairly detailed but not visual, thereby often missing quality nuances
3. Flow of questions did not take into account how mentors and monitors physically moved within the school to collect the process quality feedback

Let’s say you were still working on that essay on Microsoft Word and you just updated the table of contents. To your surprise the headings in the table of contents are not correctly ordered. Your ‘conclusion’ is above the ‘abstract’! That will introduce an unnecessary challenge for your readers to navigate when they try to read your essay.

Here our objective was to enable mentors and monitors to focus more on quality engagement with school teachers while conducting observation visits as opposed to making the data collection exercise an end in itself. Some of the points that our consulting teams, in collaboration with the administrators, instituted into the design of the data collection forms were:

• Decreased the number of questions in observation forms with a brutal focus on understanding quality of school functioning
• Clustered questions into sections to create a flow of questions mirroring potential on-ground movement of mentors and monitors
• Made conditional questions visible only when required, thereby avoiding extra information in the observation visit forms
• Included the ability to upload images to get a visual perspective of school functioning
• Also, made available past school observation data on the Samiksha mobile app and the dashboards to view the trend of school performance across indicators

These are examples of how minor process changes can improve the experience of mentors and monitors who have to conduct anywhere close to 5-10 school visits every month of the academic year.

Screenshots from the Samiksha mobile applications being used Himachal Pradesh and Haryana

Vijay Kumar, Additional Block Resource Coordinator, Sirsa, Haryana says, “Samiksha is easy to use. The mentoring form is designed well as it is thorough. I also like how it allows us to capture discussions with teachers and HMs, along with any classroom best practices being followed in the school.”

#Reviewing school observation data to take informed decisions

So administrators design the process, mentors implement the process, and reviewers who are typically at the state, district, block and cluster levels are responsible for reviewing the data collected through the school observation process. To ensure these reviews are effective some questions that reviewers look at includes:

• Data authenticity and quality

  • Is the observation feedback complete and conducted as per guidelines?

• Data inferences

  • What is the trend of observation visit feedback from these visits?

• Data visualization

  • How can the school observation data be visualized to understand basic trends?

While reviews are highly dependent on the person who is leading it and how s/he evaluates information to inform decisions, there are common challenges that we have observed across these reviews. These challenges are:

1. School visit feedback data is not summarized and available for reviews on time
2. Any unexplained deviation in data leads to it being disregarded as incorrect and useless as no one monitored and inquired about the deviation earlier when it occurred
3. Visualizations of the observation feedback on the dashboard do not inspire any further inquiry into the data

Let’s say your professor asks for a printout of your essay to review. Your heartbeat is racing as you had worked so hard and now it is time to get a decision on your grades. Unfortunately, the printer you used had low ink and certain sections of your essay are not legible on the printout! However, your supervisor continues to review the essay. You would be quite frightened to learn what decision she took to grade your essay, after reading the barely legible text on the printout!

When we were designing Samiksha from the perspective of the reviewers, we wanted to ensure that they had maximum visibility of data for driving an effective review. Some of the things we solved for, included:

• Enabled reviewers (any non-technical person) to configure their own visualizations in a few clicks as opposed to waiting for days for engineering teams to create these visualizations
• Created basic compliance dashboards for reviewers to track completeness and compliance of school visits on a real-time basis (e.g. number of visits conducted by officials, by schools, by observation type)
• Created configurable dashboards to visualize school observation data and facilitate inquiry for bringing out macro trends for decision making

These are examples of how enabling reviewers and their internal teams to create basic dashboards independently can lead to higher engagement to understand, explore and leverage school visit data to make informed decisions.

Samiksha dashboards used by the state education departments in Himachal Pradesh and Haryana to drive reviews based on data from the Samiksha app.

#Future of Samiksha

In the coming months, we hope different states will deploy the OpenSource Samiksha product to streamline and conduct school visit exercises. Our objective is to make this process of inquiry into school process performance an engaging and insightful data exercise for all stakeholders.

We continue to observe and understand this process across the states that we are working in and identify further use cases that can be enabled through Samiksha. Here are a few more things we are thinking about:

• Enabling configurable prompts for monitors while they conduct the school visit (if teaching learning material is not present then a prompt can indicate a teaching method to be shared with the teacher for teaching a topic in the absence of that material but making it as interactive for the students)
• Enabling configurable SMS/WhatsApp notifications to share school visit summary with reviewers
• Using machine learning to auto analyse image data to manage image verification at scale

Read more about the Samiksha specifications and how to deploy it for your own state.