Please, Spare Us! India Does Not Need A National Testing Service

The government recently announced the creation of a centralised testing body, the National Testing Service (NTS), to conduct national level entrance examinations for engineering, management and medical programmes. The objective of this organisation is to conduct the Common Admission Test (CAT, for IIMs), Joint Entrance Examination (JEE-Main), JEE (Advanced), Graduate Aptitude Test in Engineering (GATE), Common Management Admission Test (CMAT), National Eligibility and Entrance Test (NEET, for medical admissions), National Eligibility Test (NET) — which are currently conducted by the CBSE, IITs, IIMs and AICTE every year.

The establishment of NTS comes at a time when there is already quite a bit of confusion over the centralised medical admission examination, the NEET. Tamil Nadu wants to bypass the NEET, as the syllabus is out of sync with their higher secondary syllabus, and West Bengal wants it conducted in Bengali. ‘Reality Check India’ explains quite well, why the NEET should have been avoided. Extend the issues discussed in that article, to all the other exams which the NTS is likely to take charge of.

Why centralised testing is unfair in India

India does not need another centralised agency for examinations. The reasons for this are numerous:

1. Education lies in the concurrent list. Every state has its unique syllabus, with topics, standards and coverage adapted to local needs and relevance. The idea of a "one nation, one merit list" is not only un-achievable but also unfair, since it completely dismisses the state to state variations in the syllabus. Karnataka has the KCET system, AP and Telangana have EAMCAT.

All these systems have evolved with time and are tuned to local needs.

2. State boards often cater to a large number of first generation learners, and so the system is often adjusted to their needs. While Tamil Nadu state board largely covers similar topics as ISC or CBSE in class 11 and class 12 mathematics (just to cite an example), there is a dramatic difference between the standard of the questions asked. At times, this is due to sheer incompetence and indifference of state board officials. But often, the board has chosen a standard optimised to the needs and ability of the student body.

3. Then comes the issue of "false negatives" in an entrance exam. Many of our entrance exams, such as the JEE, test for a level higher than what might be required to study an engineering degree. Often the academic content of a B-Tech degree, might not require much more than the diluted standards of a state board physics, chemistry, and mathematics. Thus, entrance examinations should only be based on the least common denominator, for the sake of fairness to students of different boards.

This, however, has its share of issues as it dilutes academic rigour. IITs might want to have a difficult test which pulls in the Olympiad winning champion, who might not be able to distinguish himself in an easy test.

Centralization will invariably require students to join coaching centres thus encouraging a system of private gates to publicly funded institutions.

Failure of pan-India systems to handle assessments fairly

Centralised examinations have not worked out very fairly in India. For example, on multiple occasions CBSE and CISCE have been caught rigging scores, increasing pass rates arbitrarily and inflating marks by as much as 10-12 per cent in the aggregate. There is also evidence to suggest that influential schools and specific zones like Delhi are given an unfair advantage. The Times of India article linked above, suggests that a Delhi student with English and PCM would receive five marks more than others in the aggregate. CBSE, despite the pretence of a national board, is dominated by members from Delhi and North India. There is a disproportionate representation of those from the corridors of Delhi University, Jawaharlal Nehru University.

It is inevitable that any new centralised testing body will also go the same way - and run processes primarily optimised for the convenience of students from Delhi and central boards. However, one must add, that CBSE and ISC have superior academic standards to most of our state boards.

No one linear ranking of merit

The nature of entrance exams is such that even the best of them can only test for a least common denominator. One linear ranking and uniform merit list can never be a great idea for a country as vast and diverse as India. Let's try to understand this in the context of the engineering entrance as that has a very large number of takers.

- What if a particular college wants to recognise an Informatics Olympiad Winner and give him special preference in their entrance process to a Computer Science program, which is currently the case at IIIT Hyderabad?

- What if a particular institute wants to award bonus marks to students who were awarded medals in Olympiads or those who performed well in relevant extracurricular? This is currently a part of the admission at IIIT Delhi.

- BITs Pilani has a well-regarded entrance process which has not only physics, chemistry and mathematics, but also a section on language and reasoning.

These steps are required, because of our entrance examination pattern, typically based on physics, chemistry and mathematics are unable to identify talent with a special interest or ability in a particular subject. Recognition of certain accomplishments like these is impossible in a system based on a "one nation, one rank list" mirage.

Standard tests lead to a situation where one is unable to distinguish aptitude for verticals. This happens even within the IIT system where everything from Computer Science to BioTechnology to Economics uses a test based on physics, chemistry and mathematics as the only criteria. It overrides the autonomy of private players to identify and select talent of their choice.

Similarly, will the organisation be able to hold "one exam" for an MBA entrance test - a degree where different business schools value very different experiences and qualifications? The answer is obviously no.

Also, one cannot impose this system onto minority institutions which enjoy autonomy in such matters under Article 30. This immediately makes it unfair to those who do not have a minority exemption.

Increasing the blast zone of ill-conceived experiments

Centralization and concentration at a single point will also lead to centralised experimentation by future Human Resource Development (HRD) ministers. Remember the removal of the CBSE standard 10 board exam and the introduction of an ill-conceived grade-based CCE almost purely on the whims of Kapil Sibal. The system has resulted in bizarre levels of grade inflation, and 11 per cent of class 10 students scored a perfect 10/10 GPA in 2016. There has been a spiral of grade inflation and an extreme dilution of standards in the CBSE grade 10 exam.

Something similar could happen with centralised exams. Decisions will eventually be made for whimsical and non-academic reasons. The presence of a central agency will immediately spill all consequences nation-wide. The chances and implications of something going wrong increase manifold. For example, the NEET-2 question paper was leaked in Uttarakhand, in 2016.

Statistical Knowledge of assessments in India is abysmal

1. NEET-1 and NEET-2 used two different question papers (in 2016), with varying levels of difficulty. They release a combined rank list based on the absolute score or the raw marks, without any re-adjustment or re-scaling of the scores to compensate for variations in difficulty levels.

2. JEE did something similar with its online and offline versions.

3. Median CBSE score in class 12 examinations jumped by 8 per cent between 2004 and 2016 (from 61.5 to 69.5 per cent).

It is not easy to design a “one size fits all test” which has certain statistical characteristics. For example, the nature of the IIT JEE syllabus, causes it to be significantly biased towards certain states where either coaching centres abound, or the schools are affiliated to central boards like CBSE and ISC.

- This is the shape of a CBSE curve in Mathematics. Not only is there a massive bunching of scores at 95 but they have also reshaped it into something resembling a uniform distribution. One can only wonder how the Gaussian got so severely distorted.

- For decades, ICSE and ISC exams have certain numbers altogether missing from their score distributions. It is likely that there is some bug or glitch in their system. These missing gaps suggest some error where at some stage a floating point number is accidentally converted into an integer. Notice the huge gap before 40. There is a very generous distribution of “grace marks” to bump up students to the pass mark.

Multi-component testing:

We have an absurdity where students ditch subjects like economics to study physical education because one has strict grading and the other has a liberal assessment. We do not have processes in place to calibrate fairly across students with different elective subjects. This happens at the UPSC examination as well.

- The NTS is unlikely to be able to develop specialised competence across domains and conduct exams for fields as diverse as technology, management and medicine. Conducting an exam like the GATE, NEET or JEE requires domain experts and academics on board. This competence is not something which can be developed within a short span of time.

The organisation could take up responsibilities such as those of ETS and Ofqual

What might be beneficial is a general examination like the Graduate Record Examination (GRE, USA) which is a rating examination rather than a ranking examination. With the large numbers which we have, this could lead to a coarse filter, which can be used as a primary eligibility requirement by various universities, to make the first screen through lakhs of applications and converge to a few thousand who can then be put through their layer of testing.

The NTS could also serve as an exam watchdog similar to the Ofqual in the United Kingdom, which regulates examinations and qualifications. The objective could be to handle the operations and conduct of the examination to avoid mass cheating and paper leakage. It should also monitor the statistical distributions of reported scores to identify malfeasance, grade distortion and swing the axes on grade inflation time to time. The key point is that the responsibilities of the NTS should be framed in a manner which does not lead to any coercion which requires same academic systems all across the country.

The body could also help come up with guidance to central universities, on how to calibrate and compare the scores of students from different school leaving systems. There is no objective way to help us benchmark 85 per cent in Tamil Nadu board versus a similar score in CBSE or Maharashtra Board. This leads to complicated situations such as that witnessed in Delhi University in 2015 and 2016, where the bulk of those admitted were from TN, CBSE and ISC as those were the boards which graded most generously.