NCEA Level 1, MCAT Exam 2016

overview | comments regarding questions | conclusions

Overview

The two algebra Maths Common Assessment Task (MCAT) 2016 exams held on Tuesday 13 September (Day 1) and Thursday 15 September (Day 2) have come in for a lot of abuse because they were much harder than many students expected. Some were reduced to tears, while others (including those who are supposedly capable) wrote nothing at all on their exam paper because they couldn't understand what was required for any of the questions.

My own students have reported that it was much harder than expected, and that many of the questions were hard to understand what was intended, or simply didn't make sense. Foreign students with English as a second language would have this compounded, so very capable students might end up not writing anything.

The exam papers can be downloaded from the NZQA website: Day 1 and Day 2 (PDFs, ~440 KB each).

When discussing the exams with others, I compared the two 2016 exams with the same exams from 2015. One of the people I discussed them with has two university degrees and took maths to year 3 at university, but hasn't regularly used his maths skills in his line of work in the last few years. He was convinced he wouldn't be able to do the 2016 exams but was sure that he would be able to do the 2015 exams. There's clearly a significant difference between the two years.

Exams from previous years have had a much better selection of easy questions, so every student should have been able to answer at least some of them. The questions in the 2016 exams were much more difficult than previous years. The NZ Herald highlighted one particular question from a 2014 exam, which was actually for NCEA level 2, and was an Excellence question. (Individual questions in NCEA exams are rated Achieved, Merit, or Excellence. How these ratings are used and contribute to a total grade is stupidly complicated.) An almost identical question appears in the 2016 Day 1 exam, and a similar one in the Day 2 exam; the 2016 NCEA 1 questions are more diffcult than the NCEA 2 Excellence question from 2014.

From NZ Herald article:

NZQA has today also provided information on how it developed the exam – a process which started 12 months ago.

Exam papers are developed by a writing team made up of experienced teachers who are currently teaching at the relevant level, it said.

"Writers develop papers at the appropriate curriculum level, which enable assessment of knowledge and skills aligned with the standards they are assessing.

"The writing team revises initial drafts as needed, based on reports and feedback from the previous year's examinations.

"Subsequent drafts of the papers are thoroughly checked by teachers who are currently teaching at the relevant level. They also provide sample responses that they would expect from their students in order to 'test' the questions, and further professional feedback in the form of a report.

"Subject matter experts examine all aspects of examination materials from a technical point of view to ensure it is technically correct.

"All examination material is revised along the way, as needed, in light of the feedback from these checks and reviews."

Emphasis added.

Question: Was the exam really "thoroughly checked"? Clearly, obviously, blatently NO.

Comments regarding questions

I'll make some comments here regarding specific questions in the Day 2 exam (Thursday). Many of these comments also apply to the similar questions in the Day 1 exam (Tuesday).

QUESTION ONE

(a) (i).

A rectangle has an area of x² + 5x – 36.
What are the lengths of the sides of the rectangle in terms of x?

The major problem with this question is that without knowing any other information (such as the ratio of the lengths of the sides) there are an infinite number of different ways to come up with a pair of possible expressions for the lengths of the sides of the rectangle, starting from the given quadratic.

For example, 1 and the expression itself, x² + 5x – 36. There is nothing in the question to exclude this trivial solution.

Alternatively, x³ and (¹/x + ⁵/x² – ³⁶/x³) is an equally valid solution which has every term expressed in terms of x.

Although it seems the likely intended solution, there is no reason to believe that factorising the quadratic as (x + 9)(x – 4) will give the actual sides of the rectangle. There is no mathematical reason presented in the question to prefer that solution to any other. The side lengths could just as easily be one of those listed above, or a(x + 9) and (x – 4)/a, where a is any positive real number. (There is nothing in the question indicating x is not negative.)

It would therefore be incorrect to give just (x + 9)(x – 4) as the solution without any qualifying remarks.

Did the examiners intend that sort of meta analysis in order to gain an "Excellence" mark for this question? That's not NCEA Level 1!

This is the first external exam question encountered by the students sitting this exam, and the question is nonsensical.

A less important problem is that the final x in the question is not italicised. This is a careless error, of which there are too many in these two exams.

QUESTION TWO

(c).

If p is a whole number, for what values of p is 10 × 2^p–1 < 165?

The exam instructions on the front page warn that "guess and check" and "correct answer only" methods may only be used once in the exam and will not be used as evidence of solving a problem. However, to solve this question algebraically requires the use of logarithms, which are not introduced until NCEA Level 2 (normally sat in year 12).

The problem may be solved without logarithms by just plugging in some numbers at a certain point in the solving process, but according to the exam instructions that wouldn't be enough to gain more than Achievement grade. How are students expected to solve the problem and get full credit?

This has been an issue in previous years' exams as well, and even the marking schemes (or are they just a random student's model answers?) show numbers just being plugged in. Maybe all similar questions are only Achievement questions.

Similar questions in previous years have been equations, not inequalities. This year has harder questions.

(e) (i).

Investigate what happens when Janine changes the order of the numbers in Line 1. Does she get the same answer as in Line 4?

There are several issues with this question.

First, a vague instruction like "investigate" does not give enough information to the student to be able to give an appropriate answer. What sort of investigation and how much investigation is required? How much time should a student devote to an investigation which might be completely irrelevant?

We are not told how Janine changes the order of the numbers, so she might get the same total or she might not. Is it just a yes or no conclusion required? Is either answer OK if the student shows examples?

To understand why Janine might get the same or different total will require a detailed investigation. A one hour exam is not an appropriate setting for a detailed investigation, with such an investigation's open-ended consumption of time.

Furthermore, a detailed investigation will require far more working than the allowed space permits. Nine lines is not enough room for anything other than a very superficial investigation. The similar question in the Day 1 paper gets 17 lines – much more room, and possibly appropriate for the depth required for the investigation. Maybe.

QUESTION THREE

(a) (i).

The area of a rectangle is n² – 4n – 5, where n is a positive number.
If one side is has [sic] length n + 1, give the second side in terms of n.

This question is – mathematically, at least – how question ONE (a) (i) mentioned above should have been worded. It's clear what the student should do, and there is a single correct answer.

However, there's a simple grammar error in the question. It looks like it originally read "If one side is n + 1, ..." but was changed to mention that the side has length n + 1 without making the change correctly, resulting in "is has". This is a very careless correction in something so important to get right, and something that should have been picked up by many of the alleged teachers who allegedly worked through and "thoroughly checked" the "subsequent drafts of the papers".

It also seems rather hypocritical. Should we expect that students making careless errors like missing a negative sign will not be penalised?

(a) (ii).

What do you know about the values of n for this rectangle?

The space for the answer is 22 lines long. Why does this answer need so much room? Is an essay expected on the origin of the letter n?

The previous similar question on the previous page (Question Two (e) (iii): "What do you know about the numbers ... Explain your answer.") only provides an appropriate three lines to write on, with the bottom of that page left blank; three lines for that answer including explanation.

Other questions have a certain number of lines drawn, then blank space underneath. Why not this one?

When there is so little that can really be said about the values of n, the 19 or 20 extra lines are misleading and confusing.

(d).

Solve

x² – 3x – 10  (x + 5)(x – 5)

=

x 2

.

This one is very similar to the one in the Day 1 exam which the NZ Herald showed is more difficult than an NCEA Level 2 Excellence question from 2014, which just had a whole number on the right side. What is NZQA trying to achieve by using a harder-than-NCEA-2-Excellence question for NCEA 1? Was the earlier question, from 2014, only Excellence by mistake?

(e).

A game has a groove that a small ball is rolled along.
The groove can be modelled by ...

Two diagrams are given which do not relate to each other. They do not show the same groove, or the same object the groove is in.

A groove is a three dimentional object but the equation that supposedly models the groove is only two dimensions. The question should have said something like "The cross section of the groove can be modelled by ..." It doesn't make sense as written.

FWIW the image on the right is of the product on this page [broken link] (available for €15.95 plus €5 shipping).

Conclusions

The NZQA has been quoted as saying the exam "was developed by an experienced team with expert knowledge of mathematics assessment and reviewed by several current secondary school teachers" and "Subject matter experts examine all aspects of examination materials from a technical point of view to ensure it is technically correct."

For the "several current secondary school teachers" and "subject matter experts" who reviewed it, that's just embarrassing, especially for the infinite answers to the first problem in Question One and the grammar error in the first problem in Question Three.

The wording of some questions is nebulous, leaving exact answers impossible and students unclear about what they are required to do, and the amount they are required to do. This last issue is compounded by having available space for some answers which doesn't correspond with the detail probably required in the answer. This is unreasonable. The students should be doing maths, not mental gymnastics guessing about how much to do.

The easiest questions are not as simple as in previous years. For example, 2013 Day 1 started with solving a linear equation. 2016 Day 2 started with a problem with infinite solutions.

The exam was so much harder than previous years that it seems possible that it was actually an NCEA Level 2 exam. NCEA 1 exams in recent years have not required students to apply their maths knowledge to loosely defined maths-related tasks.

This exam had serious problems with its questions and structure. Serious questions need to be asked about the exam preparation process, including checking.