SmarterMaths – must know information on HSC question “difficulty bands”

Thanks for subscribing to SmarterMaths.

Teachers often ask about our methodology in allocating questions with a specific “difficulty band”. In our HSC courses, each question part is allocated a difficulty band between 1-6 which appears in a colour-coded circle in the question menu (see below).

Bands could arguably be allocated with strict adherence to “band descriptors” as defined by NESA (noting the complications added by the E1-E4 descriptors in extension courses). Alternatively, they could be a function of reverse engineering band level rankings of students. i.e. where the 90-100% band 6 ranking say, is fed into an algorithm to identify a set of questions that a student should have answered correctly to achieve that result.

Things get complicated very quickly when attempting any deep dive in this area, with competing and equally valid rationales presenting themselves.

Any subscribers interested to know exactly how we look at this issue should read on …

Principle

Provide the most useful snapshot of the question database to allow teachers to search for questions in the most efficient way.

• The most important (but not exclusive) input is the raw State mean mark.
• Our general approximation levels are: Band 6 (< 20%), Band 5 (20-50%), Band 4 (50-75%), Band 3 (75-90%), Band 2 (90-95%), Band 1 (> 95%). These approximations are adjusted for multiple choice.
• In our view, if a question is poorly answered, notwithstanding a low “band descriptor” definition, teachers should see it flagged as having caused problems i.e. with a higher band.
• Likewise, if a question is extremely well answered (say +90% state mean mark), it is more important that teachers see this as a Band 1-2 question than what might be, with strict reference to NESA’s definitions, a Band 3. In this specific example, a colour-coded circle icon indicating Band 1 or 2 difficulty will appear, identifying a question that teachers can immediately see was answered correctly by +90% of students – and where their students cannot afford to make any silly mistakes!

The above methodology has the advantage of being highly correlated with official band descriptors although, by design, we gladly depart from this band alignment when we think it will provide superior information.