Australian problem-solving skills in context

Australian problem-solving skills in context

The national report, Thinking it through: Australian students’ skills in creative problem solving, released by the Australian Council for Educational Research in September, presents the findings from the 2012 OECD Programme for International Student Assessment (PISA) problem-solving assessment.

Managed by ACER at the national and international level for the Organisation for Economic Cooperation and Development (OECD), the PISA 2012 problem-solving assessment measured 15-year-olds’ general reasoning skills, and their willingness and ability to regulate problem-solving processes.

Around 85 000 students in 28 OECD countries, and 16 partner countries and economies participated in the computer-based assessment. The Australian PISA 2012 sample consisted of more than 14 000 students from 775 schools, drawn from all jurisdictions and school sectors.

Key findings for Australia:

• Australian students performed better than expected in problem solving, based on their performance in mathematics in the main PISA survey conducted at the same time.
• While school choice in Australia has less of an impact on problem-solving achievement than across the OECD, there is considerable variation within Australia depending on which state or territory you live in.
• Australian students reported slightly higher levels of perseverance when problem solving compared to students across the OECD.

International comparisons

Australian students achieved an average score of 523 points, above the OECD average of 500 points but significantly behind the highest-achieving countries, Singapore and Korea, which averaged 562 and 561 points respectively.

The average achievement of each participating country and economy relative to Australia is displayed in Table 1. After accounting for insignificant differences between scores, three countries and four economies performed at a higher level than Australia, three countries performed at a similar level, and 32 performed at a lower level.

Table 1. Average problem-solving achievement relative to Australia

Australia was one of 19 countries that achieved a mean score that was significantly higher than the OECD average.

In Australia, the difference in achievement between the most capable problem solvers and the least capable was 320 score points, similar to the OECD average of 314. Among OECD countries, the widest difference between the most and least capable students was found in Israel (405 score points), while the narrowest difference was found in Turkey (262 score points).

Around 84 per cent of students in Australia, England, France and Italy achieved or exceeded the baseline proficiency level (Level 2), where students begin to demonstrate the competencies that will enable them to participate effectively in life situations, compared to around 93 per cent in Korea, Singapore and Japan, and 80 per cent on average across the OECD.

Figure 1. Percentage exceeding the baseline proficiency level (Level 2)

Performance within Australia

By school location

All of Australia’s states and territories performed at a statistically similar level, except Tasmania, which performed significantly lower than the other states and territories. Western Australia, the Australian Capital Territory, New South Wales, Victoria, Queensland and South Australia performed at a significantly higher level than the OECD average, the Northern Territory performed at a level not significantly different to the OECD average, and Tasmania performed significantly lower than the OECD average.

Students attending metropolitan schools outperformed students in provincial schools by an average of 18 score points, while provincial school students outperformed students in remote schools by an average of 35 score points. The spread of scores between the most and least capable students was widest for students in remote schools, and narrowest for students in provincial schools. The spread of scores in metropolitan schools was the same as across the OECD.

By student characteristics

Gender
Boys scored significantly higher than girls in 23 countries and economies, while girls performed significantly higher than boys in five countries. The largest difference in favour of boys was found in Colombia (30 score points) and the largest difference in favour of girls was found in the United Arab Emirates (26 score points). Across OECD countries, boys outperformed girls by six score points on average.

The average score for Australian boys was not significantly different to that for Australian girls. Western Australia was the only state or territory to have a significant gender difference, with boys scoring 18 points on average higher than girls – a difference three times larger than the OECD average.

Wealth
Within Australia, students in the highest socioeconomic quartile achieved an average of 73 score points higher than students in the lowest socioeconomic quartile. Ninety-one percent of Australian students in the highest socioeconomic quartile performed at or above Level 2 compared to 76 per cent of students in the lowest socioeconomic quartile.

Indigeneity
The performance of Indigenous Australian students in problem solving was significantly lower compared to non-Indigenous Australian students, by an average of 72 score points. Sixty-three per cent of Indigenous students achieved at or above Level 2 compared to 86 per cent of non-Indigenous students.

Immigrant status
Australian-born students’ performance in problem solving was similar to that of foreign-born students. First-generation students achieved at a higher level than their Australian-born and foreign-born peers, but returned a wider range of scores between their highest and lowest achievers.

Between-school and within-school differences

PISA examined whether the variation in problem-solving performance within countries is attributed to within-school (student-level) or between-school (school-level) differences.

In Australia, the amount of variation in performance within schools was 75 per cent, and was higher than the OECD average of 61 per cent, indicating that student-level factors have greater impact on problem-solving achievement in Australia than across OECD countries. Within-school differences ranged from 72 per cent in Victoria to 94 per cent in the Northern Territory.

The amount of variation in performance between schools in Australia was 28 per cent, compared to the OECD average of 38 per cent. Between-school differences ranged from 19 per cent in South Australia to 39 per cent in Tasmania, suggesting that school choice in Tasmania will influence student problem-solving performance more than school choice in other states and territories. Internationally, the lowest variation in between-school performance was found in Finland (10 per cent), where students can expect to achieve similar results regardless of the school they attend.

Skills related to problem solving

Mathematics achievement

It is expected that students who do well in the PISA problem-solving assessment are likely to do well in the PISA mathematics, science and reading assessments, and vice versa.

Across the OECD and in Australia, the strongest correlation was between problem solving and mathematics. By comparing the problem-solving performance of students from one country to the average performance observed across participating countries at a given level of proficiency in mathematics, it is possible to infer whether students’ performance in problem solving is the same as, or above or below students with similar proficiency in mathematics.

Australian students performed better than expected in problem solving, based on their performance in mathematics. The difference between observed and expected performance is particularly large among students with strong performance in mathematics.

Problem solving processes

The PISA problem-solving framework identifies four cognitive processes involved in solving a problem:

• exploring and understanding,
• representing and formulating,
• planning and executing, and
• monitoring and reflecting.

There is a pattern of high-performing countries performing relatively stronger on the exploring and understanding, and representing and formulating processes, and relatively weaker on the planning and executing, and monitoring and reflecting processes. Lower performing countries generally show the opposite of this pattern.

Based on Australia’s overall performance, Australian students are comparatively stronger on the exploring and understanding, and representing and formulating processes, and are relatively weaker on the planning and executing process.

In terms of problem-solving skills, Australian students are good at generating new knowledge and can be characterised as quick learners – questioning their knowledge and challenging assumptions, and generating and experimenting with alternatives – and good at processing abstract information. Australian students’ skills in using their knowledge to devise and execute a plan in order to solve a problem could, however, be improved.

Student attitudes

Perseverance

Perseverance relates to students’ willingness to work on tasks that are difficult, even when they encounter setbacks.

Australian students reported slightly higher levels of perseverance when problem solving compared to students across the OECD. Across the high-performing countries, students in Japan, Korea and Chinese Taipei reported the lowest levels of perseverance, while students in Shanghai–China and Singapore reported the highest levels. Of the English-speaking countries, students from the United States reported the highest levels of perseverance.

Approximately two-thirds of Australian students indicated it was not like them to give up easily when confronted with a problem and almost half indicated it was not like them to put off difficult problems. Half reported that it was like them to remain interested in the tasks they started, while one-third reported that it was like them to do more than what was expected of them when confronted with a problem.

Openness

Students need to be willing to engage with problems and to be open to new challenges in order to be able to solve complex problems and situations.

Of the high-performing countries and English-speaking countries, students from Japan reported the lowest levels of openness to problem solving, whereas students from the United States reported the highest levels of openness. Students from Australia scored below the OECD average.

In Australia, two-thirds of students indicated they sought explanations for things, approximately half indicated they could handle a lot of information, were quick to understand things and could easily link facts together, and one-third reported they liked solving complex problems.

Read the full report:
Thinking it through: Australian students’ skills in creative problem solving, by Lisa De Bortoli and Greg Macaskill, is available from the Australian PISA website < www.acer.edu.au/ozpisa >