Making internet research easier

Making internet research easier

Policy making is a complex business, but a key element is the need for information, research and evidence. While many consider peer-reviewed journals to be the most credible source of evidence, the reality is that most evidence falls into the category of ‘grey literature’ – material produced and published outside of the commercial or scholarly publishing industries.

Produced directly by organisations, including government departments and agencies, academic research centres, non-government organisations, think tanks and commercial consultants, grey literature commonly includes reports, discussion papers, working papers, briefings, literature reviews, white and green papers, submissions, evaluations, fact sheets and guides, position papers and procedures.

The huge amount of information and research published online provides unprecedented access to knowledge from a wide range of sources, enabling a much greater level of understanding and participation in public interest issues. It also brings a number of challenges: searching, sifting, evaluating and accessing information and research are time-consuming and often frustrating tasks. Digital curation of policy resources, particularly grey literature, is dispersed and fragmented, creating a black hole of resources that are being lost from online access over time.

‘grey literature is dispersed and fragmented, creating a black hole of resources that are being lost from online access over time’

In 2012 the Australian Research Council funded the first detailed study of the role and value of grey literature for public policy and practice. The Grey Literature Strategies project was a partnership between Swinburne University of Technology, Victoria University, ACER, the National Library of Australia, the National and State Libraries Australasia and the Eidos Institute. It involved online surveys of 943 information users, 144 producing organisations and 114 collecting services in Australia. A summary of findings was provided in a discussion paper released in late 2014.

At the heart of the project was the question: where is the evidence that is being used for policy and practice? The study addressed where it comes from, how it is produced and disseminated, how people find and access it, what issues they have in doing so, and how it is being managed and preserved for long-term access.

Consumption of grey literature

For those working in public policy and practice, the use of research and information is a weekly if not daily activity. The most common resources, consulted regularly or occasionally by over 80 per cent of surveyed policy information users are reports, journal articles, discussion papers, and ‘briefings, guides and research reviews’. News reports and conference papers are used regularly by 79 per cent of respondents, and two-thirds use books and data. Working papers, submissions and evaluations are used by over half of all policy workers regularly or occasionally.

Websites of key organisations, email newsletters and colleagues sharing information are the most important means of finding out about new research for over 75 per cent of information users. In terms of access, search engines are the place 95 per cent of information users turn to on a daily or weekly basis, while around two-thirds use their own organisation’s website or that of another organisation, and just over half use a subject repository or clearinghouse.

When asked to estimate, information users report that grey literature makes up 60 per cent or more of the material they consult for their work. For a quarter, grey literature constitutes more than 80 per cent of their source material.

Furthermore, those working in the policy arena estimate that around one-third of their work time each year is spent using grey literature – the equivalent of around 73 work days.

Based on occupation figures from the Australian Bureau of Statistics and other data, it can be assumed that around one-third of the workforce, or approximately 3.8 million people, use grey literature. The value to users of grey literature can therefore be estimated at $33 billion to $44 billion per annum.

Production of grey literature

Grey literature is a key method used by surveyed organisations across all sectors of society to translate and disseminate new research or policy positions. Eighty-four per cent produce grey literature on at least a quarterly basis, while 38 per cent publish at least weekly.

Staff in producing organisations estimate that they spend around a third of their weekly work time creating grey literature. This is a considerable investment of time, expertise and resources, especially when there could be 30 000 organisations currently producing grey literature in Australia. Based on reported work hours spent creating grey literature, it is estimated that production costs around $30 billion per annum.

Conference papers are the most common type of grey literature, produced by 82 per cent of organisations, followed by discussion papers (77 per cent), reports (76 per cent), briefings and reviews (67 per cent) and submissions (63 per cent).

Most surveyed producing organisations (85 per cent) ‘often or always’ use their organisation website to store and provide access to their content. Just over half of the surveyed producers based in education deposit their material with their institutional repository. Beyond this, there is little take up of external databases, libraries or other curatorial services.

It would appear from these results that producers prefer the ease and immediacy of their own websites rather than other, potentially more stable, options for content, despite the risks of losing access when content is moved or websites are changed.

Challenges

We now have a needle-in-the-haystack problem, where users have trouble finding relevant content and spend a lot of time sifting, evaluating and managing the material that is found. Finding relevant resources, including knowing what exists and where to look, and the amount of time required to sift and evaluate, are major issues for 45 per cent of information users. Just under half of surveyed information users (44 per cent) would use some types of resources more often if they were easier to find or access.

Information users estimate that on average they could save 16 per cent of their work time if grey literature were more readily accessible. Based on a 35 hour working week, this equates to 288 hours a year per person. When calculated at the national level, the efficiency impact of grey literature being more readily accessible and systematically preserved would be worth about $17 billion per annum.

Deadlinks

One of the key frustrations in accessing grey literature reported by information users is ‘reference rot’ and deadlinks. Studies estimate that the rate of loss of digital content is around 30 per cent within a few years of publication online. Around half of information users surveyed reported being unable to access an online resource on a weekly basis or more often because of a deadlink.

Deadlinks are also time-consuming, with more than half of surveyed workers continuing to search for the item after they came across a deadlink on a weekly basis, and spending an average of around 17 minutes doing so. Calculated at the national level, it is estimated that deadlinks are costing Australia $5 billion a year.

Despite posting most of their content on their own website, only 26 per cent of the producing organisations surveyed have a strategy in place to prevent deadlinks, suggesting the problem may get worse before it gets better.

Copyright

A further major issue is that Australian copyright legislation unduly limits the collection and preservation of copyrighted material, including orphan works (where the copyright owner is unknown or no longer exists), ‘unpublished’ material and non-commercial, publicly funded research and information in the public interest. There are many legal obstacles preventing libraries and archives from making available full-text copies of free, non-commercial, open access digital documents for research and public interest groups.

Two-thirds of surveyed collecting institutions felt strongly that not being able to copy, store or redistribute online information because of copyright law was a problem for their collection. Over 80 per cent of surveyed collection services believe that introducing fair use copyright law is the most important strategy for improving collection and access to grey literature.

Standardisation

More than 80 per cent of surveyed collecting services regarded standard bibliographic information on publications and agreed metadata standards as important strategies for improving the value and use of grey literature. To help organisations produce user-friendly publications, a series of one-page clear publishing guides have been developed by the Grey Literature Strategies project.

Recommendations

In summary, digital curation of policy information is inadequate, ad hoc and under-resourced.

There are clear opportunities to reduce the challenges and increase the benefits of digital grey literature. Production practices could be improved, and mandates could be created for greater access to publicly funded research. Large-scale digital collection infrastructure, collaborative cataloguing systems and shared standards could be developed for efficient collecting. Reforming copyright legislation to support fair use provisions for preservation and access to non-commercial material of public interest would make large-scale collecting more efficient and achievable.

Better management and collection of grey literature could mean an efficiency dividend of around $17 billion per annum nationwide. A very small fraction of this amount could be spent improving production and collection standards.

Read the full paper:
Where is the evidence? Realising the value of grey literature for public policy and practice, by Amanda Lawrence, John Houghton, Julian Thomas and Paul Weldon, Swinburne Institute for Social Research, 2014.