Imptact Header

Evidencing impact: The When, the What, and the How

There are no universal answers to the question of how to evidence impact. What an individual or group of researchers will need to gather to demonstrate the impact of their research will depend on both the research and impact that has been (or is hoped to be) achieved.

These pages will help you understand what could be provided as evidence of the change with some practical information on collecting it.

Collecting evidence:

When: Evidencing impact not only an end of project activity. The research use should be reviewed regularly to capture interim and unforeseen impact.
At the start of the research project or partnership gather information on the current external situation. This baseline will allow you to identify the change due to the research over time.

What: Impact is often described in terms of reach and significance.

Reach -  the spread or breadth of influence or effect on the relevant constituencies. Reach is not assessed in purely geographic terms, nor in terms of absolute numbers of beneficiaries, but rather based on the spread or breadth to which the potential constituencies have been affected.

Significance -  the intensity of the influence or effect.

Simply put, this is the breadth of potential beneficiaries that have been affected and how much they benefited or a situation changed.

Some impacts may be demonstrated through a single piece of information, however more often a credible series of independently verifiable qualitative and quantitative indicators or proxy measures need to be brought together to demonstrate impact.

Quantitative and Qualitative evidence:
Quantitative evidence, e.g. attendance data, patient numbers, sales, statistics or organisation uptake, should be collected to describe the reach.

Qualitative evidence provides the understanding as to the context of the issue and significance of the impact.

Where to keep evidence

You should have an easily accessible place to store the evidence of your impact. The impact module on MyPublications can be used to store evidence.

How: Types of Evidence

Getting started

Not sure where your impact may have occurred? Much of the information you require may already be publicly available.

What can it provide? How to get it.
Information available in the public domain e.g. online, in the news or in reports.

New or corroborating information where you research has been used.

To demonstrate change you need independent information on the previous situation or scale of the issue from websites or reports from external stakeholders.

The UK Office of National Statistics has a searchable database of national and local statistics which can help describe the context of the issue. 

If your potential impact is international then searching the UN Global Issues, relevant Charities or Government web pages may provide information on the scale of the issue.

There are different exploratory approaches that can be used to find information using internet searches.

If you believe that your research will be featured in the media or through partner press releases or on their websites you could set up a Google alert to capture that information.

Information on research use from partners

This information can provide evidence of process or organisational changes. It may also be the only way to obtain confidential or commercially sensitive data.

It is always best to discuss impact at the start of the research collaboration, that way you will know what your partner wants to achieve using the research and how they will demonstrate it.  At that point you can discuss what evidence your partner would be happy to share with you.

Even if your project has started or ended, you can approach your partners to see if they are happy to tell you about the benefit they realised from using the research or being apart of the collaboration.  They may be able to provide  evidence of this.

Advice on approaching external partners for evidence.

The planning for impact toolkit will help to get started to planning to realise impact from your research and has guidance on developing indicators to record successful activity throughout the project.

Event feedback & follow up  

What can it provide? How to get it.
Event attendance and feedback

The attendees information can provide a quantitative and qualitative baseline of your potential beneficiaries and the long term change in their perceptions and behavior.

The public engagement team have developed resources to help evaluate events

If your research was used by a partner organisation in an event or if you took part in an event that was hosted externally you could ask for the feedback they have collected.

If you have presented at a workshop/focus group or event, ask the organiser to include some of your questions in their feedback form.

Follow up feedback Impact may take time to demonstrate so being able to contact participants 6 months or a year later is useful way to find out if any has occurred. 

Here are some tips on asking the right questions and some tools to use for different events.

Awards, Independent Reviews & Media

What can it provide? How to get it.
Awards The award can either be for your research contribution to the impact or an assessment by research users of the impact e.g. product to performance. Keep all records of short listing or awards through organisation press releases, media coverage and correspondence.
Independent reviews Reviews by individuals who are external to the university but considered lay experts can provide an assessment of the significance the research contribution of the activity. Examples of Independent review used to demonstrate significance in REF2014 Impact case studies.
Media & News article News and media articles can demonstrate dissemination and can be used along side other indication of the reach of the potential audiences.

Weblinks, screen shot and dates of publication or broadcast all need to be captured.

Websites such as Statista or BARB can provide readership or viewership numbers.

Social media & online statistics

What can it provide? How to get it.
Downloads/viewing statistics and Social media comments This can be used to show the effectiveness of your engagement and a pathway to the impact you are claiming. This information depends on where share your research e.g. web pages or social media

Tracking your research or project web pages can provide numbers on

- How many people have visited the web page
- The average time spent on the web page
- Who is accessing the web page

Twitter can provide information on the location and demographics of individuals and track downloads and retweets. 

Altmetrics can be used to discover who is talking about your research in the public domain. it covers a wide range of online sources including news outlets and policy documents. However,  it may give an incomplete picture as the link to the journal article or DOI number of the research paper must be used directly in the mention. Therefore, some discussions about the research paper could be missed.

Weblinks to independent sites and visitor traffic can be measured

Google analytics is a tool which lets you track user information on websites.

twitter analytics.

Guide to using altmetrics

If you’re receiving a lot of comments on social media, you may not want to record all of them. You may just want to record a selected few that demonstrate engagement or the public debate that has been effected. Site such as Wakelet can be used for this.

Weblinks can change so you should keep the link and the date it was taken along with a screen shot of the evidnece you are citing.

Testimonials and correspondence
What can it provide? How to get it.
Testimonials

Testimonials can be both qualitative containing evidence of effect or influence of your research and who it influenced and quantitative providing dates, economic figures or numbers of appearances/attendees.

Testimonials should come from an independent, well respected figure who directly mentions the research work and how it has affected them.

Where possible, the testimonial statement should provide quantitative examples of the impact, describing how the research has led to the impact.

Reach out to individuals who are qualified (e.g. senior figures) to evidence the impact the research has made. The most effective way to do start this is via introduction email, explain the purpose of the request. where the email evidence can be saved and referenced at a later date.

Being explicit in asking what evidence you would like them to provide will help you to gather evidence which clearly supports what the impact of the research has been. For example:

  • Exactly how has the research has influenced their organisation
  • Outline of how guidelines/standards were adopted in practice resulting in efficiencies/benefits.
  • What can be done now that could not have been done before
More information on Testimonials
Correspondence

This can provide qualitative evidence of who your research influenced and the significance of this effect.

This could be an email from an organisation requesting you attend a meeting or talk to a group. The email will contain details of the organisation and event and also potentially why they want your expertise or perspective.

It could be from individual or organisation that your research has impacted (directly or indirectly).

It might be in response to a keeping in touch email you sent out to partners that has provided information of the impact your research has had for them.

You should keep emails or correspondence you receive on your impact activities. You will need to be able to review the emails quickly for relevance to the impact you are claiming.

You can do this by making a note of who this individual is and the organisation they belong to.

You can create a timeline of your activities in relation to the impact from invites to talks or meetings.

Create a file for relevant emails e.g. ‘impact emails’ or similar to be accessed at a later date.