Trialling an evaluation framework for LLM-powered careers advice - Careers Advisors
Participant Information Leaflet for Careers Advisors
|
Study Title: |
Trialling an evaluation framework for LLM-powered careers advice |
|
Investigator(s): |
Peter Dickinson, Deidre Hughes, Chris Percy, and Stef Poole – IER (Institute for Employment Research, University of º£½ÇÉçÇø); Vincent Ramos (University of Southampton). |
A downloadable version of this leaflet is available here.
Introduction
You are invited to take part in a research study. Before you decide, you need to understand why the research is being done and what it would involve for you. Please read the following information carefully. Please contact us if there is anything that is not clear or if you would like more information. Take the time to decide whether or not you wish to take part.
Who is organising and funding the study?
The study is being funded by the Nuffield Foundation and being carried out by researchers from the University of º£½ÇÉçÇø.
What is the study about?
The study is exploring how the advice given by different types of AI chatbots compares to that of careers advisers. It is also developing an evaluation framework that people can use to help them make comparisons.
Do I have to take part?
Your participation in this study is entirely voluntary and you can withdraw at any time without giving a reason.
What would taking part involve?
There are different roles that you might be assigned:
·Providing careers advice to 16-19 year old FE college students;
·Reviewing transcriptions of careers advice given to 16-19 year old FE college students;
·Reviewing the useability, accuracy and relevance of AI careers chatbots, and;
·In the role of a persona developed for the study, asking careers questions of AI careers chatbots.
·You will be invited to share your feedback on the evaluation process and how it might be improved in the future (e.g. via a post-evaluation feedback and review meeting online or survey to gather your views).
What are the possible benefits of taking part in this study?
It will help in the current and future evaluations of AI chatbots and the careers advice they provide.
What are the possible disadvantages, side effects or risks, of taking part?
There are no anticipated risks to you as a research participant.
Expenses and payments
You will paid £275 per day for taking part in the project.
Will my taking part be kept confidential?
Yes. We will follow strict ethical and legal practice and all information about you will be handled in confidence. All the information and data you provide will be pseudonymised, this means your contact details will not be stored in the same file as other data you provide. All data will be stored securely in a password-protected folder at the University of º£½ÇÉçÇø. Only the named researchers will have access to the data. No data will be transferred outside of University of º£½ÇÉçÇø.
What will happen to the data collected about me?
Any information you provide will be kept securely in a password-protected folder on University of º£½ÇÉçÇø servers, only for the purposes of the research. We will only use the data you provide in the ways needed to conduct this evaluation and analyse the data resulting from it.
We will be using information from you in order to undertake this study and University of º£½ÇÉçÇø will act as the data controller for this study. We are committed to protecting the rights of individuals in line with data protection legislation. Your contact details will be destroyed immediately after the end of the project (December 2027). Consent information will be kept securely for 10 years after the study has finished and then destroyed securely.
Research data will be pseudonymised as quickly as possible after data collection. This means all direct and indirect identifiers will be removed from the research data and will be replaced with a participant number. The key to identification will be stored separately and securely to the research data to safeguard your identity. It will be possible for us to withdraw your data up to 3 weeks after the data has been collected and this data will then be securely destroyed if you wish us to do so.
Data Sharing
Data will not be shared with any other organisation.
Your rights to access, change or move your information are limited, as we need to manage your information in specific ways in order for the research to be reliable and accurate.
The University of º£½ÇÉçÇø have in place policies and procedures to keep your data safe.
What will happen if I don’t want to carry on being part of the study?
Participation is entirely voluntary, and a decision to withdraw participation from the study without giving a reason, will not affect you in any way. If you decide to withdraw from the study, please tell us before or during your participation and we will destroy your contact details securely and will not contact you again for the purposes of this research. You can stop being involved in the study at any point without giving a reason.
After providing information you will have three weeks to withdraw your data. This can be done by contacting either of the researchers below. It will not be possible to withdraw your data 3 weeks after data has been collected. To safeguard your rights, we will use minimum personally-identifiable information and keep the data secure.
What will happen to the results of the study?
The findings of the study will be presented in a report to the Nuffield Foundation and made publicly available. All data will be presented in an aggregate form e.g. in tables and graphs. The report will be made available on the Nuffield Foundation’s and University of º£½ÇÉçÇøâ€™s website.
Who should I contact if I want further information?
If you have any questions about any aspect of the study, or your participation in it, not answered by this participant information sheet, please contact:
Peter Dickinson at p.dickinson@warwick.ac.uk
Stef Poole at stefanie.poole@warwick.ac.uk
Who should I contact if I wish to make a complaint?
Any complaint about the way you have been dealt with during the study or any possible harm you might have suffered will be addressed. Please address your complaint to the person below, who is a senior University of º£½ÇÉçÇø official entirely independent of this study:
Head of Research Governance
Research & Impact Services
University House
University of º£½ÇÉçÇø
Coventry
CV4 8UW
Email: researchgovernance@warwick.ac.uk
Tel: 024 76 522733
If you wish to raise a complaint on how we have handled your personal data, you can contact our Data Protection Officer, Information and Data Director who will investigate the matter: infocompliance@warwick.ac.uk
If you are not satisfied with our response or believe we are processing your personal data in a way that is not lawful you can complain to the Information Commissioner’s Office (ICO).
Thank you for taking the time to read this Participant Information Leaflet
PRIVACY NOTICE
How we will ensure that we are using your personal data fairly and transparently in accordance with the requirements of the General Data Protection Regulation (GDPR)
How we will use your personal data
If you agree to participate, we will ask you to confirm your name and email address. We will use the information you provide as a basis for analysis in order to develop the study’s findings.
The nature of your personal data that we will be using
The categories of personal data that we will ask for are:
·Name and contact email address when completing the consent form;
·Where payments for participating in the study can be made.
Why our use of your personal data is lawful
Under the GDPR, we are required to have a lawful basis for processing your personal data. For the purpose of this project, the lawful basis is that processing is necessary to perform a task in the public interest (e.g. providing unbiased research evidence in an article).
Who will have access to your personal data?
Access to your personal data will be restricted to members of the º£½ÇÉçÇø Institute for Employment Research (IER) project team for this research project. We will not report your name or anything that would make you personally identifiable in any outputs from the research and will use our best endeavours to ensure that data in any project outputs (e.g. reports and presentations) will not allow individuals or their organisations to be identified.
For the purposes of reporting, any identifying information will be removed to ensure confidentiality. The information you provide will be stored safely on a password-protected folder at the University of º£½ÇÉçÇø and will only be accessible by the research team. Contact details will not be stored in the same file as the information you provide. IER and its data storage procedures are fully compliant with GDPR. Aggregated research data will be made available (for example, in a published report or article) but they will not have access to any individual information, names, email addresses, telephone numbers and other identifying details of participants. The collated research findings will be published in a research report and a journal article but none of your personal data will be included in the report.
Your right to withdraw consent
Participation in this study is entirely voluntary. Refusal to participate will not affect you in any way. If you do agree to participate, you may withdraw from the study at any time: there will not be any consequence of this. You have the right to withdraw from the study completely and decline any further contact by researchers after you withdraw. However, you will not be able to withdraw any data you provide (apart from contact details) three weeks after your involvement in the study.
How long we will keep your personal data?
We will retain your pseudonymised data for ten years in accordance with the University of º£½ÇÉçÇø retention requirements for research data. Your stored contact data will be deleted as soon as is practicable after the end of the project.
Your data protection rights
Under certain circumstances, you have the right:
·to ask us for access to information about you that we hold;
·to have your personal data rectified, if it is inaccurate or incomplete;
·to request the deletion or removal of personal data where there is no compelling reason for its continued processing;
·to restrict our processing of your personal data (i.e. permitting its storage but no further processing);
·to object to direct marketing (including profiling) and processing for the purposes of scientific/historical research and statistics;
·not to be subject to decisions based purely on automated decision-making.
If you need to contact us regarding any of the above, please email the University of º£½ÇÉçÇøâ€™s Data Protection Officer at infocompliance@warwick.ac.uk. For further details see /services/idc/dataprotection/datasubjectrights
Your right to complain
If you are unhappy with our use of your personal data, please let us know by contacting:
Head of Research Governance
Research & Impact Services
University House
University of º£½ÇÉçÇø
Coventry
CV4 8UW
Email: researchgovernance@warwick.ac.uk
You can also contact the Department for Education’s Data Protection Officer via gov.uk by marking your correspondence as for the attention of the ‘DPO’.
Alternatively, you have the right to raise any concerns with the Information Commissioner’s Office (ICO) via their website at
Thank you for taking the time to read this Participant Information Leaflet