AI YOUTH COUNCIL
AI YOUTH COUNCIL
AI Scribes: What Kids Think!
Written by Nidhi Patel, AI Youth Council Secretary
An AI scribe (sometimes also called ‘Ambient Voice Technology’ (AVT for short)) can sound like a really scary term! But it’s really not as complicated as it sounds. I briefly mentioned it in my blog about the AI Youth Council launch event blog (check it out here), but in case you missed, it I’ll share a short explanation here first before I tell you want our Youth Council thought.
When someone visits their family doctor (the GP) or a specialist doctor or nurse, we need to make sure the notes are correct and really explain what people said so the next person who reads them can understand what happened. It takes a lot of time to do this well, and when there are so many people in the clinic one of two things can happen – either everyone’s appointment runs late, or the notes are just not written as well. To stop this problem being worse, a new software called an AI scribe is being used in many places. This software takes recorded speech and turns it into text, and then uses AI to summarise the conversation. It organises the conversation into subheadings to make it easier to read next time. This means the next doctor or nurse can save time having to type out or write out the entire conversation with the patient, hopefully everything is recorded better, and doctors can spend more time getting to know the patient instead of looking at the computer screen.
What we did at the AI Youth Council event
At the Launch Day event, I mentioned how Maiike and Alastair did a live demonstration of TORTUS (one brand of AI scribe). They role played a funny scenario of someone who ate something rotten at a recent party and now had a tummy ache. It was very funny as we all got the chance to come up with funny foods and symptoms the patient might have had, and checked to see how well the AI scribe would record the information. This really helped all the children and young people understand how this would actually work in the clinic if they had not seen it before.
After this, we all got into groups to share our opinions on what excited and concerned us about using this software. Many children spoke about consent (asking for permission), privacy (making sure our information wasn’t available to everyone), appropriateness (should this be done and used for everyone), and accuracy (did it really write down everything correctly?). We also thought about possible ways that could help us worry less about these issues.
What do children and young people think?
Both age groups that discussed this topic (9-11-year-olds and 12-15-year-olds) felt that children should have a say on whether AI scribes are used in their care. They also recognized that there’s a chance their personal information could be exposed if their clinician uses an AI scribe. When thinking about privacy, the 9-11-year-old age group focused on how information shared digitally can never truly be erased, and talked about how clinicians should keep patients’ information private. On the other hand, the 12-15-year-old age group thought about privacy in terms of how much access AI scribes should have to medical records.
Both groups had similar discussions about making sure AI scribes take accurate notes and are only used when appropriate. They spoke about the importance of checking for mistakes when using AI scribes and how they aren’t suitable for every single consultation.
Difference of opinion across different ages
Overall, there was more agreement in the 9-11-year-old age group compared to the 12-15-year-old age group. The 9-11-year-old group mainly shared their thoughts on the different issues the staff brought up, while the 12-15-year-old age group spent more time coming up with possible solutions. Since they had different ideas about how to address these concerns, they didn’t always agree with each other. For example, they had different opinions on how much access AI scribes should have to their medical records.
One member of the 9-11-year-old age group also spoke about their personal experience seeing their doctor use an AI scribe before in one of their consultations.
One of the main issues the 9-11-year-old age group talked about was the importance of informed consent when AI scribes are used in consultations.
Specifically, one member said:
“[A] Child should be able to say no to recording too, not just mum or dad.”
Four group members agreed with this. Another member added that it’s important for young people to be able to choose whether an AI scribe is used during their consultation. One member suggested that they’d like notification that an AI scribe is being used before the consultation starts, either by an audible message or by clear and colourful posters in the waiting area.
This group also wondered whether their consultation data would be permanently stored online. One of the younger members of this group shared that their “mum says [a message on chat or computer] will be there forever, so you have to be careful.” Many group members knew that their data would ideally be handled carefully, but they didn’t know exactly how private their data would be. Who gets involved with their data? Is the data only for the doctor or clinician? These were some of the questions the group brought up when discussing privacy.
Some group members also questioned how clinicians could make sure their notes stay accurate when they use AI scribes. One member pointed out how important it is to check for mistakes when using AI scribes, saying that clinicians shouldn’t rely on them alone to make complete and accurate notes. Another member suggested that a clinician could take a few notes during the consultation and compare them with the AI-generated notes afterwards to check for and correct any mistakes.
The 9-11-year-old age group talked about the settings where AI scribes are and aren’t appropriate to use. For example, one group member shared that they wouldn’t feel comfortable with and AI scribe being used in a school or social work setting, since they often invite sensitive and personal conversations.
In addition to discussing consent, privacy, accuracy, and appropriateness of AI scribe use, the 12-15-year-old age group also talked about how AI scribes could improve the clinician-patient relationship. One group member wrote, “We want doctors to look at me during my consultation.” Five group members agreed with this, saying that AI scribes could give their clinician more time to talk to them by reducing their workload.
Like the younger age group, the 12-15-year-old age group also thought about how important informed consent is when using AI scribes. Five group members agreed that “the doctor should always ask consent.” One group member thought that when sensitive info is shared in a consultation, the clinician should ask the young person if they want the AI scribe to stop recording. Another group member felt that clinicians should have the freedom to choose whether they want to use an AI scribe in their practice.
When the 12–15-year-old age group talked about accuracy, the group came up with some possible solutions to make AI scribe notes more accurate. One group member suggested that it should be one person’s job to check the AI scribe-generated transcript and notes at the end of the consultation. Another group member added to this and thought that a notification system alerting the clinician to review the AI scribe’s notes would be helpful.
The group also thought about the kind of situations that AI scribes are appropriate to use in. One group member said that “TORTUS shouldn’t be used everywhere. It may not be appropriate in very sick patients.” In a case where someone’s really sick, it’s best that they can speak honestly and openly to their clinician without having to worry about their conversation being recorded. Moreover, a young person who’s very sick might not have the ability to consent to having an AI scribe be used in their care.
In this group, there was some debate around how much access AI scribes should have to a person’s medical records. Seven group members agreed that AI scribes should have access to all medical records, since it can help it to generate accurate notes. One group member said that AI scribes should only have access to certain parts of a person’s medical record that the clinician chooses. They didn’t fully trust that their data would be private once an AI scribe had access to it, so they thought that limiting its data access could be useful in case there’s a data breach (where someone gets access to the data to misuse it).
Closing remarks
One of the main ideas both groups agreed on was that informed consent is incredibly important when a clinician considers using an AI scribe in a young person’s consultation. Young people know that risks can come with using this technology, and they want to be made aware of the guidelines in place to ensure that the technology is as safe as possible. Overall, they want AI scribes to be used in a way that works for them and that strengthens the relationship they have with their clinician.
