Evaluation Criteria

PatientGPT

Aspect	Verbatim Item Wording	Rating Scale
Faithful to Role	The patient’s utterances are faithful to and aligned with their role.	Strongly Disagree Disagree Neutral Agree Strongly Agree
Displays Emotions	The patient displays appropriate emotions where relevant.
Authentic	The simulated patient is believable in their role and representative of a true patient.

DoctorGPT

Aspect	Verbatim Item Wording	Rating Scale
Empathy	The doctor displayed appropriate empathy and consideration towards the patient.	Strongly Disagree
		Disagree
		Neutral
		Agree
		Strongly Agree
Communication	Please provide a score for the doctor’s communication skills using the given rubric	Poor (Does not respond to obvious patient cues or responds inappropriately. Communication prevents understanding by patient, or is inappropriate. Frustrates or antagonises the patient.)
		Average (Responds to some patient cues, but not effectively. Patient has to make some effort to understand. Some deficiencies such as passivity, self- consciousness or inappropriate aggressiveness.)
		Good (Responds effectively to patient’s needs and cues. Easily understood by the patient. Puts patient at ease.)
		Excellent (Responds to cues with a perceptive and genuine manner. Effortlessly understood by patient. Puts patient at ease. Excellent command of expression.)
Organisation	Please provide a score for the doctor’s organisational skills using the given rubric.	Poor (No recognisable plan to the interaction; tends towards random questions / process of examination.)
		Average (Organisational approach formulaic and minimally flexible.)
		Good (Well organised with flexible approach.)
		Excellent (Superior organisation, proficient and professional approach.)
Overall Grade	Please provide an overall mark for the doctor using the given rubric.	Clear fail (Responds inappropriately and/or ineffectively to significant aspects or all of the task, deficits in interpersonal or clinical skills. Performance indicates a lack of knowledge and/or undeveloped interpersonal and clinical skills.)
		Borderline (Responds effectively to the majority of the components of the task, some concerns about interpersonal and clinical skills.)
		Clear Pass (Responds effectively to the task, with integration of components. Sound interpersonal and clinical skills.)
		Very good pass (Responds precisely to the task, integrating components. Consistently displays very good interpersonal and clinical skills.)
		Excellent Pass (Responds precisely and perceptively to the task, consistently integrating all components. Outstanding interpersonal and clinical skills.)