The Medical Device Directive MDD explicitly calls that manufacturers must identify and control risks that arise from a specific context of use and by the characteristics of the user (for example level of education, intellectual and linguistic skills). The FDA does the same.

In order to demonstrate that these requirements are satisfied, it is necessary to validate usability. The IEC 62366 and the Human Factors Engineering Guidance Document of the FDA provide guidance on how this usability validation must be carried out.

1. Characterisation of the user

Thus, the usability validation must be carried out with typical representatives of the specified user group, usually in the form of participant observation. This test of course, assumes that you have specified:

• The user group, for example, based on age, education, experience with the product, physical and intellectual abilities.
• The context of use, such as "mental workload", the environment (temperature, humidity, brightness, ...) the tasks that are to be carried out and many more
• User goals, the (medical) purpose is usually referenced here.

That users and the usage environment must be specified, is a requirement of the IEC 62366 in Section 5.1.

2. Involvement of representative users in the usability validation

Only for the usability validation will you need necessarily representative users and representative context of use (usage environment).

3. Number of subjects in the usability validation

The IEC 62366 demands representative users in the validation of serviceability. How many of these there are, however, it does not say. This is hardly doable, the number also depends on how homogeneous the different user groups are.

The FDA, however, references in its Guidance Document "Applying Human Factors and Usability Engineering to Optimise Medical Device Design" a study that mentions more concrete numbers.

Usability verification: inspection

The verification of the suitability for use will be successful with an inspection. This is a verification procedure whereby one or more experts examine your products on

• whether the specified requirements, such as those found formulated in style guides and standards are implemented and / or
• whether the product (in principle) is able to meet the usage requirements. If the usage requirement specifies that the physician must identify the patients with a low hemoglobin, and the system does not display the hemoglobin value, then this requirement is not met.

There are several methods for verification, such as the Cognitive Walkthroughs.

Usability Validation: Participatory observation

However, the inspection alone is not enough. You have to run through the core tasks within a participatory observation of genuine users in an actual or simulated user environment and let the security functions run. Only when users are actually able to achieve the targets, and thus prove that the user requirements are met, will the usability of your product be validated.

This assumes that you know all the core tasks. This will lead you to the frequently used functions. Next you need to know all the safety-critical functions. These result from the risk analysis. 

Usability verification: inspection

The verification of the suitability for use will be successful with an inspection. This is a verification procedure whereby one or more experts examine your products on

• whether the specified requirements, such as those found formulated in style guides and standards are implemented and / or
• whether the product (in principle) is able to meet the usage requirements. If the usage requirement specifies that the physician must identify the patients with a low hemoglobin, and the system does not display the hemoglobin value, then this requirement is not met.

There are several methods for verification, such as the Cognitive Walkthroughs.

Usability Validation: Participatory observation

However, the inspection alone is not enough. You have to run through the core tasks within a participatory observation of genuine users in an actual or simulated user environment and let the security functions run. Only when users are actually able to achieve the targets, and thus prove that the user requirements are met, will the usability of your product be validated.

This assumes that you know all the core tasks. This will lead you to the frequently used functions. Next you need to know all the safety-critical functions. These result from the risk analysis. 

Actually, the two pairs of concepts have little to do with each other - apart from the fact that they are all related to the testing of usability. The two pairs of terms refer rather to different dimensions: usability validation and usability verification differ in the objective of the test. The terms formative and summative evaluation differ in the time of the test.

For example, examining a GUI prototype during development would be a validation and a formative evaluation. A check of serviceability of the finished product with representative users in a representative environment of use would also be a validation, but a summative evaluation.

"Classic" system / product validation versus usability validation

You will find the difference between these processes described in another article.

Usability validation versus clinical trials

I came across an article in the "European Medical Device Technology" on serviceability. In it, the author points to a paragraph of IEC 60601-1-6, which strikes me as very important:

It is exactly like that! In validating the suitability for use, it is believed that it is generally possible to meet the medical purpose with the products. That it is assumed that the clinical trial was successful.

On the other hand, the validation of serviceability checks if, the representatives of a specified user group in a specific usage context actually achieve this specified result. Because the fact that the device is generally able to do this, does not allow conclusions about the interaction of the user with the product.

In other words:

• The usability study has to validate towards the goal, if the specified user in the specified context of use can effectively and efficiently achieve the objectives. A prerequisite for the validation of usability are representative users.
• Clinical studies, however, have to prove the goal that the product has a positive benefit-risk ratio. Patients are prerequisite for a clinical trial.

Although both studies are possibly carried out in the clinic "in real terms", the objectives are nevertheless different, though not overlapping. It is often useful to separate the tests of one another.

Usability validation versus field test

One of my favourite clients had the idea of validating his software in a field test, in order to meet the relevant legal requirements for validation. He almost committed an offense.

His thought to test, whether the purpose and usage requirements are met  in the field was really good. At the end of the day, the IEC 62366 demands that one involves "real" users . But to test the application in a field test in an uncontrolled way, corresponds to placing it on the market in an uncontrolled way. That is a criminal.

"Usability, Requirements & IEC 62366" Seminar

How to plan and execute such a validation (in the form of participant observation) you will learn in the seminar "Usability, requirements and IEC 62366" with my usability guru Thomas Geis. It will change your view of your own products radically. You will learn to understand why

• there is always friction between your development department and product marketing,
• Your customers seem to constantly have new desires,
• Your company is doing harder than Apple,
• You are not yet a millionaire.

However, you’ll not only learn to understand the causes, but above all, know the way to precisely overcome these difficulties. For me the seminar was and is groundbreaking.

If you want to be there, I suggest you  secure an appointment for next year. I could only secure Thomas Geis three times to come to Konstanz. Take advantage of this rare opportunity.