A method of testing a system that does not exist. Research experiment in which subjects interact with a computer system, which is being operated or partially operated by an unseen human.
Tip's & Tricks
Test future technologies without building an expensive prototype. For example, you can test voice assistants.
Accept rapid iterations with minor changes in wording or call flow — they are immediately testable.
Your system can be evaluated at an early stage in the design process.
Colleagues who play the wizard can learn a lot about user experience.
You can predict usability problems.
Being a wizard in this test is hard — it requires significant training.
It is difficult for wizards to provide consistent responses across all sessions.
There is a risk that they will improvise beyond the programmed behavior.
It is difficult to evaluate systems with a large graphical interface element.
Computers respond differently than humans.
Playing wizard for a long test is hard due to cognitive fatigue.
You can’t define performance or recognition problems.
Historical Tips. How Is It Made?
J. F. Kelley coined the terms "Wizard of Oz" and "Oz Paradigm" when he was working on his dissertation in the early 1980s. The phrase “Wizard of Oz” has come into common usage to describe a testing or iterative design methodology. An experimenter (the “wizard”), in a laboratory setting, simulates the behavior of a theoretical intelligent computer application (often by going into another room and intercepting all communications between participant and system).
A test participant may think he is communicating with a computer using a speech interface, when the participant's words are actually being entered into the computer by a person in another room (the “wizard”) and processed as a text or an audio stream.
Wizard of Oz testing is particularly useful for testing AI-based systems before you have implemented the artificial intelligence. The human who controls the computer can simulate the AI responses based on natural intelligence