It seems almost sacrilegious to say it, but I think it is really important to maintain significant caution around data obtained from user feedback, testing or group discussions. Too often I hear people proudly say that all their design decisions are based on feedback from users, or one hears leadership asking that all the design decisions be based on documented data from user feedback.

These attitudes are, of course, well-founded, but at the same time they are seriously misguided – and for a number of different reasons …

Who did you get feedback from?

Frequently the data and feedback has come from today’s users, or from preferred customers and rarely has it come from the users that the product is intended for, or where the commercial imperative is coming from.

How many people did you get feedback from?

Frequently the data and feedback that is informing design decisions this month comes from the user feedback obtained this month, which may well have involved only a small fraction of the users being consulted across the whole development lifecycle.

Of course, this month’s feedback may be the only feedback directly addressing a particular feature or user need – but that still leaves one basing decisions on a very small amount of input

How different (or challenging) is your design vision?

Is the vision for the experience with this new release or product rooted in today’s workflows, habits and activities, or are you trying to bring about some innovation in the way tasks are done and reasoned about? If the latter, then user feedback is always liable to pull you back into the non-innovative frame of mind