How may times has this happened to you – You customers tell you everything is important and the world will fall apart if they cannot get their hands on all the features/tools they’ve requested. We all know that, in reality they probably have a few things that are “really really important” and then the rest are “really important” — The question is how do you differentiate between “really important” and “really really important.”
I was recently chatting up with Santosh Balasubramanian, a Technical Program Manager at Microsoft Research and incidentally a very close friend of mine — we went to school together at Brigham Young University. This was his basic issue:
They had a list of features that need to be implemented and wanted to find out how to deliver most “bang for the buck” — They all knew that all the features could not be implemented in the given timeframe. The idea was to find out a good way of quantifying and measuring the impact of each of the features, so that they can accommodate as many diverse users as possible.
The crux of the issue was to develop a quick model so that they could accommodate and please as many customers as possible given their limited resources. I think this is a fairly common problem and it surfaces in a few different forms.
I thought about this for some time, and initially I thought that a “Constant Sum” question would be the right and effective solution here Simply ask users to distribute 100 points over all the features they want and make sure that they only have 100 points to distribute. While the “Constant Sum” question can achieve the objective, the cognitive stress that such a question type puts on the end-user is fairly high.
I was on the lookout for a simple model. Another way of solving the same issue with very limited cognitive impact is simply ask the end users to “Pick The Top 3” features from a list of features that users want. The fact of the matter is, this effectively will capture the same psychological behavior as “Distribute 100 Points” — because we really do not think with such granularity. The same logic that applies when you go shopping for a new product/service – we basically boil our options to three or four and then make a decision.
Now, that we have the end-user picking the “Top 3” items, the next part is the determination of the “maximum” impact of the choices. It occurred to me that we can easily perform TURF Analysis instead of regular frequency analysis on the data-set and find the Top 3 Combinations that will give us the maximum reach. With TURF (Total Unduplicated Reach and Frequency) analysis, we can find out which 3 options will “affect” the most.
I think that a simple model can be an alternative to more complicated models like Conjoint Analysis when executing the complicated models are not worth the time and effort required.
