Skip to main content

Applying the ‘Wisdom of the crowd’ effect in software development

I’m currently reading Noise: A Flaw in Human Judgment, the latest book by Daniel Kahneman wo also wrote Thinking, Fast and Slow.

I’m only halfway in the book but in one chapter the authors talk about an experiment done by 2 researchers, Edward Vul and Harold Pashler where they gave a person a specific question not once but twice. The hypothesis was that the average of 2 answers was always closer to the truth than each answer independently.

And indeed they were right.

One knows more than one

Turns out that this is related to the wisdom-of-crowds-effect; if you take the average of a number of independent answers from different people it typically leads to a more correct answer.

I never heard about this effect before, but it turns out that I’m applying this principle for a long time based on something I discovered in the Righting Software book by Juval Löwy, the broadband estimation technique.

This technique allows you to estimate the implementation effort for a complex system in a fast and efficient way. The idea is that you bring a large group of people(diversity of the group is key!) together and ask them to estimate the effort to build this system.  By doing this with a large enough(think somewhere between 12 and 30 people) you are applying the wisdom-of-the-crowds effect to produce more accurate results.

UPDATE: I forgot to emphasize something when applying the wisdom-of-crowds-effect. It is really important that the estimations are done independently(!) of each other. If you allow people to align first, you end up with group-thinking and skewed estimations.

Important when applying this technique is that the outliers, values that were at least one standard deviation removed from the average, are not ignored but serve as a way to identify uncertainties. After the reasoning behind these outliers is explained a new estimation round is conducted. This process is repeated until all estimations fall within one standard deviation.

I had some very good results applying this technique but it was only when reading about the wisdom-of-crowds-effect that I started to understand the science behind it.

And now it is time to continue reading as I want to find out how I can further reduce the noise in our estimations…

More information

Noise: A Flaw in Human Judgment – Wikipedia

Wisdom of the crowd – Wikipedia

Righting Software | A Method for System and Project Design by Juval Löwy

Measuring the Crowd Within: Probabilistic Representations Within Individuals (ucsd.edu)

Labels:

Popular posts from this blog

.NET 8–Keyed/Named Services

A feature that a lot of IoC container libraries support but that was missing in the default DI container provided by Microsoft is the support for Keyed or Named Services. This feature allows you to register the same type multiple times using different names, allowing you to resolve a specific instance based on the circumstances. Although there is some controversy if supporting this feature is a good idea or not, it certainly can be handy. To support this feature a new interface IKeyedServiceProvider got introduced in .NET 8 providing 2 new methods on our ServiceProvider instance: object? GetKeyedService(Type serviceType, object? serviceKey); object GetRequiredKeyedService(Type serviceType, object? serviceKey); To use it, we need to register our service using one of the new extension methods: Resolving the service can be done either through the FromKeyedServices attribute: or by injecting the IKeyedServiceProvider interface and calling the GetRequiredKeyedServic...
Read more

Azure DevOps/ GitHub emoji

I’m really bad at remembering emoji’s. So here is cheat sheet with all emoji’s that can be used in tools that support the github emoji markdown markup: All credits go to rcaviers who created this list.
Read more

Kubernetes–Limit your environmental impact

Reducing the carbon footprint and CO2 emission of our (cloud) workloads, is a responsibility of all of us. If you are running a Kubernetes cluster, have a look at Kube-Green . kube-green is a simple Kubernetes operator that automatically shuts down (some of) your pods when you don't need them. A single pod produces about 11 Kg CO2eq per year( here the calculation). Reason enough to give it a try! Installing kube-green in your cluster The easiest way to install the operator in your cluster is through kubectl. We first need to install a cert-manager: kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.5/cert-manager.yaml Remark: Wait a minute before you continue as it can take some time before the cert-manager is up & running inside your cluster. Now we can install the kube-green operator: kubectl apply -f https://github.com/kube-green/kube-green/releases/latest/download/kube-green.yaml Now in the namespace where we want t...
Read more