The big data and small data of skills
We all talk about big data these days. Never before have we had access to so much data from so many sources. In TeamFit’s world, we are bringing in data from many, many different systems and then blending this with social interactions to get an accurate picture of the skill graph.
There are so many possible sources of data it is easier to show them than to list them.
With so much data, we need a simple scalable model of how all the pieces fit together. For us, that means the skill graph.
The skill graph has two dimensions. At one level it connects skills and people to projects. As it is a graph, we can move from a skill to a person, to another person who shares the skill, to a project where the skill was used, to other people who were on the project, and on to other skills. This is the first time anyone has taken a deep look at how skills, people and projects fit together.
One result is our approach is that we collect a lot of information about skills. So we have to look at how skills are related to one and other. We have been developing a simple scalable model to organize skills (for people who have been following our work; we recently moved from the notion of ‘parent’ and ‘child’ skills to an inference-based approach of ‘implies’ and ‘implied by’).
This let’s us do some important things.
Understand how skills fit together to deliver successful projects.
Suggest what people will work together effectively.
Recommend what skills people should develop to get on the projects where they can have the most impact.
A pure big data view will not get us where we need to go. TeamFit needs to provide insights at the individual level. This is a small data problem.
What is ‘small data’?
From Wikipedia “Small data is data that is small enough size for human comprehension” … “big data” is about machines and “small data” is about people.”
TeamFit is a mashup of big data and small data. Big data from the digital breadcrumbs we leave across the web and our enterprise and project management systems. Small data from our interactions with each other, the skills we claim or suggest, our conversations, and how we rate each other.
It is the mashup that gives TeamFit its power. On its own, big data has little to say about any particular project or individual. Small data by itself is not power enough to detect patterns that emerge after connecting 1,000 so skills and projects. By bringing them together, TeamFit will give individuals new insights into their own skills while helping organizations to put together teams that make a difference.
The top image from NASA/Chandra Observatory.
