Need Some Help with Data Mining? 

Data mining is one of my favorite types of projects when it comes to data in general. The whole goal of data mining is to figure out important information in a dataset. So if you wonder how to get new insights from your data? If you have data just sitting around collecting virtual dust? Or if you have ideas and do not know how to justify them with data? If you want to get a deeper understanding of the data in your business. If you wish to figure patterns, trends, relationships in your data that you may not have seen on your own.

According to a study published on Forbes, 53% of companies are using big data analytics today. Do not get left behind. Your competitors are analyzing their data. You should do the same to gain a deeper understanding of your activity, and leverage that knowledge to get an advantage.  

It sounds all great, but how does data mining look like? 

Data mining project depends on what you are trying to figure out and the industry you are operating in. Here are few examples of applications of data mining taken from projects that I did for clients.

Let us assume that you are in e-commerce, and you receive thousands of orders per month. One useful application can be the discovery of any patterns in what your customers buy. In the picture below, I used a pattern discovery algorithm presented in this paper, to identify sets of items bought together more often with a degree of confidence. I added a seasonal filter to differentiate the differences by season.

I presented this information on a page in an interactive dashboard. That dashboard ended up as a useful tool for the marketing team to design their seasonal campaigns. Consequently, through using those insights, they observed a 40% increase in the average count of opened emails (newsletter). As a result, a higher opening rate led to an 18% increase in conversions from email.  

Another application can be for social media, especially related to leads generation. After collecting Twitter accounts following my client account, I collected all information on who their followed accounts (friends) follow. Since people/businesses tend to follow people that share the same interest, I used that principle to find high-quality leads within my client’s network. Consequently, I managed to generate a network with millions of nodes. I then used community detection algorithms on that network to identify accounts with similar traits (accounts more similar to each other belong to the same color group). The network was finally filtered by how influential a Twitter account was. 

It resulted in a network/tool similar to the one below from which the sales team used for lead generation. Using this technique so far allowed them to get 24% more clients than blind emails and cold calls.

I could go on forever with various examples as to how beneficial data mining can be.

But, how does a data mining project usually go?

Data mining is not the most simple task to do. If it is simple, then you probably would not need me. The process I use to conduct a typical data mining project takes more or less 6 steps.

1.Requirement gatherings and assessment

During this phase, I get the sense of what you want to do, what kind of data you have. Additionally, we will discuss the project requirements. From that meeting, I will think about a plan, and make proposals. That usually takes 1-3 days.

2. Data Collections

Once the project is approved and deadlines/plan set, I will start by collecting the data. Depending on the project, this is the part in which I gather it from various sources.

3. Data Wrangling/Cleaning

Once the data are collected, I will then clean, create features, combine the data from various sources into a more understandable format.

4. Data Analysis

The data analysis phase is where I apply those complex algorithms to discover findings in the large dataset.

5. Evaluation

It is in the evaluation periods where I infer, explain, interpret and present the results of the previous step

6.Integration 

Once the previous step is approved, then I can help you integrate those insights into your existing process. Whether or not integration is required, a report is always written and presented to you.

If you have made it this far … Thank you.

I really value the relationships that I build from this platform and I hope that It can be a valuable resource for you.

I am really good at applying principles in data analysis, data science, data mining, or machine learning for small and mid-sized businesses. So if you want me to help you, do not hesitate to reach out.

Shoot me an email at malick@malicksarr.com if you have any questions. I would love to hear from you.

If you are looking for something to read, you can start here. Here’s my best content.

I have helped a lot of people make sense of their data and make data-driven decisions.