Ch.7: Challenges in Machine Learning
Inspired by: YouTube
In the previous posts, we discussed what machine learning is and explored various ways to categorize algorithms (Supervised vs. Unsupervised, Batch vs. Online, Instance-Based vs. Model-Based). As we move closer to writing actual code and building models, it is crucial to understand what goes wrong in the real world.
Building models in a controlled college project is very different from building them in a corporate environment. Today, we will discuss the 10 most common challenges you will face in machine learning projects.
Data-Related Challenges
Machine learning is entirely reliant on data. If your data is flawed, your model will fail. Most of the early challenges you will face revolve entirely around gathering and cleaning data.
1. Data Collection
When learning from tutorials, you are often handed clean CSV files on a silver platter. In a real company, data is rarely handed to you. Gathering data is an incredibly difficult process that often requires writing custom web scrapers, integrating with messy APIs, or waiting months for data engineering teams to build pipelines.
2. Insufficient Data (The Unreasonable Effectiveness of Data)
A famous research paper highlighted a fascinating phenomenon: if you have a mediocre algorithm but feed it a massive amount of data, it will almost always outperform a highly advanced algorithm (like deep learning) that has very little data.
This is known as the Unreasonable Effectiveness of Data. The challenge is that vast, enterprise-scale data is often unavailable for smaller projects, forcing you to rely on better algorithms to compensate for insufficient data.
3. Non-Representative Data (Sampling Bias and Noise)
Imagine you are building a model to predict who will win the T20 Cricket World Cup, and you gather your data by only surveying people living in India. Naturally, the vast majority will say "India". If you train your model on this data, it will conclude that India has a 99% chance of winning.
This is called Sampling Bias. Even if you survey millions of people, if your sample does not accurately represent the entire global population of cricket fans, your model's predictions will be fundamentally flawed. You must ensure your dataset is a true representation of the real-world problem you are trying to solve.
4. Poor Quality Data
Real-world data is extremely messy. It contains missing values, extreme outliers, and completely erroneous entries. In the industry, a machine learning engineer often spends 60% to 80% of their total project time just cleaning data. If you feed garbage data into a model, you will get garbage predictions out.
5. Irrelevant Features
A "feature" is simply a column in your dataset. Sometimes, features do not contribute anything to the final prediction.
For example, if you are predicting whether someone will participate in a marathon, you might look at their age, weight, and heart rate. However, if you also have a column for their "Location" (e.g., North India vs. South India), this feature likely has no correlation with fitness levels. Including irrelevant features confuses the model. The process of combining, modifying, or dropping features to improve your model is called Feature Engineering.
Algorithm-Related Challenges
Even if your data is pristine, the way your model learns can still cause massive failures.
6. Overfitting
Overfitting occurs when your model literally memorizes the training data instead of learning the underlying concepts.
Imagine visiting a city for the first time, going to one expensive movie theater, and immediately concluding that everything in that city is overpriced. You took a very small, specific data point and wrongly generalized it.
Models do this too. If a model tries to perfectly touch every single point in the training data, it creates an overly complex, chaotic decision boundary. While it will perform flawlessly on the training data, it will fail miserably when it encounters brand new data in production.
7. Underfitting
Underfitting is the exact opposite of overfitting. This happens when your model is too simplistic and fails to capture even the basic patterns in the training data. If your model gets a terrible accuracy score on the very data it trained on, it is underfitting.
Production and Engineering Challenges
Once your model is trained and performing well locally, the real engineering challenges begin.
8. Software Integration
A machine learning model is useless on its own; it must be integrated into a larger software product (like an Android app, a web dashboard, or a hardware device) to serve users.
Different platforms use different languages (Java, JavaScript, C++). Ensuring that a Python-based machine learning model runs smoothly, securely, and quickly inside a Java application is a massive software engineering hurdle. Sometimes, compatibility issues force engineers to rewrite entire models.
9. Offline Learning and Deployment
Deploying models to live servers is a complex task. While massive cloud providers like AWS, Google Cloud, and Azure provide tools to make this easier, the deployment landscape is still in its infancy. Handling the infrastructure required to host a model, ensuring it can handle thousands of real-time requests, and monitoring it for crashes requires dedicated expertise.
10. The Hidden Costs
When you build a model on your laptop, it costs nothing. When you deploy a computationally heavy machine learning model to a live server and serve 100,000 active users, your server costs can skyrocket overnight. Many models never make it to production simply because the business cannot afford the hidden computational costs required to keep them running.
This specific challenge has given rise to a massive new field in the software industry called MLOps (Machine Learning Operations). Just like DevOps handles traditional software deployment, MLOps engineers specialize in deploying, optimizing, and monitoring machine learning models cost-effectively.