Maintenance: Why the AI Job is Never Done
The ship launched. The engine is running. Cargo is being delivered. You finally sit down in the hangar, put your feet up, and pour a cup of terrible space coffee.
Then the warning lights flash.
Welcome to Maintenance.
The Scenario
When you launched the ship, it knew how to dodge asteroids. But a year later, a new asteroid belt forms. Or the sensor starts gathering cosmic dust, blurring its vision. The engine hasn’t changed—it is still doing exactly what you trained it to do. But the universe has changed.
If you aren’t actively monitoring the engine’s telemetry, you won’t realize the ship is slowly flying off course until it crashes into a moon.
The Reality
In Deep Learning, this is the Monitoring and Maintenance phase.
A machine learning model begins degrading the moment it is deployed. This isn’t software rot; it’s because the real world is constantly shifting. User behavior changes. Sensor data shifts. This is called “Data Drift.”
If you train an AI to predict housing prices in 2019 and let it run untouched until 2024, its predictions will be completely useless, even though the code is perfectly bug-free.
The Why
Deploying an AI is not a one-time project; it is an ongoing subscription to reality. You must build dashboards to track the model’s accuracy, latency, and the quality of the incoming data. When performance drops below a certain threshold, you don’t rewrite the code—you go back, gather the new data, and retrain the model.
The Takeaway
An AI model is only as smart as the data it was trained on. As the world changes, your model becomes stupid again. Monitor it, or prepare to crash.
AI specialists call it: Model Monitoring & Data Drift This is the practice of tracking a deployed model’s performance metrics over time to detect when the statistical properties of the target variable or input data change (concept drift and data drift), requiring a retraining cycle.
💬 What is a process or tool in your work that used to be perfect, but slowly degraded because the “world” around it changed?
Part 10 (Monitor) of 20 | #DLLifecycleForHumans #ai_edu Based on CS230 Stanford lectures