Basic Layers of AI Systems

We’ve had a lot of time to get used to lots of architectural approaches for building applications. In the AI world, there are a few as well. The most basic one, seems to be:

• Application Development Layer

  • AI Interface

  • Prompt engineering

  • Context construction

  • Evaluation

• Model Development Layer

  • Inference optimization

  • Dataset engineering

  • Modeling & training

  • Evaluation

• Infrastructure Layer

  • Compute management

  • Data management

  • Serving

  • Monitoring

---

Sources:

• AI Engineering by Chip Huyen (O’Reilly). Copyright 2025 Developer Experience Advisory LLC, 978-1-098-16630-4