Title: Aggregation Framework

The Aggregation Framework in MongoDB is a powerful tool for data processing and analysis. It allows you to perform complex queries that involve data transformation, filtering, and grouping. In this chapter, we'll dive deep into the fundamentals of the Aggregation Framework, explore its pipeline stages, commonly used operators, and practical use cases. By the end of this chapter, you'll be able to craft sophisticated queries that efficiently process large datasets, meeting the expectations of high-performance applications.

A. Introduction to Aggregation

Notes:

Aggregation: Aggregation in MongoDB is the process of computing results from a set of documents. It allows you to group values, calculate averages, and perform operations on data to return computed results rather than raw data.
Use Cases:
- Data summarization (e.g., total sales, average rating).
- Data transformation (e.g., converting field formats).
- Data filtering and sorting in complex ways.
Aggregation Pipeline:
- The Aggregation Framework operates using a pipeline model where documents pass through a series of stages. Each stage transforms the documents in some way.

Example:

A simple aggregation might compute the total sales per product in an e-commerce database.

B. Aggregation Pipeline Stages

Notes:

Aggregation Pipeline:
- A sequence of stages where each stage processes the input and passes the result to the next stage.
- Each stage can filter, transform, or group documents.

Common Stages:

$match:
- Filters documents by a specific condition, similar to the find() method.
- Example: Filtering orders placed in the last month.
```
db.orders.aggregate([
  { $match: { orderDate: { $gte: ISODate("2024-08-01") } } },
]);
```
$group:
- Groups documents by a specified key and applies aggregation functions (e.g., sum, avg).
- Example: Grouping sales by product category.
```
db.orders.aggregate([
  { $group: { _id: "$category", totalSales: { $sum: "$amount" } } },
]);
```
$sort:
- Sorts the documents by a specified field.
- Example: Sorting products by the number of sales.
```
db.products.aggregate([{ $sort: { totalSales: -1 } }]);
```
$project:
- Reshapes each document by including, excluding, or adding new fields.
- Example: Projecting only the product name and total sales.
```
db.orders.aggregate([
  { $project: { productName: 1, totalSales: 1, _id: 0 } },
]);
```
$limit and $skip:
- Limits the number of documents to return and skips a specified number.
- Useful for pagination in aggregation results.
```
db.orders.aggregate([
  { $sort: { orderDate: -1 } },
  { $skip: 10 },
  { $limit: 5 },
]);
```

C. Common Aggregation Operators

Notes:

Arithmetic Operators:
- $sum: Adds values together.
- $avg: Calculates the average of values.
- $min/$max: Finds the minimum/maximum value.
Array Operators:
- $push: Adds a value to an array in the output document.
- $addToSet: Adds unique values to an array (no duplicates).
Conditional Operators:
- $cond: Implements if-else logic in aggregation.
Date Operators:
- $year, $month, $dayOfMonth: Extracts components of dates.

Example:

Grouping orders by year and calculating the total sales per year:

db.orders.aggregate([
  {
    $group: {
      _id: { year: { $year: "$orderDate" } },
      totalSales: { $sum: "$amount" },
    },
  },
]);

D. Using $match, $group, $sort, $project

Notes:

These operators form the core building blocks for most aggregation queries:
- $match: Filters documents early in the pipeline, improving performance.
- $group: Aggregates data by key, applying functions like sum and avg.
- $sort: Orders documents by one or more fields.
- $project: Shapes the output, allowing the inclusion or exclusion of fields.

Example:

Find the top 5 products by sales in the last year:

db.orders.aggregate([
  { $match: { orderDate: { $gte: ISODate("2023-01-01") } } },
  { $group: { _id: "$productName", totalSales: { $sum: "$amount" } } },
  { $sort: { totalSales: -1 } },
  { $project: { productName: 1, totalSales: 1, _id: 0 } },
  { $limit: 5 },
]);

E. Aggregation Use Cases and Examples

Notes:

Data Summarization:
- Summarize user activities, sales data, or logs.
Transforming Data:
- Modify or combine fields, or create new computed fields.
Complex Filtering:
- Combine multiple conditions and operations to filter documents in advanced ways.

Example:

Calculate the average order amount for each customer who has made more than 5 orders:

db.orders.aggregate([
  {
    $group: {
      _id: "$customerId",
      totalOrders: { $sum: 1 },
      avgOrderAmount: { $avg: "$amount" },
    },
  },
  { $match: { totalOrders: { $gt: 5 } } },
  { $project: { customerId: "$_id", avgOrderAmount: 1, _id: 0 } },
]);

Conclusion

The Aggregation Framework is an essential tool in MongoDB for performing complex data analysis and processing directly within the database. Mastering its use can significantly reduce the need for additional application logic, leading to more efficient and maintainable code. This chapter covered the basics of aggregation, including key pipeline stages and operators, and provided practical examples that can be applied to real-world scenarios. As you progress, the ability to craft and optimize aggregation queries will be a valuable skill in building high-performance, data-driven applications.