Sitecore database growth: what to clean up and how

Short answerSitecore databases usually grow because the EventQueue, PublishQueue and History tables aren't being cleaned, SQL transaction logs are never backed up, unused media is still stored as blobs, and xDB keeps visitor data forever. Fix the cleanup agent settings, set the right SQL recovery model or log backups, run the built-in database cleanup, and schedule SQL index maintenance.

Database growth is a slow problem. Nothing breaks on the day it starts. Months later, the disk is nearly full, nightly backups run into the morning, publishing is slow and SQL Server uses more CPU than it should. By then the cause is hard to see because several things have grown at once.

Start by measuring

Before deleting anything, find out what's actually large. In SQL Server Management Studio, right-click each database and open Reports > Standard Reports > Disk Usage by Top Tables. Also compare the size of each database's data file with its log file. You'll usually find one or two clear offenders.

Always take a backup before any cleanup, and run large cleanups in a quiet window.

1. EventQueue, PublishQueue and History tables

These tables exist in the master, web and core databases. They record events, publish candidates and item history, and Sitecore has scheduled agents that should trim them. When those agents are misconfigured or keep failing, the tables grow to millions of rows. Large queue tables slow down publishing and the processing of remote events on content delivery servers.

Fix: in /sitecore/admin/showconfig.aspx, check the CleanupEventQueue, CleanupPublishQueue and CleanupHistory agents. Confirm they're enabled, check their intervals and how many days of data they keep, and look in the Sitecore log for errors when they run. A few days of history is usually enough for these tables.

2. SQL transaction logs that never shrink

This is the most common cause of a "suddenly full" disk. If a database uses the Full recovery model, SQL Server keeps every change in the transaction log until a log backup runs. Without log backups, the log file grows forever. It's common to find a 2 GB database with a 60 GB log file.

Fix: agree with your database administrator which model you need. If you need point-in-time restores, schedule regular log backups. If daily full backups are enough, the Simple recovery model may be fine. Once that's sorted, the log file can be shrunk once to a sensible size. Don't schedule repeated shrinking, because it fragments the database.

3. Media stored in the database

By default, Sitecore stores media library files as blobs in the database. Large videos, PDFs and image libraries add up quickly. When media items are deleted, their blobs aren't always removed straight away, so orphaned blobs stay in the Blobs table.

Fix:

  • Run Control Panel > Database > Clean up databases in a quiet window. It removes orphaned blobs and other leftover data.
  • Empty the recycle bin and archive regularly. Deleted items in the recycle bin still take up space.
  • Keep very large files, especially videos, on a video platform or CDN instead of in the media library.

4. Too many item versions

Every save in a workflow can create a new version. Pages edited daily for years can have hundreds of versions in each language. That makes the database bigger and some operations slower.

Fix: agree a version policy, such as keeping the last 10 versions, and use a version-pruning tool or script to remove the rest. Test it carefully on a copy first, and check that no one depends on old versions for compliance reasons.

5. xDB collection and processing data

On Sitecore XP, visitor data goes into the xDB shard databases and the processing pools. By default, much of it is kept indefinitely. On busy sites, xDB is often bigger than all the content databases together.

Fix: decide how long you actually need visitor data and apply Sitecore's documented options for removing old contacts and interactions. If you don't use xDB at all, see when to switch xDB off.

6. No SQL index maintenance

Sitecore's databases do a lot of inserts and deletes, so SQL indexes get fragmented and statistics go out of date. The result is slower queries everywhere: content editing, publishing and page rendering.

Fix: schedule weekly index maintenance and statistics updates. Many teams use the free and widely used Ola Hallengren maintenance scripts. If you're on Azure SQL, check that automatic tuning and maintenance are set up.

A simple monthly routine

  1. Check database and log file sizes, and free disk space.
  2. Check row counts in EventQueue, PublishQueue and History.
  3. Check the Sitecore log for cleanup agent errors.
  4. Confirm backups and SQL maintenance jobs ran successfully.

That takes about 20 minutes and prevents most database emergencies.

What to do next

Run the disk usage report and compare data and log files. That shows you which of these causes applies to you. If you'd like it handled for you, ongoing database care is part of my Sitecore support and maintenance service.

Prabh

Independent Sitecore developer and administrator. I help agencies and businesses keep Sitecore fast, patched and publishing.

Get in touch

Got a Sitecore problem that won't go away?

Tell me what's happening. You'll get a straight answer on whether I can help, usually within one business day.

Book a free 20-minute call