Why have items not been indexed and have not been sent to the indexing engine (are not listed in the ItemAdditionStatusLog)?

Problem

Archived items have not been indexed.

There are no entries for these items in the ItemAdditionStatusLog SQL table in the individual vault store databases.

Cause

Items that are sent to the indexing engine have an entry created in to the ItemAdditionStatusLog SQL table to keep track of its status. However, items that do not get sent to the indexing engine do not get entries created in the ItemAdditionStatusLog table.

 If you have items that are not indexed and do not have an entry in the ItemAdditionStatusLog table then the reasons why are detailed in the Solution section below.

Solution

1)    Indexing engine too busy
Enterprise Vault will only send data for 30 index volumes (by default) to the indexing engine simultaneously. If this is happening (as shown by the Enterprise Vault performance counter ‘Number of indexing agents’) then the index volume may not have been processed yet.

2)    Indexing engine overloaded (Throttling)
If Enterprise Vault does not receive acknowledgements for items sent to the indexing engine over a reasonable period of time, then Enterprise Vault stops sending items to the indexing engine for the affected index volume for a while. This is to allow the indexing engine time to get up-to-date before sending more data.
article 000082550 details the Dtrace entries that indicate that this has happened.

3)    Items still in the index volume queue (on the indexing engine)
Each index volume has a queue on the indexing engine where items to be indexed are placed (called the offline queue). Before Enterprise Vault starts sending items to be indexed (or updated/deleted) to the index volume it checks the queue on the indexing engine (for the specific index volume) to ensure it is empty. Enterprise Vault does not send data to an index volume unless the ‘offline queue’ is empty.
article 000082550 details the dtrace entries that indicate that this has happened.

4)    Index volume in a ‘read only’ state
If the index volume has been set to a ‘read only’ state then Enterprise Vault will not attempt to add, update or delete any items in that index volume.
Index volumes are set to ‘read only’ for the following reasons:
i)    If an index volume is being verified (this allows the index volume to be searched but not updated)
ii)    If an index volume is being rebuilt (this allows the index volume to be searched but not updated)

There is no easy way to check if an index volume is in ‘read only’ mode but the following SQL query against the EnterpriseVaultDirectory database will return all index volumes that are currently in read only mode:

SELECT ArchiveName, ComputerNameAlternate, IndexRootPath, FirstItemSequenceNumber, HighestItemSequenceNumber, FolderName
FROM view_indexvolumeserver
WHERE readonly = 1
ORDER BY archivename, indexvolumeidentity

However the indexing engine also has a read-only mode that is set internally for an index volume in the following circumstances:
i)    If an index volume is set ‘offline’ (can be done in Index Volumes Browser or via the Indexing API)
ii)    If an index volume’s server or location is placed in backup mode
iii)    If an index volume’s location is moved using the ‘Change Location’ wizard (this causes the index volume to be put into an ‘offline’ state)

5)    Index volume in an ‘offline’ state
If the index volume has been set to an ‘offline’ state then Enterprise Vault will not attempt to add, update or delete any items in that index volume. Also Enterprise Vault will not search an index volume that is in an ‘offline’ state.
To check the status of an index volume, run the Index Volumes Browser:
 

6)    Index volume in a ‘failed’ state
If the index volume has been set to a ‘failed’ state then Enterprise Vault will only attempt to send items for indexing every 6 hours (by default).
To check the status of an index volume, run the Index Volumes Browser:

The frequency that Enterprise Vault will attempt to send items to a failed index volume is set on the Advanced tab of the Enterprise Vault Server Properties:

7)    Backup mode enabled on an index volume’s location or index server
If the location or index server where an index volume exists has backup mode enabled then the index volume will not have any indexed items added, updated or deleted.
You can check the status of the index locations by selecting an Indexing server from the Vault Admin Console (if backup mode is enabled on an index server then all its index locations are placed in backup mode):

There is no easy way to check if an index volume is in ‘backup’ mode but the following SQL query against the EnterpriseVaultDirectory database will return all index volumes that are currently in backup mode:

SELECT ArchiveName, ComputerNameAlternate, IndexRootPath, FirstItemSequenceNumber, HighestItemSequenceNumber, FolderName
FROM view_indexvolumeserver
WHERE backupmode = 1
ORDERY BY archivename, indexvolumeidentity

The ‘foldername’ entry is the same name as the index volume name which can be compared to the Dtrace logs.

8)    Index volume location is closed
If the index volume location is closed then Enterprise Vault will not add any new items to the index volume. Instead a new index volume is created in the Enterprise Vault (SQL) directory database and the location for the new index volume is determined.
The index server that will host this new index volume may take a while to start processing any items for the new index volume (by default up to 1 hour but could take longer on a busy server).

9)    Index volume is full
If the index volume has reached the limit on the number of indexed items then a new index volume is created in the Enterprise Vault (SQL) directory database and the location for the new index volume is determined.
The index server that will host this new index volume may take a while to start processing any items for the new index volume (by default up to 1 hour but could take longer on a busy server).

10)    Index volume has ‘defer indexing’ enabled
FSA archives can have indexing disabled by setting ‘Defer indexing’ on the archive point.
You can check whether this is enabled in two places:
i)    Indexing tab on the (File System) archive properties:

ii)    Indexing tab on the archive point properties:

11)    Index volume is marked for deletion
If an index volume is marked for deletion then no more additions, updates or deletes will be sent to the index volume.
There is no easy way to check if an index volume is marked for deletion but the following SQL query against the EnterpriseVaultDirectory database will return all index volumes that are currently marked for deletion:

SELECT ArchiveName, ComputerNameAlternate, IndexRootPath, FirstItemSequenceNumber, HighestItemSequenceNumber, FolderName
FROM view_indexvolumeserver
WHERE beingdeleted = 1
ORDER BY archivename, indexvolumeidentity

12)    Indexing service is not running
If the indexing service responsible for the index volume is not running then (obviously) the index volume cannot be changed or searched.
You can identify the (non ‘hidden’) index volumes that exist on an index server using the Index Volumes Browser:

Alternatively you can see a complete list of the index volumes (including ‘hidden’ index volumes) and their corresponding index servers using the SQL script below:

SELECT  ArchiveName, ComputerNameAlternate, IndexRootPath, Hidden, FirstItemSequenceNumber, HighestItemSequenceNumber, FolderName
FROM view_indexvolumeserver
ORDER BY computernamealternate, archivename, indexvolumeidentity

The ‘Hidden’ value (of 1) corresponds to index volumes that are being built to replace existing (32 bit and/or 64 bit) index volumes (i.e. rebuilds or updates) but are not searchable yet.
The ‘Foldername’ value corresponds to the index volume name in the indexing engine that can be used to trace activity in the Dtrace logs.