Pending Tasksedit
There are certain tasks that only the master can perform, such as creating a new index or moving shards around the cluster. Since a cluster can have only one master, only one node can ever process cluster-level metadata changes. For 99.9999% of the time, this is never a problem. The queue of metadata changes remains essentially zero.
In some rare clusters, the number of metadata changes occurs faster than the master can process them. This leads to a buildup of pending actions that are queued.
The pending-tasks
API will show you what (if any) cluster-level metadata changes
are pending in the queue:
GET _cluster/pending_tasks
Usually, the response will look like this:
{ "tasks": [] }
This means there are no pending tasks. If you have one of the rare clusters that bottlenecks on the master node, your pending task list may look like this:
{ "tasks": [ { "insert_order": 101, "priority": "URGENT", "source": "create-index [foo_9], cause [api]", "time_in_queue_millis": 86, "time_in_queue": "86ms" }, { "insert_order": 46, "priority": "HIGH", "source": "shard-started ([foo_2][1], node[tMTocMvQQgGCkj7QDHl3OA], [P], s[INITIALIZING]), reason [after recovery from gateway]", "time_in_queue_millis": 842, "time_in_queue": "842ms" }, { "insert_order": 45, "priority": "HIGH", "source": "shard-started ([foo_2][0], node[tMTocMvQQgGCkj7QDHl3OA], [P], s[INITIALIZING]), reason [after recovery from gateway]", "time_in_queue_millis": 858, "time_in_queue": "858ms" } ] }
You can see that tasks are assigned a priority (URGENT
is processed before HIGH
,
for example), the order it was inserted, how long the action has been queued and
what the action is trying to perform. In the preceding list, there is a create-index
action and two shard-started
actions pending.