Hot Threads APIedit

Warning

This functionality is experimental and may be changed or removed completely in a future release.

The hot threads API gets the current hot threads for Logstash. A hot thread is a Java thread that has high CPU usage and executes for a longer than normal period of time.

GET /_node/hot_threads

The output is a JSON document that contains a breakdown of the top hot threads for Logstash.

Example response:

{
  "hot_threads": {
    "time": "2016-09-19T10:44:13-07:00",
    "busiest_threads": 3,
    "threads": [
      {
        "name": "LogStash::Runner",
        "percent_of_cpu_time": 0.17,
        "state": "timed_waiting",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "java.lang.Thread.join(Thread.java:1253)",
          "org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
          "org.jruby.RubyThread.join(RubyThread.java:697)",
          "org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
          "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)",
          "org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)"
        ]
      },
      {
        "name": "Ruby-0-Thread-17",
        "percent_of_cpu_time": 0.11,
        "state": "timed_waiting",
        "path": "/Users/username/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:471",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "org.jruby.RubyThread.sleep(RubyThread.java:1002)",
          "org.jruby.RubyKernel.sleep(RubyKernel.java:803)",
          "org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
          "org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)"
        ]
      },
      {
        "name": "[main]-pipeline-manager",
        "percent_of_cpu_time": 0.04,
        "state": "timed_waiting",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "java.lang.Thread.join(Thread.java:1253)",
          "org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
          "org.jruby.RubyThread.join(RubyThread.java:697)",
          "org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)",
          "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)",
          "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)"
        ]
      }
    ]
  }
}

The parameters allowed are:

threads

The number of hot threads to return. The default is 3.

human

If true, returns plain text instead of JSON format. The default is false.

ignore_idle_threads

If true, does not return idle threads. The default is true.

You can use the ?human parameter to return the document in a human-readable format.

GET /_node/hot_threads?human=true

Example of a human-readable response:

::: {}
Hot threads at 2016-07-26T18:46:18-07:00, busiestThreads=3:
================================================================================
 0.15 % of cpu usage by timed_waiting thread named 'LogStash::Runner'
        java.lang.Object.wait(Native Method)
        java.lang.Thread.join(Thread.java:1253)
        org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
        org.jruby.RubyThread.join(RubyThread.java:697)
        org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
        org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
        org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
        org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
 --------------------------------------------------------------------------------
 0.11 % of cpu usage by timed_waiting thread named 'Ruby-0-Thread-17'
 /Users/username/BuildTesting/logstash-5.0.0logstash-core/lib/logstash/pipeline.rb:471
        java.lang.Object.wait(Native Method)
        org.jruby.RubyThread.sleep(RubyThread.java:1002)
        org.jruby.RubyKernel.sleep(RubyKernel.java:803)
        org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
        org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
        rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)
        rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)
        org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)
 --------------------------------------------------------------------------------
 0.04 % of cpu usage by timed_waiting thread named '[main]-pipeline-manager'
        java.lang.Object.wait(Native Method)
        java.lang.Thread.join(Thread.java:1253)
        org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
        org.jruby.RubyThread.join(RubyThread.java:697)
        org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)
        org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)
        org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)

See Common Options for a list of options that can be applied to all Logstash monitoring APIs.