Splunk Component
Available as of Camel 2.13
The Splunk component provides access to Splunk, via the Splunk provided client Rest API, allowing you to publish and search for events in Splunk.
Maven users will need to add the following dependency to their pom.xml
for this component:
<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-splunk</artifactId> <version>${camel-version}</version> </dependency>
URI Format
splunk://[endpoint]?[options]
Producer Endpoints
Endpoint | Description |
---|---|
| Streams data to a named index, or the default index if not specified. When using stream mode be aware that Splunk has some internal buffer (about 1MB or so) before events gets to the index. |
| Uses Splunk's Rest API to publish events to a named index, or the default if not specified. |
| Streams data to a TCP port, and requires a open receiver port in Splunk. |
When publishing events the message body should contain a SplunkEvent
. See later.
Example
from("direct:start") .convertBodyTo(SplunkEvent.class) .to("splunk://submit?username=user&password=123&index=myindex&sourceType=someSourceType&source=mySource");
In this example a converter is required to convert to a SplunkEvent
class.
Consumer Endpoints
Endpoint | Description |
---|---|
| Performs normal search and requires a search query in the search option. |
| Performs search based on a search query saved in Splunk and requires the name of the query in the |
Example
from("splunk://normal?delay=5s&username=user&password=123&initEarliestTime=-10s&search=search index=myindex sourcetype=someSourcetype") .to("direct:search-result");
camel-splunk
creates a route exchange per search result with an instance of org.apache.camel.component.splunk.event.SplunkEvent
in the body.
URI Options
Name | Default Value | Context | Description |
---|---|---|---|
|
| Both | Splunk host. |
|
| Both | Splunk port. |
|
| Both | Scheme to use. Can be one of: |
|
| Both | Splunk username. |
|
| Both | Splunk password. |
|
| Both | Splunk server connection timeout, in milliseconds. |
|
| Both | When Can be useful when running in application servers to avoid application server HTTPS handling. |
|
| Both | Camel 2.16: The SSL protocol to use. Can be one of: Note: this option is ignored unless the scheme is |
|
| Producer | Splunk index to write to. |
|
| Producer | Splunk sourcetype argument. |
|
| Producer | Splunk source argument. |
| 0 | Producer | Splunk TCP receiver port when using TCP producer endpoint. |
raw |
| Producer | Camel 2.16.0 : Governs whether the body should be inserted as raw. If |
|
| Consumer | Initial start offset of the first search. Required. |
|
| Consumer | Earliest time of the search time window. |
|
| Consumer | Latest time of the search time window. |
|
| Consumer | A number that indicates the maximum number of entities to return.
This is not the same as maxMessagesPerPoll option, which currently is unsupported. |
|
| Consumer | The Splunk query to run. |
|
| Consumer | The name of the query saved in Splunk to run. |
|
| Consumer | Camel 2.14.0 : Stream exchanges as they are received from Splunk, rather than returning all of them in one batch. This has the benefit of receiving results faster, as well as requiring less memory as exchanges aren't buffered in the component. |
|
| Producer | Camel 2.17: Override the default Splunk event host field. |
Message Body
Splunk operates on data in key/value pairs. The SplunkEvent
class is a placeholder for such data, and should be in the message body for the producer. Likewise it will be returned in the body per search result for the consumer.
From Camel 2.16.0 you can send raw data to Splunk by setting raw=true
on the producer endpoint. This is useful for e.g., json/xml
and other payloads where Splunk has build in support.
Use Cases
Search Twitter for tweets with music and publish events to Splunk
from("twitter://search?type=polling&keywords=music&delay=10&consumerKey=abc&consumerSecret=def&accessToken=hij&accessTokenSecret=xxx") .convertBodyTo(SplunkEvent.class) .to("splunk://submit?username=foo&password=bar&index=camel-tweets&sourceType=twitter&source=music-tweets");
To convert a Tweet to a SplunkEvent
you could use a converter like:
@Converter public class Tweet2SplunkEvent { @Converter public static SplunkEvent convertTweet(Status status) { SplunkEvent data = new SplunkEvent("twitter-message", null); data.addPair("from_user", status.getUser().getScreenName()); data.addPair("in_reply_to", status.getInReplyToScreenName()); data.addPair(SplunkEvent.COMMON_START_TIME, status.getCreatedAt()); data.addPair(SplunkEvent.COMMON_EVENT_ID, status.getId()); data.addPair("text", status.getText()); data.addPair("retweet_count", status.getRetweetCount()); if (status.getPlace() != null) { data.addPair("place_country", status.getPlace().getCountry()); data.addPair("place_name", status.getPlace().getName()); data.addPair("place_street", status.getPlace().getStreetAddress()); } if (status.getGeoLocation() != null) { data.addPair("geo_latitude", status.getGeoLocation().getLatitude()); data.addPair("geo_longitude", status.getGeoLocation().getLongitude()); } return data; } }
Search Splunk for tweets:
from("splunk://normal?username=foo&password=bar&initEarliestTime=-2m&search=search index=camel-tweets sourcetype=twitter") .log("${body}");
Comments
Splunk comes with a variety of options for leveraging machine generated data with pre-built apps for analyzing and displaying this. For example the JMX app. could be used to publish JMX attributes, e.g., route and JVM metrics to Splunk, and displaying this on a dashboard.