THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Wiki Markup |
|---|
h2. Camel MongoDB component
*Available as of Camel 2.10*
According to Wikipedia: "NoSQL is a movement promoting a loosely defined class of non-relational data stores that break with a long history of relational databases and ACID guarantees." NoSQL solutions have grown in popularity in the last few years, and major extremely-used sites and services such as Facebook, LinkedIn, Twitter, etc. are known to use them extensively to achieve scalability and agility.
Basically, NoSQL solutions differ from traditional RDBMS (Relational Database Management Systems) in that they don't use SQL as their query language and generally don't offer ACID-like transactional behaviour nor relational data. Instead, they are designed around the concept of flexible data structures and schemas (meaning that the traditional concept of a database table with a fixed schema is dropped), extreme scalability on commodity hardware and blazing-fast processing.
MongoDB is a very popular NoSQL solution and the camel-mongodb component integrates Camel with MongoDB allowing you to interact with MongoDB collections both as a producer (performing operations on the collection) and as a consumer (consuming documents from a MongoDB collection).
MongoDB revolves around the concepts of documents (not as is office documents, but rather hierarchical data defined in JSON/BSON) and collections. This component page will assume you are familiar with them. Otherwise, visit [http://www.mongodb.org/].
h2. URI format
{code}
mongodb:connectionBean?database=databaseName&collection=collectionName&operation=operationName[&moreOptions...]
{code}
h2. Endpoint options
MongoDB endpoints support the following options, depending on whether they are acting like a Producer or as a Consumer (options vary based on the consumer type too).
{div:class=confluenceTableSmall}
|| Name || Default Value || Description || Producer || Tailable Cursor Consumer ||
| {{database}} | none | *Required.* The name of the database to which this endpoint will be bound. All operations will be executed against this database unless dynamicity is enabled and the {{CamelMongoDbDatabase}} header is set. | (/) | (/) |
| {{collection}} | none | *Required.* The name of the collection (within the specified database) to which this endpoint will be bound. All operations will be executed against this database unless dynamicity is enabled and the {{CamelMongoDbDatabase}} header is set. | (/) | (/) |
| {{operation}} | none | *Required for producers.* The id of the operation this endpoint will execute. Pick from the following:
* Query operations: {{findById}}, {{findOneByQuery}}, {{findAll}}, {{count}}
* Write operations: {{insert}}, {{save}}, {{update}}
* Delete operations: {{remove}}
* Other operations: {{getDbStats}}, {{getColStats}} | (/) | |
| {{createCollection}} | true | Determines whether the collection will be automatically created in the MongoDB database during endpoint initialisation if it doesn't exist already. If this option is {{false}} and the collection doesn't exist, an initialisation exception will be thrown. | (/) | |
| {{invokeGetLastError}} | false (behaviour may be inherited from connections WriteConcern) | Instructs the MongoDB Java driver to invoke {{getLastError()}} after every call. Default behaviour in version 2.7.2 of the MongoDB Java driver is that only network errors will cause the operation to fail, because the actual operation is executed asynchronously in the MongoDB server without holding up the client - to increase performance. The client can obtain the real result of the operation by explicitly invoking {{getLastError()}} on the {{WriteResult}} object returned or by setting the appropriate {{WriteConcern}}. If the backend operation has not finished yet, the client will block until the result is available. Setting this option to {{true}} will make the endpoint behave synchronously and return an Exception if the underlying operation failed. | (/) | |
| {{writeConcern}} | none (driver's default) | Set a {{WriteConcern}} on the operation out of MongoDB's parameterised values. See [WriteConcern.valueOf(String)|http://api.mongodb.org/java/current/com/mongodb/WriteConcern.html#valueOf(java.lang.String)]. | (/) | |
| {{writeConcernRef}} | none | Sets a custom {{WriteConcern}} that exists in the Registry. Specify the bean name. | (/) | |
| {{readPreference}} | none | Sets a {{ReadPreference}} on the connection. Accepted values: the name of any inner subclass of [ReadPreference|http://api.mongodb.org/java/current/com/mongodb/ReadPreference.html]. For example: {{PrimaryReadPreference}}, {{SecondaryReadPreference}}, {{TaggedReadPreference}}. | (/) | |
| {{dynamicity}} | false | If set to true, the endpoint will inspect the {{CamelMongoDbDatabase}} and {{CamelMongoDbCollection}} headers of the incoming message, and if any of them exists, the target collection and/or database will be overridden for that particular operation. Set to false by default to avoid triggering the lookup on every Exchange if the feature is not desired. | (/) | |
| {{persistentTailTracking}} | false | Enables or disables persistent tail tracking for Tailable Cursor consumers. See below for more information. | | (/) |
| {{persistentId}} | none | *Required if persistent tail tracking is enabled.* The id of this persistent tail tracker, to separate its records from the rest on the tail-tracking collection. | | (/) |
| {{tailTrackingIncreasingField}} | none | *Required if persistent tail tracking is enabled.* Correlation field in the incoming record which is of increasing nature and will be used to position the tailing cursor every time it is generated. The cursor will be (re)created with a query of type: tailTrackIncreasingField > lastValue (where lastValue is possibly recovered from persistent tail tracking). Can be of type Integer, Date, String, etc. NOTE: No support for dot notation at the current time, so the field should be at the top level of the document. | | (/) |
| {{cursorRegenerationDelay}} | 1000ms | Establishes how long the endpoint will wait to regenerate the cursor after it has been killed by the MongoDB server (normal behaviour). | | (/) |
| {{tailTrackDb}} | same as endpoint's | Database on which the persistent tail tracker will store its runtime information. | | (/) |
| {{tailTrackCollection}} | camelTailTracking | Collection on which the persistent tail tracker will store its runtime information. | | (/) |
| {{tailTrackField}} | lastTrackingValue | Field in which the persistent tail tracker will store the last tracked value. | | (/) |
h2. MongoDB operations - producer endpoints
h3. Query operations
h4. findById
This operation retrieves only one element from the collection whose _id field matches the content of the IN message body. The incoming object can be anything that has an equivalent to a BSON type. See [http://bsonspec.org/#/specification] and [http://www.mongodb.org/display/DOCS/Java+Types].
{code}
from("direct:findById")
.to("mongodb:myDb?database=flights&collection=tickets&operation&operation=findById")
.to("mock:resultFindById");
{code}
{tip:title=Supports fields filter}
This operation supports specifying a fields filter. See [Specifying a fields filter#FieldsFilter].
{tip}
h4. findOneByQuery
Use this operation to retrieve just one element from the collection that matches a MongoDB query. The query object is extracted from the IN message body, i.e. it should be of type {{DBObject}} or convertible to {{DBObject}}. See [Type conversions].
{code}
from("direct:findOneByQuery")
.to("mongodb:myDb?database=flights&collection=tickets&operation&operation=findOneByQuery")
.to("mock:resultFindOneByQuery");
{code}
{tip:title=Supports fields filter}
This operation supports specifying a fields filter. See [Specifying a fields filter#FieldsFilter].
{tip}
h4. findAll
The {{findAll}} operation returns all documents matching a query, or none at all, in which case all documents contained in the collection are returned.
{code}
from("direct:findAll")
.to("mongodb:myDb?database=flights&collection=tickets&operation=findAll")
.to("mock:resultFindAll");
{code}
Paging and efficient retrieval is supported via the following headers:
|| Header key || Quick constant || Description (extracted from MongoDB API doc) || Expected type ||
| {{CamelMongoDbNumToSkip}} | {{MongoDbConstants.NUM_TO_SKIP}} | Discards a given number of elements at the beginning of the cursor. | int/Integer |
| {{CamelMongoDbLimit}} | {{MongoDbConstants.LIMIT}} | Limits the number of elements returned. | int/Integer |
| {{CamelMongoDbBatchSize}} | {{MongoDbConstants.BATCH_SIZE}} | Limits the number of elements returned in one batch. A cursor typically fetches a batch of result objects and store them locally. If batchSize is positive, it represents the size of each batch of objects retrieved. It can be adjusted to optimize performance and limit data transfer. If batchSize is negative, it will limit of number objects returned, that fit within the max batch size limit (usually 4MB), and cursor will be closed. For example if batchSize is -10, then the server will return a maximum of 10 documents and as many as can fit in 4MB, then close the cursor. Note that this feature is different from limit() in that documents must fit within a maximum size, and it removes the need to send a request to close the cursor server-side. The batch size can be changed even after a cursor is iterated, in which case the setting will apply on the next batch retrieval.| int/Integer |
Additionally, you can set a sortBy criteria by putting the relevant {{DBObject}} describing your sorting in the {{CamelMongoDbSortBy}} header, quick constant: {{MongoDbConstants.SORT_BY}}.
{tip:title=Supports fields filter}
ThisThe {{findAll}} operation supportswill specifyingalso areturn fieldsthe filter.following SeeOUT [Specifyingheaders ato fields filter#FieldsFilter].
{tip}
{anchor:FieldsFilter}
h4. Specifying a field filter
Query operations will, by default, return matching objects in their entirety. If your documents are large and you are only interested in retrieving a subset of their fields, you can specify a field filter in all query operations, simply by setting the relevant {{DBObject}} (or type convertible to {{DBObject}}, such as a JSON String, Map, etc.) on the {{CamelMongoDbFieldsFilter}} header, constant shortcut: {{MongoDbConstants.FIELDS_FILTER}}.
An example that uses MongoDB's BasicDBObjectBuilder to simplify the creation of DBObjects:
{code}
DBObject fieldFilter = BasicDBObjectBuilder.start().add("_id", 0).add("fixedField", 0).get();
Object result = template.requestBodyAndHeader("direct:findAll", (Object) null, MongoDbConstants.FIELDS_FILTER, fieldFilter);
{code}
h3. Create/update operations
h4. insert
Inserts an new object into the MongoDB collection, taken from the IN message body. Type conversion is attempted to turn it into {{DBObject}} or a {{List}}.
Two modes are supported: single insert and multiple insert. For multiple insert, the endpoint will expect a List, Array or Collections of objects of any type, as long as they are - or can be converted to - {{DBObject}}. All objects are inserted at once. The endpoint will intelligently decide which backend operation to invoke (single or multiple insert) depending on the input.
Example:
{code}
from("direct:insert")
.to("mongodb:myDb?database=flights&collection=tickets&operation=insert");
{code}
The operation will return a WriteResult, and depending on the WriteConcern or the value of the {{invokeGetLastError}} option, the getLastError would have been called already or not. If you want to access the ultimate result of the write operation, you need to retrieve the {{CommandResult}} by calling {{getLastError()}} or {{getCachedLastError()}} on the {{WriteResult}}. Then you can verify the result by calling {{CommandResult.ok()}}, {{CommandResult.getErrorMessage()}} and/or {{CommandResult.getException()}}.
Note that the new object's _id must be unique in the collection. If you don't specify the value, MongoDB will automatically generate one for you. But if you do specify it and it is not unique, the insert operation will fail (and for Camel to notice, you will need to enable invokeGetLastError or set a WriteConcern that waits for the write result). This is not a limitation of the component, but it is how things work in MongoDB for higher throughput. If you are using a custom _id, you are expected to ensure at the application level that is unique (and this is a good practice too).
h4. save
The save operation is equivalent to an upsert (UPdate, inSERT) operation, where the record will be inserted if it doesn't exist, or else it will be updated. MongoDB will perform the matching based on the _id field.
Beware that in case of an update, the object will be replaced entirely and the usage of [MongoDB's $modifiers|http://www.mongodb.org/display/DOCS/Updating#Updating-ModifierOperations] is not permitted. Therefore, if you want to manipulate the object if it already exists, you have two options:
# perform a query to retrieve the entire object first along with all its fields (may not be efficient), alter it inside Camel and then save it.
# use the update operation with [$modifiersenable you to iterate through result pages if you are using paging:
|| Header key || Quick constant || Description (extracted from MongoDB API doc) || Data type ||
| {{CamelMongoDbResultTotalSize}} | {{MongoDbConstants.RESULT_TOTAL_SIZE}} | Number of objects matching the query. This does not take limit/skip into consideration. | int/Integer |
| {{CamelMongoDbResultPageSize}} | {{MongoDbConstants.RESULT_PAGE_SIZE}} | Number of objects matching the query. This does not take limit/skip into consideration. | int/Integer |
{tip:title=Supports fields filter}
This operation supports specifying a fields filter. See [Specifying a fields filter#FieldsFilter].
{tip}
{anchor:FieldsFilter}
h4. Specifying a field filter
Query operations will, by default, return matching objects in their entirety. If your documents are large and you are only interested in retrieving a subset of their fields, you can specify a field filter in all query operations, simply by setting the relevant {{DBObject}} (or type convertible to {{DBObject}}, such as a JSON String, Map, etc.) on the {{CamelMongoDbFieldsFilter}} header, constant shortcut: {{MongoDbConstants.FIELDS_FILTER}}.
An example that uses MongoDB's BasicDBObjectBuilder to simplify the creation of DBObjects:
{code}
DBObject fieldFilter = BasicDBObjectBuilder.start().add("_id", 0).add("fixedField", 0).get();
Object result = template.requestBodyAndHeader("direct:findAll", (Object) null, MongoDbConstants.FIELDS_FILTER, fieldFilter);
{code}
h3. Create/update operations
h4. insert
Inserts an new object into the MongoDB collection, taken from the IN message body. Type conversion is attempted to turn it into {{DBObject}} or a {{List}}.
Two modes are supported: single insert and multiple insert. For multiple insert, the endpoint will expect a List, Array or Collections of objects of any type, as long as they are - or can be converted to - {{DBObject}}. All objects are inserted at once. The endpoint will intelligently decide which backend operation to invoke (single or multiple insert) depending on the input.
Example:
{code}
from("direct:insert")
.to("mongodb:myDb?database=flights&collection=tickets&operation=insert");
{code}
The operation will return a WriteResult, and depending on the WriteConcern or the value of the {{invokeGetLastError}} option, the getLastError would have been called already or not. If you want to access the ultimate result of the write operation, you need to retrieve the {{CommandResult}} by calling {{getLastError()}} or {{getCachedLastError()}} on the {{WriteResult}}. Then you can verify the result by calling {{CommandResult.ok()}}, {{CommandResult.getErrorMessage()}} and/or {{CommandResult.getException()}}.
Note that the new object's _id must be unique in the collection. If you don't specify the value, MongoDB will automatically generate one for you. But if you do specify it and it is not unique, the insert operation will fail (and for Camel to notice, you will need to enable invokeGetLastError or set a WriteConcern that waits for the write result). This is not a limitation of the component, but it is how things work in MongoDB for higher throughput. If you are using a custom _id, you are expected to ensure at the application level that is unique (and this is a good practice too).
h4. save
The save operation is equivalent to an upsert (UPdate, inSERT) operation, where the record will be inserted if it doesn't exist, or else it will be updated. MongoDB will perform the matching based on the _id field.
Beware that in case of an update, the object will be replaced entirely and the usage of [MongoDB's $modifiers|http://www.mongodb.org/display/DOCS/Updating#Updating-ModifierOperations] is not permitted. Therefore, if you want to manipulate the object if it already exists, you have two options:
# perform a query to retrieve the entire object first along with all its fields (may not be efficient), alter it inside Camel and then save it.
# use the update operation with [$modifiers|http://www.mongodb.org/display/DOCS/Updating#Updating-ModifierOperations], which will execute the update at the server-side instead. You can enable the upsert flag, in which case if an insert is required, MongoDB will apply the $modifiers to the filter query object and insert the result.
For example:
{code}
from("direct:insert")
.to("mongodb:myDb?database=flights&collection=tickets&operation=save");
{code}
h4. update
Update one or multiple records on the collection. Requires a List<DBObject> as the IN message body containing exactly 2 elements:
* Element 1 (index 0) => filter query => determines what objects will be affected, same as a typical query object
* Element 2 (index 1) => update rules => how matched objects will be updated. All [modifier operations|http://www.mongodb.org/display/DOCS/Updating#Updating-ModifierOperations] from MongoDB are supported.
{note:title=Multiupdates}
By default, whichMongoDB will executeonly the update at1 theobject server-side instead. You can enableeven if multiple objects match the upsertfilter flag,query. inTo whichinstruct caseMongoDB ifto anupdate insert*all* ismatching requiredrecords, set MongoDB will apply the $modifiers to the filter query object and insert the result.
For example:
{code}
from("direct:insert")
.to("mongodb:myDb?database=flights&collection=tickets&operation=save");
{code}
h4. update
Update one or multiple records on the collection. Requires a List<DBObject> as the IN message body containing exactly 2 elements:
* Element 1 (index 0) => filter query => determines what objects will be affected, same as a typical query object
* Element 2 (index 1) => update rules => how matched objects will be updated. All [modifier operations|http://www.mongodb.org/display/DOCS/Updating#Updating-ModifierOperations] from MongoDB are supported.
{note:title=Multiupdates}
By default, MongoDB will only update 1 object even if multiple objects match the filter query. To instruct MongoDB to update *all* matching records, set the {{CamelMongoDbMultiUpdate}} IN message header to {{true}}.
{note}
Supports the following IN message headers:
|| Header key || Quick constant || Description (extracted from MongoDB API doc) || Expected type ||
| {{CamelMongoDbMultiUpdate}} | {{MongoDbConstants.MULTIUPDATE}} | | boolean/Boolean |
| {{CamelMongoDbUpsert}} | {{MongoDbConstants.UPSERT}} | | boolean/Boolean |
For example, the following will update *all* records whose filterField field equals true by setting the value of the "scientist" field to "Darwin":the {{CamelMongoDbMultiUpdate}} IN message header to {{true}}.
{note}
A header with key {{CamelMongoDbRecordsAffected}} will be returned ({{MongoDbConstants.RECORDS_AFFECTED}} constant) with the number of records updated (copied from {{WriteResult.getN()}}).
Supports the following IN message headers:
|| Header key || Quick constant || Description (extracted from MongoDB API doc) || Expected type ||
| {{CamelMongoDbMultiUpdate}} | {{MongoDbConstants.MULTIUPDATE}} | Ff the update should be applied to all objects matching. See [http://www.mongodb.org/display/DOCS/Atomic+Operations] | boolean/Boolean |
| {{CamelMongoDbUpsert}} | {{MongoDbConstants.UPSERT}} | If the database should create the element if it does not exist | boolean/Boolean |
For example, the following will update *all* records whose filterField field equals true by setting the value of the "scientist" field to "Darwin":
{code}
// route: from("direct:update").to("mongodb:myDb?database=science&collection=notableScientists&operation=update");
DBObject filterField = new BasicDBObject("filterField", true);
DBObject updateObj = new BasicDBObject("$set", new BasicDBObject("scientist", "Darwin"));
Object result = template.requestBodyAndHeader("direct:update", new Object[] {filterField, updateObj}, MongoDbConstants.MULTIUPDATE, true);
{code}
h3. Delete operations
h4. remove
Remove matching records from the collection. The IN message body is expected to be a {{DBObject}} or a type convertible to it, and will act as the removal filter query.
The following code will remove all objects whose field 'conditionField' equals true, in the science database, notableScientists collection.
{code}
// route: from("direct:updateremove").to("mongodb:myDb?database=science&collection=notableScientists&operation=updateremove");
DBObject filterFieldconditionField = new BasicDBObject("filterFieldconditionField", true);
DBObject updateObj = new BasicDBObject("$set", new BasicDBObject("scientist", "Darwin"));
Object result = template.requestBodyAndHeader("direct:update", new Object[] {filterField, updateObj}, MongoDbConstants.MULTIUPDATE, true.requestBody("direct:remove", conditionField);
{code}
h3. Delete operations
h4. remove
A header with key {{CamelMongoDbRecordsAffected}} will be returned ({{MongoDbConstants.RECORDS_AFFECTED}} constant) with the number of records deleted (copied from {{WriteResult.getN()}}).
h3. Other operations
h4. count
h4. getDbStats
h4. getColStats
h3. Dynamic operations
An Exchange can override the endpoint's fixed operation by setting the {{CamelMongoDbOperation}} header, defined by the {{MongoDbConstants.OPERATION_HEADER}} constant.
The values supported are determined by the MongoDbOperation enumeration and match the accepted values for the {{operation}} parameter on the endpoint URI.
For example:
{code}
Object result = template.requestBodyAndHeader("direct:insert", "irrelevantBody", MongoDbConstants.OPERATION_HEADER, "count");
assertTrue("Result is not of type Long", result instanceof Long);
{code}
h2. Type conversions |