...
Currently gfsh lucene query can only query with uses the default StringQueryProvider, which provides syntax as
...
To fulfill the requirement, we will introduce another specify points config map for each numeric field into the built-in TermQueryProviderStringQueryProvider.
Approach
If in gfsh, the user specify one or more --termQuery, then use the new TermQueryProvider
If user also specify --queryString, then it becomes a MUST condition to work together with other --termQuery parameters.
Assumptions
When specifying numeric range, we have to set a few reasonable assumptions to limit the scope of the work.
...
Lucene's StandardQueryParser can parse the syntax (see chapter "Gfsh command line syntax" for detail) if specify some fields are numeric.
The numeric fields could be Integer, Float, or Double.
To do that, the parser should set the PointsConfigMap with a name and type mapping.
Our index contains the indexed field list. It also saved the meta-info of each field's type somewhere. Get the meta-info of field-type mapping and create the PointsConfigMap, then set it into parser.
Challenge
- The meta-info is saved into each serializer, no generic interface. The code needs to refactor.
- Need to explicitly specify HeterogeneousLuceneSerializer as default serializer, which saved the meta-info in its private
mappers.
- The FlatFormatSerializer is implemented in different way. It has no mappers data structure.
- Pdx data type is parsed by pdxMapper, how to get the meta-info is still unknown.
Gfsh command line syntax
There's no change in gfsh. Current gfsh parameters have supported the numeric query syntax.
Code Block |
---|
# create index with 4 numeric fields
gfsh> create lucene index --name=personIndex --region=/Person --field=name,email,address,revenue,revenue_float,revenue_double,revenue_long
# find a exact match for a numeric field
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue=763000" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key763 | Person{name='Tom763 Zhou', email='tzhou763@example.com', revenue=763000, homepage='Page{id=763, c.. | 1
# use 2 SHOULD conditions, which is equivalent to "A OR B"
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue=763000 revenue=764000" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key763 | Person{name='Tom763 Zhou', email='tzhou763@example.com', revenue=763000, homepage='Page{id=763, c.. | 1
key764 | Person{name='Tom764 Zhou', email='tzhou764@example.com', revenue=764000, homepage='Page{id=764, c.. | 1
# use 2 MUST conditions, which is equivalent to "A AND B". Lucene recognizes "+" as MUST
gfsh>search lucene --region=/Person --name=personIndex --queryString="+revenue>763000 +revenue<766000" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key765 | Person{name='Tom765 Zhou', email='tzhou765@example.com', revenue=765000, homepage='Page{id=765, c.. | 1
key764 | Person{name='Tom764 Zhou', email='tzhou764@example.com', revenue=764000, homepage='Page{id=764, c.. | 1
# >=, <= are valid syntax for inclusive condition
gfsh>search lucene --region=/Person --name=personIndex --queryString="+revenue>=763000 +revenue<=766000" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key766 | Person{name='Tom766 Zhou', email='tzhou766@example.com', revenue=766000, homepage='Page{id=766, c.. | 1
key765 | Person{name='Tom765 Zhou', email='tzhou765@example.com', revenue=765000, homepage='Page{id=765, c.. | 1
key764 | Person{name='Tom764 Zhou', email='tzhou764@example.com', revenue=764000, homepage='Page{id=764, c.. | 1
key763 | Person{name='Tom763 Zhou', email='tzhou763@example.com', revenue=763000, homepage='Page{id=763, c.. | 1
# Another way to specify range query
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue:[763000 TO 766000]" --defaultField=name
# Query on float, double, long fields. All the 4 numeric types (integer, float, double, long) are supported
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue_float:[763000.0 TO 766000.0]" --defaultField=name
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue_double:[763000 TO 766000]" --defaultField=name
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue_long:[763000 TO 766000]" --defaultField=name
# Combination query to return a subset
gfsh>search lucene --region=/Person --name=personIndex --queryString="+revenue_long:[763000 TO 766000] +revenue_float:[762000 TO 765000]" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key765 | Person{name='Tom765 Zhou', email='tzhou765@example.com', revenue=765000, homepage='Page{id=765, c.. | 1
key764 | Person{name='Tom764 Zhou', email='tzhou764@example.com', revenue=764000, homepage='Page{id=764, c.. | 1
key763 | Person{name='Tom763 Zhou', email='tzhou763@example.com', revenue=763000, homepage='Page{id=763, c.. | 1
# Lucene recognizes "-" as NOT. One NOT condition will reduce results.
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue<2000 revenue>9997000 -name=Tom9998*" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key0 | Person{name='Tom0 Zhou', email='tzhou0@example.com', revenue=0, homepage='Page{id=0, content="Hel.. | 1
key1 | Person{name='Tom1 Zhou', email='tzhou1@example.com', revenue=1000, homepage='Page{id=1, content=".. | 1
key9999| Person{name='Tom9999 Zhou', email='tzhou9999@example.com', revenue=9999000, homepage='Page{id=999.. | 1
Note: name=Tom9998* equals name:Tom9998* and Tom9998* (when defaultField is name)
# Query with a numeric field in a JSON object
gfsh>search lucene --region=/Person --name=personIndex --queryString="revenue="400000" --defaultField=name
key | | score
------ | --------------------------------------------------------------------------------------------------- | -----
key400 | Person{name='Tom400 Zhou', email='tzhou400@example.com', revenue=400000, homepage='Page{id=400, c.. | 1
json1 | PDX[8776019,__GEMFIRE_JSON]{revenue=400000, address=PDX[16524384,__GEMFIRE_JSON]{city=New York, p.. | 1
# Query in numeric field in a nested object
gfsh>search lucene --region=/Customer --name=customerIndex --queryString="+contacts.revenue:[763000 TO 766000] +revenue:[762000 TO 765000]" --defaultField=name
Note: Both conditions take effect and display 3 (not 4) Customer objects |
Java API
There is no change to Java API.
...
Gfsh command line syntax
Code Block |
---|
gfsh> search lucene --name=personIndex --region=/Person --queryString=john* --termQuery=address.must("97006") --termQuery=salary.should([75000,80000]) --defaultField=name
Note: find the persons with name starts with "john" and address contain "97006", with his salary might be within 75000 to 80000 (inclusive)
gfsh> search lucene --name=personIndex --region=/Person --termQuery=name.should(john*) --termQuery=address.should("97006") --termQuery=salary.should([75000,80000]) --defaultField=name
Note: find the persons either name starts with "john" or address contains "97006" or salary are between 75000 to 80000.
|
Java API
The new TermQueryProvider can be used in java application too.
...