How Data is handled in StreamPipes
Real-time Example Primitive
Real-time Example List
Example Missing for story telling
Any Ideas?
Data Layer
Stored as a LinkedTreeMap:
A map of comparable keys to values. Unlike `TreeMap`, this class uses insertion order for iteration order. Comparison order is only used as an optimization for efficient insertion and removal.
key: s0::name value: value
why s0::name?
Time
Timestamp is always required → See Time in StreamPipes
is stored as Unix Timestamp.
What happens if YYY-MM-DD as data sourceo be converted internally to Unix timestamp?
Description Layer
Event schema; describes each individual attributes of an event. The three most important and required fields are:
SP Cass | Attribute | Type | Description |
---|---|---|---|
Class EventProperty | runtimeName | String | name of the attribute for unique identification |
runtimeType | String | primitive data type of the attribute | |
Class EventProperty | domainProperty | ArrayList<String URI> | semantic description of the attribute |
Runtime Name. The runtime name indicates the key of the property at runtime, e.g., if our JSON message contains a structure such as
{"plateNumber" : "KA-F 123"}
, the runtime name must beplateNumber
.Runtime Type. An event property must have a primitive type (we will later see how to model more complex properties such as lists and nested properties). The type must be an instance of
XMLSchema
primitives, however, the SDK provides convenience methods to provide the property type.Domain Property. The domain property indicates the semantics of the event property. For instance, the
latitude
property is linked to thehttp://www.w3.org/2003/01/geo/wgs84_pos#lat
property of the WGS84 vocabulary. The domain property should be an URI as part of an existing or domain-specific vocabulary. The SDK provides convenience methods for popular vocabularies (e.g., Schema.org, Dolce or WGS84).
SP Class | Type | Type | Description |
---|---|---|---|
Class EventProperty | label | String | |
Class EventProperty | description | String | |
measurementUnit | String | ||
String valuesSpecitication | String | ||
Class EventProperty | boolean required | boolean | |
Class EventProperty | eventPropertyQualities | ArrayList | |
Class EventProperty | requiresEventPropertyQualities | ArrayList | |
Class EventProperty | propertyScope | String | |
Class EventProperty | index | int | |
Class EventProperty | runtimeID | String |
Why
runtimeType => not Part of Class EvbentProperty Class
measurementUnit => not Part of Class EvbentProperty Class
valuesSpecitication => not Part of Class EvbentProperty Class
Event Property Types
Type | Description |
---|---|
EventPropertiyPrimitive | flat single value |
EventPropertyList | List of flat single values |
EventPropertyNested | Deprecated? |
Read Event Property
It can be called in the required Section via
Read Way | Description |
---|---|
EpRequirements |
How to read List?
Write
Write Way | Description |
---|---|
EventPropertiyPrimitiveBuilder | |
EpProperty | |
Arrays.asList(EpProperties... , EpProperties... ) |
EventPropertiyPrimitiveBuilder
This is used to build a complex Event Schema with the runtimeName, runtimeType and domainProperty and also optional attributes.
Required | Method | Description |
---|---|---|
yes | .create(Datatypes datatype, String runtimeName) | |
yes | .domainProperty(String domainProperty) | |
yes | .build() | |
optional | .description(String description) | |
optional | .label(String label) | |
optional | .scope(PropertyScope propertyScope) | |
optional | .measrumentUnit(URI measurementUnit) | |
optional | .resolution(Float resolution) | |
optional | .valueSpecification(Float min, Float max, Float step) | |
optional | .valueSpecification( String label, String description, List< String> allowedValues) |
Why is domainProperty not an Array List?
EpProperty
this is used to build a basic Event Schema with
the method also defines the runtimeType
Time
Method | Description |
---|---|
timestampProperty(String runtimeName) |
String
Method | Description |
---|---|
stringEp(Label label, String runtimeName, String domainProperty, Enumeration enumeration) | |
stringEp(Label label, String runtimeName, String domainProperty, PropertyScope propertyScope) | |
stringEp(Label label, String runtimeName, String domainProperty) | |
stringEp(Label label, String runtimeName, List <Uri> domainProperties) |
Double
Method | Description |
---|---|
doubleEp(Label label, String runtimeName, String domainProperty) | |
doubleEp(Label label, String runtimeName, String domainProperty, Float minValue, Float maxValue, Float step) |
Integer
Method | Description |
---|---|
integerEp(Label label, String runtimeName, List < Uri> domainProperties) | |
integerEp(Label label, String runtimeName, String domainProperty, QuantitativeValue valueSpecification) | |
integerEp(Label label, String runtimeName, String domainProperty, Float minValue, Float maxValue, Float step) |
Number
Method | Description |
---|---|
numberEp(Label label, String runtimeName, String domainProperty) |
General
Method | Description |
---|---|
ep(Label label, String runtimeType, String runtimeName, String domainProperty) |
Boolean
Method | Description |
---|---|
booleanEp(Label label, String runtimeName, String domainProperty) |
Long
Method | Description |
---|---|
longEp(Label label, String runtimeName, String domainProperty) | |
longEp(Label label, String runtimeName, List< URI> domainProperty) |
Image
Method | Description |
---|---|
imageProperty(String runtimeName) |
List
General
Method | Description |
---|---|
listEp(Label label, String runtimeName, Datatypes runtimeType, String domainProperty) | |
listEp(Label label, String runtimeName, EventPropety eventProperty, String domainProperty) | |
listEp(Label label, String runtimeName, EventProperty eventProperty) |
String
Method | Description |
---|---|
ListStringEp(Label label, String runtimeName, String domainProperty) |
Double
Method | Description |
---|---|
listDoupleEp(Label label, String runtimeName, String domainProperty) |
Boolean
Method | Description |
---|---|
listBoolean(Label label, String runtimeName, String domainProperty)) |
Integer
Method | Description |
---|---|
listIntegerEp(Label label, String runtimeName, String domainProperty) |
Long
Method | Description |
---|---|
listLongEp(Label label, String runtimeName, String domainProperty) |
Nested
Method | Description |
---|---|
listNestedEp(Label label, String runtimeName, List<EventProperty... eventProperties) | |
listNestedEp(Label label, String runtimeName, String domainProperty, List<Eventproperty> listItemProperties) |
Scope
There are 4 types of scope
Why needs scope to be mentioned in requiredEvent?
Type | Description |
---|---|
HEADER_PROPERTY | A property that defines meta-information about the event, for instance its occurrence time |
DIMENSION_PROPERTY | A property that describes the context of a measurement, e.g., the ID of a machine or a thing. |
MEASUREMENT_PROPERTY | A property that contains (often quantitative) measurement values |
NONE | The property scope is not further described. Use this scope for defining property requirements where the data processor's functionality does not require any specific property scope (e.g., a text filter that can filter any text-based property) |
Quality
DomainProperty
You can either use an own URI Link
or use a variety of StreamPipes implemented vocabularies.
A full list can be found under the vocabulary package:
Labels
Labels.withID(String internalID) Labels.empty()
Deprecated: Labels.from(String internalID, String label, String description) Labels.withTitle(String label, String desciption) Labels.fromResourcess(String resourceIdentifier, String resourceName)