SiteWhere

New and Noteworthy in SiteWhere 1.9.0

The SiteWhere 1.9.0 release adds 30+ new features, updates, and bug fixes and contains a number of important architectural improvements.

Tenant Management

Improved Tenant Template Support

SiteWhere 1.9.0 adds a more powerful model for using templates to create new tenants. In previous versions, there was a single tenant template which was used to bootstrap all new tenants. The new architecture provides multiple templates which provide more options for creating new tenants. The tenant templates include a base tenant configuration file and can specify data initializers for various parts of the SiteWhere data model. When creating a new tenant, a starting template is specified. Templates included out-of-the-box include:

  • Empty Tenant - This template allows a tenant to be create with no data and a very basic tenant configuration. There is no longer a need to delete the sample data that was automatically loaded with previous versions.
  • Construction Example - This template loads the construction example data included in previous SiteWhere releases.
  • Air Traffic Example - This template loads the air traffic example included in the sitewhere-examples GitHub repository. It includes additional assets for planes and tracking devices as well as a default tenant configuration that is pre-configured to ingest STOMP data sent from the air traffic web application.
In addition to the templates included OOTB, users can add their own templates to make it faster and easier to bootstrap new tenants.

Tenant Lifecycle Persistence

In previous versions, SiteWhere always started all tenants when the server was restarted. SiteWhere 1.9.0 now preserves tenant lifecycle state so tenants that have been stopped will remain stopped on server restart.

Infrastructure Updates

Better Performance in Administrative Console

Previous versions of the administrative console were based on JSP as the view technology. Spring Boot and the embedded Tomcat container have performance issues with JSP, so we updated the view technology to use Apache Velocity templates. The result is a much faster web interface and a much smaller server footprint thanks to the removed transitive dependencies.

Upgrade to Log4J 2

SiteWhere 1.9.0 now uses Log4J 2 as the basis for all system logging. This provides better integration with the existing logging in Spring boot.

Upgrade Swagger Dependencies

In the pre-1.5 Swagger version used in previous versions of SiteWhere, Scala was a transitive dependency. The footprint was huge considering it was not a core server component. SiteWhere 1.9.0 upgrades to a non-Scala version of Swagger, which greatly reduces the overall server footprint. Thanks to dependency improvements, SiteWhere 1.9.0 is almost 25% smaller than the previous version.

InfluxDB Improvements

Updated Client Jar

Updates to the core InfluxDB server caused problems with previous versions of SiteWhere due to non-backward-compatible API changes in the database. SiteWhere 1.9.0 includes the latest InfluxDB jar, which fixes the errors reported by users that upgraded InfluxDB to 1.0+.

Event Processing Fixes

The were a number of bugs in SiteWhere 1.8.0 related to the hybrid MongoDB/InfluxDB model and event processing. All of the reported issues have been fixed along with a number of improvements for the hybrid model.

Download SiteWhere 1.9.0

There are many other new features and bug fixes available in SiteWhere 1.9.0, so download it today!

SiteWhere

New and Noteworthy in SiteWhere 1.7.0

The SiteWhere 1.7.0 release includes some major architectural improvements along with many new features and bug fixes.

Upgrade to Spring Boot Architecture

Improved System Bootstrapping

SiteWhere 1.7.0 migrates the core platform to Spring Boot, supporting a more standardized approach for encapsulating the server. Previous versions of SiteWhere were bootstrapped as a web archive deployed on a standalone Tomcat instance, which gave less control over many aspects of the environment. With Spring Boot, SiteWhere provides the core application logic and uses the embedded Tomcat support to allow the web artifacts to still be deployed and accessed as with previous versions.

Updates to Folder Structure and Environment

By not packaging inside Tomcat, the folder structure has been simplified a lot. All of the Tomcat folders and configuration have been removed since they are now supplied by the embedded Tomcat instance provided by Spring Boot. The configuration folder has been moved into the root folder, but all of the content is still backward compatible with previous versions. Likewise, the database format has not changed, so you can point a 1.7.0 instance at the configuration and database from a previous installation and it will work. Since Tomcat is no longer bootstrapping, a new environment variable, SITEWHERE_HOME, has been introduced to point to the SiteWhere root folder.

Amazon SQS Integration

SiteWhere now integrates with Amazon SQS, allowing SiteWhere events to be pushed to queues in the Amazon cloud. The integration uses a configurable SiteWhere outbound event processor, which allows logic to be applied in choosing which events are pushed to Amazon. For instance, you can add an event processor that only forwards measurement events from a specific device specification into Amazon SQS.

Outbound RabbitMQ Event Processing

A new outbound event processor has been added for forwarding SiteWhere events to RabbitMQ using AMQP. As with other filtered event processors, logic (including complex scripted decision logic) can be applied to choose which events are forwarded. Any device that can listen on AMQP can receive a live stream of events as they are processed by SiteWhere. For those that prefer MQTT to AMQP, there is equivalent support for live streaming over MQTT.

Get Started!

Download SiteWhere 1.7.0 today and check out the new features for yourself! The SiteWhere team is looking forward to your feedback on the new architecture and features.

SiteWhere

New and Noteworthy in SiteWhere 1.5.0

The release of SiteWhere 1.5.0 Community Edition adds a number of new features to the platform along with some other refinements and bug fixes. One of the main areas addressed is tenant management with the addition of new administrative console functionality for tenant configuration. In addition, another high-performance option is now available for ingesting events via the AMQP protocol thanks to the new RabbitMQ event source. Finally, a new symbology subsystem was added to allow for the generation of symbols such as QR-Codes for SiteWhere entities such as devices and device assignments.

Introducing Web-based Tenant Configuration

Before SiteWhere 1.5.0 tenant configuration was managed by editing the tenant configuration file by hand. While a viable option, the process was error prone and made it hard to understand the many features available in the platform. SiteWhere 1.5.0 introduces a new web-based tenant configuration editor that allows tenants to be configured in the administrative application.

SiteWhere Tenant Configuration

The tenant configuration editor is arranged hierarchically with the same general structure as the configuration schema. It allows for the editing of existing configurations that have been directly edited in the XML file for backward compatibility while supporting a much more intuitive experience going forward. Each node in the configuration is documented along with each configurable attribute. The user interface constrains attributes based on datatype so that invalid values can not be entered. Validation functionality is also provided to ensure that it is obvious in the web interface when child components are required.

The 1.5.0 release also adds the concept of staging to SiteWhere tenant configuration. After editing the tenant configuration, the new changes are staged in a separate file from the running tenant configuration. The next time the tenant is restarted, the staged configuration takes the place of the existing one and becomes the active configuration.

RabbitMQ Event Source Support

We have received a number of requests for integrating SiteWhere with RabbitMQ to allow another high-performance option for ingesting events. SiteWhere 1.5.0 adds a new RabbitMQ event source that uses the RabbitMQ Java drivers to connect to an instance and consume events via the AMQP protocol. Characteristics such as the number of consumer threads can be configured to maximize performance.

RabbitMQ Event Source

Symbology Management and QR-Code Supprt

It is a common requirement to have symbols that represent devices or other managed entities. These can include barcodes, QR-Codes, or even custom labels with images and other symbolism. SiteWhere 1.5.0 adds a new symbol generator manager that allows various types of symbols to be configured for SiteWhere entities. The first symbology plugin supports generating QR-Codes for sites, specifications, devices, and assignments.

QR-Code Support

The symbols are generated dynamically and can be accessed via the SiteWhere REST services. For instance, if you have a custom application and need to be able add QR-Codes to the web application or product packaging, a call to the REST service will provide the QR-Code image. Since the QR-Code plugin is configurable within the new tenant configuration manager, the look-and-feel of the code (size, color, etc) can be customized on a per-tenant basis.

Video Covering 1.5.0 Features

We have also added a new video to the SiteWhere YouTube channel covering the 1.5.0 features. Take a look at the video, then download SiteWhere 1.5.0 CE to try the new features out yourself!

Dweeting with SiteWhere and Dweet.io

SiteWhere 1.3.0 introduces integration with dweet.io, a service that acts like Twitter for your devices. By sending SiteWhere data to dweet.io, you can integrate with many other services that support it as a data source. For instance freeboard.io can use dweet data as the basis of its visualizations.

Getting Started

There is no requirement to set up an account in order to send "dweets". Anyone can post data to the service by posting REST calls targeted at a unique device name. Dweet.io will allow you to "lock" a device name so that others can not post to or view data from a device unless they have a key to do so.

Configuring SiteWhere to Forward Data

Configuring SiteWhere to send device data as dweets is very simple. Open the configuration file for the tenant whose data you wish to start forwarding (the default tenant configuration is located in conf/sitewhere/default-tenant.xml). Scroll down to the outbound-processing-chain section and look for a commented entry dweet-io-event-processor. Uncomment the XML fragment, save the configuration, then restart your tenant. When the tenant restarts, data events for all devices will be forwarded as dweets.

SiteWhere Configuration

Generate Device Data with the Emulator

The SiteWhere administrative console contains an emulator application that allows data to be generated into the system as if it were coming from the device itself. This is handy for testing processing logic and will also be helpful in sending some test dweets. From the Sites page in the admin application, navigate to the device assignment list and click on the detail page for a device assignment. Click on the Emulate Assignment button at the top of the page, click Connect to connect to your local MQTT server, then click on the map to generate a new location event for the device assignment. The data will automatically be forwarded to dweet.io. Generate Data

Unique Data Identifier

Each dweet is targeted at a "thing" based on a unique identifier. Rather using device hardware ids as the identifier, SiteWhere uses the token for the current device assignment as the identifier. SiteWhere does not associate device events directly with a device since a single device can be assigned to many different assets over time. Each device assignment has a unique token that can be used to find the events generated while the assignment was active. To locate dweets, the device assignment token should be used as the "thing" name.

Viewing Data on Dweet.io

Now that SiteWhere data has been forwarded, we can open the dweet.io website and view it. When SiteWhere data is forwarded to dweet.io, the unique device name is the assignment token the event data is associated with.

The dweet.io website includes an interface for interacting with the REST services it provides, allowing you to view the data that SiteWhere has forwarded. Navigate to the play area to interact with the services. Enter the device assignment token to find the latest dweets associated with it:

Find Latest Dweet

The response will include all of the SiteWhere context data associated with the event including the device hardware id, assignment information, asset information, and detailed location information. Other event types such as measurements and alerts can be posted in the same way.

Find Latest Dweet

Next Steps

Now that you have successfully forwarded location data to dweet.io, try other information such as measurements and alerts. Play around with the REST services that dweet.io offers and see how easy it is to use them to access your SiteWhere data in a new way. Finally, take a look at the integration options that dweet.io opens up for your SiteWhere data. Have fun!

Visualizing Device Data on InitialState.com

SiteWhere 1.2.0 introduces a new outbound event processor that allows device event data to be forwarded to InitialState.com for advanced visualization. InitialState supports creating interactive dashboards with visual components such as graphs, gauges, maps, and emojis. With the new outbound processor, streams are created for each device assignment and events are published to InitialState in real time for viewing.

Creating an InitialState Account

To publish data to InitialState.com, you must first create a user account on the site. After creating a new account via the registration page, navigate to the account page and copy a Streaming Access Key that will be used to send data to InitialState:

InitialState Access Key

Configuring Connectivity to InitialState

In order to forward SiteWhere events to InitialState, a new outbound event processor must be configured in the tenant configuration file. With the tenant stopped, edit the configuration file and add the following to the outbound processor configuration:

[xml] [/xml]

After restarting the tenant, all events processed by the tenant will automatically be forwarded to InitialState.com. A separate stream is created for each device assignment as events are published. Now that SiteWhere is connected to InitialState, we need to generate some live data that will be forwarded. Open an existing device assignment and start the assignment emulator. Zoom in on the map and click on a location to add a new location event for the assignment:

Assignment Emulator

Log in to your InitialState account and view the list of data streams in the drawer on the left side of the screen. A stream should have been added for the device assignment used in the previous step. Data from the emulator should now be available in the stream for the assignment. Open the Tiles application in your InitialState console to start visualizing the new data. Since the data is geospatial, use the dropdown on the default tile to choose the Map visualization. The new location data should show up as shown below:

InitialState Tiles Application

Now that we have data successfully streaming into InitialState, we can start to visualize other types of events. Measurements and alerts can be added via the emulator or gathered from real devices. New tiles will be added for each unique type of data. For instance, adding measurements with names vehicle.speed and fuel.level will result in new tiles for the speed and fuel level of the vehicle being tracked. Alerts can also be added to track exceptional events such as critical fuel levels or movement of a vehicle outside of the work site. InitialState provides many visualizations that can bring the data to life:

Visualizations

Get Started with Your Own Data!

Integrating SiteWhere data with InitialState is easy and provides a powerful stage for visualizing your data over time. Start testing out the visualizations with your own live device data and explore the many visualizations provided by the InitialState platform. Any input that generates data into SiteWhere can be visualized, including openHAB, Android clients, or other integrations. Have fun!
SiteWhere

Introduction to Tenant Management

SiteWhere 1.2.0 introduces an advanced multiple tenant architecture that allows many IoT applications to run side by side in a single SiteWhere instance. Each tenant is isolated in terms of data storage and processing pipeline so that there is no chance of intermingling data. The new architecture is accompanied by an administrative interface for managing tenants and interacting with them.

Tenant Administrative Interface

The SiteWhere administrative application has been extended to offer tenant management capabilities. A new Tenants tab has been added to the application and is visible to system administrators. Clicking on the Tenants tab brings up a list of tenants registered in the system.

SiteWhere Tenant List

Each tenant has a unique id, a name, and a logo URL that identify it in the list of tenants. Each tenant is also associated with one or more system users, allowing them to log in an manage data for the tenant. If a user is only associated with a single tenant, the login page will proceed directly to that tenant. Otherwise, a list of associated tenants is shown and a tenant must be selected to continue into the administrative application. The logo image for the tenant is shown at the top left of the administrative application to indicate which tenant is currently being managed.

Clicking on a tenant in the list navigates to the detail page which provides information about the tenant and allows related actions to be taken.

SiteWhere Tenant Details

The detail page shows information about the lifecycle status of the tenant -- whether it is currently running or not. If the tenant is currently running, it shows the status of the entire hierarchy of components running within the tenant and the current XML configuration that is active. A running tenant also displays a button for shutting it down. Note that a running tenant does not allow its properties to be edited. A tenant may also show up as stopped as shown below:

SiteWhere Tenant Details

A stopped tenant does not display hierarchy or configuration information. The list of action buttons changes as well. A stopped tenant may be edited, deleted, or started. Note that deleting a tenant does not delete the underlying tenant configuration file. In order to reconfigure an existing tenant at runtime, stop the tenant, make changes to the underlying xxx-tenant.xml configuration file, then restart the tenant. This process will not affect other running tenants on the SiteWhere instance.

Upgrading from Previous Versions

The introduction of multitenancy resulted in significant architectural changes which affect many aspects of the system. Because of this, upgrading from previous versions is not recommended. Start with a clean installation from a downloaded bundle, then migrate your existing configuration to the new installation. Below are some of the key aspects of the system that changed to support multitenancy.

Database Structure

Before 1.2.0, all SiteWhere data was stored in one place. In MongoDB, there was a single database. In HBase, there was a single group of tables. The new architecture separates tenant data for the purpose of security, so the database storage strategy changed to match the requirements. There is now the concept of a core database (or tables in HBase) that stores the user management and tenant data. There is also a separate database (or tenant tables in HBase) for each tenant. The tenant database contains all sites, specifications, devices, assignments, etc. for the given tenant. In MongoDB, the default database is still named sitewhere and the tenant databases are named tenant-xxx where xxx is the unique tenant id. In HBase the core data is stored in the users table while tenant data is stored in the xxx-table table (e.g. default-devices specifies devices table for default tenant).

XML Configuration

Before 1.2.0, all SiteWhere configuration data was stored in a single configuration file named sitewhere-server.xml. In 1.2.0, the configuration schema has been split into two schemas, a core schema for global configuration, and a tenant schema for per-tenant configuration. The core configuration is still stored in sitewhere-server.xml and specifies the datastore for users and tenants along with configuration of global aspects such as Hazelcast. The server will also look for a xxx-tenant.xml for each tenant registered in the system. If no tenant configuration is found, one is automatically created based on the sitewhere-tenant.xml, which acts as a default template for new tenants.

Try it out!

For users that only need a single tenant, most aspects of the system should work as they did before. For users that require support for multiple tenants, the new architecture and administrative options will allow any number of tenants to run concurrently while only introducing minimal overhead. Now that you understand how SiteWhere supports multiple tenants, try it out in your own installation. SiteWhere 1.2.0 is available for download and in the cloud. The SiteWhere team looks forward to your feedback!

SiteWhere

New and Noteworthy in SiteWhere 1.1.0

The main focus of the SiteWhere 1.1.0 release is to improve usability of the SiteWhere asset management subsystem. It also addresses quite a few bugs and requested improvements in other areas of the system.

New Server Runtime Status Page

The home page for the administrative console application is now a server status page. You can quickly find which version and edition of SiteWhere is running in addition to other information about the operating system and JVM. More information will be added to the page in future releases.

SiteWhere Server Information

Manage Assets in the Administrative Console

Previous versions of SiteWhere only supported asset modules loaded from XML files or from external asset management products. While this worked for most users, it was not ideal because assets were not editable from the administrative console interface. SiteWhere 1.1.0 changes the default behavior to store assets in the datastore along with all of the other SiteWhere data. There is still the option of using XML files and external asset management solutions if needed.

In the administrative console interface, there is a new Assets tab which contains the new features. This leads to an Asset Categories page which allows categories of assets to be created. SiteWhere separates assets into four types:

  • People - Person assets contain information such as name, username, and email address.
  • Places - Location assets contain information such as latitude, longitude, and elevation.
  • Things - Hardware assets contain information such as product name, description, and SKU.
  • Devices - Device assets are a special class of hardware assets that can be used for device specifications.

Each asset category contains assets of one of the above types. Users can add any number of asset categories, then add any number of assets per category. At startup, SiteWhere creates an asset module for each category in addition to the asset modules already configured using XML files or asset management systems. Asset modules can be reloaded at runtime using the asset management REST services.

Better Support for WSO2 Identity Server

SiteWhere has supported WSO2 Identity Server as an identity management provider since before 1.0, but the module had not been updated to be configurable via the Spring XML schema. SiteWhere 1.1.0 adds Spring XML schema support and a number of new configuration options to make integration easier. A WSO2 Identity Server asset module can be configured by adding the following to the SiteWhere configuration:



	
	
	


Once configured, the WSO2 asset module will load all of the assets on startup via the SCIM protocol. If more assets are added via the WSO2 user interface, you will need to call the asset management refresh REST service. For more information about the configuration options, take a look at the documentation.

New Asset Search Options

A new REST method has been added to support queries of all assignments associated with a given asset. This can be used to get a list of devices currently associated, or to provide a history of all previous assignments for the asset. The new method is useful in applications that are more asset-centric rather than device-centric.

Multitenancy on the Way!

The SiteWhere 1.2.0 release will focus on making SiteWhere a truly multitenant system. New management features will be added to allow for the creation of tenants and the association of one or more tenants to each system user account. Each tenant will have its own SiteWhere "engine" that can configured meet its system requirements. This allows tenant isolation at the processing level.

From a data perspective, each tenant will have separate data storage with no intermingling of data. When using MongoDB as the datastore, each tenant uses a separate database. For an HBase datastore, each tenant will have their own group of tables. The administrative console will change to allow editing of data for a single tenant at a time. When a user logs in, they choose which tenant to interact with based on the list they are authorized to access.

Air Traffic

Air Traffic Monitoring Example Application

This video walks through a SiteWhere example application that emulates a real-time air traffic monitoring system. It illustrates how a developer can create a complete application with very little code using the SiteWhere platform. The application project with all source code is available on GitHub (https://github.com/sitewhere/sitewhere-examples).

Microsoft Logo

SiteWhere on Azure: Environment Setup

This is the first in a series of videos about integrating SiteWhere with Windows Azure Cloud technologies. It includes step-by-step instructions for provisioning an HBase cluster running on HDInsight and a Docker Swarm cluster running on Azure VMs. After the core environment is set up, the video provides instruction on adding SiteWhere and HiveMQ containers to the Swarm and connecting them to the HBase cluster. This video will provide a basis for moving into more advanced articles on scaling SiteWhere on the Azure Cloud using features such as EventHub, Stream Analytics and other high perforance offerings.

SiteWhere

New and Noteworthy in SiteWhere 1.0.1

SiteWhere 1.0.1 introduces many new features as well as improvements to the existing functionality. A few of the key features are discussed below.

Batch Operations

In previous versions of SiteWhere, the only method for sending commands to multiple devices was to manually create a command invocation for each device. SiteWhere 1.0.1 introduces the concept of batch operations which enable actions to target many devices at the same time. When a batch operation is submitted to the batch operation manager, an entry is created for each device and the manager proceeds to execute the desired action on each of them. The REST APIs allow the status of each operation element to be tracked, allowing for progress monitoring. Since an operation may affect millions of devices, the default batch operation manager also allows for throttling the events created by batch operations so that system traffic is not spiked when an operation is executing. For more information on batch operations, see the documentation.

The first functionality using the new batch operation APIs is for invoking commands in batch. Commands can be sent to many devices at once as long as all devices use the same device specification. Batch command invocations are supported both in the REST APIs and via the administrative console. To execute a batch command in the console, apply a device specification filter to the device list and a Batch Command button will appear in the tool bar. Click on the button and fill in command details exactly as if targeting a single device. The command will be issued to all of the devices that meet the given criteria. See the Device Filter Criteria section below to learn more about choosing which devices are targeted.

More types of batch operations are on the drawing board for future versions of SiteWhere. In particular, batch firmware updates are a common request from our users and will soon be on the roadmap.

Device Filter Criteria

Since SiteWhere is capable of managing thousands or even millions of devices, it is important to be able to filter the list of devices to find ones that meet certain criteria. SiteWhere 1.0.1 introduces advanced device filtering so that users can find devices they are interested in. This is particularly important for batch operations since a user must be able to choose the list of devices to target for an operation.

Examples of the new criteria include:

  • Including devices that implement a given device specification
  • Including devices that belong to a device group
  • Including devices that were added to the system in a given time period
  • Including devices that are not assigned to an asset

The filter criteria can be combined for queries such as ‘List all devices of specification Android that belong to the group employee tablets which were added to the system in the last week and have not been assigned to an employee’. The resulting list can be used to issue a batch command that affects all of the matching devices. For a more complete discussion of device filter criteria, see the documentation.

Simpler Device Command Implementation for Android Devices

Previously, commands sent to Java-capable platforms such as Android had to be encoded using a protocol such as Google Protocol Buffers. This imposed a requirement that adding a command to the device specification also required adding unmarshaling code on the device. SiteWhere 1.0.1 now supports dynamic commands for Java devices. The command name and parameters assigned in the device specification are used to infer a corresponding method to execute on the device implementation. The previous workflow was:

  • Copy updated proto from device specification into device project
  • Generate Java code from updated proto
  • Add unmarshal code that uses the generated Java code
  • Invoke method that executes command logic

The new workflow is:

  • Add method with name and parameters that match device specification and execute command logic within the method.

To use the new encoding scheme, the should be used on your command destination. Also, for Android, a separate sample project has been added to illustrate the new functionality.