Like many EU projects, webinos is strongly encouraged to seek out collaboration opportunities with other funded objectives. This makes eminent sense as this is public money we are using here, and it is in everyone’s interests to see this invested intelligently and without unnecessary duplication.

We have engaged in a number of these type of initiatives over the past years, and will do more so in the future. In an attempt to document this in a simple publicly accessible form, we shall run a series of blogs documenting what is going on, and what might happen in the future.

In the first of this series we shall focus on FI-WARE. This is a very large project, and as its acronym implies (Future Internet) somewhat ambitions. We shall deal with this in three parts. Part one, document what has happened to date. Part two, look at the very closely affiliated Cloud Edge Management Open Call and finally examine the Advanced WebUI set of Epics

Summary of Activity to date

FIWAREs remit is large. It defines a series of Generic Enablers that attempt to make it easier to develop Future Internet Applications, acting as foundations. The activity has been divided into a number of Chapters, of which the most relevant to core webinos activity is Interface to Networks and Devices (I2ND) in FI-WARE http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/Materializing_the_Interface_to_Networks_and_Devices_%28I2ND%29_in_FI-WARE

First thing to notice if FI-WARE lists Web Runtime and Web APIs are base line assets. Lets look in turn

  • A web runtime is a primary deliverable from webinos: webinos has a runtime. This is fully compliant with W3C standards. And they do implement some of the WAC APIs .These assets are publicly available under open source licenses.
  • WebAPIs: in summary webinos has 28 APIs defined. We will list below the exact mapping. It is interesting to note that the technology listed in FIWARE for WebAPI integration is NPAPI. This is a technique that we did use (4 years ago) when working on the earlier BONDI project. However we found it limiting. For the webinos use cases that require interdevice communication we have preferred a more loosely coupled architecture where the API host runs on a standalone server (node.js). The merits of these browser decoupled method of browse extensions have been discussed in earlier blogs.
     

The combination of WRT and API in webinos is supported on Windows, Mac , Linux, Android 2.2 -> 4.1, plus more recently ChromeOS.

When it comes to actual APIs its probably easier if we analyse them Epic by Epic (Epic is the term FIWARE uses for its coarse grained requirements)

Device Connectivity

FIWARE.Epic.I2ND.CDI.DeviceConnectivity

Read access to device connectivity is supported in webinos through the device status API and the vocabulary it exposes http://dev.webinos.org/specifications/new/devicestatus.html

The issue of how to change the connection status is more subtle. On the platform we architecturally encapsulate the Connection Manager. The issue of who can determine what connection to use (is it the application or is the user) is not simple, and needs to properly modelled in the policy framework.

Device Features

FIWARE.Epic.I2ND.CDI.DeviceFeatures

Is trivial and well supported in the existing http://dev.webinos.org/specifications/new/devicestatus.html

Device Sensors

FIWARE.Epic.I2ND.CDI.DeviceSensors

Is well supported by our newly updated Generic Sensor API http://dev.webinos.org/specifications/new/sensors.html

 

Personal Data Services

FIWARE.Epic.I2ND.CDI.PersonalDataServices

Covers PIM – Calendar, Contacts, Tasks, supported by

http://dev.webinos.org/specifications/new/contacts.html

http://www.w3.org/TR/2011/WD-calendar-api-20110419/

and pictures, videos, supported by

http://dev.webinos.org/specifications/new/MediaContent.html

The Epic also mentions access to Applications. This is partially supported by

http://dev.webinos.org/specifications/new/launcher.html

More fine grained access is possible in the webinos framework, and code exists to access it, but due to security considerations it is not exposed as a third party API

User Profile

FIWARE.Epic.I2ND.CDI.UserProfile

Is a strangely worded Epic. In terms of raw access to user data our first version of the APIs contained references to a specific UserProfiel API http://dev.webinos.org/deliverables/wp3/Deliverable32/static$02bf0c58dd455d51f29dafc0d5ed1392.html

This specification is currently being reworked to address design flaws

 

The other issues outlined in this Epic are more all pervading

I want to access to my applications through secure and easy to use identification and authentication procedures, and with consistent privacy policies supported.”

And are addressed very completely by the webinos architecture, but you need to look a the entire architecture holistically to understand how it works: how the PKI infrastructure interacts with policy and interacts with platform attestation.

 

Media Services

FIWARE.Epic.I2ND.CDI.MediaServices

The Epic as worded is quite narrow and again is addressed with

http://dev.webinos.org/specifications/new/devicestatus.html

Perhaps with a few additions to the vocabulary

But to address the full spirit of Media interaction you really need to look at the full suite of APIs available

 

http://dev.webinos.org/specifications/new/devicestatus.html

http://dev.webinos.org/specifications/new/tv.html

http://dev.w3.org/2011/webrtc/editor/getusermedia.html

http://dev.w3.org/2011/webrtc/editor/webrtc.html

 

Phone Api

FIWARE.Epic.I2ND.CDI.Phone

Is one of the few APIs not currently addressed in webinos , but planned for imminent implementation

Messaging API

FIWARE.Epic.I2ND.CDI.Messaging

One to one mapping with http://dev.webinos.org/specifications/new/messaging.html

Remote management

FIWARE.Epic.I2ND.CDI.RemoteManagement

We can break down into its two components

 

  • Discover and address a remote device: this is trivially addressed by the current webinos stack
  • Remotely configure one or multiple remote device: is not so simple. There are many control issues that need to be addressed here. Who has the right to do what? The consumer centric nature of webinos, asserts that all eventual control resides with the end user. In that sense webinos does fit with this requirement, in that is supports the synchronisation infrastructure to propagate changes between hubs and devices. A little more work is required however on actually what the typical remote configuration use cases are for end users.

 

Proximity Interfaces

FIWARE.Epic.I2ND.CDI.Proximity

Is largely about

“Application developers need to access proximity interfaces in a way as much as possible abstracted from their specific radio technologies (NFC, ZigBee, others…) and also from the specific device brand, model and OS.”

This is of course one of the primary design requirements of webinos and virtual device drivers the Peer2Peer webinos addressing model and the webinos Overlay network has been created to do exactly that.

Quality of Experience

FIWARE.Epic.I2ND.CDI.QoE

This leaves the quality of experience API, which was not anticipated in webinos. We are currently trying to asssist the FIWARE community port this API onto webinos as a new Plugin API

 

 

This should give you a good summary of where we are on base mapping of FIWARE requirement to webinos implementation.

Webinos of course does quite a bit more than this.

We have more APIs that the Epics allude to: and the reader is recommended to remind themselves of what they are here http://dev.webinos.org/specifications/new/

Also of course what is not mentioned in the FIWARE epic are the fundamental multi-device, cross device requirements which are the core of what webinos tries to achieve.

 

Cloud Edge Extended Development and Management Platform

Now we have looked at what’s happened to date, lets look a little to the future: specifically at the newly released epics which are part of the FIWARE Open Call.

Translating as best I can this means: a router or set top box that sits within a user’s home but can provide services and proxy for service you typically ascribe to cloud entities.

In webinos language this means either

Putting a PZH on a router: in scope

Hosting a PZH on a router: also in scope

Going through the epics in turn.

Client

FIWARE.Epic.Cloud.CPEDM.Client

Somewhat generically named this requirement alludes to

A library that can be used on the client side of a CP-enabled application to interact with the local CP device (discovery, deployment, resource management, etc).

Clearly discovery and deployment are already well addressed by core webinos architecture.

Resource management is more tricky and not yet covered. But an interesting problem to look at.

 

Bundles Catalog

FIWARE.Epic.Cloud.CPEDM.CPBundlesCatalog

Webinos does not support a bundles catalog; it does, however support an application a catalogue. This raises the question what is the difference between and webinos application and a bundle. Not that much it turns out. What you get in webinos is a packaging schema, manifest and certificate scheme for managing integrity and provenance of application (all based on W3C widget packages). Our App Catalogues, contain meta data to help with versioning and compatibility. And the device feature API provides a lot of information to help with negotiation of the right content for the right platform.

The difference in the main is what is inside this package. For applications resident on servers the intent is to host what we call “background applications”. These are applications written entirely in JavaScript, packaged like a widget, that generally autostart with the host devices (e.g. like a windows service/ deamon process) but instead of having and index.html entry point will start running a main.js. These type of applications can run in the cloud, on device, on a cloud proxy or indeed on an IOT device. The ubiquity of the webinos runtime means the applications are entirely portable between these devices.

The scope then is similar to OSGi bundles (this was pointed out by our reviewers a the latest webinos review meeting). We have a work item identified to reconcile OSGi bundles with webinos applications. And indeed a the same time we should review LcX container technology. So summary being this is a productive place to look for alignment, to mutual benefit.

 

App lifecycle

FIWARE.Epic.Cloud.CPEDM.CPvAppLifecycle

The details of lifecycle alignment are somewhat contingent on the nature of any potential alignment between FIWARE bundles and webinos application packages. But its useful to summarise what we already have in webinos.

  • Packaging mechanism with application identities, and addresses well defined
  • Installation mechanism
  • Set of lifecycle events defined that can be intercepted and processed by the general policy frameworks
  • A revocation scheme for rogue applications.

We have our own lifecycle specification

http://www.webinos.org/content/html/D033/Application_Lifecycle.htm

 

 

 

Data Management

FIWARE.Epic.Cloud.CPEDM.CPDataManagement

This is trivial and supported in webinos my application and user data which is automatically synced between Personal Zone Hubs and Personal Zone Proxyies

Local Storage

FIWARE.Epic.Cloud.CPEDM.CPLocalStorage

Again a simple one for webinos. Storage capable devices aare discoverable within zone, and even across zones. Moreover discovery can be local peer2peer (mDNS) or discovery can be across cloud.

A NAS drive or similar can expose many interfaces, but the principle ones for access to files and media are of course

Media libarries: http://dev.webinos.org/specifications/new/MediaContent.html

File access: http://dev.webinos.org/deliverables/wp3/Deliverable32/static$6712249f8a7460e6d8a70f021a267f2d.html

Group Communication

FIWARE.Epic.Cloud.CPEDM.CPGroupCommunication

A service running within the CP which allows the various client devices within the local environment to become aware of each other and to interact between them in a unified way. The service will provide the following capabilities: discovery, membership, messaging, etc.

This Epic simply describes what webinos does. The mapping of default functionality is pretty good as is.

 

Security

FIWARE.Epic.Cloud.CPEDM.CPSecurity

Again the epic describes “Authentication, authorization, control the permissions to access local resources, etc.”

We believe this to be well covered. But again to summarise, this is what is defined in webinos to address the device side and cloud proxy security

  • Application packaging, identity and integrity methods based on W3C Digital Certificates
  • A generic authentication mechanism that has an Open ID implementation but is extensible via Authentication API
  • A universal policy description language designed specifically for web app access to capability, based on XACML
  • The added ability for policies to be synchronised between devices, laying the foundation for multi device applications that can use device and cloud resources seamlessly
  • A PKI architecture for doing distributed user and device identity, Lays the foundation for secure peer to peer services

There are undoubtedly things that have been missed. But there definitely are areas for fruitful exploration of ideas.

 

Advanced Web-based User Interface Generic Enablers

Webinos is an Advanced Web Based User Interface, supported by Generic Enablers. It is reasonable to assume a fair degree of overlap here!

As a quick reminder of the core architecture. Webinos encapsulates all generic enablers on a separate server process on device. For this we use node.js as our host on-device server technology. The core browser extra functions which includes:

  • All API extensions
  • Policy implementations
  • Packaging implementation
  • Session management
  • Synchronisation management
  • Messaging and Routing management

All this functionality resides within this on device server. This method of decoupling extensions from the core browser serves two functions. Firstly it allows the API to be accessible from other devices without having to kick start a heavy webkit browser process. Second, it is simple good architectural practice to separate out the concerns.

The Browser, or rendering engine, must of course still attach itself and communicate with the extension process (PZP). It also needs some minor customisations to handle the pure UI centric extensions. For most platforms we do this using customised Chromium code. The exception being android, but even this may migrate to Chromium shortly.

Now moving to the Epic by Epic analysis

2D-UI

FIWARE.Epic.AdvUI.AdvWebUI.2D-UI

As currently phrased the 2D-UI epic covers to quite different types of functionalities.

The first talks about web components or reusable UI components. This is potentially a very large topic. Concretely, however, within webinos there are two obvious touch points

  • Widgets: webinos is based on widget technology as a foundation. We use widget primary as an app packaging mechanism. But they also are used to create “web components” in a gadget style. Apache Wookie project is a good example of how they can be used to create modular, configurable, reusable components by application developers
  • HTML-Templates : are interesting in a webinos context. Node.js has a number of templating technologies inbuilt. The usual context of use is for this programming paradigm,is to be used be the developer on the server. Because webinos also installs a node.js execution module on device, it raised the interesting possibility of using the same templating technology on client and server. It also raised the possibility for distributed rendering scenario. This functionality is not specifically alluded to in the Epics, but worth consideration, nonetheless

The secondary facet to the 2D-UI Epic is generic and flexible support for novel input devices. This is something already supported quite well in webinos. We have a number of APIs that abstract these inputs already and a flexible framework for adding new ones.

One compelling feature webinos supports, which is not asked for by the Epic specifically, is the ability to use input from one device, as input to another. Even perhaps sharing and input device between multiple devices. This is something that works as standard in the webinos framework by virtue of the remoting capabilities

The Epic also requires abstraction between low level input and high level UI events. This is an issue we have already hit in webinos and have demonstrations that show it working quite effectively

http://www.webinos.org/blog/2012/08/06/meet-webinos-at-ifa-2012-berlin/

http://www.webinos.org/blog/2012/07/02/webinos-demo-series-8-zap-and-shake-web-based-multi-screen-application/

In this scenario a mobile phone is being used as an input device for a TV. The raw input being processed is accelerometer and device orientation data, often coming at high velocity. What the application developer requires is a translation from this raw data to high level UI events (up down etc). In our current implementation this is done through a set of utility JavaScript libraries. The mechanics of how these work are documented more in the DataFlow section.

 

3D-UI

FIWARE.Epic.AdvUI.AdvWebUI.3D-UI

Webinos has no inbuilt support for anything 3D specific. In that we base our implementation on Chromium the developer has a relatively consistent feature set on which they can base work. This has encourage a few developers to experiment a little more using WebGl for games and 3D CSS transforms for more impactful UI -for example see webinos air hockey

http://www.webinos.org/blog/2012/02/25/webinos-demo-series-6-air-hockey-webgl-gaming/ that uses high performance webGL, overlaid high def video, on our TV platform and remote controlled with mobile devices.

Data Flow Processing

FIWARE.Epic.AdvUI.AdvWebUI.DataflowProcessing

When we first looked at the Data Flow Processing Epic is caused a fair amount of head scratching. Having thought about it some time, we have come to the conclusion that we are already doing it :)

With direct reference to the DataFlow Epic, lets work our way up the stack to see how these requirements could be addressed with webinos

Lowest: develop processing blocks

Webinos has a flexible extension regime. An extension is supplied a pluggable node module. This means there must be a JavaScript binding to the core functions. In some instances the implementation may be in JavaScript itself. But in many we will want to drop down to native or hardware optimised functions. This binding for webinos/node is via the V8 native bindings. This means in practice it is very high performance, the JavaScript binding being provided for the sole purpose of making the extension scriptable, with almost no performance impact.

As a case in point for out TV framework we have a need to create a transcoder that converts from MGEPG2-> H264 for web compatibility of content. This is supplied as a plugin module and can, depending on the hardware available run as

  • Software decode/ software encode
  • Hardware decode/ Software encode
  • Hardware decode/ Hardware encode

An API can be exposed in its raw form to the on device browser /renderer or to implement more sophisticated configurability the API can be wrapped around a webinos background application to provide more configurability and pre/post processing options.

Mid Level: requires configurability

There are two methods in webinos to provide configurability of and processing module.

First and most obvious: where the raw API or background process exposes APIs to facilitate parameter settings, and arbitrary tool can be written in HTML/JavaScript to help configure these parameters.

The second option available is to piggy back on the WebIntents architecture. This is a generic call back mechanism that allows a user to “configure” any component in a context sensitive way. This is a very loose couple mechanism of configuration, and provides a mechanism for a generic web application to call back to some deep hardware specific, deeply platform integrated configuration tools

Highest level: processing building blocks

Finally, how in webinos do we define the mechanism for creating high level building blocks from processing units? After some consideration we came to the conclusion that we were already doing this. Again our Zap and Shake application gives a concrete example of how its done.

The general model is this: a JavaScript object is created (possibly a full background application) which registers for events. These events may be the stream of accelerometer data changes for example. This object processes these “inputs” (either in hardware, native code or JavaScript) and generates a higher level analysis/higher level events. Therefore by setting up a set of daisy-chaned JavaScript objects that listed to each other’s higher level events, we have created a full DataFlow architecture than can comprise a graph of arbitrary complexity.

Importantly for webinos, because these listeners can be setup between client and cloud also peer 2 peer devices, this chain can be implemented in a highly distrusted fashion, using built in webinos features.

Concretely then for FIWARE, to piggy back on the existing technology and to address this Epic simply, it is a simple matter of defining a set of general purpose hardware optimised processing APIs that need wrapping up as webinos extensions.

The same model can be (in fact currently is) being applied to hardware optimised video processing chains.

When considering cross device (network) data stream and sinks there are some other subtle issues to consider. How do deal with real time streams effectively? How can we control flow and throttle stream? In case of overload, how do we drain the back pressure skip the stream?

Fortunately for us people have been looking at this issues for quite some time, and for us in webinos we can piggy back on some of the excellent work on going in node to address this.

https://github.com/isaacs/readable-stream

 

Cloud Rendering

FIWARE.Epic.AdvUI.AdvWebUI.CloudRendering

This is a feature very well suited to webinos capabilities. If the requisite rendering elements have been encapsulated as webinos plugins, the remoting capability comes very quickly, in that all the optimisation, addressing, messaging , invocation, and security issues are addressed.

There already exist coarse grained wholesale 3d and 2d rendereres

Finer grained hardware optimised distributed rendring can be enabled by encapsulating the right sub processing elements

Display as a Service

FIWARE.Epic.AdvUI.AdvWebUI.DisplayAsAService

We can address this in webinos at two levels

First we have a new API defined http://dev.webinos.org/specifications/new/remoteui.html which as the specification states: provides means to create and access a UI on a remote device using DOM manipulation commands.

Secondly, and somewhat as a suprise even to us. It is possible to address a large chunk of the functionality implied here as a simple third part application. Our recently ran application competition uncovered two such applications created by some very creative app developers.

AppChallenge – Webinos Tiled Image Surface http://www.youtube.com/watch?v=CO1mEXBOUGY

AppChallenge – Multi-Screen Maps http://www.youtube.com/watch?v=rNRcYMsfi9Q

These are real world applications that show device as a service in action. Watch the video closely, see how the virtual display is dynamically created through some innovative binding gestures.

Obviously not everything is solved. Some optimisation is necessary to handle screen synchronisation, data updates, etc to avoid tearing. But this should give a good idea of what is possible

This list constitutes a good first level analysis of the mapping from webinos to the requirements implied by FIWARE Advance UI Epics.

The final table summarised those mappings that are more trivial

Scene API Non webinos specific. Can be done at a coarse level though 3DXML, O3D or Collada. To support highly optimised scene sycn between devices may require a binary representation to be defined.
Synchronisation Base functionality supported as standard in webinos
Adaptable UI Currently looking at Enyo for this functionality
2D-3D capture In main supported through http://dev.w3.org/2011/webrtc/editor/getusermedia.html but may need new APIs for new devices. Image compositing handled by functionality defined elsewhere
POI Simple JavaScript library encapsulation of 3rd party source. Can use generic data flow scheme for filtering and searching

 

Conclusions

This list is a fairly arbitrary sample to demonstrate how webinos technology related to some other advance web technologies in production. As an open source project we always looking for new collaborations and opportunities to save effort.

This quick open analysis, will hopefully help stimulate further discussion and greater understanding of the opportunities that we might have going forward

Comments as always more than welcome