Like many EU projects, webinos is strongly encouraged to seek out collaboration opportunities with other funded objectives. This makes eminent sense as this is public money we are using here, and it is in everyone’s interests to see this invested intelligently and without unnecessary duplication.
We have engaged in a number of these type of initiatives over the past years, and will do more so in the future. In an attempt to document this in a simple publicly accessible form, we shall run a series of blogs documenting what is going on, and what might happen in the future.
In the first of this series we shall focus on FI-WARE. This is a very large project, and as its acronym implies (Future Internet) somewhat ambitions. We shall deal with this in three parts. Part one, document what has happened to date. Part two, look at the very closely affiliated Cloud Edge Management Open Call and finally examine the Advanced WebUI set of Epics
Summary of Activity to date
FIWAREs remit is large. It defines a series of Generic Enablers that attempt to make it easier to develop Future Internet Applications, acting as foundations. The activity has been divided into a number of Chapters, of which the most relevant to core webinos activity is Interface to Networks and Devices (I2ND) in FI-WARE http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/Materializing_the_Interface_to_Networks_and_Devices_%28I2ND%29_in_FI-WARE
First thing to notice if FI-WARE lists Web Runtime and Web APIs are base line assets. Lets look in turn
- A web runtime is a primary deliverable from webinos: webinos has a runtime. This is fully compliant with W3C standards. And they do implement some of the WAC APIs .These assets are publicly available under open source licenses.
WebAPIs: in summary webinos has 28 APIs defined. We will list below the exact mapping. It is interesting to note that the technology listed in FIWARE for WebAPI integration is NPAPI. This is a technique that we did use (4 years ago) when working on the earlier BONDI project. However we found it limiting. For the webinos use cases that require interdevice communication we have preferred a more loosely coupled architecture where the API host runs on a standalone server (node.js). The merits of these browser decoupled method of browse extensions have been discussed in earlier blogs.
The combination of WRT and API in webinos is supported on Windows, Mac , Linux, Android 2.2 -> 4.1, plus more recently ChromeOS.
When it comes to actual APIs its probably easier if we analyse them Epic by Epic (Epic is the term FIWARE uses for its coarse grained requirements)
Read access to device connectivity is supported in webinos through the device status API and the vocabulary it exposes http://dev.webinos.org/specifications/new/devicestatus.html
The issue of how to change the connection status is more subtle. On the platform we architecturally encapsulate the Connection Manager. The issue of who can determine what connection to use (is it the application or is the user) is not simple, and needs to properly modelled in the policy framework.
Is trivial and well supported in the existing http://dev.webinos.org/specifications/new/devicestatus.html
Is well supported by our newly updated Generic Sensor API http://dev.webinos.org/specifications/new/sensors.html
Personal Data Services
Covers PIM – Calendar, Contacts, Tasks, supported by
and pictures, videos, supported by
The Epic also mentions access to Applications. This is partially supported by
More fine grained access is possible in the webinos framework, and code exists to access it, but due to security considerations it is not exposed as a third party API
Is a strangely worded Epic. In terms of raw access to user data our first version of the APIs contained references to a specific UserProfiel API http://dev.webinos.org/deliverables/wp3/Deliverable32/static$02bf0c58dd455d51f29dafc0d5ed1392.html
This specification is currently being reworked to address design flaws
The other issues outlined in this Epic are more all pervading
“I want to access to my applications through secure and easy to use identification and authentication procedures, and with consistent privacy policies supported.”
And are addressed very completely by the webinos architecture, but you need to look a the entire architecture holistically to understand how it works: how the PKI infrastructure interacts with policy and interacts with platform attestation.
The Epic as worded is quite narrow and again is addressed with
Perhaps with a few additions to the vocabulary
But to address the full spirit of Media interaction you really need to look at the full suite of APIs available
Is one of the few APIs not currently addressed in webinos , but planned for imminent implementation
One to one mapping with http://dev.webinos.org/specifications/new/messaging.html
We can break down into its two components
- Discover and address a remote device: this is trivially addressed by the current webinos stack
- Remotely configure one or multiple remote device: is not so simple. There are many control issues that need to be addressed here. Who has the right to do what? The consumer centric nature of webinos, asserts that all eventual control resides with the end user. In that sense webinos does fit with this requirement, in that is supports the synchronisation infrastructure to propagate changes between hubs and devices. A little more work is required however on actually what the typical remote configuration use cases are for end users.
Is largely about
“Application developers need to access proximity interfaces in a way as much as possible abstracted from their specific radio technologies (NFC, ZigBee, others…) and also from the specific device brand, model and OS.”
This is of course one of the primary design requirements of webinos and virtual device drivers the Peer2Peer webinos addressing model and the webinos Overlay network has been created to do exactly that.
Quality of Experience
This leaves the quality of experience API, which was not anticipated in webinos. We are currently trying to asssist the FIWARE community port this API onto webinos as a new Plugin API
This should give you a good summary of where we are on base mapping of FIWARE requirement to webinos implementation.
Webinos of course does quite a bit more than this.
We have more APIs that the Epics allude to: and the reader is recommended to remind themselves of what they are here http://dev.webinos.org/specifications/new/
Also of course what is not mentioned in the FIWARE epic are the fundamental multi-device, cross device requirements which are the core of what webinos tries to achieve.
Cloud Edge Extended Development and Management Platform
Now we have looked at what’s happened to date, lets look a little to the future: specifically at the newly released epics which are part of the FIWARE Open Call.
Translating as best I can this means: a router or set top box that sits within a user’s home but can provide services and proxy for service you typically ascribe to cloud entities.
In webinos language this means either
Putting a PZH on a router: in scope
Hosting a PZH on a router: also in scope
Going through the epics in turn.
Somewhat generically named this requirement alludes to
A library that can be used on the client side of a CP-enabled application to interact with the local CP device (discovery, deployment, resource management, etc).
Clearly discovery and deployment are already well addressed by core webinos architecture.
Resource management is more tricky and not yet covered. But an interesting problem to look at.
Webinos does not support a bundles catalog; it does, however support an application a catalogue. This raises the question what is the difference between and webinos application and a bundle. Not that much it turns out. What you get in webinos is a packaging schema, manifest and certificate scheme for managing integrity and provenance of application (all based on W3C widget packages). Our App Catalogues, contain meta data to help with versioning and compatibility. And the device feature API provides a lot of information to help with negotiation of the right content for the right platform.
The scope then is similar to OSGi bundles (this was pointed out by our reviewers a the latest webinos review meeting). We have a work item identified to reconcile OSGi bundles with webinos applications. And indeed a the same time we should review LcX container technology. So summary being this is a productive place to look for alignment, to mutual benefit.
The details of lifecycle alignment are somewhat contingent on the nature of any potential alignment between FIWARE bundles and webinos application packages. But its useful to summarise what we already have in webinos.
- Packaging mechanism with application identities, and addresses well defined
- Installation mechanism
- Set of lifecycle events defined that can be intercepted and processed by the general policy frameworks
- A revocation scheme for rogue applications.
We have our own lifecycle specification
This is trivial and supported in webinos my application and user data which is automatically synced between Personal Zone Hubs and Personal Zone Proxyies
Again a simple one for webinos. Storage capable devices aare discoverable within zone, and even across zones. Moreover discovery can be local peer2peer (mDNS) or discovery can be across cloud.
A NAS drive or similar can expose many interfaces, but the principle ones for access to files and media are of course
Media libarries: http://dev.webinos.org/specifications/new/MediaContent.html
A service running within the CP which allows the various client devices within the local environment to become aware of each other and to interact between them in a unified way. The service will provide the following capabilities: discovery, membership, messaging, etc.
This Epic simply describes what webinos does. The mapping of default functionality is pretty good as is.
Again the epic describes “Authentication, authorization, control the permissions to access local resources, etc.”
We believe this to be well covered. But again to summarise, this is what is defined in webinos to address the device side and cloud proxy security
- Application packaging, identity and integrity methods based on W3C Digital Certificates
- A generic authentication mechanism that has an Open ID implementation but is extensible via Authentication API
- A universal policy description language designed specifically for web app access to capability, based on XACML
- The added ability for policies to be synchronised between devices, laying the foundation for multi device applications that can use device and cloud resources seamlessly
- A PKI architecture for doing distributed user and device identity, Lays the foundation for secure peer to peer services
There are undoubtedly things that have been missed. But there definitely are areas for fruitful exploration of ideas.
Advanced Web-based User Interface Generic Enablers
Webinos is an Advanced Web Based User Interface, supported by Generic Enablers. It is reasonable to assume a fair degree of overlap here!
As a quick reminder of the core architecture. Webinos encapsulates all generic enablers on a separate server process on device. For this we use node.js as our host on-device server technology. The core browser extra functions which includes:
- All API extensions
- Policy implementations
- Packaging implementation
- Session management
- Synchronisation management
- Messaging and Routing management
All this functionality resides within this on device server. This method of decoupling extensions from the core browser serves two functions. Firstly it allows the API to be accessible from other devices without having to kick start a heavy webkit browser process. Second, it is simple good architectural practice to separate out the concerns.
The Browser, or rendering engine, must of course still attach itself and communicate with the extension process (PZP). It also needs some minor customisations to handle the pure UI centric extensions. For most platforms we do this using customised Chromium code. The exception being android, but even this may migrate to Chromium shortly.
Now moving to the Epic by Epic analysis
As currently phrased the 2D-UI epic covers to quite different types of functionalities.
The first talks about web components or reusable UI components. This is potentially a very large topic. Concretely, however, within webinos there are two obvious touch points
- Widgets: webinos is based on widget technology as a foundation. We use widget primary as an app packaging mechanism. But they also are used to create “web components” in a gadget style. Apache Wookie project is a good example of how they can be used to create modular, configurable, reusable components by application developers
- HTML-Templates : are interesting in a webinos context. Node.js has a number of templating technologies inbuilt. The usual context of use is for this programming paradigm,is to be used be the developer on the server. Because webinos also installs a node.js execution module on device, it raised the interesting possibility of using the same templating technology on client and server. It also raised the possibility for distributed rendering scenario. This functionality is not specifically alluded to in the Epics, but worth consideration, nonetheless
The secondary facet to the 2D-UI Epic is generic and flexible support for novel input devices. This is something already supported quite well in webinos. We have a number of APIs that abstract these inputs already and a flexible framework for adding new ones.
One compelling feature webinos supports, which is not asked for by the Epic specifically, is the ability to use input from one device, as input to another. Even perhaps sharing and input device between multiple devices. This is something that works as standard in the webinos framework by virtue of the remoting capabilities
The Epic also requires abstraction between low level input and high level UI events. This is an issue we have already hit in webinos and have demonstrations that show it working quite effectively
Webinos has no inbuilt support for anything 3D specific. In that we base our implementation on Chromium the developer has a relatively consistent feature set on which they can base work. This has encourage a few developers to experiment a little more using WebGl for games and 3D CSS transforms for more impactful UI -for example see webinos air hockey
http://www.webinos.org/blog/2012/02/25/webinos-demo-series-6-air-hockey-webgl-gaming/ that uses high performance webGL, overlaid high def video, on our TV platform and remote controlled with mobile devices.
Data Flow Processing
When we first looked at the Data Flow Processing Epic is caused a fair amount of head scratching. Having thought about it some time, we have come to the conclusion that we are already doing it
With direct reference to the DataFlow Epic, lets work our way up the stack to see how these requirements could be addressed with webinos
Lowest: develop processing blocks
As a case in point for out TV framework we have a need to create a transcoder that converts from MGEPG2-> H264 for web compatibility of content. This is supplied as a plugin module and can, depending on the hardware available run as
- Software decode/ software encode
- Hardware decode/ Software encode
- Hardware decode/ Hardware encode
An API can be exposed in its raw form to the on device browser /renderer or to implement more sophisticated configurability the API can be wrapped around a webinos background application to provide more configurability and pre/post processing options.
Mid Level: requires configurability
There are two methods in webinos to provide configurability of and processing module.
The second option available is to piggy back on the WebIntents architecture. This is a generic call back mechanism that allows a user to “configure” any component in a context sensitive way. This is a very loose couple mechanism of configuration, and provides a mechanism for a generic web application to call back to some deep hardware specific, deeply platform integrated configuration tools
Highest level: processing building blocks
Finally, how in webinos do we define the mechanism for creating high level building blocks from processing units? After some consideration we came to the conclusion that we were already doing this. Again our Zap and Shake application gives a concrete example of how its done.
Importantly for webinos, because these listeners can be setup between client and cloud also peer 2 peer devices, this chain can be implemented in a highly distrusted fashion, using built in webinos features.
Concretely then for FIWARE, to piggy back on the existing technology and to address this Epic simply, it is a simple matter of defining a set of general purpose hardware optimised processing APIs that need wrapping up as webinos extensions.
The same model can be (in fact currently is) being applied to hardware optimised video processing chains.
When considering cross device (network) data stream and sinks there are some other subtle issues to consider. How do deal with real time streams effectively? How can we control flow and throttle stream? In case of overload, how do we drain the back pressure skip the stream?
Fortunately for us people have been looking at this issues for quite some time, and for us in webinos we can piggy back on some of the excellent work on going in node to address this.
This is a feature very well suited to webinos capabilities. If the requisite rendering elements have been encapsulated as webinos plugins, the remoting capability comes very quickly, in that all the optimisation, addressing, messaging , invocation, and security issues are addressed.
There already exist coarse grained wholesale 3d and 2d rendereres
Finer grained hardware optimised distributed rendring can be enabled by encapsulating the right sub processing elements
Display as a Service
We can address this in webinos at two levels
First we have a new API defined http://dev.webinos.org/specifications/new/remoteui.html which as the specification states: provides means to create and access a UI on a remote device using DOM manipulation commands.
Secondly, and somewhat as a suprise even to us. It is possible to address a large chunk of the functionality implied here as a simple third part application. Our recently ran application competition uncovered two such applications created by some very creative app developers.
AppChallenge – Webinos Tiled Image Surface http://www.youtube.com/watch?v=CO1mEXBOUGY
AppChallenge – Multi-Screen Maps http://www.youtube.com/watch?v=rNRcYMsfi9Q
These are real world applications that show device as a service in action. Watch the video closely, see how the virtual display is dynamically created through some innovative binding gestures.
Obviously not everything is solved. Some optimisation is necessary to handle screen synchronisation, data updates, etc to avoid tearing. But this should give a good idea of what is possible
This list constitutes a good first level analysis of the mapping from webinos to the requirements implied by FIWARE Advance UI Epics.
The final table summarised those mappings that are more trivial
|Scene API||Non webinos specific. Can be done at a coarse level though 3DXML, O3D or Collada. To support highly optimised scene sycn between devices may require a binary representation to be defined.|
|Synchronisation||Base functionality supported as standard in webinos|
|Adaptable UI||Currently looking at Enyo for this functionality|
|2D-3D capture||In main supported through http://dev.w3.org/2011/webrtc/editor/getusermedia.html but may need new APIs for new devices. Image compositing handled by functionality defined elsewhere|
This list is a fairly arbitrary sample to demonstrate how webinos technology related to some other advance web technologies in production. As an open source project we always looking for new collaborations and opportunities to save effort.
This quick open analysis, will hopefully help stimulate further discussion and greater understanding of the opportunities that we might have going forward
Comments as always more than welcome