Overview

The Visual Lambda Application Engine is the brain of the QWHA Controller. It is easy to use, easy to develop on, and can be as smart as one can possibly imagine.

Introduction

The Application Engine, codenamed Visual Lambda, offers the ultimate flexibility to end users to customize their home automation system.

The application engine can execute any possible control logic based on arbitrarily complex input information. End users doesn't need to do any programming. Typically end users only need to search functions from Application Store, and then schedule a task by feeding their own set of input to the function.

Examples

Let's go over some examples.

Motion Sensor Controlled Light

Motion sensor controlled light is one of typical smart home setups. Almost all IoT protocol supports the concept of binding, or linking, where one device can be linked to another device so that the event of the master device can be sent to slave device to trigger actions on slave device.

In this case, a motion sensor can bind, or link to a light. When the motion sensor is triggered, it may send a "on" signal to the linked light to turn the ligh on. In this way, the system will work even without the controller.

However, there are several limitations with the simple binding/linking method. What if the motion sensor and light are running different protocols? More importantly, what if we need two motion sensors to control the same set of lights? Simple binding/linking can no longer support setup like that.

Multiple Motion Sensor Controlled Group of Lights

Suppose you have a large room, or a room with multiple entrances. One motion sensor won't cover the entire area.

Or you may want to turn on more than one light when motion is detected.

QWHA provides a such an application function. Let's try to set up:

First, navigate to "Controller" -> "Visual Lambda" -> "Tasks" to open the tasks tab. Click "Manage" -> "Add" from upper toolbar as shown in screenshot below.

In QWHA, user creates tasks to perform control logic. Task is based on selected application function, with user provided extra information, called parameters. Parameters can be quite complex but it should usually follow common sense naturally.

Later on the user interface can be improved over time to provide more user friendly look & feel, even behavior. The task UI can be quite different on different client devices, for example, web page and smartphone.

For now, let's stick to the web UI.

Let's find the function called "Motion Sensor triggered Lights". then click "Next" button.

We need to customize the input parameters for the task. In this example, we added two motion sensors to "Motion Sensor List", "FamilyRoomMotion1" and "FamilyRoomMotion2".

We add one light to the "Light List", "FamilyRoomStair".

The logic is pretty simple, whenever a motion sensor detects a motion, the "FamilyRoomStair" light will be turned on.

When both motion sensors reports "no more motion detected", the light wil be turned off.

Note in the UI, when we add devices into the lists. We simply choose a device from all available devices of the correct type. For example, when we add a new motion sensor, we pick a motion sensor from the drop down list of all available motion sensors. It will very difficult for user to make mistake.

Finally we give the task a name, any name, and some detail descriptions.

Localization Made Easy

Ever wonder how the procedure looks like in a different language? Below is a screen shot of the same task in Simplified Chinese language. Of course we assume the user uses Chinese characters to name the devices. Nevertheless, the entire UI is turned into Chinese, including the parameter names, the detail description as help tip.

Later on we will show how easy it is to go international.

Motion Sensor Controlled Light - With Daylight Check

The previous example will turn on the light when ever the motion is detected. It is OK to have such setup for a room that is always dark, for example, a garage that has no window.

What if the room has windows? Users may want to only turn on the light after sunset and before sunrise.

Fortunately, QWHA is capable of tracing the sun position in real time. Also sunset and sunrise times can be calculated through API.

QWHA provides an application, "Motion Sensor Triggered Lights with Daytime Check", which does exactly the job.

Note user may want to adjust the actual sunset and sunrise time by plus or minus some minutes, based on the orientation of the window and terrain of the building.

Motion Sensor Controlled Light - with Ambient Light Sensor

The lighting condition inside the room not only depends on the sun position, but also the weather. A more accurate measurement can be achieved with an ambient light sensor.

A third application can be built to take a list of motion sensors, a list of lights and an ambient light sensor.

Motion Sensor Controlled Light - What Else can We Do?

What if the application integrates home security features? When user leaves home he/she can just push one button to arm the security. So that whenever the motion sensor is triggered, instead of controlling the light, it send the alarm to both user and authorities.

Motion Sensor Controlled Light - Use Your Wildest Imagination

Is it all we can do? Not even close!

Other application can be built to integrate other devices such as remotely controled shades and thermostat so that the room can automatically adjust to user's preference when user presence is detected.

Even for the lighting control. Considering a bedroom. User may want to automatically turn on the light when user enters the room at night. However, when user decides to sleep, user may want to turn off the light and the light shouldn't automatically turn on again even though user stays in the room.

So the bedroom motion controlled light shall have a feature that when user explicitly turns off the light, the light shall not be automatically turned on for the rest of the night.

The requirement for the bedroom will most likely be different that that of other rooms, such as garage.

App Engine and App Store

The motion sensors controlled lights examples in the previous chapter illustrates how complicated the real world requirement could be.

Unfortunately, while people keep talking about autonomous cars and intelligent robotics, some scenarios I depicted in the examples can still only be seen in Sci-fi movies.

As of right now, there are many smart home system in market right now. However, in order to implement any imaginable control logic, those systems are either too feature limited or too difficult to develop.

QWHA Application Engine takes a minimalistic design approach from ground up. The design achieved two goals.

  • It can perform any imaginable logic, no matter how complicated the logic is. This is functional and logical completeness that can be proved in theoretical computer science.
  • Our design is the optimal way for user to achieve maximum level of flexibility, even though the user interface may always have room to be improved.

Application Store

Application store is an online repository of all supported applications for our controller.

Developer can publish the application to Application Store by uploading application files to application store after the application is tested locally.

An application is consists a set of functions (one or more). Application is also called "Application Package" or "Application Module", or "Package", or "Module".

Users can search the application they need from application store. Once an application is found, user can download application to local controller and create tasks based on the application function.

Application store offers intelligent search to help user find application by combining the user input keywords, application documentation, applicaton input parameters, user comments/discussions and user ratings.

Local Application Store on Controller

Copies of applications downloaded by user are stored locally in Controller. Multiple versions of the same application may exists in Controller locally. Controller may automatically manage and delete unused historical versions periodically.

If user is also a developer, user developed applications may only exist in local store for testing. Once the testing is done, user may decide to publish the version to Application Store.

Application is Everything

Our QWHA Controller manages the difference devices and sensors in the home.

There are some simple control logics in controller, for example, simple device linking, software scene and timers.

Any other type of more complex control logic shall be performed by using different Applications.

The controller creates Tasks based on specific Application Function. A Task is an instance of an Application with specific user defined set of input parameters.

User creates Task using the Task Window, as illustrated in "Motion Sensor Controlled Light" example.

Task Parameters and Data Types

A task can take input parameters of very flexible parameters.

  • Number and String
  • DateTime and TimeOnly
  • Enumeration type
  • IoT hub related types, such as devices of various types. For example, a "Light" type, or more specific a "Light Dimmable", which only takes dimmable lights
  • A Table. Table may have one or more members. Each member is identified by a name. Each member also has a value. Value can be of any type
  • A List. List is an ordered collection of data of a specific type. List member can be of any type.
  • Apart from the above types. Any data may be nullable. For example if a data entry of number type is nullable, the data value can be either a number or a special "null" value.

Sounds complicated? As a matter of fact, the task GUI takes care of most complexity for end users. As a result user just need to fill in the blanks in the UI, except for the situations that user may need to manage members of a list "add or remove". The GUI can be made very intuitive for list management as well.

Below is a sample task instance that demonstrates the used of web UI for different data types.

Data Initialization Rule

  • Nullable types are initialized as null
  • Number type is initialized as default (usually 0)
  • String is initialized as default (usually empty string)
  • List is initialized as an empty list, if not nullable; otherwise as null
  • If not nullable, table is automatically populated, i.e. all members are automatically created and initialized following exactly this rule recursively
  • Some derived types (such as devices or DateTime), even though not nullable, even if they are actually primitive types natively (for example, “device” type is represented by an ID, date and time are represented as seconds since Unix epoch, enumeration type is represented as number), are initialized as null. If end user didn’t assign a valid value, the instance won’t pass validity check and the data can’t be submitted

Number and String

Number and string types can be edited in-place, as shown in screenshot below:

Number:

String:

DateTime and TimeOnly

DateTime and TimeOnly may have user friendly chooser UI:

DateTime:

TimeOnly:

Devices

Devices should prompt user with user friendly chooser UI for user to choose from all available values in the system:

In order to create a member of a list, select the list, then press the “Add” button. A new value will be added as a child of the list. The new value will be initialized (or even recursively populated if the list of a list of non-nullable table), following the initialization rule stated above.

List and Table

List members and table members can be edited:

List:

Table:

Enumeration

Enumeration type shall prompt user with all available values:

Instance from Null Value

For nullable type and the value is null, user need to first create the value (will be automatically initialized following the rule) before editing it. First select the entry, then click the button to create the value. In the screenshot below, the editor even knows the type of value to be created, it will then change the text and icon of the button (change text into “List”, “Table” or “Value” based on type):

List:

Table:

Reset to Null

A nullable data can be reset to null by clicking a button:

Flexible UI

The UI on smart phone may be quite different than Web UI in term of both look and feel and behavior. We will make specific UI to provide optimal user experience on different devices.

While the architectural design of the UI framework can be proved to be optimal. We still have to keep making improvement to the implementation on different devices for better user experience over time.

Managing Tasks

We have already learnt how to create a new Task in the previous examples. Now let's try some other task management features.

Open the task panel, select a task in the list. Then click "Manage" button from toolbar.

Below is the screenshot of Task Manager GUI tool.

Apart from adding a new Task, user can remove the selected task, or edit the parameters of the selected task or create a new task by cloning the selected task.

Task Application Versions

In tool bar, click "Manage" -> "Version" will list all versions of the Application Package the selected task is based on.

Note the current version is automatically selected at the time the version dialog pops up. The current version also has a different icon than other versions. For more details about the icons, please refer to "Task Status Icons".

Please also note that the current version doesn't necessarily have to the latest version. The philosophy of our design is: if it works, do not change it. So it is perfectly OK for user not to upgrade the task to the latest version of the application.

Enable and Disable Tasks

User can enable or disable a task at any time. All user needs to do is simply selecting the task from the list, if the task is already enabled, the button in tool bar will be "Disable"; otherwise the button will be "Enable".

Indirectly Enable Disable Tasks

Sometimes, tasks are disabled because the underlying application package failed to load. Most likely it is cause by a problematic application module. Once the application is updated, for safety reason the application will remain disabled.

Once user starts to enable a task caused by application failure, a dialog will pop up to let user enable the application instead. User will also be able to see the dependency tree that shows how many other tasks depend on this version of application and how enabling the application is going to affect the status of those tasks.

Trouble Shooting Tasks

Application code may cause runtime errors, especially for the code that is deployed for testing. There are two different kinds of failures:

  • The application fails in the load process. In this case the application can't even be loaded. All tasks that dependent on the application will be disabled.
  • The function that the task depends on caused some runtime errors. In this case only the tasks dependent on the function will be disabled.

In either cases, the task entry will show an error icon , which is slightly different than disabled icon .

Task Status Icons

Below is a list of all status icons use for applications in Lua programming language.

  • Enabled/Production Task
  • Enabled/Production Task - Not latest Version
  • Task in Error Mode
  • Task in Disabled Mode
  • Task in Error Mode - Not latest Version
  • Task in Disabled Mode - Not latest Version

Application Manager

User can manage downloaded applications (or developed applications) using Application Manager.

To open Application Manager, navigate to "Visual Lambda" -> "Applications".

below is screenshot of the application manager GUI.

Application Name and Version

Application owner is also called developer. Developer is uniquely identified with developer name.

Developer gives the application file a unique name (within developer's scope). The file name is also called package name.

User shouldn't care about the application file or package name. Each package can also have a display name. Display name is user friendly name.

An application can have one display name per localized language. For example, one display name in English, one display name in Chinese etc. Display name can be modified at any time by developer or third party authorized by developer to maintain the application documentation.

Application Functions

Application package may contain one or more functions. Function is the basic building block of Tasks.

Function may have arbitrary number of input parameters. Each parameter can be arbitrarily complex data structure. The design gives function ultimate flexibility to deal with any complex control logic.

Application Documentation

Each application, each function, each parameter, and each node of the parameter data structure tree (data structure is expressed as a tree structure) all have extra information called documentation. Documentation is user friendly text that describes the object it is associated with. Documentation is used in Task Graphic User Interface tool.

  • Documentation is used as text description, for example, field name, detail help tips etc.
  • Documentation can also be used to customize the look & feel, and behaviour of the Task GUI (or part of task GUI).

Application can have multiple documentation in different localized languages. For example, the application in the screenshot above has two locales, English and Chinese.

Application can even be improved and translated after the application is published, by third party other than the application developer. This crowd source design will help promote the application and encourage feedback from end users.

Managing Applications

User can browse the list of application in local store. From the tree view on the upper left corner. The applications are grouped by developers.

Select an application from the list, in "Function" panel the functions in the application will be listed. The first item is always the application itself. Selecting that node will display additional properties of the application in the right panel.

Function Description

Selecting a function will display the description of the function in the right panel. Function description is a combination of function signature and function documentation.

Function signature is the list of input parameters of the function and the type of each input parameter. Type can be arbitrarily complex tree structure.

Function documentation is additional information on each parameter or type node. Documentation contains information only related to user GUI, which is used for text display or customization of look & feel or behavior of the GUI.

Enable/Disable Application

User can enable/disable the currently selected application. This function can be used to control all tasks based on the selected application.

Note the enable/disable may get a little bit complicated because there may be multiple versions of the same applications running. Some tasks may be based on one version and some other tasks may be based on another version.

The dialog will list all tasks that depend on the application, grouped by versions. User can choose to enable/disable specific version of application or individual tasks.

Once the user click OK, the new enable/disable status will be sent to the Controller.

Application Version

Choose "Manage"->"History", the version dialog will pop up.

Version dialog will list all versions of the selected application in local store. User can change the "Current Version" to a different version.

After the current version is changed, the Admin tool will scan all available tasks, find tasks that use the selected application (and compatible with the current version) and prompt for a batchversion upgrade.

Note the current application version is only a recommendation. User can choose to user any compatible version with any individual task. it is however recommended that user always stick to the latest version. Current version does change the icon of tasks that depend on old versions as a reminder.

App Manager Icons

Below is a list of all status icons use for applications manager.

Note an application can be marks as library by developer. If an application package is marked as library, it cannot be used by users (to create tasks). Library can only be used by other applications.

Library has a slightly different icon.

Note an application can be both in production and development state. Imagine user is also a developer. The developer may check out a production version for further development. In this case the development copy will be a new version and while the development version is being modified, the production version can keep running in production mode.

Application can be disabled. When application is disabled, all related tasks are also automatically disabled.

If application failed to load, it will be marked as "error". When this happens, it usually indicates some serious problem in the code. If user still wants to use the application, a version upgrade/downgrade is recommended.

  • App Enabled/Production
  • Lib Production
  • App Production & Development
  • Lib Production & Development
  • App Disabled
  • App Error
  • App Disabled & Development
  • App Error & Development
  • Lib Disabled
  • Lib Error
  • Lib Disabled & Development
  • Lib Error & Development

Help Improving the Documentation

User can navigate to "Contribute" -> "Edit" to edit the documentation.

Help Translating the Documentation

User can navigate to "Contribute" -> "Translate" to translate the documentation into another language, for example, from English to Chinese. The UI will copy the documentation from the current locale and ask user to do "in-place" translation.

Other Application Examples

In QWHA, Application is everything. We depend on the developer community and user community to build a growing smart home ecosystem.

Imagine a thermostat application that running on a variety of thermostat hardware, developed by community and can be smarter than the Nest thermostat.

Imagine a smart grid cloud connected application running on the controller that can help reduce carbon emission, lower cost for both power plant and consumer, and can be both secure and user privacy preserving.

There are a a couple of other simple example applications included in the default installation.

Christmas Light Controller

Christmas light controller can be used to control fairly complex lighting sequences.

The process can be defines as finte steps, each step there are a group of lights, each light can be individually assign an on-level. On-level of 0 means off, and on-level of 255 means fully turned on.

At the end of each group, there will be wait time. The process will wait for specified time before processing the next group. Once the last group is processed, the process goes back to process the first group again.

Once started, the task will keep looping forever until it is stopped.

Below is a step by step set up of a control sequence as following:

  1. Turn on 3 lights, “Outdoor Christmas Tree Light”, “Outdoor Puppy Light” and “Outdoor Deer Light”. Note the “onLevel” of 255 means fully turn on
  2. Wait for 10 seconds (10000 milliseconds)
  3. Turn off 2 lights, “Outdoor Puppy Light” and “Outdoor Deer Light”. Note the “onLevel” of 0 means fully turn off
  4. Wait for 10 seconds (10000 milliseconds)
  5. Turn off one light “Outdoor Christmas tree Light”
  6. Wait for 10 seconds (10000 milliseconds)
  7. Repeat from step 1

The Christmas Light Controller also comes with a Chinese version of documentation. below is the screenshot of the Chinese UI. Note everything turned into Chinese. Internationalization has never been easier!

7 Day Programmerable Thermostat

Screenshot below is our version of the 7 day programmable thermostat.

It appears to be a fairly complicated set up. But it is really very easy to understand.

You think it is too complicated? Well take a look at the Honeywell 7 day programmable thermostat. And imagine the pain to set that thing up!

Please keep in mind that our version is actually more flexible and thus more powerful. You can even give your own scheduler a name and give each action a name, too.

When used in Chinese UI, the emtire task UI is turned into Chinese, automatically, because we have a Chinese version of documentation for this function.

Making Contribution

The Application Engine is crowd source enabled that allows everybody to make contributions to the user community.

Even a regular user can help making contributions by improving the documentation or translating the documentation into other languages.

Privacy and Security

The entire Controller/Hub and App Engine is built with privacy and security in mind from very beginning.

The security model of the App Engine provides maximum privacy protection for users without sacrificing system flexibility and functionality.

Creating a User Account in the Application Store

The App Store will be open to public soon. Stay tuned.

Getting Permission from Application Owner

The App Store will be open to public soon. Stay tuned.

Making Contribution and Commit Changes to App Store

Any end user can make changes to App code (if it is published in source code form) and documentation. With permission from App owner. Third party can participate in App development and documentation maintenance by publishing the changes to App Store and share with the entire world.