Overview

Visual Lambda App Engine is an OS for Internet of Things. Like IOS is an OS for Apple iPhones, and Google Android is an OS for some other smart phones, our App Engine is an OS for smart home and Internet of Things.

We released the full fledged development tools free of charge to the whole world, including IDE, source control tool and deployment and testing tool. Our development environment minimizes the effort of development, deployment and publishing of applications.

App Store is a free service for around the world to share applications.

Building Home Automation Systems that Actually Works

Home automation is one popular application of Internet of Things. While good home automation often appear in Sci-Fi movies, in reality we only got bad ones, those do work but never work well. By my definition, they don't work. I will explain why. And, most importantly, why our system will work.

Technical Background

  • App Engine is an OS for Internet of Things.
  • Third patry developer writes applications for our hub/controller.
  • Any public function in the application can be executed by App Engine, as "Virtual Process", which we call "Task".
  • Each Task takes a set of input parameters defined by end user, with graphic user interface.
  • Each task is running in a sandbox environment. The App Engine takes care of task scheduling, and real-time monitoring of tasks (e.g. CPU, memory and I/O). App Engine is secure, reliable, high performance and privacy preserving.
  • Our task scheduler can be both preemptive and cooperative, which makes it possible to schedule many tasks within one single thread.

Secure, Reliable, Simple, Simple, Simple!

  • Immune to any known attacks, the app engine can keep up and running for years
  • Realtime monitoring CPU, memory and I/O of tasks running in virtual machine
  • Simplified programming language
    • Only assignment, loop, conditional and branch statements, function and table are kept. All other fancy features disabled
    • Global variable is disabled
    • Graphical user interface assisted programming

Simple Occupancy Controlled Lighting

In Sci-Fi movies, when character returns home when it's dark, the light in room will follow the person. Light will turn on when human presence is detected and turn off when presence is no longer detected.

Assuming we have occupancy sensors that can reliably detect presence of persons in the room, what is the best way to use it to give end user optimal experience?

Task Editor

In QWHA, end user makes use of Apps by creating a "Task" based on an App function. The screen shot below is a task that uses a function called "QW: Motion Sensor Triggered Lights", from the Web interface.

This simple configuration is not so simple after all. First of all, we put two occupancy sensors in the fairly large room, the family room, "FamilyRoomMotion1" and "FamilyRoomMotion2", because in many cases one single sensor can't cover the entire area.

Secondly, we want to control a list of lights (one or more) by occupancy. In this case, there are two lights, "FamilyRoomStair" amd "FamilyRoomWindow".

The logic is very simple, any motion sensor detects occupancy, lights will be turned on, if both sensors reports no occupancy, lights will be turned off.

Advanced Occupancy Controlled Lighting

Daylight Check

What if we don't want to automatically turn on the light during day time? assuming the room has lots of windows to allow sun light in.

One possibility is to add a ambient light sensor.

Another option is to calculate the sunrise and sunset time of the current day, based on the latitude and longitude of your home, which is a required parameter during the first time hub setup.

Smarter, More Comfortable

What if the setup is in bedroom? When user wants to sleep he/she may want to click a button to turn off the light for the rest of night (with real switch, a remote controller or smartphone). We call this function "On/Off Override".

How can we implement "On/Off Override"? In the API we provided to third party developers, an App program can not only get the state update of a device, but also get the information about what caused the state update (from an end user or caused by a program).

The App

From end user's point of view, the advanced features is just another App. The name of the App is slightly different and there are more input parameters. Nevertheless the look and feel and behavior of the user interface stays the same.

Applications

In QWHA design, anything can be done with application as far as home automation is concerned.

For example, one application can use the occupancy sensor data to control lights. Another application may use the same sensors for security monitoring when user is not at home.

Applications runs on App Engine as tasks. App Engine manages hardware and software resources on the hub and provides common services for application programs. In that regard our App Engine is no different than any other operating system.

Other Examples

Christmas Light Controller

Below is a Task of the "Christmas Light Controller" application function. The task runs in an infinite loop. Based on the input data, the task will set a group of lights to its respective level (0 is completely off and 255 is fully on). Then wait for a period of time (in milliseconds). Then process the next group.

When the last group is processed and last wait time is up, the program rewinds to the first group and continue the loop on and on.

This application turns any in-wall or plugin light switch/dimmer into a twinkle light controllers for holidays.

7 Day Programmable Thermostat

Below is a task for an application function "7 Day Thermostat". It can automatically adjust the thermostat settings based on flexible user defined schedules.

It is much easier to use and more flexible than other 7 day programmable thermostat such as Honeywell.

We are confident that thermostat applications smarter than Nest can be easily implemented on our hub.

Home Brew Beer Brewery

The "Home Brew" App turns any remotely controllable refrigerator into a beer Brewery machine.

The temperature control over time of beer fermentation is actually a matter of art, and science. See the charts below (originally from Technical University of Vienna):

Image source: http://braukaiser.com/wiki/index.php?title=Fermenting_Lagers

Screenshot below is a task based on the Homebrew App. User can choose time-temperature curve for the fermentation process. Different curve results in different flavor as well as different maturation time.

Development Tools

Screenshot below is a screenshot of our integrated development environment. The IDE is web based using Web browsers as front-end and Controller/Hub as backend. It is complete free fot everyone.

Application Development

Example Code

Snippet below is the actual code for the simple motion sensors controlled lights. As one can tell it is only 20 lines of code.

The input parameters are two arrays, "motions" and "lights". And code is infinite loop. our system is able to run many infinite loop programs like that. That's why we call it an OS of Internet of Things. Each task is just like a process in conventional operating system.

Code Snippet
  1. function motionTriggerLights(motions, lights)
  2.     local prevOff = nil
  3.     while true do    -- Inifnite loop
  4.         local motionOff = true
  5.         for _,m in ipairs(motions) do    -- Iterate every motion sensor in the list   
  6.             local ret, state = QWHA_DeviceGetOnOff(m)
  7.             if (ret and state) then -- If any motion sensor reported a
  8.                 motionOff = false   -- motion, the motionOff is false
  9.                 break
  10.             end
  11.         end
  12.         if (motionOff ~= prevOff) then
  13.             prevOff = motionOff -- Motion state has changed
  14.             for _, light in ipairs(lights) do
  15.                 QWHA_DeviceSetOnOff(light, not motionOff)
  16.             end
  17.         end
  18.         QWHA_WaitDevice(motions) -- Block execution waiting for state change
  19.     end
  20. end

Code snippet below is the source code for "Christmas Light Controller". It's an infinite loop, too. And it is short, too.

Code Snippet
  1. function ChristmasLightController(groups)
  2.     while true do
  3.         for i, group in ipairs(groups) do
  4.               for j, lightState in ipairs(group.lightStates) do
  5.                 QWHA_DeviceSetLevel(lightState.light, lightState.onLevel)
  6.             end
  7.             QWHA_Sleep(group.wait)
  8.         end        
  9.     end
  10. end

Code snippet below is the production code for "7 Day Programmable Thermostat". It is a bit long at 118 lines of code but it is the production code that deals with all possible corner cases and demonstrates some unique techniques in App programming.

Code Snippet
  1. local function toUnixTime(d)
  2.     return date.diff(d, date.epoch()):spanseconds();
  3. end
  4.  
  5. local function getTimeOfDay(d) -- in seconds
  6.     local h, m, s, t = d:gettime()
  7.     return h * 3600 + m * 60 + s
  8. end
  9.  
  10. local function getSchedule(d, schedules)
  11.     local weekday = d:getweekday() - 1; -- returns 1-7; while enum is 0-6
  12.     for _, schedule in ipairs(schedules) do
  13.         for j, curWeekday in ipairs(schedule.weekDays) do
  14.             if (curWeekday == weekday) then
  15.                 return schedule
  16.             end
  17.         end
  18.     end
  19.     return nil
  20. end
  21.  
  22. local function getScheduleActions(t, schedule) -- t is seconds from midnight
  23.     if (not schedule) then    -- In case some day_of_week missing from schedule list
  24.         return
  25.     end
  26.     local curAction = nil
  27.     local curActionTime = nil
  28.     local nextAction = nil
  29.     local nextActionTime = nil
  30.     for _,action in ipairs(schedule.actions) do -- action.time is seconds from midnight
  31.         local actionTime = action.time / 1000 -- App Engine time is milliseconds since midnight
  32.         if (actionTime <= t) then -- Note must be <= not <
  33.             curAction = action
  34.             curActionTime = actionTime
  35.         else    -- actionTime > t
  36.             nextAction = action
  37.             nextActionTime = actionTime
  38.             break
  39.         end
  40.     end
  41.     return curAction, curActionTime, nextAction, nextActionTime
  42. end
  43.  
  44. local function getActions(localUnixTimeNow, schedules)
  45.     local localDateTimeNow = date(localUnixTimeNow)
  46.     local y, m, d = localDateTimeNow:getdate()
  47.     local ddd = date(y, m, d)
  48.     local localUnixDateNow = toUnixTime(ddd)
  49.     local curAction, curActionTime, nextAction, nextActionTime = getScheduleActions(
  50.         getTimeOfDay(localDateTimeNow), getSchedule(localDateTimeNow, schedules))
  51.     local dayAdjust = 0
  52.     while (not curAction) do    -- Search previous days
  53.         dayAdjust = dayAdjust - 1
  54.         curAction, curActionTime = getScheduleActions(
  55.             24 * 3600, getSchedule(localDateTimeNow:copy():adddays(dayAdjust), schedules))
  56.     end
  57.     curActionTime = curActionTime + localUnixDateNow + 24 * 3600 * dayAdjust
  58.     dayAdjust = 0
  59.     while (not nextAction) do    -- Search next days
  60.         dayAdjust = dayAdjust + 1
  61.         nextAction, nextActionTime = getScheduleActions(
  62.             0, getSchedule(localDateTimeNow:copy():adddays(dayAdjust), schedules))
  63.     end
  64.     nextActionTime = nextActionTime + localUnixDateNow + 24 * 3600 * dayAdjust
  65.     -- QWHA_Log(0, "curActionTime=" .. tostring(date(curActionTime)))
  66.     -- QWHA_Log(0, "nextActionTime=" .. tostring(date(nextActionTime)))
  67.     return curAction, curActionTime, nextAction, nextActionTime
  68. end
  69.  
  70. local function triggerAction(action, thermostats, actionTime)
  71.     -- QWHA_Log(0, "Action " .. action.actionName .. " triggered")
  72.     local t = {mode="auto",  cool_setpoint=action.coolSetPoint, heat_setpoint=action.heatSetPoint}
  73.     for _, thermostat in ipairs(thermostats) do
  74.         QWHA_DeviceSetThermostat(thermostat, t)
  75.     end
  76.     QWHA_SetSystemVariableTask("lastActionTime", actionTime, true); -- Persist the last action time
  77. end
  78.  
  79. local function scheduleActionCompare(a1, a2)
  80.     return (a1.time < a2.time)
  81. end
  82.  
  83. function SevenDayThermostat(schedules, thermostats)
  84.     local nActions = 0
  85.     for _, schedule in ipairs(schedules) do -- Sort actions by time
  86.         table.sort(schedule.actions, scheduleActionCompare)
  87.         nActions = nActions + #schedule.actions
  88.     end
  89.     if (0 == nActions) then
  90.         error("No action defined")
  91.     end
  92.     local localUnixTimeNow = toUnixTime(date(false))
  93.     local curAction, curActionTime, nextAction, nextActionTime =
  94.         getActions(localUnixTimeNow, schedules)
  95.     local lastProcessedTime = QWHA_GetSystemVariableTask("lastActionTime"); -- Load from persistence
  96.     if (not lastProcessedTime) then
  97.         lastProcessedTime = 0
  98.     end
  99.     if (curActionTime > lastProcessedTime) then -- trigger action
  100.         triggerAction(curAction, thermostats, curActionTime)
  101.         lastProcessedTime = curActionTime
  102.     end
  103.     while true do
  104.         localUnixTimeNow = toUnixTime(date(false))
  105.         local sleepSeconds = nextActionTime - localUnixTimeNow
  106.         if (sleepSeconds <= 0) then
  107.             curAction, curActionTime, nextAction, nextActionTime =
  108.                 getActions(localUnixTimeNow, schedules)  -- Recalculate cur action
  109.             triggerAction(curAction, thermostats, curActionTime)
  110.             lastProcessedTime = curActionTime
  111.         else
  112.             if (sleepSeconds > 10) then -- Maximum timedrift is 10 seconds
  113.                 sleepSeconds = 10
  114.             end
  115.             QWHA_Sleep(sleepSeconds * 1000)  -- Sleep in milliseconds
  116.         end
  117.     end
  118. end

Signature Editor

Apart from writing the code, developer is also responsible for the following:

  • Give each function a more user friendly name and detail description.
  • Define the type of each input parameter. For simple primitive type (such as number or string etc.), the type is only one node. For more complex types (such as array and key/value pair table), the type definition is actually a tree with certain depth.
  • Give a more user friendly name to each type node (including children nodes) and detail description.

In QWHA, array type is call "List", and key/value pair list (or hashtable) is called "Table".

Screenshot below is the Signature Editor of the occupancy sensor controlled light example.

Screenshot below is the signature for "Christmas Light Controller".

Documentation

The Signature Editor is also used to edit App documentation. Documentation is used by user interface (UI) for Task Editor.

Developer is also responsible for providing the initial version of documentation. Currently two pieces of information are required, the "Display Name" and "Details", as shown in the screenshot of Signature Editor.

The "Display Name" of "QW: Motion Sensor Triggered Lights", "Motion Sensor List", and "Light List" can be found as the title for input box in the "Task Editor" screenshot. The contents in "Details" can be used as additional help tip on the Task GUI UI.

Internationalization

Documentation can be later translated into other localized languages, using the same Signature Editor.

A chinese user may create a task with 100% Chinese UI like this.

Crowd Sourcing

Note the translation can not only be performed by developer, but also by anybody else in this world. With App owner (usually developer) granting appropriate permission, third party can make contribution by helping maintaining translation into a certain localized language the developer may not understand at all.

Flexible User Interface

The design of the App Engine completely separates algorithm development and UI development.

Developer is responsible for building the algorithm, deciding the type of input parameters, constraints of each parameter type node, and dependency relationship among parameters ( for example, parameter A and B shall be mutually inclusive or exclusive).

Developer is also responsible for writing the initial documentation. However, future documentation maintenance and translation can be performed by third parties, or people around the world in term of crowd sourcing.

Document can affect the look and feel or even behavior of the UI. The specifications of documentation can be extended freely by adding extra tag/value pairs. That can be achieved by adding extension of "tag/values" to the documentation.

With extensibility the front-end UI can be infinitely flexible. By defining new tags, we can even customize different UI for different devices (e.g. web v.s. cellphone). For example, because the parameter value can always be expressed as a tree, by default we use a tree view for user to edit the parameter value. But under certain conditions, the front-end UI can choose to use a wizard control instead of tree view to take input from end user.

Phone UI is quite different than web UI. In the "Outdoor Christmas Light Controller" UI we demonstrated using a mixture of tree view and dialog to take user input.

System Architecture and API

The application engine runs on a separate process to provide extra isolation. The design is independent of any particular technology or programming language.

In practice, as of now one virtual machine technology is used, LuaJIT, which provides binding to Lua programming language. Binding with other programming language and virtual machines can be added in the future, including Javascript or Java, .Net etc.

We use cooperative multitasking based on Lua coroutine.

API

Our system offers rich API to developers.

  • IoT device related API, primarily IoT device input and output, for example, turning device on and off, setting device output level or read from sensors etc.
  • OS I/O, file system, local I/O (serial port), network I/O, common protocol handler (HTTP, MTP,SOAP, REST, JSON RPC etc),
  • Application level API. Mostly network services such as local weather and traffic, financial data etc.
  • Miscellaneous, logging, debugging, etc.

More API shall be provided over time as upgrades.

Threading Model

Although we use cooperative multitasking, developer may not notice the difference if following common programming practice.

QWHA provides various wait API. We provide API that can blocking wait for certain event to happen. We also provide API to register wait on certain event. Developer can write code to register wait on multiple objects of different types (for example device I/O, network I/O and synchronization objects) and then make one "wait" call to wait for any registered event to happen. Wait API also takes an optional timeout.

For more information, please refer to "Concurrency" and "System Events and Wait API".

App Store

App store is a web service that enabled the sharing of applications among users and developers.

App store facilitates the collaboration among developers, documentation maintainers and end-users.

App store integrates a search engine that specially optimized for IoT applications. Function parameters information, documentation, and translations can all be used as additional information to optimize the search results. Also the interaction between developers and end-users through forum and comment system can not only help technical support and collecting user feedback, but also provide extra informaiton to further optimize the search engine of app store.

Only Limitation is Your Imagination

Developer can develop any home automation application based on our App Engine and API. For example, irrigating lawn based on sun position (best time is right before sunrise) and projected rain volume in the next few hours (from weather forecast data), in order to save water resource. People in California is going to love it. This App is too easy to use to be demonstrated and it takes dozens of lines of code to implement. (because system knows your longitude and latitude, it know where to get your local weather forecast and how to calculate the sun position).

Click here to read the weather API and source code of retrieving amount of rain of the near future (about 20 lines of code).

Security and Privacy Protection

Each application is running on a separate sandbox. The App Engine design implements implicit and explicit security model. Every API that accesses system resources (local and remote devices, network connection, file system, memory and CPU) is monitored by App Engine.

Implicit security model helps protecting user privacy. For example, if an application task tries to access a device that doesn't appear in the user provided input parameters, the task will be terminated because of security violation.

Performance and Stability

LuaJIT have many years of tracking record of being high performance and robust. In practice, even on a single core 700 Mhz Raspberry Pi, with dozens of tasks running, our App Engine can still manage to achieve average CPU utilization of less than 1% with milliseconds latency. Our hub with App Engine is able to keep up and running for years with maintenance.

Phone UI

Below is the phone UI under development. The App Engine seamlessly integrated with the UI. It not only gives end user the capability to manage application tasks on their smart phones, but also tightly integrates with phone in many ways. For example, the application may send alarms or notifications to the phone. It may even instruct how to render the information such as sensor readings.

Click images to see enlarged version.




Theoretical Computer Science Behind

Our application engine design can be proved to be both functionally and logically complete, which is difficult to achieve by cloud based solution at reasonable cost.

Our innovation in UI design can also be proved to be optimal and complete while still maintaining complete separation between application code and UI design.

Ultimately, it all come down to the fact that our design will offer a much better user experiences, as well as better privacy protection.

System Design Essence

System design requires taste and experience. Internet of Things and Smart Home system integrates both classic mature technologies and latest popular technologies. It directly involves all kinds of technologies, from embedded SOC to large scale cloud computing; from low level RF protocol stack to high level web services; from machine code to high level scripting language with virtual machine and JIT technology; from big data technology to information security and privacy protection.

System design reflects designer's perspective of life and world. The App Engine design clarifies all issues and details of the entire IoT and Smart Home system. The end result is an extremely simplified system for both third party developers and end-users. Simplicity brings efficiency to every part of the entire system. It improves the performance and robustness of the end product. It simplifies the project development and maintenance. It reduces the cost of end-user education and end-user support. By aiming at the whole world, we leverage any resource we can to further improve the efficiency. The App Store and crowd-sourced based translation pushed the reusability to the extreme, while in the mean time helping promoting our products to every corner of the world. Those all comes down to a product with better user experience and lower development and operational cost.

The design and implementation of the App engine, is a combination of fundamental computer science theory and meticulous engineering practice. We keep innovating in every areas such as operating system, network protocol, virtual machine technology, programming language and graphical user interface. Many technological details in our system is also hot topics in recent research and development of computer science and engineering. It is always a pleasure to do something nobody has ever done and come out some ideas nobody has ever thought of.

Intellectual Property Disclaimer

Key technologies described in this article are patent pending.