Overview

The Visual Lambda Application Engine is the brain of the QWHA Controller. It is easy to use, easy to develop on, and can be as smart as one can possibly imagine.

Prerequisites

Developer must read "Application Engine User Manual" before reading this document.

Lua Programming Language

Lua is probably the simplest programming language with OO and functional features. Our App Engine disabled a few features in Lua language:

  • Coroutine
  • User defined metatable, i.e. getmetatable and setmetatable.
  • rawget and rawset.

For people with experience in general programming, this page "Learn Lua in 15 Minutes" will be enough to master App programming in Lua. Note, one only needs to read half of the document, till "3.1 Metatables and metamethods.".

App Engine

App Engine is part of the Controller/Hub software. It allows third party to develop on the QWHA Controller.

Architecture

App Engine is implemented as one or more virtual machines. the design is programming language independent. In practice, our current implementation uses Lua programming language. In particular, we use an implementation of Lua with JIT (Just-In-Time) compiler, Luajit.

App Engine is an Operating System for Internet of Things. Developers can build any application in the application domain of IoT hub on top of our system.

Programming Model

There is not too much of programming model in the design. Our App Engine is an Operating System. Developers write program to run on the Operating System.

An Application is one Lua file. Programmer should define one or more public functions in the file.

Since our App Engine is an Operating System, it should have concept of processes, threads like any other modern operating systems.

In our system, process is called "Task". If you are familiar with programming languages such as C, Java or C#, you should know a process shall have an entry point, which is the main() function.

In our system, any public function in the application can be entry point of a Task. In other words, user can create a Task (process) from any public function.

In modern Operating Systems, a regular process may take many arguments. Each argument is a string. In our App Engine, the function can have many arguments (parameters) of any type. The arguments are provided by user during Task creation process, through Graphical User Interface (Task GUI) only.

Enable Developer Mode

From left side navigation panel, navigate to "Controller"->"Settings". The check "Enable development mode".

You need to give yourself a developer name. Note you can choose any name you want. But if you want to be able to publish the applicaiton to online app store, you have to first register a developer ID on app store and get approval. You will need to use your registered developer ID and correct password in order to publish your app.

As shown in screenshot below, you initially use a developer named as "test".

The IDE

QWHA Controller admin tool has full featured tools for development, testing, deployment and application management.

100% web based, the IDE contains several GUI tools that are tightly coupled with one another. It is provided free of charge to anybody who uses the Controller.

Tutorial 1: Flashing a Light

Our first example is a simple one. We are going to write an application to flash a light at an interval. We want to give user the option to choose a light and specify an interval.

Creating an Application

First open to the "Develop" panel (from left side nagivation panel, "Controller"->"Visual Lambda"->"Develop"). Then click "Add" button on the upper left toolbar.

A dialog box will pop up asking for the name for the new App script. In this example, we call the new App "LightTwinkler".

Click "OK" button, the new App is created and a source code editor will be open on the right side panel for you to write the source code.

Writing Code

Now let's write some code, as shown in screenshot below:

The actual code is also listed below so that you can simply copy and paste.

Code Snippet
  1. function Twinkle_light(light, interval)
  2.     while true do
  3.         QWHA_DeviceSetOnOff(light, true)
  4.         QWHA_Sleep(interval * 1000)
  5.         QWHA_DeviceSetOnOff(light, false)
  6.         QWHA_Sleep(interval * 1000)
  7.     end
  8. end

Application, Function and Task

In example above, the script "LightTwinkler" is a file. In QWHA, such as file is called an "Application", or "Package".

An application may expose multiple functions. In the above example we only exposed one function "Twinkle_light".

Unlike smart phone applications, in QWHA, the basic functional unit is function. An application is usually a collection of functions.

Technically, a Task is an instance of a function. Task works like App on smart phones. End user may run several instances of the same function simultaneously. Each instance of function is a different task.

For example, you may want to twinkle two lights simultaneously, flashing tree light every 3 seconds and flashing window light every 5 seconds. You can schedule two tasks.

Deploying the Application

Once the coding is finished, you need to deploy the application for testing. In the current editor window, click the upper tool bar "Developer" drop down menu, then choose "Deploy".

A window called "Signature Editor" will pop up.

Developer are reauired (forces actually) to give some detail information about the application on deploy.

On the left panel, there is a list. As you can tell, in this example, there are two entries on left list. The first one, "[LightTwinkler]", is the entry for the application package itself.

On the right panel, there is a tool bar on top, a tree view in the middle and a text box on the bottom.

Application Description

For the "[LightTwinkler]" application entry, developer should edit at least two entries:

  • Display Name - A user friendly name for your application. it should be a more descriptive name then the simple file name. It is recommended to keep it short and consistent among other applications the developer develops.
  • Details - A detail description about what the application contains, as well as other information such as release notes etc.

The "Display Name" and "Details" are edited as shown in screenshot below:

Function Description

The IDE has a language parser built-in. That's how the Signature Editor is able to automatically discover all public functions in the source code and list the function in the left panel.

In this example, there is only one function in the source code, "Twinkle_light". As you can see the IDE successfully parsed the code.

By the way, if there is any syntax error in the code, an error dislog will pop up and this "Signature Editor" dialog will not show up.

Now developer shall give the function a better "Display Name" and a meaningful "Details", as shown in the screenshot below.

Customize Parameters

In the source code, the function "Twinkle_light" has two parameters, named "light" and "interval" respectively. The parser can get that information correctly.

However, since Lua is dynamically typed language. There is no way to reliably induce the parameter type automatically at any rate. Right now, QWHA automatically assign a "Number" type to every parameter initially.

The most important job for developer is to make the type right.

In this case, we know the "light" parameter shall represent a light. So we pull the drop down list of the "Type" field on the "light" parameter. All currently system supported types can be found in the list. Now we need to choose the "Light" type.

Now we can set the "Display Name" and "Details" field for the "light" parameter. Note those two fields will be used in the UI (User Interface) for the end users. For example, when the end user wants to use this function to twinkle/flash a light, the end user needs to give two parameters, a light, and a interval in seconds.

For the light, the UI will prompt a user two give value to a field called "Light to twinkle", with a more detailed information "Please choose a light" as tool tip if user needs further help.

Attributes

The "interval" parameter is a number, naturally. So we don't need to change the type.

According to the code, the "interval" parameter shall be number of seconds to wait between the on/off state alternation of the light. What if the end user input a number of "0"? What if the user specifies a negative number?

We know an interval of "0" or negative seconds doesn't make any sense. Even if a very small positive number such as "0.1" second doesn't make sense either because our switch may not be able to respond that fast to continuous "on" and "off" commands at 0.1 second's interval.

We believe setting a minimum of 4 seconds for interval is reasonable. So we highlight the "interval" parameter, and choose "Min" from "Attribute" dropdown menu on tool bar.

So we added an attribute of "Min", meaning the minimum value, to the parameter "interval". We set the minimum value to "4".

Note the attributes for number types can affect the behavior and validation of user GUI, even the look & feel. For example, if we only want user to input integer number, we can set an integer "Min" and "Max, then set an integer "Step". Any user input that is not an integer will be automatically rounded to nearest integer.

Also we set the "Details" of this parameter.

Now we believe we have finished editing the description of the application. We then click "OK" button to deploy the application to the controller.

Creating a Task

Now you can actually use the "Twinkle_light" function to flash any light in your home. By creating task based on the function.

The Task Manager

From navigation panel, open "Task" panel by navigating to "Controller"->"Visual Lambda"->"Tasks". The Task Manager panel will be opened.

Adding a New Task

In the tool bar on top, click "Manage" drop down menu, then click "Add".

A Task wizard dialog will pop up. At first the user is required to choose a function. In this case, we choose "Twinkle a light tutorial" function we just created. Then click "Next" button.

Customizing Parameters

A parameter editor will show up. Note in the tree view in the middle, the "Name" column shows exactly the "Display Name" you have given. The same is true for the "Details" field on the bottom.

Initially the "Light to twinkle" is "Null" and Interval is "4", which is the minimum value we defined in attribute.

End user will have to choose a light device in his/her home. Currently the GUI provides a drop down list with all available lights. In this case, we choose "Outdoor Christmas Tree Light".

User is free to change "Interval" as well, to any number of seconds that is no less than 4.

Click "Next", user will be prompted to give the task a name and detail descriptions. Then click finish, the task will be created.

Create as Many Tasks as Needed

The "Light to twinkle" function is an infinite loop. It keeps flashing the light at an interval.

End user create an instance of the function called "task" to execute. For "Light to twinkle" function, one task flashes one light. If end user needs to flash more than one lights, he/she can create other tasks on the same function, with different set of parameters (light, interval).

As we mentioned earlier, App Engine is an operating system for Internet of Things. Task is like process on the OS. Multiple tasks will be running concurrently on the App Engine. We will explain details about how tasks are scheduled in later chapters.

Managing Tasks

As exaplained in the "User Manual" , users can Remove, Edit, Enable/Disable, even Upgrade/Downgrade tasks.

Internationalization Made Easy

Finally, we have made it extremely easy for developers to write applications for the whole world.

Open "Application Manager", by navigating to "Controller"->"Visual Lambda"->"Application".

Select the application "Tutorial Light Twinkler", as illustrated below. Then from tool bar, navigate to "Contribte "->"Translate". Then select Chinese language instead of English.

The "Signature Editor" will pop up. Note in the title bar, the localized language turned into "中文(简体)", which mean "Simplified Chinese".

Also note user can't really modify the actual signature, such as parameter types or attributes. User can only do translation on the text.

We call this kind of UI "in-place translation". Using this UI, user can simply translate the text from English into Chinese.

The information that can be translated is called "Documentation". Even though "Documentation" and "Signature" are always presented in one single UI "Signature Editor", they are actually stored separately.

Obviously, one version of function only has one signature, but it can have multiple documentations, each documentation for one localized language.

Below is the translated function documentation.

Note the above translation UI is still in English, even though the user is required to type in Chinese characters. The idea is to let people who know both English and Chinese translate documentation from English into Chinese using an English UI.

Note, not only developer can translate the documentation. Any regular user can do it too. Users can also share translated documentation on App store with the whole world. Of course making contribution on App store may require approval from App owner (developer).

User can also help maintaining existing documentation by improving it.

We separated the development of application and development of documentation. Also internationalization/localization is all about translating documentation so that is solved in an elegant way.

Now suppose a Chinese user who only knows Chinese is going to use the application. User simply fires up the Chinese version of Task UI and create the task. As illustrated in the screenshot below, everything can be in Chinese, including the tooltip text shown in the "细节"(Details) box in the lower panel.

Tutorial 2: Christmas Light Controller

In previous tutorial we demonstrated a simple light twinkler.

In this example, we want to demonstrate more complicated lighting control with more complicated input data structure.

Suppose we want to build a Christmas light controller for end users. End users are able to separate lights into groups. Each group has a set of lights, each light will have a separate "On Level", between 0 and 255, where 0 is fully turned off and 255 is fully turned on.

In each iteration, the light controller takes the current group, control each light in the group with associated "On Level" value, then sleep a customized interval of time, then proceed to next group.

After the last group is processed, the program rewind to the first group, thus keep flashing the lights continuously.

The Code

The code is listed below. It is really simple code, 11 lines. And it contains nested for-looops inside an infinite loop.

Only two API calls to App engine runtime, QWHA_DeviceSetLevel and QWHA_Sleep. Note QWHA_Sleep takes milliseconds.

Code Snippet
  1. function ChristmasLightController(groups)
  2.     while true do
  3.         -- groups is a list of control group
  4.         for i, group in ipairs(groups) do
  5.               for j, lightState in ipairs(group.lightStates) do
  6.                 QWHA_DeviceSetLevel(lightState.light, lightState.onLevel)
  7.             end
  8.             QWHA_Sleep(group.wait)
  9.         end        
  10.     end
  11. end

The Function Signature

Let's deploy the code to App Engine. At the time of the deployment, the "Signature Editor" will pop up asking developer to edit the signature. Signature Editor is able to start the process by determining that the function at issue is called “ChristmasLightController” and that it has only one parameter specified at its top-level called “groups.”

The type of groups is initialized as “Number.”

We can give the function a more user friendly display name. In this case, we simply call it “Christmas Light Controller.” An arbitrarily detailed description, about what this function does, can be entered in the “Details” field.

From the code we know the parameter “groups” is an array. So screenshot below shows a developer changing the type of the parameter to “List.” As can also be seen in the screnshot, the developer has also given the parameter a user-friendly name (“Light Groups”), and wrote a detailed description of the parameter.

The items inside the “groups” array also need to have their type specified. In the Signature Editor, the type is being set for a child node of the groups “List” type. The Signature Editor has initially set its type to “Number” (which can be a convenient default). In screenshot below, the developer is shown changing the type to “Table.” The type Table can be indicative of the corresponding data being stored as a hash table.

The Table display name is also given a user friendly name (“Light Group”).

The Signature Editor provides an “Add Member” button (although it is grayed-out in this particular snapshot). Once the selection of the Table type has been completed, the Add Member button in toolbar is once again available. Selecting this button (while Light Group adds the child node.

From the code, it can be determined that each “Light Group” Table must have two members (or sub-items). Highlighted node in screen shot below represents one of them. As can be seen, this sub-item is itself an array (like Light Groups), but one that can be accessed with the key “lightStates.”

From the code, it can be determined that the item type, of each member of the List called “List of light & on level,” is a Table (like “Light Group”). Therefore, while node "lightStates" (as native name) is still selected, the “Add Member” button can create a child node. The developer can give it a more end-user-friendly name of “Light & On Level.”

Each instance, of the Table “Light & On Level,” needs to have two fields. These two fields are:

  • light, and
  • onLevel

The developer can add these two fields by selecting the “Add Member” button twice, while parent node is highlighted.

Note member "light" shall have a type of "Light".

"onLevel" can be given attributes. This can be accomplished by selecting “Attributes” button, while onLevel is highlighted. Selection of Attributes is shown to provide the following four attributes:

  • "Max",
  • "Min",
  • "Step", and
  • "Nullable"

The developer can select to specify a value for any combination of these attributes. For example, in the case of the onLevel type, it can be useful to set the minimum and maximum onLevel numbers, that can be applied to the lights to be controlled. As shown in screenshot below, the min and max values have been set to, respectively, 0 and 255. The “Step” value, for onLevel values within the 0 to 255 range, is set to 1. Thus, in this case, 256 onLevels are possible, starting at 0 (where an IoT light is completely off) to 255 (where an light is completely on).

As discussed above, each “Light Group” Table must have two children (or sub-items). The second of these, called “wait,” is shown as added in screenshot below.

A Task

Let's suppose an end-user wants to schedule a task, based on the ChristmasLightController function, that does the following steps as a continuous, repetitive, sequence:

  1. Turn on 3 IoT-controllable lights, “Outdoor Christmas Tree Light,” “Outdoor Puppy Light” and “Outdoor Deer Light.” As discussed above, an “onLevel” of 255 means a light is to be set at its full intensity (more colloquially, the light is “on”).
  2. Wait for 10 seconds (expressed as 10,000 milliseconds).
  3. Turn off 2 of the 3 lights: “Outdoor Puppy Light” and “Outdoor Deer Light.” As discussed above, an “onLevel” of 0 means a light is to be set at its lowest intensity (more colloquially, the light is “off”).
  4. Wait for 10 seconds (10,000 milliseconds).
  5. Turn off the one light, “Outdoor Christmas tree Light,” that was still “on.”
  6. Wait for 10 seconds (10,000 milliseconds).
  7. Repeat from step 1.

First, using a Task GUI (not shown) the end-user starts creation of a new task, to be based on the ChristmasLightController function. Having started the task-creation process, screenshot below shows an example first screen. As can be seen, the “Groups” list is empty. In the “Details” box below is displayed detailed documentation for the “Groups” field. As can be seen, by highlight the “Groups” node, the “Add” button can be selected from the toolbar.

Screenshot below depicts a group (an instance of a Table) being added. As discussed above, with respect to signature definition, each instance of group has two fields:

  • “List of light & on level” and
  • “Wait for.”

These two fields are automatically created, for each group instance, because of their definition in the signature.

Screenshot below illustrates the result of an end-user highlighting “List of light & on level”, and then selecting the “Add” button. An instance #1 (at location 1522) is created, for the “List of light & on level.” According to the signature, this instance is automatically populated with the fields “Light” and “OnLevel.”

Screenshot below also shows an “Outdoor Christmas Tree Light” being selected, as the particular value for the “Light” field. As can be seen, because the “Light” field is of type light, all choices, in the menu that includes “Outdoor Christmas Tree Light”, are types of lights. No other types of devices (e.g., a garage door) can be chosen to satisfy the “Light” field.

Screenshot below shows the end-user setting the “OnLevel,” for this particular pair of “Light” and “OnLevel,” to 255. This means, of course, that the result of executing this particular pair is to set the “Outdoor Christmas Tree Light” to the fully-on state.

Screenshot below shows the result of the process of, once again, selecting “List of light & on level,” and selecting the “Add” button again. An instance #2 is created.

Screenshot below shows that three pairs, of “Light” and “OnLevel,” have been added, each setting a different light to fully on. The lights set are as follows:

  • “Outdoor Christmas Tree Light”
  • “Outdoor Deer Light”
  • “Outdoor Puppy Light”

Then set the “Wait for” value is set to 10,000 milliseconds (10 seconds).

Let's continue our task "programming". Once-again we select “Groups” and selects the “Add” button, to create an item #2 of the Groups list.

As we have seen, the net effect of item #1 is to turn fully-on all three of the outdoor Christmas lights.

Screenshot below is the what the final task looks like. As you can see, the purpose of item #2 will be to turn off two of them.

Also shown in screenshot is item #3, where the last light to be left on (“Outdoor Christmas Tree Light”) is also turned off.

System Specification

Take the "Multiple Motion Sensor Controlled Group of Lights" as a example.

  • The task entry is actually a function call. Task functions are running on the App Engine, which is a virtual machine. Task is analogous to process in operating system. The task function is like main() function of regular process.
  • Task entry function may take arbitrary number of input parameters. Each parameter may be of some complex type.
  • For example, in the function motionTriggerLights above, it takes two parameters, "motions" and "lights".
  • Both parameters, "motions" and "lights", are of array type, representing an array of motion sensors and an array of lights.

Function Signature Editor

Because Lua is dynamically type language, as most scripting languages are, we can't reliably deduct the parameter type by analyzing the code. Programmer has to explicitly specify the type of each input parameter of a function.

The Signature Editor is part of the IDE tool set. At the time of deployment, IDE can help by parsing the source code. The parser can detect any syntax error. If there is no syntax error, the parser will be able to find out every public function in the source code, number of parameters for each function, and the name of each parameter.

In QWHA, signature of a function means the data type structure of every input structure of that function.

Before the programmer deploys the code to the App Engine on controller for testing. The signature editor will pop up, so that programmer can define the signature for each public function.

The screenshot below is the signature of the function "motionTriggerLights" in signature editor.

As we can see, the parameter of "motions" is an array, which is called "List" in the editor tool. The parameter "lights" is also an array.

No matter how complex a type can be, in QWHA, a type can always be defined as a tree. A simple primitive type is just a single tree node.

We will cover types in details inlater chapters. Now we only need to know "motions" is an array (List) of "OccupancySensor" and "lights" is an array (List) of "Light". Both "OccupancySensor" and "Light" are special types that represents the actual device types.

Function Signature

In the above screenshot, there are three columns in the tree view (upper part of the window), "Display Name", "Type" and "Native Name". There is also a text box named "Details" in the lower half of the window.

Only "Type" and "Native Name" contains the information for the function signature. The signature defines the tree structure of the input parameters and their types. Not all node have "Native Name". Native name shall be presented in the following types of nodes:

  • Parameter Node - Parameter shall have a parameter name. The name is defined in the source code. By parsing the source code the system can programmatically deduct the name. For example, assuming we have a function(a, b), the parser will know function "f" has two parameters, named "a" and "b" respectively.
    Note the parameter name can actually be ignored by runtime for most languages because the parameters are pushed onto stack one by one in order.
  • Member of Table - If the node is a member of "Table" type, which is a native "Hash Table", the member shall be referenced by a key in source code. QWHA mandates that the key has to be of "String" type. The key name and value type has to be defined by developer to matches the source code.

In summary, the "Native Name" for parameter node cannot be modified, they are automatically deducted by source code parser. The "Native Name" for "Table member" node has to be provided by developer and it cannot be empty. The actual name shall match the actual name used in source code.

Documentation

In previous paragraph, we learn that only "Type" and "Native Name" contains the information for the function signature.

The "Display Name" and "Details" are user friendly description of each type node (short version and long version). Those two fields are part of "Application Documentation".

In the QWHA App Engine design, "Documentation" is clearly distinguished from "Signature". "Documentation" contains information that only matters for GUI (Graphical User Interface). The two required fields, "Display Name" and "Details", are all of string type. So both fields defines what text string should be displayed on user front-end GUI too.

As a matter of fact, the App Engine runtime only cares about signature. It takes Documentation as a file and simply saves the file on persistent storage. And it will deliver the Documentation to front-end tool over network on demand.

Nevertheless, it by no means indicates that "Signature" is more important than "Documentation". "Documentation" is the only thing that matters to end users. End users doesn't have to know anything about signature. End user brings a uniform standard GUI to create a Task. He/she only needs to follow the instruction displayed on the GUI to complete the operation.

In the "motion sensors controlled lights" example, User creates a new Task, adds two motion sensors, "FamilyRoomMotion1" and "FamilyRoomMotion2" to the "Motion Sensor List", adds one light "FamilyRoomStair" to the "Light List". That's it. All the text displayed on the GUI are from the Documentation.

Signatures affects the UI, as well. In the above example, the "Motion Sensor List" is a list of "OccupancySensors", as we can see from the screenshot of the "Signature Editor". So user can only add available motion sensors from the system to the list. Still, users doesn't have to know anything about the signature. They just follows the UI and UI makes sure they can't make serious mistakes. For example, users can only add "OccupancySensors" to the list, not some other devices such as "Thermostat".

Signature may also contain additional information called "Attributes". Some Attributes may also affect the behavior of UI. For example, a type node is "Number" may have Attributes auch as "Max", "Min", "Step", which may affect the look & feel and behavior of the corresponding UI (a number spinbox).

Separation of Concerns and Crowd Sourcing Approach

Initially, at the first time the developer deploys the application to Controller for testing. The "Signature Editor" dialog is pop up to "force" developer provide the initial signature and documentation of the application.

Once the application is tested successfully, developer can publish the Application to the online App Store. Other people can download the Application from App Store and create tasks on it.

Third parties can also help improving or maintaining the documentation, using the same Signature Editor. However, later editing the documentation will have the signature locked. User can only edit the documentation part, in this case, the "Display Name" and "Details" fields.

Third parties can not only keep the documentation for themselves, but also publish the improved documentation to the application store, so that the improvement can be shared with people all over the world. Of course in order to do so, third party has to get permission from Application owner (i.e. the developer). Application store offers fine grained access control features for developer to assign access permissions to other accounts so that people can collaborate on Application development.

Internationalization and Localization

Apart from editing the documentation, third party can also translate the documentation into another localized language. Developer only needs to write the application and provide initial version of documentation in his current localized language. Then the application can be translated into languages the developer doesn't understands at all by people all over the world.

Our design encourages people all over the world to work on the same Application project in the most efficient and reliable way.

The screenshot below shows the same signature translated into Chinese. The translation can be done by replacing text in-place from one language to another language. Anybody who is bilingual can be part of the translation team.

Data Types

A data type is expressed as a tree. Primitive types (Number, Boolean, String) are presented as single node, while complex types (array or table) are collections of items with certain type.

System defines special derived types, for example, “On/Off Light”, “Dimmable Light”, or generic “Light”. Natively those are “Number” type, which is the unique system device ID. However, they are treated specially in admin and development tools. A “Dimmable Light” GUI component may only allow user choose from available system dimmable light devices instead of prompting for a number. Special derived types can be string (for example, user) or more complex types, depending on the implementation.

Attributes

Attributes are additional information associated with Application, Application function or node on a type tree.

For data type tree nodes, an Attribute may put additional constraints on the data type, for example, a “Number” type may have further constraints such as range (maximum, minimum), default value and minimum incremental step.

Attributes shall be part of Application Signature.

Some important Attributes are listed below:

  • Built-in Attributes for Application
    • Reference – If an Application references functions in another Application, the reference information shall be part of the Application signature. The reference shall contain both Application name (including developer name) and version.
    • Singleton – All functions defined in the Application is a singleton function. For details please refer to the definition of Singleton attribute on Application Functions below.
    • Library – Every function in this application is library function.
  • Built-in Attributes for Application Functions
    • Singleton – At most one instance of the function (Task), across all versions, is allowed to be made available. Singleton may have impact on resource management.
    • Library – The function shall be referenced by other functions. Users shall not have access to this function (cannot schedule tasks on the function). Library is not required to have a meaningful description.
    • Permission – The special permission the function may need, for example, file or network access. When user creates a Task on the Function, Permission Attributes are inspected. A GUI will pop up for user to explicitly grant each permission request.
  • Built-in Attribute for Type Node
    • Nullability is a predefined attribute for types. If a type is nullable, then the value of the type can be special “Nil type”.
    • Min, Max and Step Attribute for Number types
    • Default attribute defines default value.
    • Format attribute defines type specific format string.
    • MutuallyInclusive Attribute may be used along with “Nullable” Attribute. The value of this attribute is a list of sibling type nodes identified by “Native Names”. When this attribute is added, value of the field is not null if and only if values of all specified sibling nodes are not null. Multiple MutuallyInclusive attributes can be specified for one type node, if any MutuallyInclusive attribute evaluates a result of “Not Null”, the node shall not have a “Null” value.
    • MutuallyExclusive Attribute may be used along with “Nullable”, When this attribute is added, value of the field is “Null” if and only if values of all specified sibling nodes are not null. Multiple MutuallyExclusive attributes can be specified for one type node, if any MutuallyExclusive attribute evaluates a result of “Null” for this node, the node shall have a “Null” value.

More attributes may be added in the future.

Type Specialization

With Lua virtual machine. The only native types are Boolean, Number, String, Array and Table. All other types are specialized types.

System defines Special Derived Types. One example is specific device types, such as Light, or more specific, Dimmable Light etc. Special types can be other system objects, such as User, Physical Location (e.g. Living Room, Bed Room etc.). Special types can be expanded in the future to add new types.

Special Derived Types receives special treatment in the Task GUI. In the GUI tool, when user is required to give a value of Special Derived Types, user shall be prompted with a GUI component with available system objects of the special type as possible choices.

Special Derived Types, combined with Task function and task GUI, allows user to easily configure complex logics. For example, if a specific detects presents of a specific user, adjust the light in that room to specific color and luminance.

Nullability

Null, or Nil, is a special value in Lua, and in many other languages as well. A type is nullable if the value of that type can be Nil.

In Signature Editor, a type can be set as nullable. The only exception is that the element of an array can't be nullable.

nullability is an attribute.

Type List

Below is a list of currently supported types:

Type Name Description
Boolean true or false.
Number Number type can be either integer or floating number.
String A text string.
List Native array type. List type shall have exactly one child type node representing the type of array element. All elements in the array shall be of the specified type and shall not be null.
Table Native hash table type. Table type shall have one or more children nodes. Each child node shall have a "native name", which is the key, and a "value type", which is the type of the value. The key must be a non-empty string unique within the table scope.
Enumeration Natively represents a number type. Enumeration type shall have a list of enumeration names. Enumeration names are stored in Documentation for internationalization purpose. For example, a DayOfWeek enumeration type will have enumeration names from "Sunday" to "Saturday", for English localization. For Chinese localization the names should be "星期日" to "星期六". value of Enumeration types starts from 0.
DateTime Natively represents a number type, which is the Unix time value, number of seconds since Unix epoch.
TimeOnly Natively represents a number type, seconds from midnight.
Device An opaque type, although natively represents an integer, which the ID of the device. This type represents any device in the system.
LoadDevice An opaque type, although natively represents an integer, which the ID of the device. This type represents a load device. For example, a Light is a load device, while a light switch is not.
Scene An opaque type, although natively represents an integer, which the ID of the device. This type represents a software scene.
Light An opaque type, although natively represents an integer, which the ID of the device. This type represents a light device.
LightSwitch An opaque type, although natively represents an integer, which the ID of the device. This type represents a light switch.
LightOnOff An opaque type, although natively represents an integer, which the ID of the device. This type represents a on/off light.
LightDimmable An opaque type, although natively represents an integer, which the ID of the device. This type represents a dimmable light.
LightColorDimmable An opaque type, although natively represents an integer, which the ID of the device. This type represents a color dimmable light.
LightSwitchOnOff An opaque type, although natively represents an integer, which the ID of the device. This type represents an on/off light switch.
LightSwitchDimmable An opaque type, although natively represents an integer, which the ID of the device. This type represents a dimmable light switch.
LightSwitchColorDimmable An opaque type, although natively represents an integer, which the ID of the device. This type represents a color dimmable light switch.
Output An opaque type, although natively represents an integer, which the ID of the device. This type represents a generic output device.
OutputOnOff An opaque type, although natively represents an integer, which the ID of the device. This type represents an on/off output device.
OutputLevel An opaque type, although natively represents an integer, which the ID of the device. This type represents a level output device.
Switch An opaque type, although natively represents an integer, which the ID of the device. This type represents a generic switch device.
SwitchOnOff An opaque type, although natively represents an integer, which the ID of the device. This type represents a generic on/off switch device.
SwitchLevel An opaque type, although natively represents an integer, which the ID of the device. This type represents a generic level control switch device.
Thermostat An opaque type, although natively represents an integer, which the ID of the device. This type represents a thermostat.
OccupancySensor An opaque type, although natively represents an integer, which the ID of the device. This type represents an occupancy sensor.
Shade An opaque type, although natively represents an integer, which the ID of the device. This type represents a shade.
WindowCovering An opaque type, although natively represents an integer, which the ID of the device. This type represents a window cover.
LightSensor An opaque type, although natively represents an integer, which the ID of the device. This type represents a light sensor.
User Natively represents a string, which the ID of a system user.

More types may be added in the future.

Version Control

The App Engine on Controller implements a version control system. Version control is a very important feature. Every time the developer deploys a source code for test, a snapshot of the source code is save as a production version. Every time developer wants to modify the production code, he/she has to check the code out from version control. Checking out will create a development version with new version number. Developer can also commit extra snapshot of development version to version control.

Local controller may keep several different versions of the same application. Different version can execute at the same time.

Developer may choose to publish a mature version to the App Store to share it with the whole world. App Store may also keep a list of versions for the same application.

There will always be a version marked as "Current Version".

Version control makes it easier for developer to keep track of the development and code changes. It also gives developer the ability to keep improving the application by publishing upgraded versions.

Version control gives user the ability to upgrade/downgrade the application, and the ability to use Application that uses different version of the same library without worrying about the "DLL hell".

Version Compatibility and Consistency

Version control gives programmers the ability to deploy/publish new version of application. There are several concerns about this feature:

  • If the signature of the Application Functions are not changed in the new version. Do programmer have to edit the signature again from scratch?
  • What if the signature has been changed? What can the IDE tools do to notify programmer and help programmer solve the potential problem?
  • What will happen to the documentation? Some documentation may not even be written by the programmer. In particular, documentation may be in some localized language the developer doesn't understand at all.
  • What should the IDE tools do to docukmentation if the signature has not been changed? What if the signature did change?
  • If the signature did change, what should the IDE tool do to make sure the new version won't break the existing tasks that use the old version?

The IDE tool provide solutions to solve all the concerns.

Signature Version Compatibility Checking

IDE automatically tracks function signature changes between versions, and blocks any deployment of new version with signature change that is not compatible with currently published version (in which case developer has to rename the application to avoid conflict). As long as deployment is blocked, publishing will never happen.

Version check may generate warnings or errors. Warning indicates potential incompatibility with existing Tasks. Error indicates that new version is not compatible. User may explicitly proceed with deployment in case warning. However with error the deployment will be aborted.

Below is the general signature compatibility rule:

  • New version of application may have more functions
  • As long as a function is defined in a version that has been published to Application Store, it shall not be removed from the newer version; otherwise error or warning will be generated
  • As long as a function is defined in a version that has been published to the Application Store, the signature of function shall not be modified. There are a few exceptions:
    • For special derived types such as device types
      • It is permissible that a type in new version has broader coverage than the old version. For example, if a type in the old version is “Dimmable Light” while in the new version is changed into “Light”, it is permissible because “Light” is generic device type which includes “On/Off Light”, “Color Changeable Light” and “Dimmable Light”.
      • If a type in new version has narrower coverage than old version. A warning will be generated and it is up to the developer. For example, a “Light” type in old version is changed into “Dimmable Light” in new version.
      • Developer can append more enumerations to an existing enumeration type
      • New version of a function may have more parameters than old version, as long as new parameters all have “Nullable” attribute.

Note there are some flexibility to determine what kind of incompatibility is considered error and what kind of incompatibility is considered warning, which is subject to implementation.

QWHA uses different compatibility checking policy for deployment and publishing. Deployment is used for development testing. Programmers may make mistakes in signature editing and may need to make corrections. Publishing is to publish the application to the App Store so that application can be used by everybody. It is understandable that publishing may require more stringent compatibility checking policy.

Task Compatibility Checking

Also Note, even with the strictest compatibility checking policy, unless the function signature of new version is the same as that of old version, there will be chances that a task is not compatible with some historical version.

So the task compatibility checking must be performed during task version upgrade or downgrade.

Below are the task compatibility check rules:

  • The parameter count of task matches that of the signature (except for the trailing Nullable parameters)
  • Each value matches the corresponding data type and values are valid
    • Value of special device types represents a valid device of the correct type
    • The constraints set by attributes shall be met
    • The value of enumeration type is within the range of the corresponding definition of the particular Documentation
  • For Table(Key/Value pair) data type, the required keys shall be defined in corresponding value in the task parameters (Note Nullable key/value pair is not required)

Documentation Inheritance

Once a new version of Application is deployed, its version number will be updated. Although the new Application is required to be “backward compatible” with the old published version, it may contain new functions. The function signatures of new version may be compatible with the old version but not exactly the same as in old version. The new version shall come with updated Application Documentation in all localized languages available. The development tool shall synchronize all localized Documentation from Application Store to local Application Engine storage before deployment. During the new version deployment process, all localized Documentation shall be updated with the following rule.

  • Because backward compatibility is ensured with compatibility check, for functions that are backward compatible, the function Documentation can be simply copied from the specific localized version. Only exception is the expanded enumerations or extra Nullable type nodes. The extra items shall be copied from the Documentation of the developer’s initial locale, which may need to be translated later on.
  • For new functions introduced in the new version, the Documentation of functions will be cloned from the developer’s initial localized Documentation.
  • Anything extra in the new Documentation may need to be translated later on by somebody.

Sandboxing

As of now, the Application Engine is a Lua virtual machine running in a separate process (for isolation purpose). In case of failure, such as application unresponsiveness or runtime failure such as out of memory, the Application Engine can mark the culprit application as error and restart the engine process instead of restarting the entire system.

Each application will be running in a separate sandbox within the virtual machine. Sandbox provides security isolation.

System Variables

Controller implements a concept called system variable. System variables are a collection of key/value pairs managed by the core of the controller system, which can be accessed by Application code through API.

The key shall be a string. The value can be of any type or complex type.

System variable can be ephemeral, i.e. the data will be lost after a system restart/reboot and initially will be null on startup until the value is assigned.

System variable can be persistent also, every time the value changes, the new value is written into persistent storage. At the time of the system start up, the persistent system variables will be loaded from persistent storage.

If system variable is persistent, the value has to be JSON serializable. In order to be JSON serializable, the value shall be either a primitive type, or a complex type (array or table) that doesn't have circular reference. Also the type or any member of the type shall not have custom metatable; otherwise the deserialized value may lose custom functions.

System variable can also be used with user interface. For example, in user interface, a sensor output may be linked to a system variable. That system variable can be a string value that is managed by an Application task that interprets the sensor reading into user friendly text in localized language. For example, a task can be scheduled to monitor an air quality sensor and modify the associated system variable based on the sensor reading. The output could be more user friendly text such as “Excellent”, “Good”, “Fair” and “Poor” for air quality. The output to the system variable will be displayed on user interface, such as a smartphone. When the value of system variable changes, the controller will automatically send the new value to user interface to reflect the change.

System variable can be used for inter-task communication. The access (read/write) of the system variable shall be atomic. And the system provides API that can block and wait for the change of the value of system variable, for example, QWHA_WaitMultiple API shall have an entry of a list of system variable names.

System Variable Scope

A system variable shall have a scope. Scope is identified by application. Each application creates a unique scope. There are 3 types of scopes.

  • Application scope - System variable is bound to the application that creates the system variable.
  • Task scope - System variable is bound to the task that creates the system variable.
  • Global scope - Global scope is a special scope and it is the scope for some well known system variables built in the system.

The concept of scope enables access control for security reasons and completely eliminates the possibility of name conflict.

Application code may access system variable created by another application or task. There are API calls that allow application to access system variable. Scope is one parameter of the API calls.

MasterState System Variable

One system variable defined in Global scope is named “MasterState”, with at least two possible values, “Occupied” and “Vacant”, which indicate the occupancy status of the house. The “MasterState” can be set by user and read from Application code.

The application code may incorporate the “MasterState” into the control logic. For example, when the house if vacant, automatically adjust the set-point of the heating and air-conditioning unit to conserve energy. Here is another example. When the house is vacant and occupancy sensor detects motion of human, take appropriate action to notify user and other agencies such as police station.

QWHA leaves users flexibility in "MasterState" definition. The extended states must have either "Occupied." or "Vacant." as prefix. For example, "Occupied.BedTime", "Occupied.GoodMorning", "Vacant.ComingHome" etc. This specification will help applications that only cares about general states correctly parse the state string.

Concurrency

As we mentioned before, user creates Tasks based on application functions. And App Engine is an operating system of IoT. Each active task is a process running on the App Engine as an OS.

As you may have noticed in the sample applications source code, lot of Applications are essentially infinite loop. The App Engine is able to schedule the Tasks so that all tasks appear to be running concurrently just like the processes running concurrently on modern operating systems.

Scheduler

In our App Engine we implemented a cooperative scheduler, based on Lua coroutine. As a result the Lua coroutine related API are banned in user code.

While a cooperative scheduler may sound like going backward to 1980's, it actually suits our application very well.

First of all, all Tasks can be scheduled within only one thread, which makes our system very efficient especially on underpowered processors. On a first generation Raspberry Pi with only single 700 Mhz CPU, with dozens of sample Tasks active, the CPU utilization of App Engine is well below 1%. The response latency is within single digit of miliseconds.

Secondly, programmer usually won't tell difference of the scheduler. As long as in the code the API that start with "QWHA_" keep getting called (those API should be called anyway), the code will implicitly trap into the scheduler and yield CPU to other Tasks.

Programmers only need to be careful when the code is going to do some very CPU intensive job, for example, doing calculation on fairly large matrices. In that case, a "QWHA_Yield()" call shall be inserted somewhere in the loop to explicitly yield CPU to keep the system responsive.

For whatever reason, if a Task is unresponsive for over 10 seconds, the Task will be marked as "Error" and it will not be executed again. Then the host App Engine process is going to restart and recover. Note this is classic Turing Halting Problem and clock counting is the best we can do.

The same rule applies to Application code itself. Note the Application loading process is simply executing the Application code to get variables initialized. So if an application can't load within 10 seconds, it will be marked as error and engine process will recycle.

Try to insert a simple infinite loop "while true do end" in to a function or an application code, start the task, you will see what is going to happen. Note the error detail will be written to system log as well.

10 seconds is already pretty generous. In practice every task shall keep responsive within 10 milliseconds.

App engine employs mechanism that dynamically monitors the execution of every task and thread. If any thread consistently blocks execution for over 10 milliseconds, developer shall take measures to improve the responsiveness (and it's easy to do, simply inserting QWHA_Yield() in long loop of your source code).

Threads

Task is a process. The App Engine also introduces the concept of thread. When a task is started, a thread as "Task main thread" is automatically created for entry function exeuction. The App Engine provides API for task code to create more threads.

Synchronization

As of now, the best way to achieve synchronization among threads and tasks is to use system variable. We offer both blocking and non-blocking API to for a thread to "wait for" update of system variable.

Resource Management

QWHA Controller is design to run forever. The App Engine design must be robust to deal with all possible scenarios.

  • Task creation, deletion, enabling and disabling
  • Application add, remove, development
  • Task application upgrade/downgrade
  • task/application runtime errors

Note user can perform application/task operations at any time. The system is designed to favor responsiveness over any other factors.

File and Persistent System Variable

App Engine provides API for file and persistent system variable access. Both can be used to store persistent information to storage. There are similarities and differences.

  • Both are isolated within sandbox. For example, Task A and Task B both create create a file "f.txt". Those two files are actually located within different virtual root directory.
  • File is associated with Task. Once Task is removed, the file the task created will be automatically cleaned.
  • Persistent System Variable can be associated with Application. When it does, even though there are more than one version of Application in the system, if code of more than one versions access a system variable with same name, the same system variable will be accessed.
  • Persistent System Variable can also be associated with Global Scope.
  • Further more, multiple Tasks that based on the same function may also possibly access the same system variable within system or application scope.
  • System variables are serialized into and deserialized from JSON. Programmer is responsible for the storage format of the file. Please consider the performance impact for each method.

File Management

App Engine runtime keeps track of the open files for each Task. After the last thread of a Task is terminates, all open files will be automatically closed.

System Variable Management

System variables within global scope are available all the time. Generally applications only have read access to those variables.

System variables within application scope are only available if at least one version of application in active. In other words, application is active if at least one active instance of task that is based on at least one version of application is running.

System variables within task scope are only available if task is running.

If no active application is running, all system variables within that application scope will be garbage collected and any read access to associated system variable will result in "Nil" value. Likewise if a task is not running, all system variables within that task scope will be garbage collected and any read access to associated system variable will result in "Nil" value.

Memory Management

Task and Application Foot Print

A version of Application is not longer active if no Task based on any of the function defined in this version of application is currently running. When that happens, that version of application will be subject for garbage collection.

No Global Variables

App Engine disables global variable creation after the application module is loaded. Of course if you are familiar with Lua, the module loading process is the process of creating global variables of public functions (within the environment of sandbox). So we only disable new variable creation after the loading.

Disabling the global access improves safety a lot. After all because functions has to execute as instance of tasks. Tasks are independent from each other. So there is really no reason to have global variables shared among functions.

Task and Thread Local Storage

Application is required to maintain its own task and thread local storage. It is easy to be achieved, simply declaring a table inside the task or thread entry functions.

Network and Other Resources

App Engine provides API to access network and other local hardware, such as serial port. Those resource are also tracked and will be automatically cleaned when Task is terminated

Safety, Security and Privacy

QWHA takes user's security and privacy very seriously. The security and privacy model gives end user complete control over their privacy in the most user friendly way.

There will be needs to interacting with outside world (for example cloud services) for very good reason. For example, interacting with power plant with information such as your projected near term energy usage will benefit both end user and power plant, and will help reduce carbon emission greatly. Our design can achieve this goal without sacrificing user's privacy or losing user's control over privacy.

Below is list of safety, security and privacy features:

  • Each task is running in a sandbox. Application engine tracks the usage of different resources, such as CPU, memory, file system, network connections. If any resource usage exceeds the predefined limit, the Task will be terminated and marked as “error”.
  • Application engine can also track task activities, such as device I/O access rate, log message writing rate. If the rate exceeds the predefined limit, the Task will be terminated and marked as “error”.
  • App Engine can use implicit device access control, for example, Task can only access devices specified in Task parameters; otherwise the Task will be terminated for access violation.
  • Attributes can be used to request explicit access permission. For example, a Task may request to connect to a specific host identified by hostname or IP address. If Task tries to connect to other site other than the request hostnames, the Task will be terminated for access violation. A listening socket port shall also be represented as an attribute type.
  • When user deploys a Task with explicit access permission request attributes. A warning dialog shall prompt with all request listed for user to grant the access explicitly.

System Events and Wait API

Most of the application shall be an infinite loop. Actually all application shall be an infinite loop in QWHA.

Most of the time, a task shall be in "wait" state, either waiting for certain events to happen, or simply sleeping for a given interval (or waiting for certain events with a timeout).

Simple Blocking Wait

The QWHA_WaitXXXX API are used for waiting on certain type of objects. For example:

All wait functions above are blocking with optional timeout value in miilseconds.

Multiple Wait Events Registration

QWHA offers API that registers events on waitable objects. Application code can register wait on multiple objects of any type, then call QWHA_Wait() function to wait for any registered to happen (with optional timeout).

Code snippet below demonstrates how to wait on multiple objects of different types.

Wait On Multiple
  1. QWHA_SetWaitBegin()
  2. QWHA_SetWaitDevice(some_devices)
  3. QWHA_SetWaitSystemVariableTask(some_name)
  4. QWHA_SetWaitNio(some_nio_fd)
  5. QWHA_Wait()  -- or QWHA_Wait(time_out_milliseconds)

NIO Library

App engine offers it's own version I/O library, QWHA_NIO.

HTTP Client

HTTP client supports both HTTP and HTTPS.

Code snippet below demonstrates how to use HTTP client to download weather forecast data from National Weather Service website.

HTTP Client Demo
  1. local QWHA_HTTP_USER_AGENT = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36 QWHA"
  2. local lat, lon = QWHA_GetCoordinates()
  3. local query = "/MapClick.php?lat=" .. lat ..
  4.     "&lon=" .. lon .. "&unit=0&lg=english&FcstType=json"
  5. local headers = {}
  6. table.insert(headers, {"Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"});
  7. table.insert(headers, {"Cache-Control", "max-age=0"});
  8. table.insert(headers, {"Connection", "keep-alive"});
  9. table.insert(headers, {"User-Agent", QWHA_HTTP_USER_AGENT});
  10. local weatherNWSFd = QWHA_NioNewHttpClient(
  11.     "forecast.weather.gov",
  12.     80,
  13.     false,
  14.     "GET",
  15.     query,
  16.     headers)
  17. QWHA_WaitNio(weatherNWSFd, 1)
  18. local success, response = QWHA_NioGetHttpResponse(weatherNWSFd)
  19. QWHA_NioClose(weatherNWSFd)     -- Note, must close the fd.

Support for more network protocols, including raw TCP and UDP client/server, is going to be released soon.

QWHA Builtin Information Services

Using the NIO HTTP client, QWHA offer API to get some commonly used information service. Currently weather service from National Weather Service (for 50 United States and Puerto Rico) and openweathermap (http://openweathermap.org).

QWHA_WeatherNWS and QWHA_WeatherOWM, download data from the NWS or OWM website. Our API have cache mechanism to avoid flooding the websites with excessive requests even when called concurrently from several tasks simultaneously. In fact NWS data only updates for every hour.

Application code can call weather API to get weather information as frequent as it requires. However, our API only makes actual requests to the weather websites no frequently than every 30 minutes.

Note if any error occurs (lost internet connection or drastic data format change), the weather API may return nil or stale data.

NWS Weather Service

QWHA_WeatherNWS returns a structure transformed from raw NOAA XML data. The transformed data is much easier to process.

The data is a collection of many time series object. Each time series object has an option "units" member. Table below is a sample of time series names and units.

Time Series Name Uints
temperature_dew_point Fahrenheit
temperature_minimum Fahrenheit
wind_speed_incremental50 percent
humidity_minimum_relative percent
probability_of_precipitation_12_hour percent
wind_speed_gust knots
wind_speed_sustained knots
precipitation_liquid inches
fire_weather_risk_from_wind_and_relative_humidity
conditions_icon_forecast_NWS
precipitation_snow inches
fire_weather_risk_from_dry_thunderstorms
cloud_amount_total percent
wind_speed_cumulative64 percent
humidity_relative percent
wind_speed_cumulative50 percent
wind_speed_cumulative34 percent
wind_speed_incremental34 percent
wind_speed_incremental64 percent
humidity_maximum_relative percent
temperature_hourly Fahrenheit
direction_wind degrees true
temperature_maximum Fahrenheit
precipitation_ice inches
temperature_apparent Fahrenheit

Each time series object has a member named "data", data is an array of objects. Rach object has 3 members.

  • startTime - The start time of the period.
  • endTime - Optional. The end time of the period.
  • value - a number, or nil

Code snippet below retrieves rain amount (in inches) for next few hours (customizable as input parameter) from NWS web service.

Rain Amount from NWS
  1. function WeatherRainAmountNWS(nextHours)
  2.     local data = QWHA_WeatherNWS()
  3.     local rainAmount = nil
  4.     if (data) then
  5.         local ts = data.precipitation_liquid
  6.         if (ts) then
  7.             local curTimeUtc = os.time()
  8.             local endTimeUtc = curTimeUtc + nextHours * 3600
  9.             rainAmount = 0
  10.             for _, p in ipairs(ts.data) do
  11.                 local time = p.endTime
  12.                 if (not p) then
  13.                     time = p.startTime
  14.                 end
  15.                 if (time > curTimeUtc and time < endTimeUtc) then
  16.                     if (p.value) then
  17.                         rainAmount = rainAmount + p.value
  18.                     end
  19.                 end
  20.             end
  21.         end
  22.     end
  23.     return rainAmount
  24. end

OWM Weather Service

QWHA_WeatherOWM returns a JSON deserialized data from OpenWeatherMap 5 day 3 hour data. For more information about the format, please refer to http://openweathermap.org/forecast5.

Note OWM service requires an App ID, which can be requested for free from OWM website. The ID must be specified in the "Settings" dialog from QWHAAdmin management GUI tool.

Code snippet below demonstrates how to retrieve rain amount (in centimeters) for next few hours (customizable as input parameter).

Rain Amount from OWM
  1. function WeatherRainAmountOWM(nextHours)
  2.     local rainAmount = nil
  3.     local data = QWHA_WeatherOWM()
  4.     if (data) then
  5.         data = data.list
  6.     end
  7.     if (data) then
  8.         local curTimeUtc = os.time()
  9.         local endTimeUtc = curTimeUtc + nextHours * 3600
  10.         rainAmount = 0
  11.         for _, p in ipairs(data) do
  12.             if (p.dt and p.dt > curTimeUtc and p.dt < endTimeUtc) then
  13.                 if (p.rain and p.rain['3h']) then
  14.                     rainAmount = rainAmount + p.rain['3h']
  15.                 end
  16.             end
  17.         end
  18.     end
  19.     return rainAmount
  20. end

Simple Task Life Cycle

The hub and App Engine are designed to keep up and running for months or even for years. On the other hand, the hub may be powered off any time by user or power outrage. Some Apps may be problematic. Once the hub detects a problematic App, the hub may have to kill the App Engine process and restart it.

As of right now we choose the simple life cycle of tasks. User can mark a task as in "production" mode so that the App Engine can start the task at start up.

While the hub is up and running, end user can always manually stop and restart a task, from the Task Manager GUI.

Developer is solely responsible for the state management of the application code, if there is any state to maintain. Keep in mind a task is like a process in traditional OS. So state initialization code should be at the beginning of the task entry function, and usually before the infinite loop of the task code.

The task can be "stopped" by end user at any time, and there is no "clean up" callback features so far. That fact may affect the state persistence mechanism of the application. However, so far we don't see any problem with this simple life cycle. Please do keep in mind that no OS will guarantee that the process will always have a chance to bail out gracefully.

Also please keep in mind that our hub is equipped with flash memory, which may wear out pretty quickly with excessive writes. So even if application is designed to collect some data periodically, it shall not write to files or system variable too frequently.

Also the application design shall be robust against the fact that the task may be terminated before some data have a chance to be persisted.

If application is designed to write to a file, under rare occasions the file may be corrupted because of events such as power failure. Using the create temp file, write to temp, rename old file, rename temp, delete old file technique will alleviate the chance of damage.

Debugging

The App Engine is tightly integrated with the system log. The system log can be used as output device. QWHA exposes API function QWHA_Log.

For example, the code below:

Code Snippet
  1. function TestLog()
  2.     QWHA_Log(0, "Hello world!");
  3. end

The above code, when scheduled as a task named "Test Log", will produce an entry in system log viewer:

More debugging tools will be released soon.

API References

Lua Builtin Functions

Following Lua builtin functions are exposed:

bit, ipairs, next, pairs, pcall, tonumber, tostring, type, select, pack, unpack, error,

string functions

string.byte, string.char, string.find, string.format, string.gmatch, string.gsub, string.len, string.lower, string.match, string.rep, string.reverse, string.sub, string.upper

table functions

tableInsert, tableRemove, table.maxn, table.sort, table.getn, table.pack, table.unpack

math functions

math.abs, math.acos, math.asin, math.atan, math.atan2, math.ceil, math.cos, math.cosh, math.deg, math.exp, math.floor, math.huge, math.fmod, math.frexp, math.ldexp, math.log, math.log10, math.max, math.min, math.modf, math.pi, math.pow, math.rad, math.random, math.sin, math.sinh, math.sqrt, math.tan, math.tanh

os functions

os.clock, os.difftime, os.time, os.date

io functions

io.write, io.open

Third Party API

lom module

LuaExpat is a third party library for XML parsing. It is exposed as lom module. Functions below are exposed.

lom.parse, lom.encode

Note lom.encode can be used to encode a lom format table into XML. For more informaiton about lom, please visit the LuaExpat website.

json module

A jason module from http://regex.info/blog/lua/json is exposed to App Engine with name "json". Below is the usage:

local lua_value = json:decode(raw_json_text)
local raw_json_text = json:encode(lua_table_or_value)

LuaDate

Documentation of LuaDate API can be found here: https://github.com/Tieske/date.

LuaData is exposed with name "date".

QWHA API

Below is the table of current public QWHA API. Note the information may be subject to change and more API will be added over time.

QWHA_DeviceGetLevel( id )

Getting the device level. Note this call may block for a while for certain type of devices in order to poll the device state through a round trip of request/response. Note this call is subject to permission check.

Parameters:
  • id – the device ID
Returns:
  • status – true if the return value is valid
  • result – a value indicating the current device level, typically between 0 and 255
  • counter – device state counter
  • source – the source that caused the last device state change

QWHA_DeviceGetOnOff( id )

Getting the device on/off state. Note this call may block for a while for certain type of devices in order to poll the device state through a round trip of request/response. Note this call is subject to permission check.

Parameters:
  • id – the device ID
Returns:
  • status – true if the return value is valid
  • result – a value indicating the current device on/off state; true is on
  • counter – device state counter
  • source – the source that caused the last device state change

QWHA_DeviceGetThermostat( id )

Getting the current state of thermostat. Note this call may block for a while for certain type of devices in order to poll the device state through a round trip of request/response. Note this call is subject to permission check.

Parameters:
  • id – the device ID
Returns:
  • status – true if the return value is valid
  • result – the result table, with possible members listed below:
    • mode - current mode of thermostat. Following valid values: "off", "heat", "cool", "auto", "fan", "program", "program heat", "program cool", "emergency heating", "precooling",
    • heat_setpoint - terperature threshold that triggers the heating
    • cool_setpoint - terperature threshold that triggers the cooling
    • temperature - current terperature
  • counter – device state counter
  • source – the source that caused the last device state change

QWHA_DeviceSetLevel( id, state [, sync ] )

Setting the level of the device. Note this call is subject to permission check.

Parameters:
  • id – the device ID
  • state – a value indicating a valid level, typically between 0 and 255
  • sync – Optional. true indicating the function shall be blocking and only returns after receiving an acknowledgement from device or some error occurs (e.g. device timeout)
Returns:

true if success; otherwise false. Note a return of success only means the device is valid and supports setting level.


QWHA_DeviceSetOnOff( id, state [, sync ] )

Setting the on/off state of the device. Note this call is subject to permission check.

Parameters:
  • id – the device ID
  • state – true indicates on; false indicates off; or any number less than or equals to 0 means off and number greater than 0 means on
  • sync – Optional. true indicating the function shall be blocking and only returns after receiving an acknowledgement from device or some error occurs (e.g. device timeout)
Returns:

true if success; otherwise false. Note a return of success only means the device is valid and supports setting level.


QWHA_DeviceSetThermostat( id, state [, sync ] )

Setting the thermostat state. Note this call is subject to permission check.

Parameters:
  • id – the device ID
  • state – a table with thermostat state information.
    • mode - current mode of thermostat. Following valid values: "off", "heat", "cool", "auto", "fan", "program", "program heat", "program cool", "emergency heating", "precooling",
    • heat_setpoint - terperature threshold that triggers the heating
    • cool_setpoint - terperature threshold that triggers the cooling
  • sync – Optional. true indicating the function shall be blocking and only returns after receiving an acknowledgement from device or some error occurs (e.g. device timeout)
Returns:

true if success; otherwise false. Note a return of success only means the device is valid and supports setting level.


QWHA_DeviceSetThermostatMode( id, mode [, sync ] )

Setting the mode of thermostat only. Note this function differs than QWHA_DeviceSetThermostat in that QWHA_DeviceSetThermostat also take heat and cool set points if necessary while this function only sets the mode. The thermostat will be the most recently heat and cool set points in its memory. Note this call is subject to permission check.

Parameters:
  • id – the device ID
  • mode – a valid value indicating the thermostat mode. possible values: "off", "heat", "cool", "auto", "fan", "program", "program heat", "program cool", "emergency heating", "precooling"
  • sync – Optional. true indicating the function shall be blocking and only returns after receiving an acknowledgement from device or some error occurs (e.g. device timeout)
Returns:

true if success; otherwise false. Note a return of success only means the device is valid and supports setting level.


QWHA_SetWaitBegin( )

Clears previously registered waits. This function shall be called first before a series of other QWHA_SetWaitXXXX calls followed by a QWHA_Wait() call.

Returns:

none.


QWHA_SetWaitDevice( ... )

Register wait on given devices. Note this call is subject to permission check.

Parameters:
  • ... – input is variable argument list. Each element in the list can be either a device ID or another array of device IDs.
Returns:

none.


QWHA_SetWaitSystemVariableGlobal( name )

Register wait on specified system variable in global scope. Note this call is subject to permission check.

Parameters:
  • name – system variable name.
Returns:

none.


QWHA_SetWaitSystemVariableApp( name, [, app ] )

Register wait on specified system variable in App scope. Note this call is subject to permission check.

Parameters:
  • name – system variable name.
  • app – optional. the app ID, if absent use QWHA_GetCurrentApp()
Returns:

none.


QWHA_SetWaitSystemVariableTask( name [, task ] )

Register wait on specified system variable in task scope. Note this call is subject to permission check.

Parameters:
  • name – system variable name.
  • task – optional. the app ID, if absent use QWHA_GetCurrentTask()
Returns:

none.


QWHA_SetWaitNio( fd, events )

Register wait on specified NIO object. Note this call is subject to permission check.

Parameters:
  • fd – fd of the NIO object
  • events – 1-Read; 2-Write; 3-Both
Returns:

none.


QWHA_Wait( [ timeout ] )

Blocking the current thread execution until at least one of previously registered QWHA_WaitXXXX event happened.

Parameters:
  • timeout – optional. timeout in milliseconds
Returns:

value of positive number indicates number of events happened; 0 indicates timeout


QWHA_WaitDevice( ... )

Blocking the current thread execution until there is a state change on at least one device in the list. Note this call is subject to permission check.

Parameters:
  • ... – input is variable argument list. each element in the list can be either a device ID or another array of device IDs
Returns:

value of positive number indicates number of events happened


QWHA_WaitDeviceTimeout( timeout ... )

Blocking the current thread execution until there is a state change on at least one device in the list; or timeout has been reached. Note this call is subject to permission check.

Parameters:
  • timeout – timeout in milliseconds
  • ... – input is variable argument list. each element in the list can be either a device ID or another array of device IDs
Returns:

value of positive number indicates number of events happened; 0 indicates timeout


QWHA_WaitSystemVariableGlobal( name [, timeout ] )

Blocking the current thread execution until the system variable is updated; or timeout has been reached. Note, system may update the system variable with the same value as the old one. As long as some thread made a "set" call to the system variable it is counted as an update. Also note if one thread makes more than one "set" call to the same system variable in a row, other thread may only receive one "update" notification and may only get the latest updated value. System variable is used as a synchronization object. This call is subject to permission check.

Parameters:
  • name – system variable name
  • timeout – optional. timeout in milliseconds
Returns:

value of 1 indicates number of events happened; 0 indicates timeout


QWHA_WaitSystemVariableApp( name [, app [, timeout ] ] )

Blocking the current thread execution until the system variable is updated; or timeout has been reached. Note, system may update the system variable with the same value as the old one. As long as some thread made a "set" call to the system variable it is counted as an update. Also note if one thread makes more than one "set" call to the same system variable in a row, other thread may only receive one "update" notification and may only get the latest updated value. System variable is used as a synchronization object. This call is subject to permission check.

Parameters:
  • name – system variable name
  • app – optional. the app ID, if absent use QWHA_GetCurrentApp()
  • timeout – optional. timeout in milliseconds
Returns:

value of 1 indicates number of events happened; 0 indicates timeout


QWHA_WaitSystemVariableTask( name [, task [, timeout ] ] )

Blocking the current thread execution until the system variable is updated; or timeout has been reached. Note, system may update the system variable with the same value as the old one. As long as some thread made a "set" call to the system variable it is counted as an update. Also note if one thread makes more than one "set" call to the same system variable in a row, other thread may only receive one "update" notification and may only get the latest updated value. System variable is used as a synchronization object. This call is subject to permission check.

Parameters:
  • name – system variable name
  • task – optional. the task ID, if absent use QWHA_GetCurrentTask()
  • timeout – optional. timeout in milliseconds
Returns:

value of 1 indicates number of events happened; 0 indicates timeout


QWHA_GetSunriseSunset( )

Getting the sunrise and sunset time of the current day.

returns:
  • sunrise – sunrise time of current day, seconds from midnight in local time
  • sunset – sunset time of current day, seconds from midnight in local time

QWHA_GetTimeOfDay( )

Getting time of day in seconds from midnight in local time.

Returns:

time of day in seconds from midnight in local time


QWHA_GetCurrentApp( )

Getting the current App ID.

Returns:

a number indicating the current App ID. Note different versions of the same App share the same AppID


QWHA_GetCurrentTask( )

Getting the current Task ID.

Returns:

a number indicating the current task ID


QWHA_GetCurrentThread( )

Getting the current thread ID.

Returns:

a number indicating the current thread ID. Note the thread ID is ephemeral and may change after controller restarts


QWHA_GetSystemVariableGlobal( name )

Getting the system variable in global scope. This call is subject to permission check.

Parameters:
  • name – name of the global system variable
Returns:

value of the system variable; or nil if the value is not defined. Note the value is immutable. If the value is a table, the content of the table cannot be modified


QWHA_GetSystemVariableApp( name [, app ] )

Getting the system variable in app scope. This call is subject to permission check.

Parameters:
  • name – name of the system variable
  • app – optional. the app id, if not presented it will be result of QWHA_GetCurrentApp()
Returns:

value of the system variable; or nil if the value is not defined. Note the value is immutable. If the value is a table, the content of the table cannot be modified


QWHA_GetSystemVariableTask( name [, task ] )

Getting the system variable in task scope. This call is subject to permission check.

Parameters:
  • name – name of the system variable
  • app – optional. the task id, if not presented it will be result of QWHA_GetCurrentTask()
Returns:

value of the system variable; or nil if the value is not defined. Note the value is immutable. If the value is a table, the content of the table cannot be modified


QWHA_SetSystemVariableApp( name, value [, persistent ] )

Setting the system variable in app scope. This call is subject to permission check.

Parameters:
  • name – name of the system variable
  • value – a value. note if the persistent is true the value has to be JSON serializable
  • persistent – optional. boolean value. the app id, if true the value will be persisted on disk or flash
Returns:

a immutable version of the value if the value is a table; otherwise the original value


QWHA_SetSystemVariableTask( name, value [, persistent ] )

Setting the system variable in task scope. This call is subject to permission check.

Parameters:
  • name – name of the system variable
  • value – a value. note if the persistent is true the value has to be JSON serializable
  • persistent – optional. boolean value. the app id, if true the value will be persisted on disk or flash
Returns:

a immutable version of the value if the value is a table; otherwise the original value


QWHA_NewThread( func, [, ... ] )

Creating a new thread with specific entry function and input arguments.

Parameters:
  • name – an entry function
  • ... – variable argument list of the input parameters of the entry function
Returns:

none.


QWHA_NioNewHttpClient( hostname, port, tls, method, query, headers [, body ] )

Creating a new HTTP client and start sending request and then downloading response. Note the call may raise error if the maximum concurrent open connections per task is reached. Also note this call is subject to permission check.

Parameters:
  • hostname – a host name. Note this value is not an URL. An example is "google.com".
  • port – port number. Typically it is 80 for HTTP and 443 for HtTPS
  • tls – boolean value, true if uses HTTPS
  • method – HTTP method. Example: "GET" or "POST"
  • query – query string. Example: "/" or "/index.html"
  • headers – list of headers. Example:
    local headers = {}
    table.insert(headers, {"Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"});
    table.insert(headers, {"Cache-Control", "max-age=0"});
  • body – optional; string representing the HTTP request body
Returns:

an fd (integer) representing the new HTTP client.


QWHA_NioGetHttpResponse( fd )

Getting the HTTP response. Note, QWHA_NioClose must be called to close the fd when the client is no longer needed. Also note this call is subject to permission check.

Parameters:
  • fd – the fd returned from previous QWHA_NioNewHttpClient() call
Returns:
  • success – boolean value; true if the HTTP response is received successfully
  • response – HTTP response object if succeeded; otherwise a string representing the reason of failure.
    The response is a table with the following fields:
    version - the HTTP version. Should be 1.1
    code - the HTTP response code. For example "200" or "404" etc.
    status - the HTTP response status. For example "OK" for code "200"; or "not found" for code "404" etc.
    headers - the HTTP response header. It's same structure as headers in QWHA_NioNewHttpClient.
    body - the string representing the response body or nil if no body.

QWHA_NioClose( fd )

Close the HTTP client session.

Parameters:
  • fd – the fd returned from previous QWHA_NioNewHttpClient() call.
Returns:

none.


QWHA_Sleep( milli )

Blocking the execution of current thread for specified milliseconds.

Parameters:
  • milli – integer value of milliseconds
Returns:

none.


QWHA_Yield( )

Yielding CPU to other threads. Note this function shall be called in the loop of very CPU intensive jobs. It is more efficient than QWHA_Sleep(0).

Returns:

none.


QWHA_Log( level, message )

Writing an entry to system log.

Parameters:
  • level – a level between 0 and 5; indicating TRACE, DEBUG, INFO, WARN, ERR, FATAL
  • message – message string
Returns:

none.


QWHA_WeatherNWS( )

Getting current weather forecast data from Nation Weather Service.

Returns:

a data structure represents the weather data, or nil if error occurs. For more information please refer to NWS Weather Service.


QWHA_WeatherOWM( )

Getting current weather forecast data from OpenWeatherMap.org.

Returns:

a data structure represents the weather data, or nil if error occurs. For more information please refer to OWM Weather Service.