Dependability Competition
Following the success of last year's edition, the International Conference on Embedded Wireless Systems and Networks (EWSN 2017)
will host also this year a dependability competition to compare the performance of WSN and IoT communication protocols in environments rich of radio interference.
Contestants must first submit an abstract describing their approach in detail by the contest registration deadline. The authors of the selected abstracts will then be invited to participate in a two-days dedicated competition session and asked to deploy their software on wireless sensor nodes deployed in an environment with controlled radio interference.
EVALUATION SCENARIO
The competition rules and evaluation scenario will resemble the ones of last year's edition.
Contestants will be asked to deploy their software on a few off-the-shelf Maxfor MTM-5000 sensor nodes (TelosB replicas operating in the 2.4 GHz ISM band) and to dependably communicate despite the surrounding radio interference. This hardware choice allows contestants to prepare by testing their protocols on most public WSN testbeds.
One of the sensor nodes (sensing node) will be placed in proximity of a light source and should monitor its brightness using the embedded light sensors. Any sudden variation in the lighting condition should be promptly communicated to a sink node that will trigger one of its I/O pins accordingly. The sink node is not in the communication range of the sensing node and additional forwarding nodes are available in the surroundings in order to reach the sink in a multi-hop fashion.
The light source will be turned off and on according to a secret schedule. The delay with which each change in the lighting condition will be reported to the sink are going to be measured and considered in the evaluation metric.
RF interference will be generated in the competition area in a repeatable fashion to ensure fairness across multiple experiments. All sensor nodes will be USB powered and can be reprogrammed remotely. The number of nodes available and their exact position will not be revealed to the contestants and the ID of the nodes can be read from the on-board 1 MB external flash.
EVALUATION PROCEDURE
Competitors will be asked to provide a compatible binary file that will be uploaded on the sensor nodes and to let the sink communicate the status of the light source using the expansion pins or common serial tools. To ensure compatibility with the testbed setup, additional information and code examples will be provided at a later stage.
Solutions will be evaluated based on (i) the reliability of transmissions, i.e., on the number of changes in the lighting condition correctly reported to the sink, (ii) on the end-to-end latency in communicating each change in the lighting condition to the sink node, and (iii) on the energy-efficiency of the solution. The latter will be measured at each node using external hardware. For further information please visit the EWSN 2016 competition Website.
CATEGORIES AND PRIZE:
Depending on the nature and number of submissions, multiple categories may be defined and awards will be given to the
winners in each category. Accepted abstract will appear in the ACM digital library (upon approval of the authors) and the
top-classified contestants in each category will also be invited to present their approach in a dedicated session at the conference.
ELIGIBILITY:
Both academia and industry submissions are encouraged. All communication protocols used in low-power wireless sensor networks
are welcome, and there is no restriction on the operating system used to program the sensor nodes.
SUBMISSION INSTRUCTIONS:
Formatting requirements. Contestants must submit an abstract describing their approach in detail by the contest registration deadline. Submissions must be written in English and can have a maximum length of 2 pages including figures, tables, and references.
Pages must have 8.5" x 11" (letter) two-column format, using 10-point type on 11-point leading, with a maximum text block of 7" wide x 9" deep with an inter-column spacing of .25". Authors may use the LaTeX templates provided here. Abstracts should prominently include the name of the authors as well as their affiliation and contact information. The abstracts will be pre-selected and the accepted ones will appear in the ACM Digital Library, unless authors explicitly mark the abstract as confidential.
Submission. To submit your solution, please email your abstract to with the following subject line: "EWSN 2017 Competition - Submission".
IMPORTANT DATES:
Contest Registration and Abstract Deadline [extended!]: November 24, 2016, 23:59 EST.
Notification of acceptance: December 1, 2016
Competition: February 18-19, 2017
ORGANIZERS:
Carlo Alberto Boano, TU Graz
Markus Schuss, TU Graz
FAQ – Frequently Asked Questions
Where will the competition take place?
The competition will take place indoors in the Polacksbacken campus of Uppsala University (old military building, 3rd floor). The exact location and number of sensor nodes will not be disclosed beforehand, in order to avoid solutions that are specifically engineered in advance for the evaluation scenario. However, all contestants are provided with an upper bound on the number of nodes and their density, so that they can correctly dimension their solution. In particular, contestants can expect not more than 20 wireless sensor nodes that will be deployed over an area of approximately 350-400 m2. Compared to the previous year's edition, the sensor nodes will be deployed over a larger area, hence expect a sparser setup.
Is there going to be only one sensing node generating “changes in the lighting condition” events (light on / light off)?
Yes, there is going to be only one sensing node that generates events to be captured and communicated to the sink. No data needs to be collected from the other nodes that will act as forwarders only.
How often will the “changes in the lighting condition” be scheduled?
Changes in the lighting condition will follow a predefined pattern that will not be disclosed beforehand. The light source will, however, not change in sub-second timescales.
Can the “changes in the lighting condition” already occur at time 0?
No, there will be some slack time from boot time in order to allow contestants to carry out node and topology discovery. The duration of this slack time will be approximately 15 seconds.
How is interference practically generated?
Interference will be generated using JamLab. Multiple wireless sensor nodes will act as interferers across the whole 2.4 GHz ISM band. The reproduced interference will resemble the interference patterns of common appliances (e.g., Wi-Fi devices and microwave ovens). Contestants cannot assume that some IEEE 802.15.4 channel will be constantly interference-free. Please note that JamLab records and emulates interference patterns from real-world devices, but does not back-off and stop interfering in case one of the sensor nodes is transmitting strong messages nearby (as, for example, Wi-Fi access points do [ref]).
Will JamLab be already active since time 0?
Yes, JamLab will be active since the beginning of the experiment, i.e., also during node and topology discovery (if any). However, interference may vary over time, and the one recorded at time 0 is not necessarily representative of the one that may occur later.
Can additional interference be present in the area?
During the competition days, the Wi-Fi access points of the building should be disabled and no particular sources of interference are expected. We will nevertheless monitor the radio interference in the surroundings and, in case some particular disturbance is detected, the experiment will be repeated.
How is the energy consumption of the nodes measured? Is it going to be measured on all nodes?
Power consumption of nodes will be measured using additional hardware. In principle, the measurement will be carried out on all nodes and will start at boot time, i.e., node and topology discovery (if any) will account for the total power consumption of the sensor nodes.
Can the submission be kept confidential?
Yes, there is the possibility to keep the submission confidential (no publication in the ACM digital library, and no disclosure of ranking after the competition). Each contestant will be specifically asked about this at a later stage.
Software upload: is a single version of the firmware necessary?
Contestants will be asked to provide a single firmware for all nodes (source, sink, relays).
Is the reading of the sensor given or should it be implemented by each contestant?
As the sampling strategy may affect all three metrics (reliability, latency, and energy-efficiency), each team should come up with its own strategy. In order to fine-tune the operation of the light sensor (e.g., select a correct threshold), a dedicated setup and preparation session will be available before the competition. The sensor nodes used are Maxfor / Advanticsys MTM-5000 sensor nodes, and the light sensor properly works with common software (e.g., using Contiki’s sensor reading on the sky platform).
How does a node knows its identity (source, sink, relay)?
The ID of each node can be read from the on-board external flash. In this case, the ID is an unsigned short (16 bits) number, and an example program in Contiki on how to read it from flash is available here. Alternatively, also the 48-bit unique ID chip (DS2411) can be used.
How does the sink node signal it received a packet to the outside world?
The sink will turn one of the GPIO pins of the expansion header high (light on) or low (light off) according to received packets. The exact pin specifications will be communicated on a later stage.
Is there any limitation in the number of packets sent? Is repeated signaling for the same packet allowed?
The sink only reproduces on the GPIO pins the light pattern of the source node. There is no constraint on the number of packets sent.
Will the contestants have USB access to the nodes in the testbed during the competition?
During the preparation phase, the contestants will be able to log serial output and will receive it via e-mail. During the competition, however, the FTDI interface of the nodes should be disabled to ensure minimal energy consumption.
Where will the competition take place?
The competition will take place indoors in the Polacksbacken campus of Uppsala University (old military building, 3rd floor). The exact location and number of sensor nodes will not be disclosed beforehand, in order to avoid solutions that are specifically engineered in advance for the evaluation scenario. However, all contestants are provided with an upper bound on the number of nodes and their density, so that they can correctly dimension their solution. In particular, contestants can expect not more than 20 wireless sensor nodes that will be deployed over an area of approximately 350-400 m2. Compared to the previous year's edition, the sensor nodes will be deployed over a larger area, hence expect a sparser setup.
Is there going to be only one sensing node generating “changes in the lighting condition” events (light on / light off)?
Yes, there is going to be only one sensing node that generates events to be captured and communicated to the sink. No data needs to be collected from the other nodes that will act as forwarders only.
How often will the “changes in the lighting condition” be scheduled?
Changes in the lighting condition will follow a predefined pattern that will not be disclosed beforehand. The light source will, however, not change in sub-second timescales.
Can the “changes in the lighting condition” already occur at time 0?
No, there will be some slack time from boot time in order to allow contestants to carry out node and topology discovery. The duration of this slack time will be approximately 15 seconds.
How is interference practically generated?
Interference will be generated using JamLab. Multiple wireless sensor nodes will act as interferers across the whole 2.4 GHz ISM band. The reproduced interference will resemble the interference patterns of common appliances (e.g., Wi-Fi devices and microwave ovens). Contestants cannot assume that some IEEE 802.15.4 channel will be constantly interference-free. Please note that JamLab records and emulates interference patterns from real-world devices, but does not back-off and stop interfering in case one of the sensor nodes is transmitting strong messages nearby (as, for example, Wi-Fi access points do [ref]).
Will JamLab be already active since time 0?
Yes, JamLab will be active since the beginning of the experiment, i.e., also during node and topology discovery (if any). However, interference may vary over time, and the one recorded at time 0 is not necessarily representative of the one that may occur later.
Can additional interference be present in the area?
During the competition days, the Wi-Fi access points of the building should be disabled and no particular sources of interference are expected. We will nevertheless monitor the radio interference in the surroundings and, in case some particular disturbance is detected, the experiment will be repeated.
How is the energy consumption of the nodes measured? Is it going to be measured on all nodes?
Power consumption of nodes will be measured using additional hardware. In principle, the measurement will be carried out on all nodes and will start at boot time, i.e., node and topology discovery (if any) will account for the total power consumption of the sensor nodes.
Can the submission be kept confidential?
Yes, there is the possibility to keep the submission confidential (no publication in the ACM digital library, and no disclosure of ranking after the competition). Each contestant will be specifically asked about this at a later stage.
Software upload: is a single version of the firmware necessary?
Contestants will be asked to provide a single firmware for all nodes (source, sink, relays).
Is the reading of the sensor given or should it be implemented by each contestant?
As the sampling strategy may affect all three metrics (reliability, latency, and energy-efficiency), each team should come up with its own strategy. In order to fine-tune the operation of the light sensor (e.g., select a correct threshold), a dedicated setup and preparation session will be available before the competition. The sensor nodes used are Maxfor / Advanticsys MTM-5000 sensor nodes, and the light sensor properly works with common software (e.g., using Contiki’s sensor reading on the sky platform).
How does a node knows its identity (source, sink, relay)?
The ID of each node can be read from the on-board external flash. In this case, the ID is an unsigned short (16 bits) number, and an example program in Contiki on how to read it from flash is available here. Alternatively, also the 48-bit unique ID chip (DS2411) can be used.
How does the sink node signal it received a packet to the outside world?
The sink will turn one of the GPIO pins of the expansion header high (light on) or low (light off) according to received packets. The exact pin specifications will be communicated on a later stage.
Is there any limitation in the number of packets sent? Is repeated signaling for the same packet allowed?
The sink only reproduces on the GPIO pins the light pattern of the source node. There is no constraint on the number of packets sent.
Will the contestants have USB access to the nodes in the testbed during the competition?
During the preparation phase, the contestants will be able to log serial output and will receive it via e-mail. During the competition, however, the FTDI interface of the nodes should be disabled to ensure minimal energy consumption.
