ICE 6: End to End Test of i3 Architecture
Share |


November 9-14 2014
Final Report

The NENA i3 architectural standard for NG9-1-1 defines functional elements, their interfaces, and their interactions to provide end to end delivery of emergency calls. There are many vendors who provide functional elements and interfaces. In order to build reliable, secure, inter-operable networks and NG9-1-1 services, vendors are asked to work together to ensure the introduction of NG9-1-1 goes smoothly.


NENA is now inviting vendors to participate in the next NG9-1-1 interoperability Industry Collaboration Event (ICE). At recent ICE events, we tested various components and aspects of the i3 architecture, however, the various aspects were not tested simultaneously. ICE 6 focused on comprehensive end-to-end functionality, interaction between vendor elements (external interfaces) and interoperability testing.

More information on ICE:

  • Delaine Arnold – NENA ICE Testing Coordination Manager - 813.960.1698
  • Roger Hixson – NENA Technical Issues Director - 202.618.4405


Planning Committee

·        Brian Knueppel – (Oracle) Chair
·        Wolfgang Kampichler – (Frequentis) Vice Chair


Chris Flynn

Jay Malin

Ameel Kamboh
Airbus DS Communications

Dave Staub

Dan Banks
Digital Data Technologies

Michael Smith
DSS Corporation

Matthew Daughtrey

Hal Koch

Wolfgang Kampichler

Brooks Shannon

Robert Bowman

James Kinney

Christian Militeau

Suresh Gursahaney

Phil Reichl

Simon Smith

Brian Knueppel

Jerry Eisner
RedSky Technologies

Rob Plaza

Simon Farrow

Patrick Voigt

Tom Dong

Matt Hampton
Voice Print International


ICE 6 primarily focuses on processing calls end to end. This requires core functional elements which process calls and queries, and it also includes supporting elements which record and report. Testing might include elements which provide additional data, MIS, CAD and bridging.

·        Baseline testing includes dialing 911 and PSAP callback.

·        Transfer/conference

·        Location verification. Exercise various PIDF-LO formats and content. Forest Guide top-level.

·        Routing based on calling party. This will exercise multiple vendors equipment and various interop scenarios.

·        Reroute or alternate routing based on failure scenarios.

·        Handling floods or overloads.

·        Interface testing verifies SIP messaging is consistent. Verify URI/URN, headers, cause codes.

·        Multimedia testing includes voice, video, text (includes TTY).

·        Recording multimedia: SRC and SRS.

·        Logging events.

·        Elements in the i3 architecture which provide or process additional call data (ACDR).

·        Legacy equipment and interfaces (ie TDM) with SIP via gateways.

·        Prioritization/QoS, ie Diffserv.


The ICE 6 Planning Committee will develop appropriate Scenarios, Test Plans, and Data Collection and Reporting mechanisms for executing the interoperability tests. The collection/reporting mechanisms for this event will be developed with the assistance of Devery Thumann of the ICE Steering Committee, who headed the Data Collection and Reporting Committee in previous ICE events.



Plan and execute an event that:

1.     Attracts and encourages the widest possible participation of vendors that have products in the focus area (even those outside of the traditional 9-1-1 marketplace).

2.     Allows all participating vendors to test all valid architectures and configurations requested.

3.     Delivers relevant feedback on standards to the appropriate NENA Technical Committee(s) and may identify gaps in existing standards work.

4.     Allows vendors to better understand the interoperability between their implementations and products developed by other vendors.

5.     Allows for the gathering of relevant results data that will be reported to the NENA Technical Committees, to other SDOs, and, at a high level, to all interested parties, all in formats appropriate to the audience in question.




ICE 6 Planning Process


ICE planning is conducted in phases as listed below.  The ICE planning process, this document, is owned and updated by the ICE planning committee and is reviewed for continued applicability at the start of each ICE planning cycle. This document, the planning process, serves as an overview and repository of what should occur at each phase and is intended to live across subsequent ICE events.


Phase 1- Scenario Planning

The scenario planning phase is used to work out what configurations and call flows will be tested as part of the ICE. All Planning Committee members are invited and encouraged to propose scenarios and call flows.


The scenario plans should include sufficient detail to determine which nodes are required and what their high-level functions are. Once the high-level scenarios are determined, explicit interfaces, message-types as well as success and failure paths can be identified.


Scenarios are built around the inventory of vendor functional elements, for which a template is provided and is filled out at the start of each ICE. The actual vendor functional elements available will vary between events based on which vendors choose to participate. If some functional elements are not available from participating vendors, a plan is needed to address this deficit. If a satisfactory plan cannot be agreed to then the scenario is dropped from the formal ICE test plan. If this occurs, individual vendors may still agree to test the scenario at ICE, but their results are not considered in the overall ICE summary.


The person or company proposing a scenario is responsible for planning and managing specifying the actual test prior to the  event.  All scenarios must adhere to published (draft or final) architectures and standards and bench marks for draft specifications must be tabled, agreed to and anchored prior to the start of bilateral test planning.  Where there is a void in the published architectures or standards, proposals can be made but must not be proprietary. Again, if agreements cannot be reached then the scenario or aspects of the scenario are dropped from the formal test set.


Once all the scenarios are documented, an evaluation is conducted to determine if any of the scenarios can be combined into a single scenario with multiple permutations and combinations.  Once the evaluation and/or consolidation is completed, the final set of scenarios forms the basis for the ICE testing.


An inventory of required infrastructure is developed based on the agreed scenarios.  Once this inventory is agreed on, a search is done to locate a suitable site or sites for the event.  


Only vendors that have functional elements required in one or more scenarios are permitted to participate in ICE. Additional elements such as CAD, MIS etc are welcome. It is preferred if these products operate in a passive mode so as to avoid interruption of the actual scenario under test.



- All participating vendors must provide a list of the functional elements they plan to test at the event.



Phase 2 – Bilateral and Group Test Planning

The scenarios developed in Phase 1 document a number of point-to-point or device-to-device interactions.  Scenarios are likely to have many of these for an end-to-end test.  To build up to conducting a full scenario test, it is proposed to test each individual point-to-point connection first, whereas some scenarios, especially those including media, may already require end-to-end testing.  An inventory is developed of all possible vendor combinations of point-to-point or ‘bilateral tests’ that could be conducted. 


Many bilateral test cases have been written and executed for previous ICE events. These tests will be considered simple enough that they do not have to be conducted.  Some, simply cannot be conducted on a bilateral basis.  All bilateral tests called for in the scenarios will be categorized using these designations (additional categories may be created as required).  A plan is developed to conduct and track all bilateral tests that are deemed to be necessary prior to full scenario testing. 


Recently, the complexity of scenarios, especially those that comprise end-to-end testing, has increased and in addition the correct configuration of functional elements is getting more important or else is key to successfully complete test cases. Therefore, the planning process recommends extending bilateral testing with group testing. Pre-condition for group testing is a shared network infrastructure that interconnects devices required for group testing (preferably VPN). Further all configurations that are required in order to route end-to-end shall comprise data that is identical to what is envisioned for the event. This ensures baseline interoperability among involved elements and validates the correctness of configuration data (e.g. location sets, mappings etc.). A plan is developed to conduct and track all group tests that are deemed to be necessary prior to full scenario testing.


Execution of bilateral tests and group tests prior to the ICE itself is a precondition of participation in scenario testing. That is, priority for scenario testing is given to those vendors that indicate that bilateral testing has been completed. Once all these vendors have completed their scenario testing then vendors yet to complete bilateral testing are able to attempt the scenarios if time permits. This ensures that maximum time is available to complete the complex scenario tests.


If appropriate, bilateral test scripts/tools developed in previous events may be used for each of the point-to-point combinations.  These scripts/tools can be used as a starting point for test scripts in new ICE, and repository for these tools will be created.  Each ICE undoubtedly introduces new functionality and/or functional elements thus requiring the development of new test scripts/tools.  Vendors bringing these new FEs to the event are asked to collaborate on the creation of new bilateral test scripts/tools.


ACTION required by the Planning Committee and/or members

- The PC selects a person to lead this effort. He/she works with the teams planning the scenarios to develop the list of bilateral tests and categorize them. Once the list is complete and approved by the Planning Committee, this person manages the plan and tracks progress.

- Vendors bringing in FEs that were not exercised in previous events need to collaborate in the development of detailed bilateral test scripts.



Phase 3 – Participant preparation

This phase is set aside for vendor participants to prepare for the bilateral and full scenario testing.  While it is hoped that no custom development or effort is required by the participants, there are several reasons this may be necessary.  Some of them include:

-          To support bilateral testing prior to the event, some vendors may want to make their FEs/products available on the Internet.  If these products are not already available remotely, some work may be required.  In previous events, several vendors had staff attend the actual event but left their equipment back in their labs.

-          Some tests may involve standards or architectures that are allowable under published documents but may not have been considered by the participants and may require additional development.

-          Through Phase 1 and Phase 2, some participating vendors may become aware of standards or architectures that they were previously unaware of or the publication of new versions of documents they were aware of.

-          Some participating vendors, as a result of talking to other participating vendors, discover their interpretation of a standard is different than the interpretation of the standard by other vendors.  If there is a correct interpretation, all vendors should modify their systems to adhere to it.

-          Where there is a void in the standards or the published architectures, vendors may propose a reasonable, non-proprietary substitute.  This proposal, if accepted by the Planning Committee, may require modifications to one or more functional elements.  Any vendor providing one of these FEs may opt to participate in the test by doing the necessary development work. 

-          In the spirit of ICE, all vendors are encouraged to participate in as many bilateral and full scenario tests as possible.  Since this is a technical event, participants are discouraged from avoiding tests for business reasons.  No results will be published externally so participation and non-participation in tests is confidential.



Phase 4 – Implementation

Implementation begins with bilateral tests that can be conducted prior to the event using the Internet and continues on through to the actual event.  During this phase, the leader of the bilateral testing oversees that set of tests and each of the scenario leads oversees the execution of their tests.


ACTION required by the Planning Committee and/or members

- The Planning Committee selects a target date for the event. If the general suggested timelines are reasonable, the          Planning Committee targets dates in mid second or fourth quarter.

- The committee develops a master schedule for all tests occurring prior to the event and at the event.

- Each team lead (each scenario and the bilateral testing) implements their test plans.


Parallel efforts

There are four other efforts that are undertaken in parallel with the phase outlined above.


Parallel effort 1 – Data Gathering and Reporting

The Chair of the Data Planning and Reporting committee is engaged in the planning from the beginning.  All teams described above need to keep this Chair involved in their activities.   The Chair may provide guidance to the various teams on what is needed to meet the reporting committee’s objectives.  All Reporting Committee Chair requests should be accommodated.


Parallel effort 2 – Host location selection

Illinois Institute of Technology, IIT, has been selected to host ICE-6.


Parallel effort 3 – Event logistics

Some amount of work needs to be done to plan for event logistics.  Event credentialing, acknowledgement of the Code of Conduct, group meals, and other items need to be planned. This role is generally performed by NENA staff and the role is appointed at the start of the ICE planning phase.


There will be a $75, per person, Participant Registration Fee for ICE 6 that will aid in covering the cost of the event. This will assist in covering the Wednesday night social, break food items and other associated expenses with the event. Coffee and tea will be provided throughout the day.  Participants are requested to use the IIT vending machines for soft drinks and water or bring your own.  All Participants must be registered by October 1, 2014.  There will be no refunds, but you may substitute a person for someone already paid, if necessary, by providing Delaine Arnold the name registered and the substitute’s name.

The link to register is


Parallel effort 4 – Feedback collection

Feedback and comments on standards that apply to ICE will be collected from PC members by the PC chair. The collected feedback and comments will be analyzed and provided to relevant standardization committees together with the overall result of the ICE after the event takes place.





Planning Committee Chair

Brian Knueppel

Planning Committee Vice-Chair

Wolfgang Kampichler

Data Planning and Reporting Committee Chair

Devery Thumann

NENA Logistics staff

Delaine Arnold





Scenario Planning start date:


Scenario Planning completion date:


Bilateral test planning start date:


Bilateral test planning completion date:


Bilateral test execution start date:


Bilateral test execution completion date:


Group testing start date


Group testing completion date


ICE start date:

11/10/2014 (setup Sunday 11/9/2014)

ICE conclusion date:


ICE Test Report date:




Illinois Institute of Technology – IIT

Wheaton Illinois