Workshop on “Monitoring and Benchmarking of Quality of Service and

Workshop on “Monitoring and Benchmarking of Quality of Service and Quality of Experience of Multimedia Services in Mobile Networks” (Buenos Aires, Argentina, 24-25 July 2014) Quality of Service Measurements of Mobile Networks Yvonne UMUTONI Quality of Service Development Group (QSDG) Chairperson [email protected] 250 782052394

EXECUTIVE SUMMARY This presentation focuses on the quality of service monitoring and enforcement for mobile networks from the Regulatory point of view. It covers the following: purposes of QoS regulations Requirements for a Regulator in order to enforce Quality of service Purpose of Auditing the QoS and auditing methods prerequisites so as to conduct a QoS Measurement Campaign QoS Parameters of some mobile services such as voice, sms, data Buenos Aires, Argentina, 24-25 July 2014

QUALITY OF SERVICE REGULATION QoS is defined as the “Collective effect of service performance, which determines the degree of satisfaction of a user of the service” [ITU-T Recommendation E.800]. QoS regulation is part of customer protection; But customer protection is broader than QoS regulation and QoS is not the same as network performance, which is more concerned with standards for the network, not user experience. Buenos Aires, Argentina, 24-25 July 2014

QUALITY OF SERVICE REGULATION (Cont ) Main purposes of QoS regulation are [ITU-T Supp. 9 of E.800 Series]: Helping customers be aware of the Quality of service provided by Telecom Operators/ ISPs through networks (mobile & fixed), so that to make their own choices; Checking claims by operators; Understanding the state of the market; Maintaining / improving the QoS in presence of competition; Maintaining / improving the QoS in absence of competition; Helping operators to achieve fair competition; and Making interconnected networks work well together. Buenos Aires, Argentina, 24-25 July 2014

ENFORCEMENT OF QUALITY OF SERVICE Requirements so as to enforce the quality of service: i. ICT Law: Adopted and published in the Official Gazette ii. Regulations / Guidelines of Quality of Service in Mobile networks: With a purpose: Improve / maintain service quality; make QoS Information available to customers; assist the development of telecommunication markets and; improve operation and performance of interconnected networks. Guidance about: Methodologies for measuring, reporting and recording. QoS parameters definitions and thresholds. Buenos Aires, Argentina, 24-25 July 2014

ENFORCEMENT OF QUALITY OF SERVICE (Cont.) iii. Reports of QoS submitted Monthly or Quarterly by Telecom Operators to the Regulatory Authority: Include technical and non technical parameters iv. QoS Monitoring Tools for auditing the QoS of mobile networks independently v. Staff in charge well trained Buenos Aires, Argentina, 24-25 July 2014

AUDIT OF QUALITY OF SERVICE Purpose of QoS Auditing : Verify the QoS of Mobile Networks experienced by customers and Compare the Results (from audit exercise) against the licence obligations Methods to audit telecom operators’ mobile networks are, but not limited to: Drive Test (performed on quarterly basis or any time required) Consumer survey Data submitted on monthly or quarterly basis by Mobile Telecom Operators Etc. Buenos Aires, Argentina, 24-25 July 2014

METHODOLOGIES FOR MEASUREMENT This presentation focuses on Drive Test Methodology QoS Measurement Campaign prerequisites: Specify Locations (e.g.: City, sector) Prepare maps for those locations Calculate Samples (attempts) required for each location based on population: For more information, please refer to ITU-T Recommendation E.804. Calculate number of days/ hours required Calculate number of hotspots (for measurement of data services) and locate those hotspots Prepare a script for each service (e.g: Voice, FTP, HTTP ) For Voice service measurement, specify the Mode (e.g: GSM, 3G or Dual mode) Buenos Aires, Argentina, 24-25 July 2014

METHODOLOGIES FOR MEASUREMENT (Cont.) Example of Voice call sequence: Start loop Make a voice call from MOC to MTC Call duration: wait 90, or 120 seconds MOC disconnects the call (Hang-up) Pause (Idle Mode) time: wait 20 seconds End loop (go back to step 2) Buenos Aires, Argentina, 24-25 July 2014

QUALITY OF SERVICE PARAMETERS VOICE KPIs Call Setup Success Rate (CSSR) Call Setup time (CST) Call Drop Rate (CDR) Speech Quality Service coverage Area (Signal level): -95 dBm. Buenos Aires, Argentina, 24-25 July 2014 Definitions CSSR denotes the probability that the end-user can access the mobile telephony service when requested if it is offered by display of the network indicator on the user equipment CST describes the time period between sending of complete address information by the MOC and receipt of call set-up notification or an alerting message. CDR denotes the probability that a successful established attempt is ended unintentionally by MOC or MTC party Speech quality on sample basis: is an indicator representing the quantification of the end-to-end speech transmission quality of the mobile telephony service. The Service coverage area verification depends on Operator’s coverage plan.

QUALITY OF SERVICE PARAMETERS (Cont ) KPIs for SMS SMS Service Accessibility SMS Completion Ratio SMS End-to-End Delivery Time [s] Definitions SMS service accessibility denotes the probability that the enduser can access the SMS when requested while it is offered by display of the network indicator on the user equipment. SMS Completion Ratio is the ratio of successfully received and sent messages from the sending UE (user equipment) to the receiving UE, excluding duplicate received and corrupted messages. SMS end-to-end delivery time is the time period between sending a short message to the network and receiving the very same short message at another UE. Buenos Aires, Argentina, 24-25 July 2014

QUALITY OF SERVICE PARAMETERS (Cont ) KPIs Data services Definitions Attach failure ratio Attach failure ratio denotes the probability that a subscriber cannot attach to the Packet Switched (PS) network. Attach Setup time Attach setup time describes the time period needed to attach to the PS network PDP Context Activation Failure Ratio PDP context activation failure ratio denotes the probability that the PDP context cannot be activated. It is the proportion of unsuccessful PDP context activation attempts and the total number of PDP context activation attempts. PDP Context Activation Time PDP context activation time describes the time period needed for activating the PDP context. PDP Context Cut-off Ratio PDP context cut-off ratio denotes the probability that a PDP context is deactivated without being initiated. Buenos Aires, Argentina, 24-25 July 2014

PUBLICATION For Consumer protection and Awareness purposes, the Quality of Service results from Audit Campaign (drive test, consumer survey.) are published by Regulators, on website, or in magazine for a regulator; Buenos Aires, Argentina, 24-25 July 2014

Thank you for your attention Buenos Aires, Argentina, 24-25 July 2014