Premium networks are best evaluated by comprehensively analyzing the intrinsic links between key performance indicators (KPIs), key quality indicators (KQIs), and quality of experience (QoE). Linking KPIs, KQIs, and QoE enhances the accuracy of network planning and optimization capabilities, and helps carriers build and evaluate high-quality networks.
The development of mobile broadband, especially in the LTE era, drives the rapid growth of data services. GSMA statistics reveal that LTE users consume twice as much data traffic per month as 3G users. At a time when user experience is of paramount importance, how can carriers tackle the challenges posed by big data traffic and build premium networks? McKinsey suggests that carriers in the big data era can prioritize the following targets: optimize network performance and service quality to improve user experience; analyze network data for more accurate traffic forecasts and network capacity assessments; and utilize new technologies and solutions to enhance operation and maintenance (O&M) efficiency.
How can we make good use of quantitative indicators such as KPIs, KQIs, user experience, capacity, and O&M efficiency to evaluate top-quality networks? Like a physical examination, assessing network quality requires rigorous systems and experience. With advanced methods and extensive experience, carriers can focus on key indicators that genuinely reflect network quality and user experience to arrive at scientific and persuasive conclusions.
Optimal seven-dimension network evaluation model
Based on successful partnerships with various carriers, Huawei has designed an optimal seven-dimension evaluation program that incorporates the following indicators – coverage, capacity, interference, network KPIs, NE health, O&M, and KQIs.
The seven-dimension model incorporates not only technical indicators, but also non-technical O&M indicators. Meanwhile, the focus of O&M has shifted from NE assurance to service assurance. Only key indicators that are closely related to user experience are used as criteria for network quality assessments. Each dimension is divided into sub-indicators, so the entire evaluation system covers all the important quantitative indicators that can be used to assess the quality and experience offered by a mobile broadband network. By grading each network indicator as optimal, good, moderate or poor, and giving an overall score, we can identify the gap between a current network and a top-quality network.
Improving network KPIs for a better user experience
The scores of key indicators reflect the health of the network. Network assessments can expose many pain points for telcos such as weak coverage, high loads, high interference, and high failure. The "bucket effect" tells us that the shortest bucket board determines the capacity of a bucket. So, for telcos, it is crucial to reduce network loads, interference and failure rate by repairing the shortest board. Enhancing user experience begins with locating network problems and optimizing key indicators.
Network coverage:Coverage is a key indicator in the seven-dimension network model for quality evaluation; for example, it ensures that users can readily access services. Although telcos in different countries face different network conditions, coverage remains the basic guarantee for service provision. Therefore, many telcos prioritize coverage when seeking to build top-quality networks.
For example, Carrier A from Europe considers coverage, especially key areas, its top priority for modernizing networks and constructing a leading LTE network. By strengthening LTE deployment in key cities coupled with precise network planning and optimization, the carrier significantly improved coverage and markedly reduced the call drop rate. As a result, it receives fewer user complaints due to a greatly improved user experience. Now, Carrier A plans to expand LTE coverage to over 70% nationwide in urban areas and to 80% in hotspots such as airports, CBDs and highways.
In network development, high loads, interference and failure rates greatly compromise user experience and require immediate solutions. An evaluation on a high-quality network project run by an African carrier revealed numerous problems, including low download rates, poor voice quality and high complaint rates. A more thorough analysis found that these problems were caused by poor indoor coverage because the number of HSDPA users in CBD areas was too high, causing high loads and interference between 2G and 3G signals. Based on the assessment result, the project team introduced coverage and capacity enhancement features, and carried out refined network optimization in areas of high interference. As a result, both network performance and user satisfaction improved.
Comprehensive analysis:We select key network indicators that reflect the problems with user experience as the criteria for quality assessments. For example, for coverage evaluation we choose the proportion of weak coverage; for NE assessment we look at the number of faulty NEs; and for network quality we select the proportion of low-rate links and links with low CSFB rates. These network assessment indicators are often correlated and interactive. Therefore, we need to conduct a holistic analysis on key indicators, including KPIs, coverage, NEs, capacity, interference and O&M. We can then evaluate their combined influence on network performance and user experience. In summary, we should focus on delivering accurate network optimization and strengthening weak links to enhance overall network performance.
Service KQI assessment to enhance application performance
Currently, KQIs – such as Europe’s P3 and U.K.’s ROOT tests – are the most commonly used indicators to assess network quality. Service KQIs are mainly QoS parameters used to assess the network performance and user experience of different services. Based on industry standards and field experience, Huawei has built its own KQI system that incorporates tests on voice, SMS, web browsing and video. This system conforms to ETSI standards, covers the KQI scenarios of the P3 test, and includes an indicator for improving user experience.
For example, mobile Operator A from Europe came last in a benchmark network quality test conducted by a third party in 2013, falling far behind its competitors. At that time, Operator A’s management was experiencing considerable difficulties. Based on its quality criteria system, Huawei analyzed network requirements and decided to focus on improving user experience and network quality. By analyzing service quality and other indicators, Huawei helped the customer accurately locate network problems. With its advanced KQI system, accurate planning capabilities, experience and tools, Huawei carried out deep service and network optimization for the telco, holistically improving numerous network indicators and greatly enhancing the customer's network and application performance. In later tests, the carrier surpassed its competitors and topped the ranking lists.
Future evolution of network quality evaluation
Due to differences in mobile network complexity, area, and the competitive environment, mobile operators differ in terms of mainstream services, competitiveness, customer requirements and quality standards. Therefore, the indicators of the quality assessment system should be customized (indicator addition/deletion and threshold adjustment) for different networks.
In the big data era, top-quality networks that focus on user experience have become a key point for competitiveness among mobile operators. As the top-quality networks become increasingly centered on user experience, quality evaluation will continue to evolve. For example, the focus on assessments will shift from the quality of service types (video and web) to the service quality offered by certain apps (e.g., YouTube and WeChat). In turn, voice quality assessments will shift from CS-centered voice to voice over IP (VoIP).
As the O&M model of global carriers transforms from network-focused to service-focused, they will gradually take service quality indicators as key indicators for assessing premium networks. With the service operations center (SOC) as the key and the top-quality network quality assessment system as the quantitative standard, carriers will be able to precisely improve network planning and optimization capabilities, guarantee a superior user experience and carry out precision marketing. A premium network that features quality services and enhanced network performance not only delivers excellent user experience and profitability, but also increases ROI for carriers and maintains their competitiveness in the market.
