FCC Releases Draft Broadband Performance Testing Order On Reconsideration
October 11, 2019 – The Federal Communications Commission has released a draft of the broadband network performance testing Order On Reconsideration that the Commission will vote on at its next open meeting set for October 25, 2019.[1] The draft is not final, and is subject to change prior to the meeting.
The draft Order On Reconsideration revises and clarifies the July 2018 Performance Measures Order, which sets a framework for measuring speed and latency performance that applies to fixed broadband service providers receiving high-cost universal service fund (USF) support to serve fixed locations.[2] Those broadband providers must measure speed and latency from customer premises to a remote test server located at or reached by passing through an FCC-designated Internet Exchange Point (IXP). Providers must test their networks for compliance pursuant to one of three specific methods, and certify and report the results to the Universal Service Administrative Company (USAC) and relevant state or Tribal governments on an annual basis. Results are subject to audit, and failure to comply will result in a loss of USF support. A full summary of the FCC’s Performance Measures Order is available here.
First things first, does the draft Order announce when testing starts? Yes, and for many broadband providers, testing will not start for a few years. Each diferent high-cost USF/Connect America Fund support program has a specific start date for testing. Below is the schedule for the start of pre-testing and official testing:
CAF Phase II (Price-cap carrier funding): January 1, 2020 & July 1, 2020
Rural Broadband Experiments: January 1, 2021 & January 1, 2022
Alaska Plan: January 1, 2021 & January 1, 2022
A-CAM I: January 1, 2021 & January 1, 2022
A-CAM I Revised: January 1, 2021 & January 1, 2022
ACAM II: January 1, 2022 & January 1, 2023
Legacy Rate-of-Return; January 1, 2022 & January 1, 2023
CAF II Auction: January 1, 2022 & January 1, 2023
New NY Broadband Program January 1, 2022 & January 1, 2023
In the draft Order On Reconsideration, the Commission reviews the overall approach to performance testing “to ensure that those standards strike a balance between promoting accountable and effective use of universal service funds while providing the flexibility carriers need in light of carriers’ different sizes, networks, and technical abilities.” The Draft Order addresses petitions for reconsideration and applications for review of the Performance Measures Order in the following three ways: (1) Rejects proposed changes to the network performance testing methodology; (2) Makes changes the network performance testing rules; and (3) Clarifies the network performance testing methodology to resolve confusion.
No Change To End Points for Testing
The end points for testing remain the same – all high-cost support recipients serving fixed locations must perform speed and latency tests from the customer premises of an active subscriber to a remote test server located at or reached by passing through an FCC-designated IXP.[3]
The FCC disagreed with those arguing for testing over a shorter span – to the edge of a carrier’s network. This has been a major point of contention for rural carriers who must pass their traffic on to other networks to reach an IXP, meaning testing will include network elements a USF-supported carrier does not control. Once again, the FCC said a carrier should “purchase transport of a sufficient quality that enables it to provide the requisite level of service expected by consumers and required by the Commission’s rules.” And added, carriers can seek a waiver if the only level of transport available causes them to fail to meet the FCC’s network performance obligations.
New Definition of FCC-Designated IXP: Public Internet Gateway With Active Interface To Qualifying Internet Autonomous System
The FCC designated 16 IXPs in the Performance Measures Order, all large cities. However, the FCC has now revised the definition of IXP after concluding that using cities as IXPs is too ambiguous because “[i]t is not clear where the boundaries of a designated IXP metropolitan area begin and end.” In reality, the FCC probably just liked the revised definition of IXP proposed by ITTA and USTelecom better than its own.
For broadband network testing purposes, an FCC-designated IXP is defined as any building, facility, or location housing a public Internet gateway that has an active interface to a qualifying Internet Autonomous System (ASN). This could be either within a provider’s own network or outside of it.[4] The term “qualifying ASN” means the ASN is considered a connection to the public Internet, rather than another intervening connection point.
More specifically, an ASN is determined to be qualifying if it appears on the Center for Applied Internet Data Analysis’s (CAIDA) AS Rank List and meets the following criteria: (1) it is a “transit/access” ASN; (2) it is flagged in the U.S., Canada, or Mexico; (3) it has a Transit Degree of 100 or greater; (4) it peers with two or more of the top 300 USA-flagged ASNs on CAIDA’s AS Rank List; and (5) at least one of these peered ASNs is ranked in the top 100. There are 4 ASNs that meet these criteria but are excluded because analysis of PeeringDB and their websites indicate the ASNs do not peer with a significant portion of the public Internet – Akamai (32787); Florida International University (20080); WoodyNet (Packet Clearing House) (42); and eBay (backbone for eBay Inc.) (62955).[5]
The FCC has initially designated 44 major North American ASNs using the CAIDA ranking of Autonomous Systems and other publicly available resources. The initial list, which will be updated periodically, is available as Appendix A to the draft order.
Consequently, broadband providers may test from a customer location to any building, facility, or location housing a public Internet gateway that has an active interface to one of the qualifying ASNs or may petition the Bureaus to add additional ASNs to the list.[6]
No Change To Daily Test Period – Testing Must Be Conducted Between 6:00 p.m. & 12:00 a.m. Local Time
The Performance Measures Order established a daily testing period for speed and latency tests, requiring carriers to conduct tests between 6:00 p.m. and 12:00 a.m. local time, including weekends. Despite many challenges to the time in which broadband providers must perform network testing, the FCC declined to revisit the daily testing period.[7]
Clarification: Flexibility For Providers To Determine When To Commence Speed Tests
The Performance Measures Order requires a specified number of speed tests during each testing window – a minimum of one download test and one upload test per testing hour at each subscriber test location. The order required broadband providers to start separate download and upload speed tests at the beginning of each test hour window, and, after deferring a test due to crosstalk, providers were required to reevaluate crosstalk levels every minute until the speed test can be run or the one-hour test window ends. USTelecom, ITTA, and WISPA requested clarification that broadband providers are afforded flexibility in commencing hourly tests – providers should only be required to complete the test within the hour, and should be able to retry tests as frequently as their systems allow until a successful test is administered.
The FCC has clarified that broadband providers do not have to begin speed tests at the beginning of each test hour. However, providers must still conduct and report at least one download test and one upload speed test per testing hour at each subscriber test location, subject to the following:
A broadband provider that begins attempting speed tests within the first fifteen minutes of a testing hour, and repeatedly retries and defers the test at one-minute intervals due to consumer load meeting the adopted crosstalk thresholds (i.e., 64 Kbps for download tests or 32 Kbps for upload tests), may report that no test was successfully completed during the test hour because of crosstalk.
A provider that does not attempt a speed test within the first 15 minutes of the hour and/or chooses to retry tests in greater than one-minute intervals must, however, conduct and report a successful speed test for the testing hour regardless of crosstalk.[8]
No Change To Specific Latency Test Requirements
The Performance Measures Order requires a minimum of one discrete latency test per minute (i.e., 60 tests per hour), for each of the testing hours, at each subscriber test location, with the results of each discrete test recorded separately.[9] Many parties expressed concern and requested reconsideration of the latency testing framework, while a few parties supported the requirements. The FCC has declined to revise the requirements that broadband providers must conduct latency testing once per minute.[10]
Clarification: What Is Considered A Sufficient Latency Test
The FCC has provided additional explanation of what is considered a sufficient latency test to assuage concerns about the number of latency tests per hour. A latency “test” constitutes a “single, discrete observation or measurement of speed or latency.” While broadband providers may choose to continuously start and stop latency testing every minute and record the specific result, the FCC has clarified that there is no requirement to conduct latency testing in this manner. Instead, providers may continuously run latency testing software over the course of a test-hour and record an observation or measurement every minute of that test-hour. If a provider transmits one packet at a time for a one-minute measurement, the provider should report the result of that packet as one observation. However, some applications, such as ping, commonly send three packets and only report summarized results for the minimum, mean, and maximum packet round trip time and not individual packet round trip time. If this is the case, the provider should report the mean as the result of this observation. If the provider sends more than one packet and the testing application allows for individual round trip time results to be reported for each packet, then the broadband provider must report all individual measurements for each packet. In sum, providers have the flexibility to choose how to conduct their latency testing, so long as one separate, discrete observation or measurement is recorded each minute of the specific test-hour.[11]
No Change To Number Of Required Test Locations – Providers Having Difficulty Finding Sufficient Number Of Testing Locations May File A Waiver
The Performance Measures Order requires broadband providers to test a maximum of 50 subscriber locations per required USF-supported service tier offering per state, depending on the number of subscribers a provider has in a state, randomly selected every two years. Smaller broadband providers – those with fewer than 500 subscribers in a state and particular service tier – are required to test 10% of the total subscribers in the state and service tier, except for the smallest carriers – those with 50 or fewer subscribers – which must test 5 subscriber locations. If a broadband provider serving 50 or fewer subscribers in a state and particular service tier cannot find five active subscribers to test, the provider may test existing, non-USF-supported active subscriber locations within the same state and service tier to satisfy the requirement to test five active subscriber locations.
Many parties representing very small broadband providers argued in their petitions for reconsideration and applications for review that the required number of subscriber test locations is too high. Nevertheless, the FCC declined to modify the adopted sample sizes for testing speed and latency.[12]
The FCC did note that it recognizes that a few broadband providers facing unique circumstances may find it extraordinarily difficult to find a sufficient number of subscriber locations to test, and accordingly, they may file a waiver if they are unable to meet the requirements.
Change For Small Providers: No Testing Of Non-USF-Supported Locations
The FCC has revised the option given to broadband providers serving 50 or fewer subscribers in a state and particular service tier to test non-USF-supported locations, if a provider cannot find five active USF-supported subscriber locations to test.[13] Testing and reporting speed and latency for non-USF-supported locations, the FCC believes, adds unnecessary complexity to the broadband network testing requirements.
Accordingly, any non-compliant broadband provider testing fewer than five USF-supported subscriber locations because more are not available would be subject to verification that more customers are not available, rather than requiring that all providers testing fewer than five USF-supported subscriber locations find non-USF-supported locations to test.[14]
New Website Explaining Nature & Purpose Of Testing To Help Obtain Customer Participation
Many broadband providers have explained to the FCC that “obtaining customer consent for testing which requires placement of testing equipment on customer premises may prove difficult.” To help broadband providers, the FCC has directed its Wireline Competition Bureau to publish information on the FCC’s website explaining the nature and purpose of the required testing, and urging the public’s participation. The FCC “expect[s] that providing such information in an easy-to-understand format will help alleviate subscribers’ potential concerns.”[15]
Clarification: Same Sample Size For Speed & Latency, Same Locations For Speed & Latency
The FCC has clarified that the same sample sizes adopted for speed will also apply to latency. Therefore, broadband providers will test all of the locations in the random sample for both speed and latency.[16]
No Change To Requirement To Test To USF Obligation Speeds, Even If Locations Require Upgrades
There is no change to the requirement that broadband providers meet and test to their USF obligation speeds regardless of whether their subscribers purchase Internet service offerings with speeds matching the USF-required speeds for those USF-eligible locations.[17] The FCC acknowledges this may require some providers to upgrade individual subscriber locations, at least temporarily, to conduct speed testing.
Providers are required to test all locations in the random sample even if the provider has other customers that subscribe to the required service level. Requests to test only new locations were rejected by the FCC. Additionally, the FCC reminds broadband providers that they may not charge customers for any upgrades, new modems, or other testing equipment required to comply with testing obligations. Performance testing is a requirement of receiving USF support and is the responsibility of broadband providers, not customers.
No Change To Quarterly Testing Requirement
The Performance Measures Order requires broadband providers to conduct quarterly testing for speed and latency, which is intended to “capture any seasonal effects and differing conditions throughout the year that can affect a carrier’s broadband performance.” The FCC declined to adjust the quarterly testing requirement.[18]
Broadband Providers May Choose Among Three Testing Methods
The Commission confirmed that broadband providers may use any of the three methodologies outlined in the Performance Measures Order to demonstrate their compliance with network performance requirements. Broadband providers subject to network performance obligations are permitted to conduct testing by employing either: (1) Measuring Broadband America testing infrastructure (MBA testing), (2) existing network management systems and tools (off-the-shelf testing), or (3) provider-developed self-testing configurations (provider-developed self-testing or self-testing).
Clarification: Standards for Full Compliance
To achieve full compliance with latency and speed standards, the Performance Measures Order required that 95% of latency measurements during testing windows fall below 100 milliseconds round-trip time, and that 80% of speed measurements be at 80% of the required network speed. High-latency providers that will receive CAF Phase II auction support, must certify that 95 percent or more of all testing hours measurements are at or below 750 ms. There were petitions opposing, supporting, and seeking clarification of these standards for full compliance.
The Commission declined to modify the longstanding latency standard requiring that 95% of round-trip measurements be at or below 100 ms.[19]
As for speed tests, the Commission clarified that broadband providers are not required to provide speeds beyond what they are already obligated to deploy as a condition of their receipt of high-cost support. Thus, for a location where a provider is obligated to provide 10/1 Mbps service, the Commission only requires testing to ensure that the location provides 10/1 Mbps service, even if the customer there has ordered and is receiving 25/3 Mbps service.[20]
There is a change to the trimming of data in calculating compliance with the speed standard. The Commission rejected the Bureaus’ decision to exclude from compliance calculations any speed test results with values over 150% of the advertised speed for the location. The Commission has directed the Bureaus to study data collected from providers’ pre-testing and testing, and determine how best to implement a more sophisticated procedure using multiple statistical analyses to exclude outlying data points from the test results.[21]
No Change To Support Reduction Penalty For Non-Compliance
The Performance Measures Order established a framework of USF support reductions in the event that a broadband provider’s performance testing did not demonstrate compliance with its speed and latency standards. The Commission generally declined to revise the compliance and certification frameworks adopted by the Bureaus, disagreeing that the consequences for failure to meet performance measures are greater than that for failure to meet deployment obligations. As it reasoning, the Commission noted the lengthy pre-testing period, and the 95% and 80% standards for full compliance with latency and speed requirements, respectively.[22]
Clarification: Failure To Meet Speed And Latency Requirements Will Be Considered Failure To Deploy
The Commission clarified that, at the conclusion of a broadband provider’s buildout term, any failure to meet the speed and latency requirements will be considered a failure to deploy. In other words, a failure to comply with all performance measure requirements will result in the Commission determining that the provider has not fully satisfied its broadband deployment obligations at the end of its buildout term and subjecting the provider to the appropriate broadband deployment non-compliance support reductions.
Thus, if a broadband provider has deployed to 100% of its required locations, but only 90% of those locations meet the required speed and latency measurements, USAC will recover the percentage of the provider’s support equal to 1.89 times the average amount of support per location received in the state for that provider over the term of support for the relevant number of locations that do not meet the speed and latency requirements, plus 10 percent of the provider’s total relevant high-cost support over the support term for that state.[23]
Clarification: Penalties For Both Lack Of Deployment And Non-Compliance With Speed And Latency Standards
The Commission has clarified that at the end of the support term when USAC has performed the calculation to determine the total lack of deployment based on the numbers of locations to which a broadband provider has built out facilities and the number of locations that are in compliance with the performance measures, USAC will ensure that the total amount of support withheld from the provider because of failure to meet deployment milestones and performance requirements does not exceed the requirements of Section 54.320(d)(2). To facilitate this calculation, the Commission has revised the decision allowing carriers to recover only the support withheld for non-compliance for 12 months or less. Accordingly, when a non-compliant broadband provider comes into a higher level of compliance, USAC will now return the withheld support up to an amount reflecting the difference between the levels’ required withholding. By returning all the support USAC may have withheld from a provider for non-compliance, the non-compliance framework will continue to provide an incentive to broadband providers to return to full compliance with the speed and latency standards.[24]
Change: End of Buildout Term Failure Due To Small Sample Size
The Commission has provided additional flexibility at the conclusion of a broadband provider’s buildout term for any provider that has failed to meet its performance requirements and believes that its failure to do so is the result of a small sample size. Thus, if at the end of its term, a broadband provider is shown not to have met its deployment obligations due to a failure in meeting the speed and latency requirements, the provider can submit a request to the Bureaus for an increased size of random samples that will produce an estimate with a margin of error of 5% or less and conduct further testing to show that the provider is compliance with the Commission’s performance requirements. If, after this further testing, the broadband provider is able to demonstrate that it fully complies with the required speed and latency benchmarks, then the provider will be considered to have met the deployment obligations.[25]
Change: Revised Testing Schedule
The Commission has modified the scheduled start of performance testing to link speed and latency testing to the deployment obligations for broadband providers receiving support from each of the various high-cost support mechanisms.[26] Below is the schedule for the start of pre-testing and official testing for each USF/CAF program:
CAF Phase II (Price-cap carrier funding): January 1, 2020 – July 1, 2020
Rural Broadband Experiments: January 1, 2021 – January 1, 2022
Alaska Plan: January 1, 2021 – January 1, 2022
A-CAM I: January 1, 2021 – January 1, 2022
A-CAM I Revised: January 1, 2021 – January 1, 2022
ACAM II: January 1, 2022 – January 1, 2023
Legacy Rate-of-Return; January 1, 2022 – January 1, 2023
CAF II Auction: January 1, 2022 – January 1, 2023
New NY Broadband Program January 1, 2022 – January 1, 2023
Change: Testing Requirements Only For Alaska Plan Carriers With Defined Build-Out Requirements
NTCA–The Rural Broadband Association requested clarification that the Performance Measures Order applies only to high-cost recipients with mandatory build-out obligations. Some Alaskan rate-of-return carriers are subject to defined build-out obligations, while some are not. Consistent with NTCA’s request, the Commission has clarified that only carriers subject to defined build-out requirements are required to test the speed and latency of their networks. Accordingly, Alaskan rate-of-return carriers that have committed to maintaining existing service levels therefore are not subject to the broadband network performance measures.[27]
Alaskan rate-of-return carriers that have committed to defined build-out obligations, however, must conduct speed and latency testing of their networks.
Alaska Plan carriers using satellite or satellite backhaul must certify that 95% or more of all testing hour measurements of network round trip latency are at or below 750 ms for any locations using satellite technology. These carriers must certify annually that no terrestrial backhaul options exist, and that they are unable to satisfy the standard performance measures due to the limited functionality of the available satellite backhaul facilities. To the extent that new terrestrial backhaul facilities are constructed, or existing facilities improve sufficiently to meet the public interest obligations, funding recipients must meet the standard performance measures within twelve months of the new backhaul facilities becoming commercially available.[28]
**********
[1] Connect America Fund, WC Docket No. 10-90, Order On Reconsideration (Draft), FCC-CIRC1910-09 (Oct. 4, 2019) (Draft Order On Reconsideration).
[2] Connect America Fund, WC Docket No. 10-90, Order, DA 18-710 (July 6, 2018) (Performance Measures Order). The requirement that high-cost USF recipients meet certain performance obligations was put in place in 2011. See Connect America Fund et al., WC Docket No. 10-90 et al., Report and Order and Further Notice of Proposed Rulemaking, FCC 11-161, ¶ 86 (Nov. 18, 2011).
[3] Draft Order On Reconsideration at ¶¶ 13-16.
[4] Draft Order On Reconsideration at ¶ 19. The revised and expanded definition of FCC-designated IXP will supposedly account for the way Internet traffic is routed.
[5] Draft Order On Reconsideration at fn. 34. See AS Rank, Center for Applied Internet Data Analysis (Mar. 1, 2019), http://as-rank.caida.org.
[6] Draft Order On Reconsideration at ¶ 19. The Bureaus will determine whether any ASN included in a carrier petition is sufficiently similar to qualifying ASNs that it should be added to the list of qualifying ASNs.
[7] Draft Order On Reconsideration at ¶¶ 22-23.
[8] Draft Order On Reconsideration at ¶ 26.
[9] Performance Measures Order at ¶ 27.
[10] Draft Order On Reconsideration at ¶¶ 27-37.
[11] Draft Order On Reconsideration at ¶ 38.
[12] Draft Order On Reconsideration at ¶ 44.
[13] Performance Measures Order at ¶ 36.
[14] Draft Order On Reconsideration at ¶ 44.
[15] Draft Order On Reconsideration at ¶ 46.
[16] Draft Order On Reconsideration at ¶ 47. The FCC has reconsidered the requirement that providers replace latency testing locations that are no longer actively subscribed after 12 months with another actively subscribed location. The same replacement requirements will now apply to both speed and latency. Therefore, broadband providers are required to replace non-actively subscribed locations with another actively subscribed location by the next calendar quarter testing. Id.
[17] Draft Order On Reconsideration at ¶ 48.
[18] Draft Order On Reconsideration at ¶ 52.
[19] Draft Order On Reconsideration at ¶ 61.
[20] Draft Order On Reconsideration at ¶ 63.
[21] Draft Order On Reconsideration at ¶ 63.
[22] Draft Order On Reconsideration at ¶ 67.
[23] Draft Order On Reconsideration at ¶ 69.
[24] Draft Order On Reconsideration at ¶ 70.
[25] Draft Order On Reconsideration at ¶ 71.
[26] Draft Order On Reconsideration at ¶¶ 72-87.
[27] Draft Order On Reconsideration at ¶ 89.
[28] Draft Order On Reconsideration at ¶ 90.
