Interconnect Evolution An African IXP and colocation experience

Interconnect Evolution An African IXP and colocation experience in scaling for growth

What commercial people say

What technical people say

Colocation has become more than space & power Interconnection is one of the most important features to look for in a data centre provider, these are the key aspects to review: Access - Data centre interconnection provides access to multiple Carrier/ISPs Convenience - Data centre interconnection providers also offer the benefit of a direct, physical connection between servers, providers, and customers. Interconnection Policies - Carrier-neutral data centres are not confined to any one service provider and generally colocate a successful IXP Pricing - non-carrier-neutral environment high pricing or access only allowed the carrier’s service options

Neutral colocation client vs interconnect growth YEAR INTERCONNECT S 2010 300 23 2 2011 879 53 2 2012 1163 93 2 2013 2017 132 2 2014 3392 180 3 2015 5578 219 4 2016 7122 280 4 2017 8624 348 4 CLIENTS AVE. INTERCONNECT/CAB

Increased Market Diversity Complex Cabling Interconnection 2016 - 2017 9% 4% 3% 56% 28% Interconnection 2015 - 2016 Financial, 2% Enterprise, 5% MSP, 18% Content, 2% Telco/ISP Enterprise MSP Finance Content Telecoms/ISP, 73% Telecoms/ISP MSP Enterprise Content Financial

NAPAfrica overview: AS37195 3 Regions: Johannesburg (Avg: 126Gbps, Peak: 205Gbps), Cape Town (30G), Durban (3G) Members: 270 unique ASN’s Port Sizes: 1G/10G/100G 559Active Ports – 2.4Tbps connected capacity Pricing: Free (Ports, Membership, Cross Connects) Policy: Open to all Multi-lateral peering arrangements available; Bi-lateral peering arrangements available IPv4 & IPv6 enabled

NAPAfrica Growth 2016 to 2017 July 2016 IXP 1G Ports 10G Ports Total Connected Capacity NAPAfrica IX JHB 177 88 265 1057G NAPAfrica IX CPT 87 28 115 367G NAPAfrica IX DBN 41 8 49 121G Total 305 124 429 1545G July 2017 IXP 1G Ports 10G Ports Total Connected Capacity NAPAfrica IX JHB 199 156 355 1759G NAPAfrica IX CPT 97 46 143 557G NAPAfrica IX DBN 50 11 61 160G Totals 346 213 559 2476G

Massive growth operational challenges

Infrastructure considerations Continued growth through new membership and member migration to higher port speeds, increased structured cabling, larger MMR’s Large Capex Global Content operators requiring 100G infrastructure New product requirements through virtual cross connection Long term scalability at the least cost – Capex, ongoing support and licensing Purely focused on switching infrastructure and no requirement for full feature sets System Automation and integration requirements Reduce complexity due to limited staff resources Great existing reference sites required

IXP: Vendor considerations All major vendors where reviewed e.g. Cisco, Extreme, Arista, Juniper How did Arista win? Ticked all the infrastructure considerations boxes at the best bang per port buck

IXP: Migration experience Why did we migrate? Virtual chassis environment could not scale to the massive growth, new product and membership requirements. Bundling multiple 10Gbps ports Biggest bottleneck was interconnecting old platform to new platform at sufficient bandwidth. 80Gpbs AE limitation on old platform meant we had to carefully plan and move networks to ensure we remained below 90% capacity. Moving too much content or eyeballs at one go meant congestion would shift in opposite direction. At the same time, we took on a number of content networks on the new platform, and at times were running 90% capacity between platforms. Used sflow to plan optimal moves – combination of content and eyeballs at the same time. Currently at 30% utilisation between platforms

Lessons learned 100G is real and there is demand for it now (Price drops have made 40G redundant for anything other than parallel breakouts for 10G density) Multimode optics are cheap, but make absolutely no sense at 100Gbps given structured cabling requirements (Unless you are using AOC for short distances) Shop around for your SFP’s and remember to make sure vendor of choice does NOT lock you in Make sure you understand the costs of upgrade before partnering with a vendor Various standards available: LR4, SR4, IR4, PLR4, CRL4, etc. (IEEE, MSA). Make sure you know what standard your members use. Some optic vendors mix IR4 with CWDM4 or CLR4.

Virtual Cross connection a is reality for Colocation Operators Layer 2 overlay, Layer 3 underlay with ECMP Can be point-to-point or point-to-multipoint Instant, client controlled activation of virtual circuits Multiple circuits per physical cable Fewer ports, fewer physical cables, less power, more efficient cooling Per circuit monitoring using industry standard sFlow Client controlled bandwidth allocation No waiting for facility cabling Less people inside the facility, meaning less chance of physical

Virtual Cross Connection Benefits The benefits are: Immediate and real time activation of services between customers Less physical infrastructure Ability to proactively monitor connections Improved service levels Improved performance Cost effective No Contractual Obligations – Great solution for POC’s and back up route for Primary Physical Cabling

Virtual Cross Connect Interface Demo Live demonstration of turning up a circuit the icon below click on

Thank you Andrew Owens [email protected]