You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Dell Enterprise Sonic Evaluation: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Cathal Mooney
 
imported>Cathal Mooney
Line 1: Line 1:
=Background=
=Background=


For many years Wikimedia have used Juniper equipment for all networking requirements (currently edge/WAN routers, datacenter switches, management firewalls).  While we are broadly happy with Juniper it is also imperative to assess alternatives, and ensure the foundation is getting value for money.
For many years Wikimedia have used Juniper equipment for all networking requirements (currently edge/WAN routers, datacenter switches, management firewalls).  While we are broadly happy with Juniper, it is also imperative to assess alternatives, ensuring the foundation gets value for money and the best performance possible.


==Cost==
==Foundation Costs==


Recent years have seen the cost of datacenter switches in particular increasing.  This has partially been driven by a gradual move from 1G to faster connections to end-hosts, with the newer equipment supporting 10G+ speeds being pricier.  But also there have been increased costs for software feature licenses which in the part were part of the 'base' system, pushing up overall costs.  The supply-chain / chip shortage problems that we've seen from 2020 onwards have only accelerated this trend.
Recent years have seen the cost of datacenter switches in particular increasing.  This has partially been driven by a gradual move from 1G to faster connections to end-hosts, with the newer equipment supporting 10G+ speeds being pricier.  But there have also been increased costs for software licenses, which in the past were part of the 'base' system, pushing up overall costs.  The supply-chain / chip shortage problems that emerged from 2020 onwards have only accelerated this trend.


==Open Source==
==Open Source==


JunOS, Juniper's operating system, stands out in the foundation as one of the largest closed-source / proprietary software systems we use.  In many respects this is standard for network devices.  These typically use custom [https://en.wikipedia.org/wiki/Application-specific_integrated_circuit ASICs] for packet forwarding, and are not based on the largely open x86/amd64 architecture which server operating systems target.  The specialized and proprietary nature of such hardware has resulted in vendors typically offering "vertically integrated" software/hardware stacks since the dawn of the industry.
JunOS, Juniper's operating system, stands out in the foundation as one of the largest closed-source / proprietary software systems in use.  In many respects this is standard for network devices.  These typically use custom [https://en.wikipedia.org/wiki/Application-specific_integrated_circuit ASICs] for packet forwarding, and are not based on the largely open x86/amd64 architecture which server operating systems target.  The specialized and proprietary nature of such hardware has seen vendors typically offering "vertically integrated" software/hardware stacks since the dawn of the industry.


In more recent years there has been some movement away from this, however.  Driven initially by the large web-scalers, disaggregated or [https://community.fs.com/blog/white-box-switch-will-it-be-the-great-choice-for-next-generation-network.html  white box] switching has risen to prominence.  In this model the switching hardware is provided by one company, and the operating-system is sourced elsewhere (much like one buys a Dell server and runs Debian or Windows on it).  Such an approach offers many advantages, such as being able to change vendors but keep the same operating system.  Or change the OS in use on existing hardware.  "White box" switch hardware is typically available for a substantially lower cost than brand-name alternatives.  There can be drawbacks, however, such as not having a "one stop shop" for support. 
===White Box===


Another caveat is that a small number of ASIC vendors, notably Broadcom, have the switching market carved upThese vendors often gate access to their designs and SDKs, limiting the scope for independent parties to create software for them.  In one famous case Broadcom ceased licensing its SDK to Cumulus Networks, after they were acquired by rival hardware manufacturer nVidiaThis left some customers forced to choose another hardware supplier or move to another OS when they had to upgrade.   
In more recent years there has been some movement away from thisDriven initially by the large web-scalers, disaggregated or [https://community.fs.com/blog/white-box-switch-will-it-be-the-great-choice-for-next-generation-network.html  white box] switching has risen to prominence.  In this model the switching hardware is provided by one company, and the operating-system is sourced elsewhere (much like one buys a Dell server and runs Debian or Windows on it without consulting Dell).  Such an approach offers many advantages, such as being able to change vendors but keep the same operating system.  Or change the OS in use on existing hardware.  "White box" switch hardware is typically available for a substantially lower cost than brand-name alternativesThere can be drawbacks, however, such as not having a "one stop shop" for support.


Nevertheless the space has opened up and there are now several "white box" NOS's available, even if the space is unlikely to ever be as open as that for server hardware.  Options include commercial offerings such as [https://www.pica8.com/picos-software/ PicOS], [https://www.arrcus.com/connected-edge/arcos/ ArcOS] and [https://www.ipinfusion.com/products/ocnos/?2#odc OcNOS], as well as open-source projects such as [https://www.danosproject.org/ DANOS] and [https://www.openswitch.net/ OpenSwitch].  Of the open-source options [https://en.wikipedia.org/wiki/SONiC_(operating_system) SONiC], initially released by Microsoft and now with contributions from many others, has become a clear [https://www.nextplatform.com/2020/05/12/is-microsofts-sonic-winning-the-war-of-the-noses/ leader in the space].
===Ecosystem===
 
Another caveat is that a small number of ASIC vendors, notably Broadcom, have the switching market carved up.  These vendors often gate access to their designs and SDKs, limiting the scope for independent parties to create software for them.  In one famous case Broadcom ceased licensing its SDK to Cumulus Networks, after they were acquired by rival hardware manufacturer nVidia.  This left some customers forced to choose another hardware supplier or move to another OS when they had to upgrade.  The reality right now is that it is not possible to produce an operating system for switching hardware without permission from the ASIC vendors.
 
Nevertheless the space has opened up and there are several "white box" NOS's available, even if things won't ever be as open as server hardware.  Options include commercial offerings such as [https://www.pica8.com/picos-software/ PicOS], [https://www.arrcus.com/connected-edge/arcos/ ArcOS] and [https://www.ipinfusion.com/products/ocnos/?2#odc OcNOS], as well as open-source projects such as [https://www.danosproject.org/ DANOS] and [https://www.openswitch.net/ OpenSwitch].  Of the open-source options [https://en.wikipedia.org/wiki/SONiC_(operating_system) SONiC], initially created by Microsoft and now with contributions from many others, has become a definite [https://www.nextplatform.com/2020/05/12/is-microsofts-sonic-winning-the-war-of-the-noses/ leader in the space].


=Dell Enterprise SONiC=
=Dell Enterprise SONiC=


Of the various open-source options SONiC is one of the most popular, with significant industry support.  As such it makes good sense
Of the various open-source options SONiC is one of the most popular, with significant industry support.  Significantly Broadcom have been supportive of the project and have [https://github.com/Broadcom-Switch/SAI released].


=Test Criteria=
=Test Criteria=

Revision as of 11:42, 4 August 2022

Background

For many years Wikimedia have used Juniper equipment for all networking requirements (currently edge/WAN routers, datacenter switches, management firewalls). While we are broadly happy with Juniper, it is also imperative to assess alternatives, ensuring the foundation gets value for money and the best performance possible.

Foundation Costs

Recent years have seen the cost of datacenter switches in particular increasing. This has partially been driven by a gradual move from 1G to faster connections to end-hosts, with the newer equipment supporting 10G+ speeds being pricier. But there have also been increased costs for software licenses, which in the past were part of the 'base' system, pushing up overall costs. The supply-chain / chip shortage problems that emerged from 2020 onwards have only accelerated this trend.

Open Source

JunOS, Juniper's operating system, stands out in the foundation as one of the largest closed-source / proprietary software systems in use. In many respects this is standard for network devices. These typically use custom ASICs for packet forwarding, and are not based on the largely open x86/amd64 architecture which server operating systems target. The specialized and proprietary nature of such hardware has seen vendors typically offering "vertically integrated" software/hardware stacks since the dawn of the industry.

White Box

In more recent years there has been some movement away from this. Driven initially by the large web-scalers, disaggregated or white box switching has risen to prominence. In this model the switching hardware is provided by one company, and the operating-system is sourced elsewhere (much like one buys a Dell server and runs Debian or Windows on it without consulting Dell). Such an approach offers many advantages, such as being able to change vendors but keep the same operating system. Or change the OS in use on existing hardware. "White box" switch hardware is typically available for a substantially lower cost than brand-name alternatives. There can be drawbacks, however, such as not having a "one stop shop" for support.

Ecosystem

Another caveat is that a small number of ASIC vendors, notably Broadcom, have the switching market carved up. These vendors often gate access to their designs and SDKs, limiting the scope for independent parties to create software for them. In one famous case Broadcom ceased licensing its SDK to Cumulus Networks, after they were acquired by rival hardware manufacturer nVidia. This left some customers forced to choose another hardware supplier or move to another OS when they had to upgrade. The reality right now is that it is not possible to produce an operating system for switching hardware without permission from the ASIC vendors.

Nevertheless the space has opened up and there are several "white box" NOS's available, even if things won't ever be as open as server hardware. Options include commercial offerings such as PicOS, ArcOS and OcNOS, as well as open-source projects such as DANOS and OpenSwitch. Of the open-source options SONiC, initially created by Microsoft and now with contributions from many others, has become a definite leader in the space.

Dell Enterprise SONiC

Of the various open-source options SONiC is one of the most popular, with significant industry support. Significantly Broadcom have been supportive of the project and have released.

Test Criteria

Test Results

Conclusions

Pros

Cons

Costs

Verdict