You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

SONiC: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Cathal Mooney
imported>Ayounsi
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=== Configuration ===
== Evaluation ==
Currently tested on the April 2021 release.
[[SONiC/Dell Enterprise Sonic Evaluation]]


==== Manual ====
[[SONiC/Leaf-spine testing]]
All configuration change done with <code>sudo config</code> need at some point a <code>sudo config save</code> to make them permanent (eg. device reboot)


'''Hostname'''
[[SONiC/VXLAN-EVPN Network Testing - Sonic on Dell switches]]


<code>sudo config hostname dell-spine1</code>
== Production ==
[[SONiC/initial-setup]]


'''Management'''
== Cheat-sheet ==
 
[[SONiC/cheatsheet|SONiC/Cheat-sheet]]
SONiC supports having the management interface in a dedicated VRF, so the default management route (to mr1) doesn't risk impacting production traffic.
 
<code>sudo config vrf add mgmt</code>
 
<code>sudo config interface ip add eth0 10.x.x.x/16 10.x.0.1</code>
 
<code>show management_interface address</code>
 
https://github.com/Azure/SONiC/blob/master/doc/mgmt/sonic_stretch_management_vrf_design.md
 
https://github.com/Azure/sonic-utilities/blob/master/doc/Command-Reference.md#configuring-management-interface
 
'''SNMP'''
 
Have SNMP listen on its mgmt IP
 
<code>sudo config snmpagentaddress add 10.x.x.x -v mgmt -p 161</code>
 
In the tested release, <code>sudo config snmp</code> is not a valid configuration keyword.
 
Workaround to set the SNMP community:
 
<code>sudo vim /etc/sonic/snmp.yml</code>
 
<code>sudo systemctl restart snmp</code>
 
https://github.com/Azure/SONiC/wiki/How-to-Check-SNMP-Configuration
 
'''NTP'''
 
<code>sudo config ntp add <server IP></code>
 
===Some commands===
 
<code> show ip interface  </code>
Interface    Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet4              10.0.0.2/31          up/up        ARISTA02T2      10.0.0.3
Ethernet8              10.0.0.4/31          up/up        ARISTA03T2      10.0.0.5
Ethernet12            10.0.0.6/31          up/up        ARISTA04T2      10.0.0.7
Ethernet16            10.0.0.8/31          up/up        ARISTA05T2      10.0.0.9
 
<code>sudo config interface  shutdown Ethernet8 </code>
Interface    Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet4              10.0.0.2/31          up/up        ARISTA02T2      10.0.0.3
Ethernet8              10.0.0.4/31          down/down    ARISTA03T2      10.0.0.5
Ethernet12            10.0.0.6/31          up/up        ARISTA04T2      10.0.0.7
 
<code>sudo config interface  startup  Ethernet8 </code>
 
=== Leaf/spine configuration ===
We are going to use the diagram below to setup the leaf/spine. Each lead switch has 1 connection to each spine switch. There is no connection between the spines. The first setup is to identify the interfaces on each switch.
 
[[File:Sonic_leaf_spine1.png|400px]]
 
====Interface mapping ====
<code>show interfaces description</code>
 
on S5232F-ON (Spine)
  Interface    Oper    Admin            Alias    Description
  -----------  ------  -------  ---------------  -------------
  Ethernet0    down      up  hundredGigE1/1            N/A
  Ethernet4    down      up  hundredGigE1/2            N/A
  Ethernet8    down      up  hundredGigE1/3            N/A
  -
  -
  Ethernet120    up      up  hundredGigE1/31            N/A
  Ethernet124    up      up  hundredGigE1/32            N/A
  Ethernet128  down      up      tenGigE1/33            N/A
  Ethernet129  down      up      tenGigE1/34            N/A
 
on both spines switches, we will use interfaces Ethernet120 on Sonic side which is interface E1/31 on Dell switch and Ethernet124 which is E1/32
 
On S5248F-ON (leaf)
<code>leaf1# show interface description</code>
  Interface    Oper    Admin                Alias    Description
  -----------  ------  -------  --------------------  -------------
  Ethernet0    down      up  twentyfiveGigE1/1/1            N/A
  Ethernet1    down      up  twentyfiveGigE1/1/2            N/A
  Ethernet2    down      up  twentyfiveGigE1/1/3            N/A
  -
  -
  Ethernet54    up      up      hundredGigE1/55            N/A
  Ethernet55    up      up      hundredGigE1/56            N/A
On the leaf's we will use interfaces Ethernet54 and Ethernet55 which are E1/55 and E1/56 on the Dell switch
 
Now that we have the interface mapping from Sonic to Dell it is time to setup IP address on each of those interfaces. Below is the diagram showing the IP's and network that we will use for the leaf/spine setup.
 
[[File:Sonic_leaf_spine2.png| 400px]]
 
==== Interface configuration ====
By default the Sonic OS comes with all the interfaces already preconfigured with some IP's.You can check that by looking at the "config_db.json"
 
<code> admin@dell-spine1:/etc/sonic$ vi config_db.json</code>
"Ethernet120|10.0.0.60/31": {},
"Ethernet124|10.0.0.62/31": {},
"Ethernet128|10.0.0.64/31": {},
"Ethernet129|10.0.0.66/31": {},
"Ethernet12|10.0.0.6/31": {},
"Ethernet16|10.0.0.8/31": {},
We see the two(2) interfaces Ethernet120 and 124  that we are going to use have already an IP address. When setting up the interface IP we need to delete as well the existing IP address.
 
'''spine1'''
sudo config interface ip add  Ethernet120 172.16.0.9/30
sudo config interface ip remove  Ethernet120 10.0.0.60/31
sudo config interface ip add  Ethernet124 172.16.0.1/30
sudo config interface ip remove  Ethernet120 10.0.0.62/31
Sudo config save -y
 
'''spine2'''
sudo config interface ip add Ethernet124 172.16.0.5/30
sudo config interface ip add Ethernet120 172.16.0.13/30
sudo config interface ip remove Ethernet120 10.0.0.60/30
sudo config interface ip remove Ethernet124 10.0.0.62/30       
Sudo config save -y
'''leaf1'''
sudo config interface ip add  Ethernet54 172.16.0.6/30
sudo config interface ip remove  Ethernet54 10.0.0.108/31
sudo config interface ip add  Ethernet55 172.16.0.2/30
sudo config interface ip remove  Ethernet55 10.0.0.110/31
Sudo config save -y
 
'''leaf2'''
sudo config interface ip add  Ethernet54 172.16.0.14/30
sudo config interface ip remove  Ethernet54 10.0.0.108/31
sudo config interface ip add  Ethernet55 172.16.0.10/30
sudo config interface ip remove  Ethernet55 10.0.0.110/31
Sudo config save -y
 
Check connectivity between spine1 and leaf1
admin@dell-spine1:/etc/sonic$ ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data.
64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=0.246 ms
64 bytes from 172.16.0.2: icmp_seq=2 ttl=64 time=0.235 ms
 
admin@leaf1:/etc/sonic$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.268 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.241 ms
Check connectivity between spine2/leaf1/leaf2
 
admin@dell-spine2:~$ ping 172.16.0.14
PING 172.16.0.14 (172.16.0.14) 56(84) bytes of data.
64 bytes from 172.16.0.14: icmp_seq=1 ttl=64 time=0.429 ms
64 bytes from 172.16.0.14: icmp_seq=2 ttl=64 time=0.248 ms
admin@dell-spine2:~$ ping 172.16.0.6
PING 172.16.0.6 (172.16.0.6) 56(84) bytes of data.
bytes from 172.16.0.6: icmp_seq=1 ttl=64 time=0.264 ms
Now that we have connectivity, the next step is to setup BGP
 
====BGP configuration====
There are two(2) ways to configure BGP
 
- using command line (shell for FRR daemons). I am still testing this method
 
<code>sudo vtysh</code>
 
- Make the changes directly from the config_db.json file. If you make the changes from this file you need to run
 
<code> sudo config reload -y </code> to save the changes to the Database.
 
Sonic comes with some BGP neighbor's settings preconfigured. We have the option to delete those settings and add new once or just add new once. Note: The preconfigured settings are not active.See below
"BGP_NEIGHBOR": {
    "10.0.0.1": {
        "asn": "65200",
        "holdtime": "180",
        "keepalive": "60",
        "local_addr": "10.0.0.0",
        "name": "ARISTA01T2",
        "nhopself": "0",
        "rrclient": "0"
    },
    "10.0.0.3": {
    "asn": "65200",
    "holdtime": "180",
    "keepalive": "60",
    "local_addr": "10.0.0.2",
    "name": "ARISTA02T2",
    "nhopself": "0",
    "rrclient": "0"
For our testing, we are going to replace the first two(2) preconfigured neighbors with our settings. First we need to setup the Loopback0 interface IP since we will use this in the BGP configuration for the router identifier (router-id).
like for the Ethernetx configuration, the Loopback0 interface also comes with a default IP address of 10.1.0.1/32
sudo config interface ip add Loopback0 10.0.1.13/32 (spine1)
sudo config interface ip remove Loopback0 10.1.0.1/32
sudo config save -y
 
sudo config interface ip add Loopback0 10.0.1.14/32 (spine2)
sudo config interface ip remove Loopback0 10.1.0.1/32
sudo config save -y
 
sudo config interface ip add Loopback0 10.0.1.24/32 (leaf1)
sudo config interface ip remove Loopback0 10.1.0.1/32
sudo config save -y
 
sudo config interface ip add Loopback0 10.0.1.25/32 (leaf2)
sudo config interface ip remove Loopback0 10.1.0.1/32
sudo config save -y
 
'''spine1'''
"BGP_NEIGHBOR": {
    "172.16.0.2": {
        "asn": "65032",
        "holdtime": "180",
        "keepalive": "60",
        "local_addr": "10.0.1.13",
        "name": "leaf1",
        "nhopself": "0",
        "rrclient": "0"
    },
    "172.16.0.10": {
    "asn": "65033",
    "holdtime": "180",
    "keepalive": "60",
    "local_addr": "10.0.1.13",
    "name": "leaf2",
    "nhopself": "0",
    "rrclient": "0"
Under "DEVICE_METADATA" change "bgp_asn": "65100" to spine1 asn number
 
"DEVICE_METADATA": {
    "localhost": {
    "bgp_asn": "65030",
Close the file and run <code> sudo config reload -y</code>
 
Repeat the same steps for spine2, leaf1 and leaf2 by replacing the "IP addresses" and "asn"
 
''' verification'''
 
<code>show ip bgp summary</code>
 
'''spine1'''
Neighbhor      V    AS    MsgRcvd    MsgSent    TblVer    InQ    OutQ  Up/Down      State/PfxRcd  NeighborName
-----------  ---  -----  ---------  ---------  --------  -----  ------  ---------  --------------  --------------
172.16.0.2    4  65032      1566      1569        0      0      0  1d01h57m                3  leaf1
172.16.0.10    4  65033      1534      1543        0      0      0  1d01h27m                2  leaf2
Total number of neighbors 2
'''spine2'''
Neighbhor      V    AS    MsgRcvd    MsgSent    TblVer    InQ    OutQ  Up/Down      State/PfxRcd  NeighborName
-----------  ---  -----  ---------  ---------  --------  -----  ------  ---------  --------------  --------------
172.16.0.6    4  65032      1548      1548        0      0      0  1d01h40m                2  leaf1
172.16.0.14    4  65033      1543      1544        0      0      0  1d01h28m                3  leaf2
Total number of neighbors 2
 
'''leaf1'''
Neighbhor      V    AS    MsgRcvd    MsgSent    TblVer    InQ    OutQ  Up/Down      State/PfxRcd  NeighborName
-----------  ---  -----  ---------  ---------  --------  -----  ------  ---------  --------------  --------------
172.16.0.1    4  65030      1562      1561        0      0      0  1d01h53m                2  Spine1
172.16.0.5    4  65031      1543      1647        0      0      0  1d01h35m                2  spine2
Total number of neighbors 2
 
'''leaf2'''
Neighbhor      V    AS    MsgRcvd    MsgSent    TblVer    InQ    OutQ  Up/Down      State/PfxRcd  NeighborName
-----------  ---  -----  ---------  ---------  --------  -----  ------  ---------  --------------  --------------
172.16.0.9    4  65030      1539      1537        0      0      0  1d01h30m  3              spine1
172.16.0.13    4  65031      1538      1537        0      0      0  1d01h30m  3              spine2
 
=== Known limitations ===
 
* It is not possible to configure a DNS resolver, all configuration needs to be done using IPs
* It is not possible to change the syslog facility
* <code>cgexec -g l3mdev:mgmt</code> is supposed to run commands from the mgmt VRF, but fails with "cgroup change of group failed"
** "ip vrf exec <name> <command>" does seem to work for this. (name being the vrf name, "mgmt" in this case.)

Latest revision as of 11:52, 19 April 2023

Evaluation

SONiC/Dell Enterprise Sonic Evaluation

SONiC/Leaf-spine testing

SONiC/VXLAN-EVPN Network Testing - Sonic on Dell switches

Production

SONiC/initial-setup

Cheat-sheet

SONiC/Cheat-sheet

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=SONiC&oldid=600247"