Shu Sugimoto, profile picture
Uploaded byShu Sugimoto
PPTX, PDF23,609 views

Tutorial: Using GoBGP as an IXP connecting router

This document provides a tutorial on using GoBGP as an IXP connecting router, discussing its features, setup, and commands. It highlights GoBGP's performance, automation capabilities, and its integration with Zebra for fib manipulation. Additionally, it details installation steps, configuration, and various use cases for GoBGP in production environments.

Downloaded 328 times
INTERNET MULTIFEED CO.Copyright © 2016 Tutorial: Using GoBGP as an IXP connecting router Shu Sugimoto JPNAP / INTERNET MULTIFEED CO. IX.br Forum 10 2016/12/07(Wed)
INTERNET MULTIFEED CO.Copyright © 2016 About this presentation • Show you how GoBGP can be used as a software router in conjunction with quagga • (Tutorial) Walk through the setup of IXP connecting router using GoBGP • This is going be/was spoken at IX.br Forum 10 • Slides available at SlideShare 2
INTERNET MULTIFEED CO.Copyright © 2016 GoBGP • New BGP implementation • OSS developed by NTT Lab SIC • https://github.com/osrg/gobgp • Written in Golang • High performance • exploits multi-cores natively • Automation friendly • API first principle • CLI on top of API 3 gobgpd CLI gobgp Your Software API (gRPC)
INTERNET MULTIFEED CO.Copyright © 2016 Key features 1/2 • full-featured CLI • Multiprotocol support • IPv4, IPv6, Labeled IPv4/IPv6, VPN IPv4/IPv6, EVPN, flowspec IPv4/IPv6/L2 • Flexible Policy • Graceful Restart • both restarting/helper speaker role • Route Reflector • Route Server 4
INTERNET MULTIFEED CO.Copyright © 2016 Key features 2/2 • MRT dumping • BMP • RPKI validation • FIB manipulation • gRPC API • Standard configuration format • structured based on OpenConfig • supports toml/yaml/json/hcl 5
INTERNET MULTIFEED CO.Copyright © 2016 Components • gobgpd • main daemon process which implements BGP • can be controlled via gRPC API • configuration file also supported • gobgp • full-featured CLI • convert human friendly commands into gRPC API call • and vice versa • configuration file (optional) • popular way to define the behavior of gobgpd • written in toml/yaml/json/hcl 6
INTERNET MULTIFEED CO.Copyright © 2016 gobgp CLI command example • show list of neighbors • show RIB 7 $ gobgp neighbor Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 3d 08:25:02 Establ | 1 1 10.173.176.103 65003 3d 08:25:00 Establ | 1 1 10.173.176.211 64686 never Active | 0 0 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 3d 08:06:32 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 3d 08:34:01 [{Origin: i} {Communities: 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 00:00:03 [{Origin: i} {Communities: 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 3d 08:34:03 [{Origin: i} {LocalPref: 100}]
INTERNET MULTIFEED CO.Copyright © 2016 gobgp CLI command example • neighbor operations 8 ### perform peer softresetin $ gobgp neighbor 10.1.0.101 softresetin ### perform peer reset $ gobgp neighbor 10.1.0.101 reset ### perform peer disable (shutdown) $ gobgp neighbor 10.1.0.101 disable ### perform peer enable $ gobgp neighbor 10.1.0.101 enable
INTERNET MULTIFEED CO.Copyright © 2016 gobgp CLI command example • You can even add/delete peers/routes/policies on the fly • Some easy use case can be accomplished without using configuration file 9 ### launch gobgpd (need privilege to listen on tcp 179) $ sudo gobgpd ### (open new terminal) ### set AS and router-id $ gobgp global as 1 router-id 1.1.1.1 ### add neighbor $ gobgp neighbor add 192.0.2.2 as 2 ### add route into RIB, which will then advertised to peers $ gobgp global rib add -a ipv4 10.0.0.0/24 med 10 community 100:100
INTERNET MULTIFEED CO.Copyright © 2016 gobgp CLI command example • `-j` support • Every command supports json output 10 $ gobgp global rib 10.4.0.0/16 -j {"10.4.0.0/16":[{"nlri":{"prefix":"10.4.0.0/16"},"attrs":[{"type":1,"val ue":0},{"type":2,"as_paths":[{"segment_type":2,"num":1,"asns":[65004]}]} ,{"type":3,"nexthop":"10.1.14.104"},{"type":5,"value":100},{"type":8,"co mmunities":[4259907539]}],"age":1480845275,"validation":"not- found","source-id":"10.1.0.101","neighbor-ip":"10.1.0.101"}]} ### pretty print using python $ gobgp global rib 10.4.0.0/16 -j | python -mjson.tool { "10.4.0.0/16": [ { "age": 1480845275, "attrs": [ { "type": 1, : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 gobgp CLI command example • Event monitoring • The data will be sent from gobgpd through the gRPC connection channel when events occur • Push notification • One implementation of the event driven application 11 $ gobgp monitor global rib [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.4.0.0/16 via 10.1.14.104 aspath [65004] attrs [{Origin: i} {LocalPref: 100} {Communities: 65001:2003}] [DELROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] (waiting for further events…)
INTERNET MULTIFEED CO.Copyright © 2016 Who use GoBGP? • IXP Route Server • JPNAP • Monitoring compoment • FastNetMon • DoS/DDoS analyzer • BGPmon • BGP routing information monitor • Cloudwatt • is an OpenStack based public cloud service • They seems to be using it as a component of Looking Glass 12
INTERNET MULTIFEED CO.Copyright © 2016 Who use GoBGP? • SDN solution component • Project Calico • Contiv netplugin • Both are network plugin for containers • Test virtualization • ex.) One is using GoBGP to virtualize the Lab environment for testing BGP routing policies • Only test target is a real router, everything else is VM • GoBGP is used as a route originator • gRPC API is used to generate arbitrary routes • http://www.slideshare.net/ssuser6a8d29/gobgp • in Japanese :P 13
INTERNET MULTIFEED CO.Copyright © 2016 Why use GoBGP? • Automation Friendliness • It’s definitely easier to implement whatever comes up in your mind • Everything is exposed through API • `gobgp -j` can be a good start point of “thinking about automation” • can skip the painful “output parsing” part, which broke a lot of people’s motivation • much easier to begin with for operators  • Performance • Especially in larger deploy 14
INTERNET MULTIFEED CO.Copyright © 2016 GoBGP performance 15 Quagga won’t scale if the policy is huge Comparing the convergence time in Y-axis (between beginning of first peer up and the last update sent) X-axis = # of IPs in prefix list / fixed # of peers = 100, number of prefixes per peer = 100, Route Server setup Graph generated using bgperf ( https://github.com/osrg/bgperf ), in Jan 2016
INTERNET MULTIFEED CO.Copyright © 2016 Why not? Then… • “Because it’s not stable yet, isn’t it?” • “Because no one is using it as yet, right?” • (many many many reasons…) 16
INTERNET MULTIFEED CO.Copyright © 2016 Why not? Then… • I can’t say 100% “Yes it’s stable” but… • We use it in production as a Route Server at JPNAP and so far not facing any big issue • Development team are amazingly fast • in response, finding causes, and providing patches • I really would like some of you to consider trying GoBGP • as a software router • There’s no one still AFAIK • Need help? Find bug? Anything you want to discuss? • Open an issue at GitHub • or you can join open slack channel • https://slackin-gobgp.mybluemix.net/ 17
INTERNET MULTIFEED CO.Copyright © 2016 Bugs resolved • Here are the list of bugs found and fixed while I was working on making this tutorial… • server: fix bug of deleteNeighbor() #1184 • https://github.com/osrg/gobgp/pull/1184 • zebra: add flags for recursive nexthop lookup if necessary #1179 • https://github.com/osrg/gobgp/pull/1179 • fix several bugs related to rpki and policy #1178 • https://github.com/osrg/gobgp/pull/1178 • gobgpd: support global policy assignment update via configuration file #1177 • https://github.com/osrg/gobgp/pull/1177 • Some other bugs still under discussion • Many thanks to @wataru and @tomo 18
INTERNET MULTIFEED CO.Copyright © 2016 FIB manipulation • GoBGP is just a bgp daemon and itself does not contain any functionality to modify routing table • If you like to use GoBGP as a component of software router and do packet forwarding, you need to implement that • There are two options to achieve FIB manipulation with GoBGP • Use built-in zebra integration • Write your own code using gRPC API • In this tutorial I’ll introduce zebra integration and show you how we can use it 19
INTERNET MULTIFEED CO.Copyright © 2016 cf.) GoPlane • https://github.com/osrg/goplane • Utilize gobgpd gRPC API and perform things like FIB manipulation on Linux platform • Can create EVPN/VxLAN fabric • Also can modify iptables rules based on received FlowSpec routes • Can’t do any complex routing • ex.) Recursive next-hop resolving not supported • Not suitable for the use case in this tutorial 20
INTERNET MULTIFEED CO.Copyright © 2016 Quagga architecture 21 zserv.api zebra routing table Linux kernel bgpdospfd vtysh vtysh vtysh CLI operation tcp / unix socket zclient(s) more protocols available ripd, isisd...Quagga zebra protocol
INTERNET MULTIFEED CO.Copyright © 2016 Zebra integration of GoBGP 22 zserv.api zebra routing table Linux kernel gobgpdospfd vtysh API gobgpd acts as one of zclient vtysh CLI operation gRPC gobgp CLI
INTERNET MULTIFEED CO.Copyright © 2016 Tutorial • Target: IXP connecting router • You will walk through • Installation of GoBGP • gobgpd administration via systemd • Writing configuration file • Adding eBGP/iBGP peers • Applying policy • Including example use case to control route advertising over Route Server • FIB manipulation with zebra integration • RPKI setup • MRT/BMP setup • InfluxDB integration setup 23
INTERNET MULTIFEED CO.Copyright © 2016 Topology 24 lo: 10.3.0.103 AS65004 10.4.0.0/16 AS65001 10.1.0.0/16 lo: 10.1.0.102 lo: 10.1.0.101 10.1.12.0/24 10.173.176.0/24 .211 .103.102 .102 .101 10.1.14.0/24 .101 .104 g2 IX r3 r1 r4 rs OSPF Area 0 eth2 eth3 eth1 ge-0/0/1 ge-0/0/1 ge-0/0/2 ge-0/0/1 redistribute direct lo: 10.4.0.104 AS65003 10.3.0.0/16 eBGP eBGP iBGP Route Server AS65686 Target Host OS Softwares r1 vSRX 12.1 g2 Debian 8 GoBGP Quagga (zebra/ospfd) r3 vSRX 12.1 r4 vSRX 12.1 rs Debian 8 GoBGP (Route Server) util Debian 8 Ryu BMP server influxdb g2 util eth1 eth1 .102 .250 10.254.0.0/24
INTERNET MULTIFEED CO.Copyright © 2016 gobgp-tutorial • You can instantly build the demo topology using Vagrant • Vagrantfile available at GitHub • https://github.com/s2ugimot/gobgp-tutorial • Follow the instructions in README.md • Everything except the GoBGP in g2 will be set up 25
INTERNET MULTIFEED CO.Copyright © 2016 Demo setup • MacBook Pro 13r (Early 2015) • Mac OS X 10.11.6 (El Capitan) • 3.1GHz Intel Core i7 • 16GB RAM (at least > 8GB) • Vagrant 1.8.1 • vagrant-host-shell 0.0.4 • vagrant-junos 0.2.1 • VirtualBox 5.0.28 • Should work on Windows/Linux too • haven’t tested though :P 26
INTERNET MULTIFEED CO.Copyright © 2016 System consideration in real world • RAM • > 16GB • Recommend 32GB or more if you handle IPv4 full routes = 600k routes with multiple eBGP peers • This is by design • No extensive tweaks, keep it simple, just buy memory • CPU • > 2cores • The more, the better performance • VM is fine • If forwarding is not the issue 27
INTERNET MULTIFEED CO.Copyright © 2016 Tutorial: Step by step 28
INTERNET MULTIFEED CO.Copyright © 2016 Install Go • Just follow the instruction on official web • https://golang.org/doc/install • It’s quite simple, just extract tar.gz and add to $PATH • Choose go1.5 or above 29
INTERNET MULTIFEED CO.Copyright © 2016 Build GoBGP • Built binaries are portable • Libraries are statically linked into binary • Should preserve the $GOPATH • To make it reproducible • Go itself does not provide any good solution still • ex.) build in container and keep the image 30 ### build binaries g2 $ go get github.com/osrg/gobgp/gobgpd g2 $ go get github.com/osrg/gobgp/gobgp ### copy them to somewhere under $PATH g2 $ cp $GOPATH/bin/* /usr/local/sbin ### optional: install shell completion for gobgp command g2 $ cp $GOPATH/src/github.com/osrg/gobgp/tools/completion/*.bash /etc/bash_completion.d/
INTERNET MULTIFEED CO.Copyright © 2016 Setup systemd unit file • Prepare systemd unit file for gobgpd process to let it managed by systemd 31
INTERNET MULTIFEED CO.Copyright © 2016 Setup systemd unit file 32 ### create a unit file for gobgpd g2 $ cat << EOF > /etc/systemd/system/gobgpd.service [Unit] Description=gobgpd After=network.target syslog.target [Service] Type=simple PermissionsStartOnly=yes User=quagga ExecStartPre=/sbin/setcap 'cap_net_bind_service=+ep' /usr/local/sbin/gobgpd ExecStart=/usr/local/sbin/gobgpd -f /etc/gobgp/gobgpd.conf -t yaml -- cpus=2 ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s TERM $MAINPID [Install] WantedBy=multi-user.target EOF g2 $ systemctl daemon-reload
INTERNET MULTIFEED CO.Copyright © 2016 Setup systemd unit file • Set process user same as that of quagga service • because gobgpd needs rw permission to zserv.api to speak to zebra • Use `setcap` to let unprivileged user listen on port < 1024 • Security concern: you should not run any important gobgpd on shared servers! • Anyone has access to full-control through gRPC API • At least you should block access to API port (default tcp 50051) by applying iptables rules from other hosts • Exposing `setcap`-ed binary to others might compromise your server seucrity 33
INTERNET MULTIFEED CO.Copyright © 2016 Adjusting options • Update `ExecStart` section in unit file to fit your needs • You can specify how many cores you want to use 34 Application Options: -f, --config-file= specifying a config file -t, --config-type= specifying config type (toml, yaml, json) (default: toml) -l, --log-level= specifying log level -p, --log-plain use plain format for logging (json by default) -s, --syslog= use syslogd --syslog-facility= specify syslog facility --disable-stdlog disable standard logging --cpus= specify the number of CPUs to be used --api-hosts= specify the hosts that gobgpd listens on (default: :50051) -r, --graceful-restart flag restart-state in graceful-restart capability -d, --dry-run check configuration --pprof-host= specify the host that gobgpd listens on for pprof (default: localhost:6060) --pprof-disable disable pprof profiling
INTERNET MULTIFEED CO.Copyright © 2016 Create the first configuration • Minimal configuration includes AS and router-id • gobgpd supports toml/yaml/json/hcl format • I would recommend using toml • used in official document • Here I will use yaml through this tutorial • It’s easier to read 35 global: config: as: 65001 router-id: 10.1.0.102
INTERNET MULTIFEED CO.Copyright © 2016 toml configuration example • toml is better in two reasons • It is easy to “copy & paste” config fragments • Element ordering does not matter • ex.) You can define a neighbor, a policy for it, then other neighbor • Arbitrary ordering helps you organize the configuration 36 [global.config] as = "65001" router-id = "10.1.0.102” [[neighbors]] [neighbors.config] neighbor-address = 10.173.176.103 peer-as = 65003 [[neighbors]] [neighbors.config] neighbor-address = 10.1.0.101 peer-as = 65001 [neighbors.transport.config] local-address = 10.1.0.102
INTERNET MULTIFEED CO.Copyright © 2016 Launch gobgpd 37 g2 $ systemctl start gobgpd g2 $ systemctl status gobgpd ● gobgpd.service - gobgpd Loaded: loaded (/etc/systemd/system/gobgpd.service; disabled) Active: active (running) since Thu 2016-12-01 05:56:17 UTC; 7s ago Process: 5987 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS) Process: 5981 ExecReload=/bin/kill -s HUP $MAINPID (code=exited, status=0/SUCCESS) Process: 6057 ExecStartPre=/sbin/setcap cap_net_bind_service=+ep /usr/local/sbin/gobgpd (code=exited, status=0/SUCCESS) Main PID: 6061 (gobgpd) CGroup: /system.slice/gobgpd.service └─6061 /usr/local/sbin/gobgpd -f /etc/gobgp/gobgpd.conf -t yaml --cpus=2 Dec 01 05:56:17 g2 gobgpd[6061]: {"level":"info","msg":"gobgpd started","time":"2016-12-01T05:56:17Z"} Dec 01 05:56:17 g2 gobgpd[6061]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-01T05:56:17Z"}
INTERNET MULTIFEED CO.Copyright © 2016 Logging • Default to output in json format • easy for computers • (a bit) hard for humans :P • You can switch to plain text mode by passing `-p/--log- plain` • All logs will be shown in stdout • can be disabled by `--disable-stdlog` • Syslog is also supported • exactly the same log will be produced 38
INTERNET MULTIFEED CO.Copyright © 2016 Logging 39 ### check log via journald g2 $ journalctl -xn -f -u gobgpd -- Logs begin at Mon 2016-12-05 05:05:34 UTC. -- Dec 05 05:21:57 g2 gobgpd[1352]: {"level":"info","msg":"gobgpd started","time":"2016-12-05T05:21:57Z"} Dec 05 05:21:57 g2 gobgpd[1352]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-05T05:21:57Z"} ### check log via file (via rsyslogd through systemd/journald) g2 $ tailf /var/log/syslog Dec 5 05:21:57 g2 gobgpd[1352]: {"level":"info","msg":"gobgpd started","time":"2016-12-05T05:21:57Z"} Dec 5 05:21:57 g2 gobgpd[1352]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-05T05:21:57Z"}
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer • Establish eBGP peer between g2 and r3 • Receive routes from r3 • Apply policy to routes received from r3 • r3 is a peer over IXP • will `tag` to identify the routes received from IXP peers using community • Use ”65000:1000” 40
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer • neighbor/policy configuration can be dynamically applied by sending SIGHUP to the gobgpd process • We define `reload` to send SIGHUP in systemd unit file • If there is a syntax error in configuration file, gobgpd just abort reloading and remain on the current state • You can see exact location causing error in log 41 global: config: as: 65001 router-id: 10.1.0.102 neighbors: - config: neighbor-address: 10.173.176.103 peer-as: 65003 g2 $ systemctl reload gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer 42 g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.173.176.103 65003 never Active | 0 0 ### wait for a while... g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.173.176.103 65003 00:00:36 Establ | 1 1
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer 43 g2 $ gobgp neighbor 10.173.176.103 BGP neighbor is 10.173.176.103, remote AS 65003 BGP version 4, remote router ID 10.3.0.103 BGP state = established, up for 00:03:43 BGP OutQ = 0, Flops = 0 Hold time is 90, keepalive interval is 30 seconds Configured hold time is 90, keepalive interval is 30 seconds Neighbor capabilities: multiprotocol: ipv4-unicast: advertised and received route-refresh: advertised and received graceful-restart: received 4-octet-as: advertised and received cisco-route-refresh: received Message statistics: Sent Rcvd Opens: 1 1 Notifications: 0 0 Updates: 0 1 Keepalives: 8 10 : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer • Caveats: you need to use `-j` option and see in json format to see more detailed attributes like router-id or source (from which neighbor the route has received) • Feel free to open an issue at GitHub  44 ### check received routes ### junos: show route receiving-protocol bgp 10.173.176.103 g2 $ gobgp neighbor 10.173.176.103 adj-in Network Next Hop AS_PATH Age Attrs 10.3.0.0/16 10.173.176.103 65003 00:12:08 [{Origin: i}] ### check BGP RIB ### junos: show route protocol bgp g2 $ gobgp global rib gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.3.0.0/16 10.173.176.103 65003 00:17:34 [{Origin: i}]
INTERNET MULTIFEED CO.Copyright © 2016 Configure first eBGP peer 45 ### see detailed information using `-j` g2 $ gobgp global rib –j | python -mjson.tool { "10.3.0.0/16": [ { "age": 1480923494, "attrs": [ : (snip) ], "neighbor-ip": "10.173.176.103", "nlri": { "prefix": "10.3.0.0/16" }, "source-id": "10.3.0.103" } ] }
INTERNET MULTIFEED CO.Copyright © 2016 Apply policy to received routes 46 policy-definitions: - name: tag-ixp-neighbors statements: - conditions: match-neighbor-set: neighbor-set: ixp-neighbors match-set-options: any actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:1000" defined-sets: neighbor-sets: - neighbor-set-name: ixp-neighbors neighbor-info-list: - 10.173.176.103
INTERNET MULTIFEED CO.Copyright © 2016 Apply policy to received routes 47 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - tag-ixp-neighbors default-import-policy: accept-route ### apply policy g2 $ systemctl reload gobgpd ### see what happens g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.3.0.0/16 10.173.176.103 65003 01:43:37 [{Origin: i} {Communities: 65001:1000}]
INTERNET MULTIFEED CO.Copyright © 2016 Policy definition structure • Policy consists of statements • Each statement has condition(s) and action(s) • condition specifies the state of NLRIs to match • ex.) prefix, neighbor, AS_PATH, community, … • actions specifies what to do with the NLRIs • accept / reject • modify path attributes • community, MED, local-pref, AS_PATH, next-hop • Some condition refers to defined-sets • ex.) prefix-set, neighbor-set, community-set, … 48
INTERNET MULTIFEED CO.Copyright © 2016 Policy definition structure 49
INTERNET MULTIFEED CO.Copyright © 2016 Policy configuration basics • First you define policies • and defined-sets if needed to achieve your needs • Then attach it to global config level • not in neighbor level • beware! There is a nob under neighbor level which is only for Route Server setup • You can specify neighbor using `neighbor-set` • in import policy: from which neighbor • in export policy: to which neighbor 50
INTERNET MULTIFEED CO.Copyright © 2016 Where policies are applied 51
INTERNET MULTIFEED CO.Copyright © 2016 Other policy examples • see the official doc for more details • https://github.com/osrg/gobgp/blob/master/docs/sour ces/policy.md 52
INTERNET MULTIFEED CO.Copyright © 2016 Install routes into Linux FIB / zebra integration • gobgpd can act as a replacement of bgpd in quagga • Setup zebra integration • And first we see BGP received route from r3 installed in the routing table 53
INTERNET MULTIFEED CO.Copyright © 2016 Install routes into Linux FIB / zebra integration • Specify the path to `zserv.api` created by zebra daemon • gobgpd process needs rw access to it • Check permission if you encounter any problem • `redistribute-route-type-list` specifies from which protocol gobgpd imports routes into BGP RIB • At this time we do not redistribute any route from zebra so just leave it blank 54 zebra: config: enabled: true url: "unix:/var/run/quagga/zserv.api" redistribute-route-type-list: []
INTERNET MULTIFEED CO.Copyright © 2016 Install routes into Linux FIB / zebra integration • You need to restart gobgpd • some configuration change need restart of gobgpd to take effect • zebra integration is one of them • Booting order is important! • zebra process must be started before gobgpd starts 55 ### apply configuration change g2 $ systemctl restart gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Install routes into Linux FIB / zebra integration 56 g2 $ vtysh -d zebra -c "show ip route" Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, A - Babel, > - selected route, * - FIB route K>* 0.0.0.0/0 via 10.0.2.2, eth0 S>* 10.0.0.0/8 [1/0] is directly connected, Null0, bh C>* 10.0.2.0/24 is directly connected, eth0 S>* 10.1.0.0/16 [1/0] is directly connected, Null0, bh O>* 10.1.0.101/32 [110/10] via 10.1.12.101, eth3, 1d02h44m O 10.1.0.102/32 [110/10] is directly connected, lo, 1d02h44m C>* 10.1.0.102/32 is directly connected, lo O 10.1.12.0/24 [110/10] is directly connected, eth3, 1d02h44m C>* 10.1.12.0/24 is directly connected, eth3 O>* 10.1.14.0/24 [110/0] via 10.1.12.101, eth3, 1d02h44m B>* 10.3.0.0/16 [20/0] via 10.173.176.103, eth2, 00:16:10 C>* 10.173.176.0/24 is directly connected, eth2 C>* 10.254.0.0/24 is directly connected, eth1 C>* 127.0.0.0/8 is directly connected, lo S>* 172.16.0.0/12 [1/0] is directly connected, Null0, bh S>* 192.168.0.0/16 [1/0] is directly connected, Null0, bh
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra • Inject routes from zebra into gobgpd then advertise to r3 • We use static null route • Common way to originate your own prefixes • Set import policy to select only routes that we want to import 57
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra • `redistribute-route-type-list` • Specify from which protocol gobgpd imports routes • If you like to also redistribute connected routes and ospf routes then just append them to the list 58 zebra: config: enabled: true url: "unix:/var/run/quagga/zserv.api" redistribute-route-type-list: - static
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra 59 ### apply configuration change, need restart g2 $ systemctl restart gobgpd ### all static routes are imported g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.0.0.0/8 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 10.1.0.0/16 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 172.16.0.0/12 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 192.168.0.0/16 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] g2 $ vtysh -d zebra -c "show ip route static“ S>* 10.0.0.0/8 [1/0] is directly connected, Null0, bh S>* 10.1.0.0/16 [1/0] is directly connected, Null0, bh S>* 172.16.0.0/12 [1/0] is directly connected, Null0, bh S>* 192.168.0.0/16 [1/0] is directly connected, Null0, bh
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra • Reject any routes from zebra except my-prefixes • `route-type: local` matches routes from zebra 60 defined-sets: prefix-sets: - prefix-set-name: my-prefixes prefix-list: - ip-prefix: 10.1.0.0/16 policy-definitions: - name: zebra-import-my-prefixes statements: - conditions: bgp-conditions: route-type: local match-prefix-set: prefix-set: my-prefixes match-set-options: invert actions: route-disposition: reject-route
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra • This seems to be just a policy change but you still need to restart gobgpd to get routes injected from zebra installed into BGP RIB • currently a kind of `soft reset in` from zebra is not supported! • If you really don’t like to restart gobgpd, then delete/re-add those static routes in zebra console • and open an issue at GitHub  61 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors default-import-policy: accept-route ### apply configuration change g2 $ systemctl restart gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Originate routes from zebra • We can see my route 10.1.0.0/16 appears in BGP RIB and advertised to r3 62 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:37:38 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 00:37:20 [{Origin: i} {Communities: 65001:1000}] root@r3> show route protocol bgp inet.0: 11 destinations, 13 routes (11 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 10.1.0.0/16 *[BGP/170] 00:39:15, MED 0, localpref 100 AS path: 65001 I > to 10.173.176.102 via ge-0/0/1.0
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • Establish iBGP peer between g2 and r1 using local loopback addresses • Apply export policy • to do `next hop self` • to all routes received from r3 which is peer on IXP 63
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP 64 neighbors: - config: neighbor-address: 10.1.0.101 peer-as: 65001 transport: config: local-address: 10.1.0.102 ### apply configuration change g2 $ systemctl reload gobgpd ### check establishment g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 00:07:41 Establ | 1 1 10.173.176.103 65003 00:08:16 Establ | 1 1
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP 65 ### check RIB g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:09:02 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 00:08:47 [{Origin: i} {Communities: 65001:1000}] *> 10.4.0.0/16 10.1.14.104 65004 00:08:12 [{Origin: i} {LocalPref: 100}] ### check routing table g2 $ # vtysh -d zebra -c "show ip route" : (snip) O>* 10.1.14.0/24 [110/0] via 10.1.12.101, eth3, 1d04h50m B>* 10.3.0.0/16 [20/0] via 10.173.176.103, eth2, 00:15:05 B> 10.4.0.0/16 [200/0] via 10.1.14.104 (recursive), 00:14:30 * via 10.1.12.101, eth3, 00:14:30 : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • We‘d like to change next-hop… • because IXP subnet address 10.172.176.0/24 is not in IGP (in this example it is OSPF) • r1 cannot reach 66 ### check adj-out g2 $ gobgp neighbor 10.1.0.101 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.1.0.102 [{Origin: i} {Med: 0} {LocalPref: 100}] 10.3.0.0/16 10.173.176.103 65003 [{Origin: i} {LocalPref: 100} {Communities: 65001:1000}] root@r1> show route protocol bgp : (snip) 10.3.0.0/16 *[BGP/170] 00:21:05, localpref 100, from 10.1.0.102 AS path: 65003 I Discard : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • Not possible to specify “from who” in export policy • instead we can use community which we applied previously 67 defined-sets: bgp-defined-sets: community-sets: - community-set-name: from-ixp-neighbors community-list: - "65001:1000" policy-definitions: - name: set-next-hop-self statements: - conditions: bgp-conditions: match-community-set: community-set: from-ixp-neighbors actions: bgp-actions: set-next-hop: self
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • When modifying export policy, you need to call `softresetout` manually • contrary to `softresetin` which will be performed automatically • Caveats: currently after `softresetout` gobgpd will send ALL NLRIs in the RIB to peers, not only updated NLRIs • Maybe problematic with huge number of prefixes • Open an issue at GitHub :P 68 global: apply-policy: config: export-policy-list: - set-next-hop-self default-export-policy: accept-route ### apply configuration change g2 $ systemctl reload gobgpd ### perform soft reset out g2 $ gobgp neighbor 10.1.0.101 softresetout
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • next-hop attribute is modified as intended to point the loopback address of g2 69 g2 $ gobgp neighbor 10.1.0.101 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.1.0.102 [{Origin: i} {Med: 0} {LocalPref: 100}] 10.3.0.0/16 10.1.0.102 65003 [{Origin: i} {LocalPref: 100} {Communities: 65001:1000}] root@r1> show route receive-protocol bgp 10.1.0.102 inet.0: 16 destinations, 18 routes (16 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 10.1.0.0/16 10.1.0.102 0 100 I * 10.3.0.0/16 10.1.0.102 100 65003 I
INTERNET MULTIFEED CO.Copyright © 2016 Establish iBGP • Now you can reach from r4 loopback address to r3 loopback address that all RIBs/FIBs in between them are properly set up 70 root@r4> ping 10.3.0.103 source 10.4.0.104 count 3 PING 10.3.0.103 (10.3.0.103): 56 data bytes 64 bytes from 10.3.0.103: icmp_seq=0 ttl=62 time=10.496 ms 64 bytes from 10.3.0.103: icmp_seq=1 ttl=62 time=2.417 ms 64 bytes from 10.3.0.103: icmp_seq=2 ttl=62 time=2.586 ms --- 10.3.0.103 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.417/5.166/10.496/3.769 ms root@r3> ping 10.4.0.104 source 10.3.0.103 count 3 PING 10.4.0.104 (10.4.0.104): 56 data bytes 64 bytes from 10.4.0.104: icmp_seq=0 ttl=62 time=10.624 ms 64 bytes from 10.4.0.104: icmp_seq=1 ttl=62 time=1.948 ms : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Establish eBGP peer between g2 and rs, which is Route Server at IXP • Control advertisement policy • by adding specific communities • Here we add community value which will reject advertising our prefix only to r3 from route server 71
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Nothing new here except md5 password authentication 72 neighbors: - config: neighbor-address: 10.173.176.211 peer-as: 64686 auth-password: pass65001 ### apply configuration change g2 $ systemctl reload gobgpd ### check establishment g2 $ gobgp n Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 03:16:11 Establ | 1 1 10.173.176.103 65003 03:16:46 Establ | 1 1 10.173.176.211 64686 00:05:32 Establ | 1 1
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Receiving the same route for AS65003 from rs • It’s actually coming from rs that AS_PATH does not include 64686 which is the ASN of rs • No communities for IXP peers 65001:1000 added yet • We’ll do that later 73 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 03:21:07 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 03:20:52 [{Origin: i} {Communities: 65001:1000}] * 10.3.0.0/16 10.173.176.103 65003 00:09:38 [{Origin: i}] *> 10.4.0.0/16 10.1.14.104 65004 03:20:17 [{Origin: i} {LocalPref: 100}]
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Receiving routes from rs at r3 • AS_PATH does not contain 64686 • next-hop is not the address of rs which is 10.173.176.211 74 root@r3> show route receive-protocol bgp 10.173.176.211 inet.0: 12 destinations, 16 routes (12 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 10.1.0.0/16 10.173.176.102 0 65001 I 10.4.0.0/16 10.173.176.102 65001 65004 I
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Add the same community to tag routes `coming from IXP peers` as well as r3 • We’ve already have policy for IXP peers applied to r3 • What we need to do here is just add the neighbor address of the rs to the neighbor-set 75 defined-sets: neighbor-sets: - neighbor-set-name: ixp-neighbors neighbor-info-list: - 10.173.176.103 - 10.173.176.211 ### apply configuration g2 $ systemctl reload gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Add peer to Route Server • Routes received from rs are also tagged with community 65001:1000 76 ### apply configuration change g2 $ systemctl reload gobgpd g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 03:51:52 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 03:51:37 [{Origin: i} {Communities: 65001:1000}] * 10.3.0.0/16 10.173.176.103 65003 00:40:23 [{Origin: i} {Communities: 65001:1000}] *> 10.4.0.0/16 10.1.14.104 65004 03:51:02 [{Origin: i} {LocalPref: 100}]
INTERNET MULTIFEED CO.Copyright © 2016 Control Route Server policy with communities • IXP in this tutorial provides following policy at Route Server to control advertisement to other peers • ex.) If you like to advertise to all peers EXCEPT AS65003 • Add “0:65003” • ex.) If you like to advertise ONLY TO AS65003 and AS65123 • Add “0:64686 64686:65003 64686:65123” 77 ASN meaning 0:N Do not advertise to N 64686:N Advertise to N 0:64686 Do not advertise to ANY peer
INTERNET MULTIFEED CO.Copyright © 2016 Control Route Server policy with communities 78 defined-sets: neighbor-sets: - neighbor-set-name: ixp-rs neighbor-info-list: - 10.173.176.211 policy-definitions: - name: rs-no-export-to-as65003 statements: - conditions: match-neighbor-set: neighbor-set: ixp-rs actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "0:65003"
INTERNET MULTIFEED CO.Copyright © 2016 Control Route Server policy with communities • Remember that you need to call `softresetout` manually when you change export policy 79 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors default-import-policy: accept-route export-policy-list: - set-next-hop-self - rs-no-export-to-as65003 default-export-policy: accept-route g2 $ systemctl reload gobgpd g2 $ systemctl neighbor 10.173.176.211 softresetout
INTERNET MULTIFEED CO.Copyright © 2016 Control Route Server policy with communities • We can confirm that r3 does not receive any routes from rs because of the community based policy control has taken place 80 g2 $ gobgp neighbor 10.173.176.211 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.173.176.102 65001 [{Origin: i} {Med: 0} {Communities: 0:65003}] 10.3.0.0/16 10.173.176.102 65001 65003 [{Origin: i} {Communities: 65001:1000, 0:65003}] 10.4.0.0/16 10.173.176.102 65001 65004 [{Origin: i} {Communities: 0:65003}] root@r3> show route receive-protocol bgp 10.173.176.211 inet.0: 12 destinations, 14 routes (12 active, 0 holddown, 0 hidden)
INTERNET MULTIFEED CO.Copyright © 2016 RPKI • Configure route validation with RPKI • Apply policy based on validation result • tag with communities • We will use following value 81 RPKI validation result community to add Valid 65001:2001 Invalid 65001:2002 Not found 65001:2003
INTERNET MULTIFEED CO.Copyright © 2016 RPKI • What we need is only the address of ROA cache server • Here we use open ROA cache server operated by INTERNET MULTIFEED CO. • see http://www.mfeed.ad.jp/rpki/en/ • This change requires restart of gobgpd 82 rpki-servers: - config: address: 210.173.170.254 port: 323 g2 $ systemctl restart gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 RPKI • “N” indicates record not found in ROA table • Of course it’s a private IP! • Local originating route will not be validated, only received routes are 83 g2 $ gobgp rpki server Session State Uptime #IPv4/IPv6 records 210.173.170.254:323 Up 00:05:38 24977/3522 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:05:48 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 00:05:38 [{Origin: i} {Communities: 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 00:05:35 [{Origin: i} {Communities: 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 00:05:30 [{Origin: i} {LocalPref: 100}]
INTERNET MULTIFEED CO.Copyright © 2016 RPKI • You can use `rpki—validation-result` to match against RPKI validation status of the route 84 policy-definitions: - name: tag-rpki-validation statements: - conditions: bgp-conditions: rpki-validation-result: valid actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2001“ # ...
INTERNET MULTIFEED CO.Copyright © 2016 RPKI 85 # cont‘d - conditions: bgp-conditions: rpki-validation-result: invalid actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2002" - conditions: bgp-conditions: rpki-validation-result: not-found actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2003"
INTERNET MULTIFEED CO.Copyright © 2016 RPKI 86 global: apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors - tag-rpki-validation g2 $ systemctl restart gobgpd g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 02:20:52 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 02:20:42 [{Origin: i} {Communities: 65001:2003, 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 02:20:39 [{Origin: i} {Communities: 65001:2003, 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 02:20:34 [{Origin: i} {LocalPref: 100} {Communities: 65001:2003}]
INTERNET MULTIFEED CO.Copyright © 2016 MRT • Configure MRT dump to monitor • RIB periodically (TABLE_DUMPv2) • incoming UPDATE messages (BGP4MP) 87
INTERNET MULTIFEED CO.Copyright © 2016 MRT • Dump of update messages can be rotated automatically by specifying interval and file name format • 2006-01-02_1504 = YYYY-MM-DD_HHMM • Seems weird but this is how to specify dates format • Table dump currently does not support rotating • content will be replaced every `dump-interval` second 88 mrt-dump: - config: dump-type: updates file-name: /tmp/updates.2006-01-02_1504.mrt rotation-interval: 180 - config: dump-type: table file-name: /tmp/table.mrt dump-interval: 60 g2 $ systemctl restart gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 MRT • Dumps are created • Use your favorite tool to see them • `bgpdump` from bgptools • `bgpreader` from BGPStream • etc… 89 g2 $ ls -al /tmp/*.mrt -rw-r--r-- 1 quagga quagga 1768 Dec 3 11:36 /tmp/table.mrt -rw-r--r-- 1 quagga quagga 241 Dec 3 11:28 /tmp/updates.2016-12- 03_1128.mrt -rw-r--r-- 1 quagga quagga 0 Dec 3 11:31 /tmp/updates.2016-12- 03_1131.mrt -rw-r--r-- 1 quagga quagga 0 Dec 3 11:34 /tmp/updates.2016-12- 03_1134.mrt
INTERNET MULTIFEED CO.Copyright © 2016 BMP • BMP is a protocol to monitor BGP • Abbreviation of BGP Monitoring Protocol • RFC7854 • GoBGP can export BMP messages • Several implementation for BMP server can be found • GoBGP itself can be simple BMP server • Here we use Ryu BMP Server • http://osrg.github.io/bmp/ 90
INTERNET MULTIFEED CO.Copyright © 2016 BMP • What we need is only the address of BMP Server • We will run Ryu BMP Server on util • Here again it requires the restart of gobgpd • Launch Ryu BMP Server within docker container 91 bmp-servers: - config: address: 10.254.0.250 port: 11019 g2 $ systemctl restart gobgpd util $ docker run -it -p 11019:11019 osrg/ryu /bin/bash util(container) # ryu run --verbose ./ryu/ryu/app/bmpstation loading app ./ryu/ryu/app/bmpstation instantiating app ./ryu/ryu/app/bmpstation of BMPStation BRICK bmpstation listening on 0.0.0.0:11019
INTERNET MULTIFEED CO.Copyright © 2016 BMP • We can see BMP message received at util 92 g2 $ gobgp neigh 10.173.176.103 reset ### (cont’d util(container) console) 2016 Dec 03 13:43:29 | 10.254.0.102 | BMPInitiation(info=[],len=6,type=4,version=3) 2016 Dec 03 13:43:29 | 10.254.0.102 | BMPPeerUpNotification(is_post_policy=False,len=158,local_address='0.0.0.0',local_port=49071,pee r_address='0.0.0.0',peer_as=64686,peer_bgp_id='10.173.176.211',peer_distinguisher=0,peer_type=0 ,received_open_message=BGPOpen(bgp_identifier='10.173.176.211',hold_time=90,len=45,my_as=64686, opt_param=[BGPOptParamCapabilityRouteRefresh(cap_code=2,cap_length=0,length=2,type=2), BGPOptParamCapabilityMultiprotocol(afi=1,cap_code=1,cap_length=4,length=6,reserved=0,safi=1,typ e=2), BGPOptParamCapabilityFourOctetAsNumber(as_number=64686,cap_code=65,cap_length=4,length=6,type=2 )],opt_param_len=16,type=1,version=4),remote_port=179,sent_open_message=BGPOpen(bgp_identifier= '10.1.0.102',hold_time=90,len=45,my_as=65001,opt_param=[BGPOptParamCapabilityRouteRefresh(cap_c ode=2,cap_length=0,length=2,type=2), BGPOptParamCapabilityMultiprotocol(afi=1,cap_code=1,cap_length=4,length=6,reserved=0,safi=1,typ e=2), BGPOptParamCapabilityFourOctetAsNumber(as_number=65001,cap_code=65,cap_length=4,length=6,type=2 )],opt_param_len=16,type=1,version=4),timestamp=1480772609.0,type=3,version=3) : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Store BGP event log into InfluxDB • InfluxDB is one of the time series DB • like RRD • GoBGP can export BGP related event logs directly into InfluxDB • This function is not documented yet 93
INTERNET MULTIFEED CO.Copyright © 2016 Store BGP event log into InfluxDB • Create new database 94 ### prepare influxdb on util util $ docker run --name=influxdb -d -p 8086:8086 influxdb util $ docker run --rm --net=container:influxdb -it influxdb influx - host localhost Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. Connected to http://localhost:8086 version 1.1.0 InfluxDB shell version: 1.1.0 > create database gobgpd > show databases name: databases name ---- _internal gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Store BGP event log into InfluxDB • What we need is only the address of InfluxDB • Here again it requires the restart of gobgpd • Note that you first need to launch InfluxDB because gobgpd will not boot if it fails to connect • There is a bug! • If you enable this, import policy against zebra will not work properly 95 collector: config: url: http://10.254.0.250:8086 db-name: gobgpd g2 $ systemctl restart gobgpd
INTERNET MULTIFEED CO.Copyright © 2016 Store BGP event log into InfluxDB • You can see records inserted into InfluxDB 96 ### util(docker)/influx console (cont’d) > use gobgpd Using database gobgpd > select * from peer name: peer time PeerAS PeerAddress PeerID State ---- ------ ----------- ------ ----- 1480776860991000000 64686 10.173.176.211 10.173.176.211 Established 1480776861015000000 65001 10.1.0.101 10.1.0.101 Established 1480776867995000000 65003 10.173.176.103 10.3.0.103 Established > select * from updates : (snip)
INTERNET MULTIFEED CO.Copyright © 2016 Wrap up • You can find further information in official docs • https://github.com/osrg/gobgp/tree/master/docs/sour ces 97

More Related Content

PDF
High-Performance Networking Using eBPF, XDP, and io_uring
byScyllaDB
 
PDF
Boost UDP Transaction Performance
byLF Events
 
PDF
Linux Linux Traffic Control
bySUSE Labs Taipei
 
PDF
Fun with PRB, VRFs and NetNS on Linux - What is it, how does it work, what ca...
byMaximilan Wilhelm
 
PDF
Faster packet processing in Linux: XDP
byDaniel T. Lee
 
PDF
VLAN vs VXLAN
byGLC Networks
 
PPTX
OpenvSwitch Deep Dive
byrajdeep
 
PDF
Accelerating Envoy and Istio with Cilium and the Linux Kernel
byThomas Graf
 
High-Performance Networking Using eBPF, XDP, and io_uring
byScyllaDB
 
Boost UDP Transaction Performance
byLF Events
 
Linux Linux Traffic Control
bySUSE Labs Taipei
 
Fun with PRB, VRFs and NetNS on Linux - What is it, how does it work, what ca...
byMaximilan Wilhelm
 
Faster packet processing in Linux: XDP
byDaniel T. Lee
 
VLAN vs VXLAN
byGLC Networks
 
OpenvSwitch Deep Dive
byrajdeep
 
Accelerating Envoy and Istio with Cilium and the Linux Kernel
byThomas Graf
 

What's hot

PDF
Linux Networking Explained
byThomas Graf
 
PPTX
Linux Network Stack
byAdrien Mahieux
 
PPTX
Vxlan control plane and routing
byWilfredzeng
 
PPTX
Vxlan deep dive session rev0.5 final
byKwonSun Bae
 
PPTX
MP BGP-EVPN 실전기술-1편(개념잡기)
byJuHwan Lee
 
PDF
LinuxCon 2015 Linux Kernel Networking Walkthrough
byThomas Graf
 
PDF
検証環境をGoBGPで極力仮想化してみた
byToshiya Mabuchi
 
PDF
VXLAN BGP EVPN: Technology Building Blocks
byAPNIC
 
PDF
DPDK: Multi Architecture High Performance Packet Processing
byMichelle Holley
 
PPTX
OVN - Basics and deep dive
byTrinath Somanchi
 
PDF
EVPN Introduction
byBangladesh Network Operators Group
 
PDF
Cilium - Fast IPv6 Container Networking with BPF and XDP
byThomas Graf
 
PDF
DevConf 2014 Kernel Networking Walkthrough
byThomas Graf
 
PDF
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017
byBruno Teixeira
 
PPTX
DPDK KNI interface
byDenys Haryachyy
 
PDF
Receive side scaling (RSS) with eBPF in QEMU and virtio-net
byYan Vugenfirer
 
PPTX
BGP Flowspec (RFC5575) Case study and Discussion
byAPNIC
 
PDF
The linux networking architecture
byhugo lu
 
PDF
MPLS L3 VPN Deployment
byAPNIC
 
PDF
OVS VXLAN Network Accelaration on OpenStack (VXLAN offload and DPDK) - OpenSt...
byVirtualTech Japan Inc.
 
Linux Networking Explained
byThomas Graf
 
Linux Network Stack
byAdrien Mahieux
 
Vxlan control plane and routing
byWilfredzeng
 
Vxlan deep dive session rev0.5 final
byKwonSun Bae
 
MP BGP-EVPN 실전기술-1편(개념잡기)
byJuHwan Lee
 
LinuxCon 2015 Linux Kernel Networking Walkthrough
byThomas Graf
 
検証環境をGoBGPで極力仮想化してみた
byToshiya Mabuchi
 
VXLAN BGP EVPN: Technology Building Blocks
byAPNIC
 
DPDK: Multi Architecture High Performance Packet Processing
byMichelle Holley
 
OVN - Basics and deep dive
byTrinath Somanchi
 
EVPN Introduction
byBangladesh Network Operators Group
 
Cilium - Fast IPv6 Container Networking with BPF and XDP
byThomas Graf
 
DevConf 2014 Kernel Networking Walkthrough
byThomas Graf
 
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017
byBruno Teixeira
 
DPDK KNI interface
byDenys Haryachyy
 
Receive side scaling (RSS) with eBPF in QEMU and virtio-net
byYan Vugenfirer
 
BGP Flowspec (RFC5575) Case study and Discussion
byAPNIC
 
The linux networking architecture
byhugo lu
 
MPLS L3 VPN Deployment
byAPNIC
 
OVS VXLAN Network Accelaration on OpenStack (VXLAN offload and DPDK) - OpenSt...
byVirtualTech Japan Inc.
 

Similar to Tutorial: Using GoBGP as an IXP connecting router

PPT
Bgp For Presentation
byAlp isik
 
PDF
Bgp Toc
bybigb0206
 
PDF
07.bgp
byJuan Carlos Añazco Millar
 
PDF
Bgp attribute-case study
bySalvatoreFILORIZZO
 
PDF
Picobgp - A simple deamon for routing advertising
byClaudio Mignanti
 
PPT
BSCI30S06 border gateway protocol BG ppt
byhodeve9961
 
PDF
BGP (border gateway routing protocol)
byNetwax Lab
 
DOCX
CCNP Route 642 902 BGP
byIT Tech
 
PDF
GoBGP : yet another OSS BGPd
byPavel Odintsov
 
PDF
Border Gateway Protocol BGP Info abot dynamic
byLakshmi101519
 
PDF
PLNOG15: BGP New Advanced Features - Piotr Wojciechowski
byPROIDEA
 
PDF
Cisco ccnp 642-902 practice tests for quick preparation
byCisco-642-902
 
PDF
08 ip oc304 2_e1_1 zxr10 m6000 bgp configuration 24
bylegasu zemene
 
PDF
Computer network (14)
byNYversity
 
DOCX
Cisco BGP Exam 642-661 Review Notes
byDuane Bodle
 
PDF
Bgp
byw78ww
 
PDF
Bgp
byDgeison Alexander Carvalho
 
PDF
Bgp
byHải Quân
 
PDF
Ccnp enterprise workbook v1.0 bgp zero to hero
bySagarR24
 
PDF
Bgp
bySatheesh Kirupai
 
Bgp For Presentation
byAlp isik
 
Bgp Toc
bybigb0206
 
07.bgp
byJuan Carlos Añazco Millar
 
Bgp attribute-case study
bySalvatoreFILORIZZO
 
Picobgp - A simple deamon for routing advertising
byClaudio Mignanti
 
BSCI30S06 border gateway protocol BG ppt
byhodeve9961
 
BGP (border gateway routing protocol)
byNetwax Lab
 
CCNP Route 642 902 BGP
byIT Tech
 
GoBGP : yet another OSS BGPd
byPavel Odintsov
 
Border Gateway Protocol BGP Info abot dynamic
byLakshmi101519
 
PLNOG15: BGP New Advanced Features - Piotr Wojciechowski
byPROIDEA
 
Cisco ccnp 642-902 practice tests for quick preparation
byCisco-642-902
 
08 ip oc304 2_e1_1 zxr10 m6000 bgp configuration 24
bylegasu zemene
 
Computer network (14)
byNYversity
 
Cisco BGP Exam 642-661 Review Notes
byDuane Bodle
 
Bgp
byw78ww
 
Bgp
byDgeison Alexander Carvalho
 
Bgp
byHải Quân
 
Ccnp enterprise workbook v1.0 bgp zero to hero
bySagarR24
 
Bgp
bySatheesh Kirupai
 

Recently uploaded

PPTX
Your Enterprise’s Digital Hub for Connection and Collaboration
byarchanaaam99
 
PDF
Delegating Behaviors in C++: A Practical Tour of the Available Mechanisms
byDaniele Pallastrelli
 
DOCX
Digital Dominance_ Why You Need a Top Web Design Agency in Dubai.docx
bydigpal3
 
PPTX
The Impact of GST on Medicine and the Healthcare Sector
byMarg Books
 
PDF
How App Developers in London UK Empower Digital Businesses – US APP Developer...
byIndia App Developer
 
PPTX
White Label Travel Portal Solutions for B2C, B2B & B2E
byRayds Services Limited
 
PDF
Chpetwr two Health Survey for - (2).pdf
byAhmedAlhadiAbduselam
 
PDF
Test Driven Development with mORMot 2 with Delphi and FPC
byArnaud Bouchez
 
PDF
Agentic AI meets Serverless on AWS - AWS User Group Cologne 2025
byVadym Kazulkin
 
PPTX
Building smarter with AI - From Idea to Market at Warp Speed
byLuca Grulla
 
PPTX
Digital Twins for RolandRust U Maryland 20251030 v14.pptx
byhome
 
PDF
AI and Generative AI: Dual-Use Risks and Autonomous Threats.pdf
byCyberneticGI
 
PDF
One HTTPS server in Modern Pascal to Rule Them All
byArnaud Bouchez
 
PDF
git - from commit to merge - semcomp beta 2018 workshop
byJoao Marins
 
PPTX
Empower Your Business with Busy Software from MovingCloud360.pptx
byMoving Cloud
 
PPTX
Trailblazer Community Session - Patterns & Anti-Patterns_ The Good, The Bad, ...
bymansoorahmad941764
 
PDF
Do customers need Windows Server Device CALs for devices if they have User CA...
byQ-Advise
 
PPTX
BUSY Software – Smart Accounting & Inventory Solution.pptx
byMoving Cloud
 
PPTX
Meridian 2025 - The Intent-Driven Future With NEAR Intents
byDejan Radic
 
PPTX
ETH Belgrade 2025 - AI Agent Custody and Fund Access
byDejan Radic
 
Your Enterprise’s Digital Hub for Connection and Collaboration
byarchanaaam99
 
Delegating Behaviors in C++: A Practical Tour of the Available Mechanisms
byDaniele Pallastrelli
 
Digital Dominance_ Why You Need a Top Web Design Agency in Dubai.docx
bydigpal3
 
The Impact of GST on Medicine and the Healthcare Sector
byMarg Books
 
How App Developers in London UK Empower Digital Businesses – US APP Developer...
byIndia App Developer
 
White Label Travel Portal Solutions for B2C, B2B & B2E
byRayds Services Limited
 
Chpetwr two Health Survey for - (2).pdf
byAhmedAlhadiAbduselam
 
Test Driven Development with mORMot 2 with Delphi and FPC
byArnaud Bouchez
 
Agentic AI meets Serverless on AWS - AWS User Group Cologne 2025
byVadym Kazulkin
 
Building smarter with AI - From Idea to Market at Warp Speed
byLuca Grulla
 
Digital Twins for RolandRust U Maryland 20251030 v14.pptx
byhome
 
AI and Generative AI: Dual-Use Risks and Autonomous Threats.pdf
byCyberneticGI
 
One HTTPS server in Modern Pascal to Rule Them All
byArnaud Bouchez
 
git - from commit to merge - semcomp beta 2018 workshop
byJoao Marins
 
Empower Your Business with Busy Software from MovingCloud360.pptx
byMoving Cloud
 
Trailblazer Community Session - Patterns & Anti-Patterns_ The Good, The Bad, ...
bymansoorahmad941764
 
Do customers need Windows Server Device CALs for devices if they have User CA...
byQ-Advise
 
BUSY Software – Smart Accounting & Inventory Solution.pptx
byMoving Cloud
 
Meridian 2025 - The Intent-Driven Future With NEAR Intents
byDejan Radic
 
ETH Belgrade 2025 - AI Agent Custody and Fund Access
byDejan Radic
 

Tutorial: Using GoBGP as an IXP connecting router

  • 1.
    INTERNET MULTIFEED CO.Copyright© 2016 Tutorial: Using GoBGP as an IXP connecting router Shu Sugimoto JPNAP / INTERNET MULTIFEED CO. IX.br Forum 10 2016/12/07(Wed)
  • 2.
    INTERNET MULTIFEED CO.Copyright© 2016 About this presentation • Show you how GoBGP can be used as a software router in conjunction with quagga • (Tutorial) Walk through the setup of IXP connecting router using GoBGP • This is going be/was spoken at IX.br Forum 10 • Slides available at SlideShare 2
  • 3.
    INTERNET MULTIFEED CO.Copyright© 2016 GoBGP • New BGP implementation • OSS developed by NTT Lab SIC • https://github.com/osrg/gobgp • Written in Golang • High performance • exploits multi-cores natively • Automation friendly • API first principle • CLI on top of API 3 gobgpd CLI gobgp Your Software API (gRPC)
  • 4.
    INTERNET MULTIFEED CO.Copyright© 2016 Key features 1/2 • full-featured CLI • Multiprotocol support • IPv4, IPv6, Labeled IPv4/IPv6, VPN IPv4/IPv6, EVPN, flowspec IPv4/IPv6/L2 • Flexible Policy • Graceful Restart • both restarting/helper speaker role • Route Reflector • Route Server 4
  • 5.
    INTERNET MULTIFEED CO.Copyright© 2016 Key features 2/2 • MRT dumping • BMP • RPKI validation • FIB manipulation • gRPC API • Standard configuration format • structured based on OpenConfig • supports toml/yaml/json/hcl 5
  • 6.
    INTERNET MULTIFEED CO.Copyright© 2016 Components • gobgpd • main daemon process which implements BGP • can be controlled via gRPC API • configuration file also supported • gobgp • full-featured CLI • convert human friendly commands into gRPC API call • and vice versa • configuration file (optional) • popular way to define the behavior of gobgpd • written in toml/yaml/json/hcl 6
  • 7.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp CLI command example • show list of neighbors • show RIB 7 $ gobgp neighbor Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 3d 08:25:02 Establ | 1 1 10.173.176.103 65003 3d 08:25:00 Establ | 1 1 10.173.176.211 64686 never Active | 0 0 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 3d 08:06:32 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 3d 08:34:01 [{Origin: i} {Communities: 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 00:00:03 [{Origin: i} {Communities: 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 3d 08:34:03 [{Origin: i} {LocalPref: 100}]
  • 8.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp CLI command example • neighbor operations 8 ### perform peer softresetin $ gobgp neighbor 10.1.0.101 softresetin ### perform peer reset $ gobgp neighbor 10.1.0.101 reset ### perform peer disable (shutdown) $ gobgp neighbor 10.1.0.101 disable ### perform peer enable $ gobgp neighbor 10.1.0.101 enable
  • 9.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp CLI command example • You can even add/delete peers/routes/policies on the fly • Some easy use case can be accomplished without using configuration file 9 ### launch gobgpd (need privilege to listen on tcp 179) $ sudo gobgpd ### (open new terminal) ### set AS and router-id $ gobgp global as 1 router-id 1.1.1.1 ### add neighbor $ gobgp neighbor add 192.0.2.2 as 2 ### add route into RIB, which will then advertised to peers $ gobgp global rib add -a ipv4 10.0.0.0/24 med 10 community 100:100
  • 10.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp CLI command example • `-j` support • Every command supports json output 10 $ gobgp global rib 10.4.0.0/16 -j {"10.4.0.0/16":[{"nlri":{"prefix":"10.4.0.0/16"},"attrs":[{"type":1,"val ue":0},{"type":2,"as_paths":[{"segment_type":2,"num":1,"asns":[65004]}]} ,{"type":3,"nexthop":"10.1.14.104"},{"type":5,"value":100},{"type":8,"co mmunities":[4259907539]}],"age":1480845275,"validation":"not- found","source-id":"10.1.0.101","neighbor-ip":"10.1.0.101"}]} ### pretty print using python $ gobgp global rib 10.4.0.0/16 -j | python -mjson.tool { "10.4.0.0/16": [ { "age": 1480845275, "attrs": [ { "type": 1, : (snip)
  • 11.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp CLI command example • Event monitoring • The data will be sent from gobgpd through the gRPC connection channel when events occur • Push notification • One implementation of the event driven application 11 $ gobgp monitor global rib [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.4.0.0/16 via 10.1.14.104 aspath [65004] attrs [{Origin: i} {LocalPref: 100} {Communities: 65001:2003}] [DELROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] [ROUTE] 10.3.0.0/16 via 10.173.176.103 aspath [65003] attrs [{Origin: i} {Communities: 65001:1000, 65001:2003}] (waiting for further events…)
  • 12.
    INTERNET MULTIFEED CO.Copyright© 2016 Who use GoBGP? • IXP Route Server • JPNAP • Monitoring compoment • FastNetMon • DoS/DDoS analyzer • BGPmon • BGP routing information monitor • Cloudwatt • is an OpenStack based public cloud service • They seems to be using it as a component of Looking Glass 12
  • 13.
    INTERNET MULTIFEED CO.Copyright© 2016 Who use GoBGP? • SDN solution component • Project Calico • Contiv netplugin • Both are network plugin for containers • Test virtualization • ex.) One is using GoBGP to virtualize the Lab environment for testing BGP routing policies • Only test target is a real router, everything else is VM • GoBGP is used as a route originator • gRPC API is used to generate arbitrary routes • http://www.slideshare.net/ssuser6a8d29/gobgp • in Japanese :P 13
  • 14.
    INTERNET MULTIFEED CO.Copyright© 2016 Why use GoBGP? • Automation Friendliness • It’s definitely easier to implement whatever comes up in your mind • Everything is exposed through API • `gobgp -j` can be a good start point of “thinking about automation” • can skip the painful “output parsing” part, which broke a lot of people’s motivation • much easier to begin with for operators  • Performance • Especially in larger deploy 14
  • 15.
    INTERNET MULTIFEED CO.Copyright© 2016 GoBGP performance 15 Quagga won’t scale if the policy is huge Comparing the convergence time in Y-axis (between beginning of first peer up and the last update sent) X-axis = # of IPs in prefix list / fixed # of peers = 100, number of prefixes per peer = 100, Route Server setup Graph generated using bgperf ( https://github.com/osrg/bgperf ), in Jan 2016
  • 16.
    INTERNET MULTIFEED CO.Copyright© 2016 Why not? Then… • “Because it’s not stable yet, isn’t it?” • “Because no one is using it as yet, right?” • (many many many reasons…) 16
  • 17.
    INTERNET MULTIFEED CO.Copyright© 2016 Why not? Then… • I can’t say 100% “Yes it’s stable” but… • We use it in production as a Route Server at JPNAP and so far not facing any big issue • Development team are amazingly fast • in response, finding causes, and providing patches • I really would like some of you to consider trying GoBGP • as a software router • There’s no one still AFAIK • Need help? Find bug? Anything you want to discuss? • Open an issue at GitHub • or you can join open slack channel • https://slackin-gobgp.mybluemix.net/ 17
  • 18.
    INTERNET MULTIFEED CO.Copyright© 2016 Bugs resolved • Here are the list of bugs found and fixed while I was working on making this tutorial… • server: fix bug of deleteNeighbor() #1184 • https://github.com/osrg/gobgp/pull/1184 • zebra: add flags for recursive nexthop lookup if necessary #1179 • https://github.com/osrg/gobgp/pull/1179 • fix several bugs related to rpki and policy #1178 • https://github.com/osrg/gobgp/pull/1178 • gobgpd: support global policy assignment update via configuration file #1177 • https://github.com/osrg/gobgp/pull/1177 • Some other bugs still under discussion • Many thanks to @wataru and @tomo 18
  • 19.
    INTERNET MULTIFEED CO.Copyright© 2016 FIB manipulation • GoBGP is just a bgp daemon and itself does not contain any functionality to modify routing table • If you like to use GoBGP as a component of software router and do packet forwarding, you need to implement that • There are two options to achieve FIB manipulation with GoBGP • Use built-in zebra integration • Write your own code using gRPC API • In this tutorial I’ll introduce zebra integration and show you how we can use it 19
  • 20.
    INTERNET MULTIFEED CO.Copyright© 2016 cf.) GoPlane • https://github.com/osrg/goplane • Utilize gobgpd gRPC API and perform things like FIB manipulation on Linux platform • Can create EVPN/VxLAN fabric • Also can modify iptables rules based on received FlowSpec routes • Can’t do any complex routing • ex.) Recursive next-hop resolving not supported • Not suitable for the use case in this tutorial 20
  • 21.
    INTERNET MULTIFEED CO.Copyright© 2016 Quagga architecture 21 zserv.api zebra routing table Linux kernel bgpdospfd vtysh vtysh vtysh CLI operation tcp / unix socket zclient(s) more protocols available ripd, isisd...Quagga zebra protocol
  • 22.
    INTERNET MULTIFEED CO.Copyright© 2016 Zebra integration of GoBGP 22 zserv.api zebra routing table Linux kernel gobgpdospfd vtysh API gobgpd acts as one of zclient vtysh CLI operation gRPC gobgp CLI
  • 23.
    INTERNET MULTIFEED CO.Copyright© 2016 Tutorial • Target: IXP connecting router • You will walk through • Installation of GoBGP • gobgpd administration via systemd • Writing configuration file • Adding eBGP/iBGP peers • Applying policy • Including example use case to control route advertising over Route Server • FIB manipulation with zebra integration • RPKI setup • MRT/BMP setup • InfluxDB integration setup 23
  • 24.
    INTERNET MULTIFEED CO.Copyright© 2016 Topology 24 lo: 10.3.0.103 AS65004 10.4.0.0/16 AS65001 10.1.0.0/16 lo: 10.1.0.102 lo: 10.1.0.101 10.1.12.0/24 10.173.176.0/24 .211 .103.102 .102 .101 10.1.14.0/24 .101 .104 g2 IX r3 r1 r4 rs OSPF Area 0 eth2 eth3 eth1 ge-0/0/1 ge-0/0/1 ge-0/0/2 ge-0/0/1 redistribute direct lo: 10.4.0.104 AS65003 10.3.0.0/16 eBGP eBGP iBGP Route Server AS65686 Target Host OS Softwares r1 vSRX 12.1 g2 Debian 8 GoBGP Quagga (zebra/ospfd) r3 vSRX 12.1 r4 vSRX 12.1 rs Debian 8 GoBGP (Route Server) util Debian 8 Ryu BMP server influxdb g2 util eth1 eth1 .102 .250 10.254.0.0/24
  • 25.
    INTERNET MULTIFEED CO.Copyright© 2016 gobgp-tutorial • You can instantly build the demo topology using Vagrant • Vagrantfile available at GitHub • https://github.com/s2ugimot/gobgp-tutorial • Follow the instructions in README.md • Everything except the GoBGP in g2 will be set up 25
  • 26.
    INTERNET MULTIFEED CO.Copyright© 2016 Demo setup • MacBook Pro 13r (Early 2015) • Mac OS X 10.11.6 (El Capitan) • 3.1GHz Intel Core i7 • 16GB RAM (at least > 8GB) • Vagrant 1.8.1 • vagrant-host-shell 0.0.4 • vagrant-junos 0.2.1 • VirtualBox 5.0.28 • Should work on Windows/Linux too • haven’t tested though :P 26
  • 27.
    INTERNET MULTIFEED CO.Copyright© 2016 System consideration in real world • RAM • > 16GB • Recommend 32GB or more if you handle IPv4 full routes = 600k routes with multiple eBGP peers • This is by design • No extensive tweaks, keep it simple, just buy memory • CPU • > 2cores • The more, the better performance • VM is fine • If forwarding is not the issue 27
  • 28.
    INTERNET MULTIFEED CO.Copyright© 2016 Tutorial: Step by step 28
  • 29.
    INTERNET MULTIFEED CO.Copyright© 2016 Install Go • Just follow the instruction on official web • https://golang.org/doc/install • It’s quite simple, just extract tar.gz and add to $PATH • Choose go1.5 or above 29
  • 30.
    INTERNET MULTIFEED CO.Copyright© 2016 Build GoBGP • Built binaries are portable • Libraries are statically linked into binary • Should preserve the $GOPATH • To make it reproducible • Go itself does not provide any good solution still • ex.) build in container and keep the image 30 ### build binaries g2 $ go get github.com/osrg/gobgp/gobgpd g2 $ go get github.com/osrg/gobgp/gobgp ### copy them to somewhere under $PATH g2 $ cp $GOPATH/bin/* /usr/local/sbin ### optional: install shell completion for gobgp command g2 $ cp $GOPATH/src/github.com/osrg/gobgp/tools/completion/*.bash /etc/bash_completion.d/
  • 31.
    INTERNET MULTIFEED CO.Copyright© 2016 Setup systemd unit file • Prepare systemd unit file for gobgpd process to let it managed by systemd 31
  • 32.
    INTERNET MULTIFEED CO.Copyright© 2016 Setup systemd unit file 32 ### create a unit file for gobgpd g2 $ cat << EOF > /etc/systemd/system/gobgpd.service [Unit] Description=gobgpd After=network.target syslog.target [Service] Type=simple PermissionsStartOnly=yes User=quagga ExecStartPre=/sbin/setcap 'cap_net_bind_service=+ep' /usr/local/sbin/gobgpd ExecStart=/usr/local/sbin/gobgpd -f /etc/gobgp/gobgpd.conf -t yaml -- cpus=2 ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s TERM $MAINPID [Install] WantedBy=multi-user.target EOF g2 $ systemctl daemon-reload
  • 33.
    INTERNET MULTIFEED CO.Copyright© 2016 Setup systemd unit file • Set process user same as that of quagga service • because gobgpd needs rw permission to zserv.api to speak to zebra • Use `setcap` to let unprivileged user listen on port < 1024 • Security concern: you should not run any important gobgpd on shared servers! • Anyone has access to full-control through gRPC API • At least you should block access to API port (default tcp 50051) by applying iptables rules from other hosts • Exposing `setcap`-ed binary to others might compromise your server seucrity 33
  • 34.
    INTERNET MULTIFEED CO.Copyright© 2016 Adjusting options • Update `ExecStart` section in unit file to fit your needs • You can specify how many cores you want to use 34 Application Options: -f, --config-file= specifying a config file -t, --config-type= specifying config type (toml, yaml, json) (default: toml) -l, --log-level= specifying log level -p, --log-plain use plain format for logging (json by default) -s, --syslog= use syslogd --syslog-facility= specify syslog facility --disable-stdlog disable standard logging --cpus= specify the number of CPUs to be used --api-hosts= specify the hosts that gobgpd listens on (default: :50051) -r, --graceful-restart flag restart-state in graceful-restart capability -d, --dry-run check configuration --pprof-host= specify the host that gobgpd listens on for pprof (default: localhost:6060) --pprof-disable disable pprof profiling
  • 35.
    INTERNET MULTIFEED CO.Copyright© 2016 Create the first configuration • Minimal configuration includes AS and router-id • gobgpd supports toml/yaml/json/hcl format • I would recommend using toml • used in official document • Here I will use yaml through this tutorial • It’s easier to read 35 global: config: as: 65001 router-id: 10.1.0.102
  • 36.
    INTERNET MULTIFEED CO.Copyright© 2016 toml configuration example • toml is better in two reasons • It is easy to “copy & paste” config fragments • Element ordering does not matter • ex.) You can define a neighbor, a policy for it, then other neighbor • Arbitrary ordering helps you organize the configuration 36 [global.config] as = "65001" router-id = "10.1.0.102” [[neighbors]] [neighbors.config] neighbor-address = 10.173.176.103 peer-as = 65003 [[neighbors]] [neighbors.config] neighbor-address = 10.1.0.101 peer-as = 65001 [neighbors.transport.config] local-address = 10.1.0.102
  • 37.
    INTERNET MULTIFEED CO.Copyright© 2016 Launch gobgpd 37 g2 $ systemctl start gobgpd g2 $ systemctl status gobgpd ● gobgpd.service - gobgpd Loaded: loaded (/etc/systemd/system/gobgpd.service; disabled) Active: active (running) since Thu 2016-12-01 05:56:17 UTC; 7s ago Process: 5987 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS) Process: 5981 ExecReload=/bin/kill -s HUP $MAINPID (code=exited, status=0/SUCCESS) Process: 6057 ExecStartPre=/sbin/setcap cap_net_bind_service=+ep /usr/local/sbin/gobgpd (code=exited, status=0/SUCCESS) Main PID: 6061 (gobgpd) CGroup: /system.slice/gobgpd.service └─6061 /usr/local/sbin/gobgpd -f /etc/gobgp/gobgpd.conf -t yaml --cpus=2 Dec 01 05:56:17 g2 gobgpd[6061]: {"level":"info","msg":"gobgpd started","time":"2016-12-01T05:56:17Z"} Dec 01 05:56:17 g2 gobgpd[6061]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-01T05:56:17Z"}
  • 38.
    INTERNET MULTIFEED CO.Copyright© 2016 Logging • Default to output in json format • easy for computers • (a bit) hard for humans :P • You can switch to plain text mode by passing `-p/--log- plain` • All logs will be shown in stdout • can be disabled by `--disable-stdlog` • Syslog is also supported • exactly the same log will be produced 38
  • 39.
    INTERNET MULTIFEED CO.Copyright© 2016 Logging 39 ### check log via journald g2 $ journalctl -xn -f -u gobgpd -- Logs begin at Mon 2016-12-05 05:05:34 UTC. -- Dec 05 05:21:57 g2 gobgpd[1352]: {"level":"info","msg":"gobgpd started","time":"2016-12-05T05:21:57Z"} Dec 05 05:21:57 g2 gobgpd[1352]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-05T05:21:57Z"} ### check log via file (via rsyslogd through systemd/journald) g2 $ tailf /var/log/syslog Dec 5 05:21:57 g2 gobgpd[1352]: {"level":"info","msg":"gobgpd started","time":"2016-12-05T05:21:57Z"} Dec 5 05:21:57 g2 gobgpd[1352]: {"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2016-12-05T05:21:57Z"}
  • 40.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer • Establish eBGP peer between g2 and r3 • Receive routes from r3 • Apply policy to routes received from r3 • r3 is a peer over IXP • will `tag` to identify the routes received from IXP peers using community • Use ”65000:1000” 40
  • 41.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer • neighbor/policy configuration can be dynamically applied by sending SIGHUP to the gobgpd process • We define `reload` to send SIGHUP in systemd unit file • If there is a syntax error in configuration file, gobgpd just abort reloading and remain on the current state • You can see exact location causing error in log 41 global: config: as: 65001 router-id: 10.1.0.102 neighbors: - config: neighbor-address: 10.173.176.103 peer-as: 65003 g2 $ systemctl reload gobgpd
  • 42.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer 42 g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.173.176.103 65003 never Active | 0 0 ### wait for a while... g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.173.176.103 65003 00:00:36 Establ | 1 1
  • 43.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer 43 g2 $ gobgp neighbor 10.173.176.103 BGP neighbor is 10.173.176.103, remote AS 65003 BGP version 4, remote router ID 10.3.0.103 BGP state = established, up for 00:03:43 BGP OutQ = 0, Flops = 0 Hold time is 90, keepalive interval is 30 seconds Configured hold time is 90, keepalive interval is 30 seconds Neighbor capabilities: multiprotocol: ipv4-unicast: advertised and received route-refresh: advertised and received graceful-restart: received 4-octet-as: advertised and received cisco-route-refresh: received Message statistics: Sent Rcvd Opens: 1 1 Notifications: 0 0 Updates: 0 1 Keepalives: 8 10 : (snip)
  • 44.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer • Caveats: you need to use `-j` option and see in json format to see more detailed attributes like router-id or source (from which neighbor the route has received) • Feel free to open an issue at GitHub  44 ### check received routes ### junos: show route receiving-protocol bgp 10.173.176.103 g2 $ gobgp neighbor 10.173.176.103 adj-in Network Next Hop AS_PATH Age Attrs 10.3.0.0/16 10.173.176.103 65003 00:12:08 [{Origin: i}] ### check BGP RIB ### junos: show route protocol bgp g2 $ gobgp global rib gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.3.0.0/16 10.173.176.103 65003 00:17:34 [{Origin: i}]
  • 45.
    INTERNET MULTIFEED CO.Copyright© 2016 Configure first eBGP peer 45 ### see detailed information using `-j` g2 $ gobgp global rib –j | python -mjson.tool { "10.3.0.0/16": [ { "age": 1480923494, "attrs": [ : (snip) ], "neighbor-ip": "10.173.176.103", "nlri": { "prefix": "10.3.0.0/16" }, "source-id": "10.3.0.103" } ] }
  • 46.
    INTERNET MULTIFEED CO.Copyright© 2016 Apply policy to received routes 46 policy-definitions: - name: tag-ixp-neighbors statements: - conditions: match-neighbor-set: neighbor-set: ixp-neighbors match-set-options: any actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:1000" defined-sets: neighbor-sets: - neighbor-set-name: ixp-neighbors neighbor-info-list: - 10.173.176.103
  • 47.
    INTERNET MULTIFEED CO.Copyright© 2016 Apply policy to received routes 47 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - tag-ixp-neighbors default-import-policy: accept-route ### apply policy g2 $ systemctl reload gobgpd ### see what happens g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.3.0.0/16 10.173.176.103 65003 01:43:37 [{Origin: i} {Communities: 65001:1000}]
  • 48.
    INTERNET MULTIFEED CO.Copyright© 2016 Policy definition structure • Policy consists of statements • Each statement has condition(s) and action(s) • condition specifies the state of NLRIs to match • ex.) prefix, neighbor, AS_PATH, community, … • actions specifies what to do with the NLRIs • accept / reject • modify path attributes • community, MED, local-pref, AS_PATH, next-hop • Some condition refers to defined-sets • ex.) prefix-set, neighbor-set, community-set, … 48
  • 49.
    INTERNET MULTIFEED CO.Copyright© 2016 Policy definition structure 49
  • 50.
    INTERNET MULTIFEED CO.Copyright© 2016 Policy configuration basics • First you define policies • and defined-sets if needed to achieve your needs • Then attach it to global config level • not in neighbor level • beware! There is a nob under neighbor level which is only for Route Server setup • You can specify neighbor using `neighbor-set` • in import policy: from which neighbor • in export policy: to which neighbor 50
  • 51.
    INTERNET MULTIFEED CO.Copyright© 2016 Where policies are applied 51
  • 52.
    INTERNET MULTIFEED CO.Copyright© 2016 Other policy examples • see the official doc for more details • https://github.com/osrg/gobgp/blob/master/docs/sour ces/policy.md 52
  • 53.
    INTERNET MULTIFEED CO.Copyright© 2016 Install routes into Linux FIB / zebra integration • gobgpd can act as a replacement of bgpd in quagga • Setup zebra integration • And first we see BGP received route from r3 installed in the routing table 53
  • 54.
    INTERNET MULTIFEED CO.Copyright© 2016 Install routes into Linux FIB / zebra integration • Specify the path to `zserv.api` created by zebra daemon • gobgpd process needs rw access to it • Check permission if you encounter any problem • `redistribute-route-type-list` specifies from which protocol gobgpd imports routes into BGP RIB • At this time we do not redistribute any route from zebra so just leave it blank 54 zebra: config: enabled: true url: "unix:/var/run/quagga/zserv.api" redistribute-route-type-list: []
  • 55.
    INTERNET MULTIFEED CO.Copyright© 2016 Install routes into Linux FIB / zebra integration • You need to restart gobgpd • some configuration change need restart of gobgpd to take effect • zebra integration is one of them • Booting order is important! • zebra process must be started before gobgpd starts 55 ### apply configuration change g2 $ systemctl restart gobgpd
  • 56.
    INTERNET MULTIFEED CO.Copyright© 2016 Install routes into Linux FIB / zebra integration 56 g2 $ vtysh -d zebra -c "show ip route" Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, A - Babel, > - selected route, * - FIB route K>* 0.0.0.0/0 via 10.0.2.2, eth0 S>* 10.0.0.0/8 [1/0] is directly connected, Null0, bh C>* 10.0.2.0/24 is directly connected, eth0 S>* 10.1.0.0/16 [1/0] is directly connected, Null0, bh O>* 10.1.0.101/32 [110/10] via 10.1.12.101, eth3, 1d02h44m O 10.1.0.102/32 [110/10] is directly connected, lo, 1d02h44m C>* 10.1.0.102/32 is directly connected, lo O 10.1.12.0/24 [110/10] is directly connected, eth3, 1d02h44m C>* 10.1.12.0/24 is directly connected, eth3 O>* 10.1.14.0/24 [110/0] via 10.1.12.101, eth3, 1d02h44m B>* 10.3.0.0/16 [20/0] via 10.173.176.103, eth2, 00:16:10 C>* 10.173.176.0/24 is directly connected, eth2 C>* 10.254.0.0/24 is directly connected, eth1 C>* 127.0.0.0/8 is directly connected, lo S>* 172.16.0.0/12 [1/0] is directly connected, Null0, bh S>* 192.168.0.0/16 [1/0] is directly connected, Null0, bh
  • 57.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra • Inject routes from zebra into gobgpd then advertise to r3 • We use static null route • Common way to originate your own prefixes • Set import policy to select only routes that we want to import 57
  • 58.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra • `redistribute-route-type-list` • Specify from which protocol gobgpd imports routes • If you like to also redistribute connected routes and ospf routes then just append them to the list 58 zebra: config: enabled: true url: "unix:/var/run/quagga/zserv.api" redistribute-route-type-list: - static
  • 59.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra 59 ### apply configuration change, need restart g2 $ systemctl restart gobgpd ### all static routes are imported g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.0.0.0/8 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 10.1.0.0/16 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 172.16.0.0/12 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] *> 192.168.0.0/16 0.0.0.0 00:00:05 [{Origin: i} {Med: 0}] g2 $ vtysh -d zebra -c "show ip route static“ S>* 10.0.0.0/8 [1/0] is directly connected, Null0, bh S>* 10.1.0.0/16 [1/0] is directly connected, Null0, bh S>* 172.16.0.0/12 [1/0] is directly connected, Null0, bh S>* 192.168.0.0/16 [1/0] is directly connected, Null0, bh
  • 60.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra • Reject any routes from zebra except my-prefixes • `route-type: local` matches routes from zebra 60 defined-sets: prefix-sets: - prefix-set-name: my-prefixes prefix-list: - ip-prefix: 10.1.0.0/16 policy-definitions: - name: zebra-import-my-prefixes statements: - conditions: bgp-conditions: route-type: local match-prefix-set: prefix-set: my-prefixes match-set-options: invert actions: route-disposition: reject-route
  • 61.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra • This seems to be just a policy change but you still need to restart gobgpd to get routes injected from zebra installed into BGP RIB • currently a kind of `soft reset in` from zebra is not supported! • If you really don’t like to restart gobgpd, then delete/re-add those static routes in zebra console • and open an issue at GitHub  61 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors default-import-policy: accept-route ### apply configuration change g2 $ systemctl restart gobgpd
  • 62.
    INTERNET MULTIFEED CO.Copyright© 2016 Originate routes from zebra • We can see my route 10.1.0.0/16 appears in BGP RIB and advertised to r3 62 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:37:38 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 00:37:20 [{Origin: i} {Communities: 65001:1000}] root@r3> show route protocol bgp inet.0: 11 destinations, 13 routes (11 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 10.1.0.0/16 *[BGP/170] 00:39:15, MED 0, localpref 100 AS path: 65001 I > to 10.173.176.102 via ge-0/0/1.0
  • 63.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • Establish iBGP peer between g2 and r1 using local loopback addresses • Apply export policy • to do `next hop self` • to all routes received from r3 which is peer on IXP 63
  • 64.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP 64 neighbors: - config: neighbor-address: 10.1.0.101 peer-as: 65001 transport: config: local-address: 10.1.0.102 ### apply configuration change g2 $ systemctl reload gobgpd ### check establishment g2 $ gobgp neigh Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 00:07:41 Establ | 1 1 10.173.176.103 65003 00:08:16 Establ | 1 1
  • 65.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP 65 ### check RIB g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:09:02 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 00:08:47 [{Origin: i} {Communities: 65001:1000}] *> 10.4.0.0/16 10.1.14.104 65004 00:08:12 [{Origin: i} {LocalPref: 100}] ### check routing table g2 $ # vtysh -d zebra -c "show ip route" : (snip) O>* 10.1.14.0/24 [110/0] via 10.1.12.101, eth3, 1d04h50m B>* 10.3.0.0/16 [20/0] via 10.173.176.103, eth2, 00:15:05 B> 10.4.0.0/16 [200/0] via 10.1.14.104 (recursive), 00:14:30 * via 10.1.12.101, eth3, 00:14:30 : (snip)
  • 66.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • We‘d like to change next-hop… • because IXP subnet address 10.172.176.0/24 is not in IGP (in this example it is OSPF) • r1 cannot reach 66 ### check adj-out g2 $ gobgp neighbor 10.1.0.101 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.1.0.102 [{Origin: i} {Med: 0} {LocalPref: 100}] 10.3.0.0/16 10.173.176.103 65003 [{Origin: i} {LocalPref: 100} {Communities: 65001:1000}] root@r1> show route protocol bgp : (snip) 10.3.0.0/16 *[BGP/170] 00:21:05, localpref 100, from 10.1.0.102 AS path: 65003 I Discard : (snip)
  • 67.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • Not possible to specify “from who” in export policy • instead we can use community which we applied previously 67 defined-sets: bgp-defined-sets: community-sets: - community-set-name: from-ixp-neighbors community-list: - "65001:1000" policy-definitions: - name: set-next-hop-self statements: - conditions: bgp-conditions: match-community-set: community-set: from-ixp-neighbors actions: bgp-actions: set-next-hop: self
  • 68.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • When modifying export policy, you need to call `softresetout` manually • contrary to `softresetin` which will be performed automatically • Caveats: currently after `softresetout` gobgpd will send ALL NLRIs in the RIB to peers, not only updated NLRIs • Maybe problematic with huge number of prefixes • Open an issue at GitHub :P 68 global: apply-policy: config: export-policy-list: - set-next-hop-self default-export-policy: accept-route ### apply configuration change g2 $ systemctl reload gobgpd ### perform soft reset out g2 $ gobgp neighbor 10.1.0.101 softresetout
  • 69.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • next-hop attribute is modified as intended to point the loopback address of g2 69 g2 $ gobgp neighbor 10.1.0.101 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.1.0.102 [{Origin: i} {Med: 0} {LocalPref: 100}] 10.3.0.0/16 10.1.0.102 65003 [{Origin: i} {LocalPref: 100} {Communities: 65001:1000}] root@r1> show route receive-protocol bgp 10.1.0.102 inet.0: 16 destinations, 18 routes (16 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 10.1.0.0/16 10.1.0.102 0 100 I * 10.3.0.0/16 10.1.0.102 100 65003 I
  • 70.
    INTERNET MULTIFEED CO.Copyright© 2016 Establish iBGP • Now you can reach from r4 loopback address to r3 loopback address that all RIBs/FIBs in between them are properly set up 70 root@r4> ping 10.3.0.103 source 10.4.0.104 count 3 PING 10.3.0.103 (10.3.0.103): 56 data bytes 64 bytes from 10.3.0.103: icmp_seq=0 ttl=62 time=10.496 ms 64 bytes from 10.3.0.103: icmp_seq=1 ttl=62 time=2.417 ms 64 bytes from 10.3.0.103: icmp_seq=2 ttl=62 time=2.586 ms --- 10.3.0.103 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.417/5.166/10.496/3.769 ms root@r3> ping 10.4.0.104 source 10.3.0.103 count 3 PING 10.4.0.104 (10.4.0.104): 56 data bytes 64 bytes from 10.4.0.104: icmp_seq=0 ttl=62 time=10.624 ms 64 bytes from 10.4.0.104: icmp_seq=1 ttl=62 time=1.948 ms : (snip)
  • 71.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Establish eBGP peer between g2 and rs, which is Route Server at IXP • Control advertisement policy • by adding specific communities • Here we add community value which will reject advertising our prefix only to r3 from route server 71
  • 72.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Nothing new here except md5 password authentication 72 neighbors: - config: neighbor-address: 10.173.176.211 peer-as: 64686 auth-password: pass65001 ### apply configuration change g2 $ systemctl reload gobgpd ### check establishment g2 $ gobgp n Peer AS Up/Down State |#Received Accepted 10.1.0.101 65001 03:16:11 Establ | 1 1 10.173.176.103 65003 03:16:46 Establ | 1 1 10.173.176.211 64686 00:05:32 Establ | 1 1
  • 73.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Receiving the same route for AS65003 from rs • It’s actually coming from rs that AS_PATH does not include 64686 which is the ASN of rs • No communities for IXP peers 65001:1000 added yet • We’ll do that later 73 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 03:21:07 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 03:20:52 [{Origin: i} {Communities: 65001:1000}] * 10.3.0.0/16 10.173.176.103 65003 00:09:38 [{Origin: i}] *> 10.4.0.0/16 10.1.14.104 65004 03:20:17 [{Origin: i} {LocalPref: 100}]
  • 74.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Receiving routes from rs at r3 • AS_PATH does not contain 64686 • next-hop is not the address of rs which is 10.173.176.211 74 root@r3> show route receive-protocol bgp 10.173.176.211 inet.0: 12 destinations, 16 routes (12 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 10.1.0.0/16 10.173.176.102 0 65001 I 10.4.0.0/16 10.173.176.102 65001 65004 I
  • 75.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Add the same community to tag routes `coming from IXP peers` as well as r3 • We’ve already have policy for IXP peers applied to r3 • What we need to do here is just add the neighbor address of the rs to the neighbor-set 75 defined-sets: neighbor-sets: - neighbor-set-name: ixp-neighbors neighbor-info-list: - 10.173.176.103 - 10.173.176.211 ### apply configuration g2 $ systemctl reload gobgpd
  • 76.
    INTERNET MULTIFEED CO.Copyright© 2016 Add peer to Route Server • Routes received from rs are also tagged with community 65001:1000 76 ### apply configuration change g2 $ systemctl reload gobgpd g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 03:51:52 [{Origin: i} {Med: 0}] *> 10.3.0.0/16 10.173.176.103 65003 03:51:37 [{Origin: i} {Communities: 65001:1000}] * 10.3.0.0/16 10.173.176.103 65003 00:40:23 [{Origin: i} {Communities: 65001:1000}] *> 10.4.0.0/16 10.1.14.104 65004 03:51:02 [{Origin: i} {LocalPref: 100}]
  • 77.
    INTERNET MULTIFEED CO.Copyright© 2016 Control Route Server policy with communities • IXP in this tutorial provides following policy at Route Server to control advertisement to other peers • ex.) If you like to advertise to all peers EXCEPT AS65003 • Add “0:65003” • ex.) If you like to advertise ONLY TO AS65003 and AS65123 • Add “0:64686 64686:65003 64686:65123” 77 ASN meaning 0:N Do not advertise to N 64686:N Advertise to N 0:64686 Do not advertise to ANY peer
  • 78.
    INTERNET MULTIFEED CO.Copyright© 2016 Control Route Server policy with communities 78 defined-sets: neighbor-sets: - neighbor-set-name: ixp-rs neighbor-info-list: - 10.173.176.211 policy-definitions: - name: rs-no-export-to-as65003 statements: - conditions: match-neighbor-set: neighbor-set: ixp-rs actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "0:65003"
  • 79.
    INTERNET MULTIFEED CO.Copyright© 2016 Control Route Server policy with communities • Remember that you need to call `softresetout` manually when you change export policy 79 global: config: as: 65001 router-id: 10.1.0.102 apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors default-import-policy: accept-route export-policy-list: - set-next-hop-self - rs-no-export-to-as65003 default-export-policy: accept-route g2 $ systemctl reload gobgpd g2 $ systemctl neighbor 10.173.176.211 softresetout
  • 80.
    INTERNET MULTIFEED CO.Copyright© 2016 Control Route Server policy with communities • We can confirm that r3 does not receive any routes from rs because of the community based policy control has taken place 80 g2 $ gobgp neighbor 10.173.176.211 adj-out Network Next Hop AS_PATH Attrs 10.1.0.0/16 10.173.176.102 65001 [{Origin: i} {Med: 0} {Communities: 0:65003}] 10.3.0.0/16 10.173.176.102 65001 65003 [{Origin: i} {Communities: 65001:1000, 0:65003}] 10.4.0.0/16 10.173.176.102 65001 65004 [{Origin: i} {Communities: 0:65003}] root@r3> show route receive-protocol bgp 10.173.176.211 inet.0: 12 destinations, 14 routes (12 active, 0 holddown, 0 hidden)
  • 81.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI • Configure route validation with RPKI • Apply policy based on validation result • tag with communities • We will use following value 81 RPKI validation result community to add Valid 65001:2001 Invalid 65001:2002 Not found 65001:2003
  • 82.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI • What we need is only the address of ROA cache server • Here we use open ROA cache server operated by INTERNET MULTIFEED CO. • see http://www.mfeed.ad.jp/rpki/en/ • This change requires restart of gobgpd 82 rpki-servers: - config: address: 210.173.170.254 port: 323 g2 $ systemctl restart gobgpd
  • 83.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI • “N” indicates record not found in ROA table • Of course it’s a private IP! • Local originating route will not be validated, only received routes are 83 g2 $ gobgp rpki server Session State Uptime #IPv4/IPv6 records 210.173.170.254:323 Up 00:05:38 24977/3522 g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 00:05:48 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 00:05:38 [{Origin: i} {Communities: 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 00:05:35 [{Origin: i} {Communities: 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 00:05:30 [{Origin: i} {LocalPref: 100}]
  • 84.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI • You can use `rpki—validation-result` to match against RPKI validation status of the route 84 policy-definitions: - name: tag-rpki-validation statements: - conditions: bgp-conditions: rpki-validation-result: valid actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2001“ # ...
  • 85.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI 85 # cont‘d - conditions: bgp-conditions: rpki-validation-result: invalid actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2002" - conditions: bgp-conditions: rpki-validation-result: not-found actions: bgp-actions: set-community: options: add set-community-method: communities-list: - "65001:2003"
  • 86.
    INTERNET MULTIFEED CO.Copyright© 2016 RPKI 86 global: apply-policy: config: import-policy-list: - zebra-import-my-prefixes - tag-ixp-neighbors - tag-rpki-validation g2 $ systemctl restart gobgpd g2 $ gobgp global rib Network Next Hop AS_PATH Age Attrs *> 10.1.0.0/16 0.0.0.0 02:20:52 [{Origin: i} {Med: 0}] N*> 10.3.0.0/16 10.173.176.103 65003 02:20:42 [{Origin: i} {Communities: 65001:2003, 65001:1000}] N* 10.3.0.0/16 10.173.176.103 65003 02:20:39 [{Origin: i} {Communities: 65001:2003, 65001:1000}] N*> 10.4.0.0/16 10.1.14.104 65004 02:20:34 [{Origin: i} {LocalPref: 100} {Communities: 65001:2003}]
  • 87.
    INTERNET MULTIFEED CO.Copyright© 2016 MRT • Configure MRT dump to monitor • RIB periodically (TABLE_DUMPv2) • incoming UPDATE messages (BGP4MP) 87
  • 88.
    INTERNET MULTIFEED CO.Copyright© 2016 MRT • Dump of update messages can be rotated automatically by specifying interval and file name format • 2006-01-02_1504 = YYYY-MM-DD_HHMM • Seems weird but this is how to specify dates format • Table dump currently does not support rotating • content will be replaced every `dump-interval` second 88 mrt-dump: - config: dump-type: updates file-name: /tmp/updates.2006-01-02_1504.mrt rotation-interval: 180 - config: dump-type: table file-name: /tmp/table.mrt dump-interval: 60 g2 $ systemctl restart gobgpd
  • 89.
    INTERNET MULTIFEED CO.Copyright© 2016 MRT • Dumps are created • Use your favorite tool to see them • `bgpdump` from bgptools • `bgpreader` from BGPStream • etc… 89 g2 $ ls -al /tmp/*.mrt -rw-r--r-- 1 quagga quagga 1768 Dec 3 11:36 /tmp/table.mrt -rw-r--r-- 1 quagga quagga 241 Dec 3 11:28 /tmp/updates.2016-12- 03_1128.mrt -rw-r--r-- 1 quagga quagga 0 Dec 3 11:31 /tmp/updates.2016-12- 03_1131.mrt -rw-r--r-- 1 quagga quagga 0 Dec 3 11:34 /tmp/updates.2016-12- 03_1134.mrt
  • 90.
    INTERNET MULTIFEED CO.Copyright© 2016 BMP • BMP is a protocol to monitor BGP • Abbreviation of BGP Monitoring Protocol • RFC7854 • GoBGP can export BMP messages • Several implementation for BMP server can be found • GoBGP itself can be simple BMP server • Here we use Ryu BMP Server • http://osrg.github.io/bmp/ 90
  • 91.
    INTERNET MULTIFEED CO.Copyright© 2016 BMP • What we need is only the address of BMP Server • We will run Ryu BMP Server on util • Here again it requires the restart of gobgpd • Launch Ryu BMP Server within docker container 91 bmp-servers: - config: address: 10.254.0.250 port: 11019 g2 $ systemctl restart gobgpd util $ docker run -it -p 11019:11019 osrg/ryu /bin/bash util(container) # ryu run --verbose ./ryu/ryu/app/bmpstation loading app ./ryu/ryu/app/bmpstation instantiating app ./ryu/ryu/app/bmpstation of BMPStation BRICK bmpstation listening on 0.0.0.0:11019
  • 92.
    INTERNET MULTIFEED CO.Copyright© 2016 BMP • We can see BMP message received at util 92 g2 $ gobgp neigh 10.173.176.103 reset ### (cont’d util(container) console) 2016 Dec 03 13:43:29 | 10.254.0.102 | BMPInitiation(info=[],len=6,type=4,version=3) 2016 Dec 03 13:43:29 | 10.254.0.102 | BMPPeerUpNotification(is_post_policy=False,len=158,local_address='0.0.0.0',local_port=49071,pee r_address='0.0.0.0',peer_as=64686,peer_bgp_id='10.173.176.211',peer_distinguisher=0,peer_type=0 ,received_open_message=BGPOpen(bgp_identifier='10.173.176.211',hold_time=90,len=45,my_as=64686, opt_param=[BGPOptParamCapabilityRouteRefresh(cap_code=2,cap_length=0,length=2,type=2), BGPOptParamCapabilityMultiprotocol(afi=1,cap_code=1,cap_length=4,length=6,reserved=0,safi=1,typ e=2), BGPOptParamCapabilityFourOctetAsNumber(as_number=64686,cap_code=65,cap_length=4,length=6,type=2 )],opt_param_len=16,type=1,version=4),remote_port=179,sent_open_message=BGPOpen(bgp_identifier= '10.1.0.102',hold_time=90,len=45,my_as=65001,opt_param=[BGPOptParamCapabilityRouteRefresh(cap_c ode=2,cap_length=0,length=2,type=2), BGPOptParamCapabilityMultiprotocol(afi=1,cap_code=1,cap_length=4,length=6,reserved=0,safi=1,typ e=2), BGPOptParamCapabilityFourOctetAsNumber(as_number=65001,cap_code=65,cap_length=4,length=6,type=2 )],opt_param_len=16,type=1,version=4),timestamp=1480772609.0,type=3,version=3) : (snip)
  • 93.
    INTERNET MULTIFEED CO.Copyright© 2016 Store BGP event log into InfluxDB • InfluxDB is one of the time series DB • like RRD • GoBGP can export BGP related event logs directly into InfluxDB • This function is not documented yet 93
  • 94.
    INTERNET MULTIFEED CO.Copyright© 2016 Store BGP event log into InfluxDB • Create new database 94 ### prepare influxdb on util util $ docker run --name=influxdb -d -p 8086:8086 influxdb util $ docker run --rm --net=container:influxdb -it influxdb influx - host localhost Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. Connected to http://localhost:8086 version 1.1.0 InfluxDB shell version: 1.1.0 > create database gobgpd > show databases name: databases name ---- _internal gobgpd
  • 95.
    INTERNET MULTIFEED CO.Copyright© 2016 Store BGP event log into InfluxDB • What we need is only the address of InfluxDB • Here again it requires the restart of gobgpd • Note that you first need to launch InfluxDB because gobgpd will not boot if it fails to connect • There is a bug! • If you enable this, import policy against zebra will not work properly 95 collector: config: url: http://10.254.0.250:8086 db-name: gobgpd g2 $ systemctl restart gobgpd
  • 96.
    INTERNET MULTIFEED CO.Copyright© 2016 Store BGP event log into InfluxDB • You can see records inserted into InfluxDB 96 ### util(docker)/influx console (cont’d) > use gobgpd Using database gobgpd > select * from peer name: peer time PeerAS PeerAddress PeerID State ---- ------ ----------- ------ ----- 1480776860991000000 64686 10.173.176.211 10.173.176.211 Established 1480776861015000000 65001 10.1.0.101 10.1.0.101 Established 1480776867995000000 65003 10.173.176.103 10.3.0.103 Established > select * from updates : (snip)
  • 97.
    INTERNET MULTIFEED CO.Copyright© 2016 Wrap up • You can find further information in official docs • https://github.com/osrg/gobgp/tree/master/docs/sour ces 97