You will need to install
brew install dep golang protobuf shellcheck
All of the actual code in Network Service Mesh builds as pure go:
go generate ./... go build ./...
But to really do interesting things in NSM, you will want to build various Docker containers, and deploy them to K8s. All of this is doable via normal Docker/K8s commands, but to speed development, some make machinery has been added to make things easy.
You can build all of the containers needed for NSM, including a bunch of handle Network Service Endpoints (NSEs) and NSCs (Network Service Clients) that are useful for testing, but not part of the core with:
If you are using the vagrant machinery to run your K8s cluster (described a bit further down), you really want to use:
make k8s-save will build your containers and save them in scripts/vagrant/images where they can be loaded by the vagrant K8s cluster.
You can also selectively rebuild any component, say the nsmd, with:
Network Service Mesh provides a handy vagrant setup for running a two node K8s cluster. Once you’ve done
make k8s-save, you can deploy to it with:
By default this will: 1. Spin up a two node K8s cluster from scripts/vagrant if one is not already running. 2. Delete old instances of NSM config if present 3. Load all images from scripts/vagrant/images into the master and worker node 2. Deploy the nsmd and vppagent-dataplane Daemonsets 3. Deploy a variety of Network Service Endpoints and Network Service Clients 4. Deploy the crossconnect-monitor (a useful tool for debugging)
You can check to see things working by typing:
which will try pinging from NSCs to NSEs.
You can remove the effects of k8s-deploy with:
As in the case with save and build, you can always do this for a particular component, like
make k8s-nsc-deploy or
The described quick start method works for fast deployments and quick tests. However, the build infrastructure provides a fine-grained control over the deployments.
To spin the default 2 node vagrant setup with Kubernetes on top type:
At any point, you can
make vagrant-suspend and
make vagrant-resume to pause and restore the spawn virtual nodes. If for some you need to rebuild or completely destroy the vagrant environment, use
make vagrant-restart and
To point your
kubectl to the Kubernetes deployment in the virtual nodes, use:
Network Service Mesh consists of a number of system pods, which take care of service registration, provide the dataplane functionality, do monitoring and observability. Once you have configured your
kubectl to the desired Kubernets master (may or may not be set through vagrant), you can initiate the NSM infrastructure deployment and deletion using
make k8s-infra-deploy and
The project comes with a simple, ready to test ICMP example. It deploys a number of ICMP responder NSEs and connects NSCs to them. This shows same and cross-node communication and is good for visualising it with the provided monitoring tools.
The commands to deploy and delete it are
make k8s-icmp-deploy and
make k8s-icmp-delete. Checking the operability of the ICMP example is done through
One of the big advantages on Network Service Mesh is NS composition, i.e. forming a complex service out of a number of simple NSEs. The project comes with an example that implements the “secure-intranet-connectivity” Network Service which connects together a simple ACL based packet filtering firewall and a simulated VPN gateway NSEs. Deploying it is done through
make k8s-vpn-deploy and to uninstall it run
make k8s-vpn-delete. Checking VPN’s operability is done with
In the course of developing NSM, you will often find yourself wanting to look at logs for various nsm components.
will dump all the logs for all running nsmd Pods in the cluster (you are going to want to redirect these to a file). This works for any component in the system.
Of particular utility:
dumps the logs from the crossconnect-monitor, which has been logging new crossconnects as they come into existence and go away throughout the cluster.
If you change types.go or any of the .proto files you will need to be able to run
go generate ./... to regenerate the code.
In order to be able to do that you need to have installed:
go install ./vendor/github.com/golang/protobuf/protoc-gen-go/
go install ./vendor/k8s.io/code-generator/cmd/deepcopy-gen/
Then just run:
go generate ./...
If you need to add new dependencies to the vendor/ directory.
1. Install dep
As part of our CI, we run shellcheck on all shell scripts in the repo. If you want to run it locally, you need to:
The .circleci/config.yml file is the canonical source of how to build Network Service Mesh in case this file becomes out of date.