1.3. How To

1.3.1. Create and Run an Experiment

Experiments are defined in a declarative fashion inside Python modules using classes. Basically, create a .py file and add a global variable experiments, a list which contains multiple instances of the class Experiment, each one describing a standalone experiment. This is very helpful if you wish to evaluate your work in different environments, for example, you may want to swap out some simulator or investigate multiple topologies with different scale.

The class Experiment provides methods to add the simulators you wish to run. All available simulators can be found in the module orchestration/simulators.py. Instantiating HostSim requires you to specify a NodeConfig, which contains the configuration for your host, for example, its networking settings, how much system memory it should have, and most importantly, which applications to run by assigning an AppConfig. You can find predefined classes for node and app configs in the module orchestration/nodeconfig.py. Feel free to add new ones or just create a new class locally in your experiment’s module. For more details, see SimBricks Orchestration.

The last step to complete your virtual testbed is to specify which virtual components connect to each other. You do this by invoking the respective methods on the simulators you have instantiated. See the different simulator types’ base classes in the module orchestration/simulators.py for more information. A simple and complete experiment module in which a client host pings a server can be found below.

If you plan to simulate a topology with multiple hosts, it may be helpful to take a look at the module orchestration/simulator_utils.py in which we provide some helper functions to reduce the amount of code you have to write.

Finally, to run your experiment, invoke experiments/run.py and provide the path to your experiment module. In our docker containers, you can also just use the following command from anywhere:

$ simbricks-run --verbose --force <path_to_your_module.py>

--verbose prints all simulators’ output to the terminal and --force forces execution even if there already exist result files for the experiment. If simbricks-run is not available, you can always do

$ cd experiments
$ python run.py --verbose --force <path_to_your_module.py>

While running, you can interrupt the experiment using Ctrl+C in your terminal. This will cleanly stop all simulators and collect their output in a JSON file in the directory experiments/out/<experiment_name>. These are the necessary basics to create and run your first experiment. Have fun.

Listing 1.1 A simple experiment with a client host pinging a server, both are connected through a network switch. The setup of the two hosts could be simplified by using simbricks.orchestration.simulator_utils.create_basic_hosts().

from simbricks.orchestration.experiments import Experiment
from simbricks.orchestration.nodeconfig import (
    I40eLinuxNode, IdleHost, PingClient
)
from simbricks.orchestration.simulators import Gem5Host, I40eNIC, SwitchNet

e = Experiment(name='simple_ping')
e.checkpoint = True  # use checkpoint and restore to speed up simulation

# create client
client_config = I40eLinuxNode()  # boot Linux with i40e NIC driver
client_config.ip = '10.0.0.1'
client_config.app = PingClient(server_ip='10.0.0.2')
client = Gem5Host(client_config)
client.name = 'client'
client.wait = True  # wait for client simulator to finish execution
e.add_host(client)

# attach client's NIC
client_nic = I40eNIC()
e.add_nic(client_nic)
client.add_nic(client_nic)

# create server
server_config = I40eLinuxNode()  # boot Linux with i40e NIC driver
server_config.ip = '10.0.0.2'
server_config.app = IdleHost()
server = Gem5Host(server_config)
server.name = 'server'
e.add_host(server)

# attach server's NIC
server_nic = I40eNIC()
e.add_nic(server_nic)
server.add_nic(server_nic)

# connect NICs over network
network = SwitchNet()
e.add_network(network)
client_nic.set_network(network)
server_nic.set_network(network)

# set more interesting link latencies than default
eth_latency = 500 * 10**3  # 500 us
network.eth_latency = eth_latency
client_nic.eth_latency = eth_latency
server_nic.eth_latency = eth_latency

experiments = [e]

1.3.2. Add a Node or Application Config

A host’s configuration and the workload to run are defined via Node and App Config. SimBricks already comes with a few examples in the module orchestration/nodeconfig.py. If they don’t fit your use-case, you need to add your own by overriding the pre-defined member functions of NodeConfig and AppConfig. The most important one is run_cmds(), which defines the commands to execute for your workload/application. Further information can be found in the module orchestration/nodeconfig.py directly.

1.3.3. Add a Custom Image

First off, make sure you actually need a custom image. You can find more information on this under Images. We have a tutorial in our examples repository that highlights how to add a custom, out-of-tree image for a simple Memcached experiment. Further, we are currently reworking our orchestration framework to greatly simplify the process of defining your own custom image, abstracting away the need to manually build and manage it.

For the moment, here is some additional information on the inner details of the image building process: We use Packer, which essentially starts a QEMU VM with internet access, boots up the kernel and then runs an installation script inside to change configs and install required packages either through apt or from source. You can find examples of installation scripts for our in-tree images under images/scripts. The commands for building them reside in the file images/rules.mk.

The produced images are stored as images/output-<image_name>/<image-name>. By default, we produce disk images in the compressed qcow2 format. Some of our host simulators, e.g. gem5 and Simics, require raw disk images though, which are substantially larger. The qcow2 can be easily converted to raw with qemu-img convert. For our in-tree images there’s a Makefile target for this, which stores the converted images as images/output-<image_name>/<image-name>.raw.

$ make convert-images-raw

1.3.4. Integrate a New Simulator

The first step is to implement a SimBricks adapter in the simulator you want to integrate. This adapter on one side uses the simulator’s extension API to act as a native device and on the other side sends and receives SimBricks messages. You can find more information on adapters in our Architectural Overview.

To make running experiments and setting up the SimBricks communication channels to other simulators convenient, add a class for the simulator in orchestration/simulators.py` that inherits either from Simulator or one of the more specialized base classes in. In this class, you define the command(s) to execute the simulator together with further parameters, for example, to connect to the communication channels with other simulators. Below is an example of what this looks like.

Listing 1.2 Orchestration framework class for WireNet, a simple Ethernet wire that attaches to the SimBricks Ethernet adapters of two simulators.

class WireNet(NetSim):

  def run_cmd(self, env):
      connects = self.connect_sockets(env)
      assert len(connects) == 2
      cmd = (
          f'{env.repodir}/sims/net/wire/net_wire {connects[0][1]}'
          f' {connects[1][1]} {self.sync_mode} {self.sync_period}'
          f' {self.eth_latency}'
      )
      if env.pcap_file:
          cmd += ' ' + env.pcap_file

      return cmd