Booting the cluster involves starting your hosts, running the circuit on each of them, and making sure that all circuit daemons are connected.
For the tutorial we need two hosts ideally, but you can start any desired number 1, 2, 3, …
Begin by starting a new EC2 instance, using the image we created.
This and all subsequent instances will belong to the same virtual private network inside EC2. EC2 will, as default, give two IP addresses to each host — one private, one public.
Host instances on your network will be able to connect to all ports on other host instances, using their private IP address. Hosts outside EC2 can connect to a restricted set of ports on the public IP addresses.
When configuring the first host instance before launch, in addition to the requisite SSH port, make sure to leave open another TCP port (say 11022) if you would like to be able to connect to the circuit directly from your notebook. These configurations are accomplished in the “security group” section on EC2.
Once the host instance is running, connect into it using SSH.
Discover the private address of the EC2 host instance, and save it into a variable:
# ip_address=`curl http://169.254.169.254/latest/meta-data/local-ipv4`
Start the circuit server, instructing it to listen on the private IP address of this host, on port 11022. (The port choice is arbitrary.)
# circuit start -a ${ip_address}:11022 1> /var/circuit/address 2> /var/circuit/log &
When the server starts it will print its circuit address (a URL-type string that
looks like circuit://…) to standard output and we
save this into the host-local file /var/circuit/address for future use.
The server logs all commands and other events that happen to it to standard error.
Respectively, we redirect it to a host-local log file named /var/circuit/log.
This start-up procedure can be summarized in a shell script, which you can locate in the source repo at
$GOPATH/src/github.com/gocircuit/circuit/tutorial/ec2/start-first-server.sh
Note that we are starting the circuit server as a singleton server, without specifying an automatic method for discovering other servers. This is because Amazon EC2 does not support Multicast UDP, which is needed for automatic server-server discovery.
Launch the remaining hosts and their respective circuit servers in the manner described here. Naturally if you are building a production system, you would preconfigure the host image to start the circuit server automatically upon booting.
Suppose you launch a total of three hosts using this method. You now have three hosts with circuit server running on each. At this stage each circuit server is a singleton member in a network of one. Our next step will be to join all three of them in a single network of three circuit servers.
Obtain the circuit address of each of the running circuit servers. On any given host you can accomplish this by saying
# cat /var/circuit/address
You should expect to see a URL-type string that looks like this in return:
circuit://174.51.10.12:11022/12477/Q15d828c92f4c90ff
This URL uniquely identifies the circuit server process that created it, as well as describes how to connect to the server.
Next we are going to use the circuit client to log into one of the servers, say host1,
and instruct it to join the network of each of the other two servers.
Note that joining is a symmetric operation. If server A is instructed to join server B,
the result is that the networks that A and B are part of are merged into one. (And
if A and B were already part of the same network, no change occurs.)
Log into host1 and, for convenience, save the circuit address of the local
server into a shell variable H1:
host1# H1=$(cat /var/circuit/address)
Use the circuit client to connect into the local circuit server and list all servers that it sees in its own network:
host1# circuit ls -d $H1 /
You should see a single member — the circuit server running on host1 itself —
listed by its name in the circuit's virtual file system.
The output should look something like:
/Xfea8b5b798f2fc09
Place the circuit addresses of the other two hosts, whatever they might be in your case,
in the variables H2 and H3. Using the circuit client again,
instruct the local circuit server (the on host1) to join the networks of the other two:
host1# circuit join -d $H1 /Xfea8b5b798f2fc09 $H1 host1# circuit join -d $H1 /Xfea8b5b798f2fc09 $H2
Let us break down these command lines. The part -d $H1
tells the client how to connect into the local server (the circuit
address $H1 contains the host and port of the running server).
Then it instructs the server with virtual name /Xfea8b5b798f2fc09 —
which happens to be the local server we are connected into — to join
its network into that of $H1 and $H2 respectively.
As a result, all three servers will become part of the same network, and you can verify this by listing the members of the network again.
host1# circuit ls -d $H1 /
This time you should see three entries, along the lines of:
/X2987b5b023f2f988 /Xca2b345798112c09 /Xfea8b5b798f2fc09
If you were to log into any of the other hosts and use the circuit client from there, say from the second host:
host2# circuit ls -d $H2 /
You should see the exact same list of three members.
At this stage the circuit cluster is connected and ready to be used. You should not have to restart and rejoin circuit servers unless a host dies and/or you are adding a new one to the cluster.