A simple three-tier demo application, fully instrumented with Prometheus, Jaeger and Loki logging.
The "TNS" name comes from "The New Stack", where the original demo code was used for an article.
There are a set of tools you will need to download and install.
Note: You can run this app without kubernetes. See the instructions here.
This demo assumes you have Docker installed. Follow instructions here for more details.
Note: The k3d kubernetes distribution used for this demo runs as a single node cluster inside docker. Make sure
your docker daemon has at least 2.5 GB of total memory available for all pods in this deployment to be scheduled.
To run this demo, you need a Kubernetes cluster. While the demo should work against any
Kubernetes cluster, these docs will assume a locally running k3d cluster. Download and
install k3d from here.
kubectl is used to interact with Kubernetes clusters. Follow the instructions
here to install it.
Tanka uses the Jsonnet language to interact with Kubernetes, via the kubectl tool.
Download and install it from here.
Jsonnet bundler downloads Jsonnet dependencies. Download and install it from here.). Rename the downloaded binary to jb and move it to the location where $PATH points. Also make sure the binary is executable:
$ chmod +x /usr/local/bin/jb
If you wish to use a Kubernetes cluster other than a local k3d one, please adjust these
instructions accordingly.
- Clone TNS repository
$ git clone https://github.com/grafana/tns
$ cd tns- Install K3D Cluster
$ ./create-k3d-cluster
$ export KUBECONFIG=$(k3d kubeconfig write tns)If you see any error like,
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
then try to add your user to docker group to run docker without sudo.
$ sudo usermod -aG docker <your-user>you should logout and login again for the changes to take effect.
- Install TNS applications
This step will ask you to confirm
yesfour times.
$ ./install- Wait It will take some time to install the demo - there's a lot of downloading to do. It is not unreasonable for it to take over ten minutes for everything to download then start up.
This command will show you the status of the cluster:
$ kubectl get pods -AAll pods should be listed as either running or completed. If this is the case,
your cluster should be ready for use.
You should now be able to access the demo via http://localhost:8080/.
- Go to the TNS dashboard
- Zoom in on a section with failed requests if you are so inclined
- Panel Drop Down -> Explore
- Datasource Drop Down -> Loki
- Choose a log line with a traceID -> Tempo
- Go to Explore
- Choose Datasource prometheus-exemplars
- Run this query
histogram_quantile(.99, sum(rate(tns_request_duration_seconds_bucket{}[1m])) by (le)) - Click an exemplar
- Click the log icon on a span line
- Go to Explore
- Choose Datasource Loki
- Run this query
{job="tns/app", level="info"} | logfmt | status>=500 and status <=599 and duration > 50ms - Choose a log line with a traceID -> Tempo
This installation will have created a tanka directory in your TNS checkout. This
directory contains all of the Jsonnet resources used to install this demo.
You will now have a tanka directory within your checkout that contains all of the
Jsonnet resources that were needed to deploy this monitoring stack. To find out more
about Tanka, visit https://tanka.dev.
Should you wish to disable your cluster, use this command:
$ k3d cluster stop tnsTo re-enable it, do this:
$ k3d cluster start tnsOnce you have finished with the cluster, this should remove it and leave you ready to recreate it on another occasion:
$ k3d cluster delete tns
$ rm -rf tanka- Tempo 404's when trying to load traces
- This is likely because the jaeger agent is not running correctly. Check that all pods were successfully scheduled