… been in Google land and found an Atom!

Hey all,

so… the past weeks I have been in Google land, specifically: working on Google Compute Engine (GCE).  As part of the containerization of IT I started using an Atomic Host on GCE. This was basically and exercise to learn some of the GCE tools. But the important part: it gives a reusable Atomic Host with endless compute power. I put together a basic example of a web server running as a container, orchestrated by kubernetes

In the upcoming weeks, I will use this type of set up to expolore

  1. multi instance networking with kubernetes and
  2. troubleshooting containers, aka. sosreport and docker


Posted at 11° 47' on 2014-10-21 | Filed Under: docker, kubernetes, Red Hat | read on

on debugging and gathering information about containers

After having set up the basic environment for kubernetes and docker, I will explore into options on getting some information (eg logfiles, configurations) of containers. The main focus will be debugging of container configuration and/or containerized applications, so I am not primarily interested in the application’s logfile. And I assume that /dev/log is bound to the host via docker -v /dev/log:/dev/log.

First of all, let me give a short overview of some of the options available for “logging with docker”. There has also been a long discussion on the docker-dev mailing list

For the purpose of debugging a container’s configuration or the abnormal termination of an application I will try to gather some information:

  • from outside the container, like network configuration, disks, cgroups, other processes, other logs
  • from inside the container, docker’s own logfiles of the container
  • and from inside the container, like network configuration, cgroups, …

Gathering different kinds of information for an operating system environment has been solved before, and I will choose sosreport as the base for my work. sosreport is included in every major Linux distribution, including Debian GNU/Linux, Fedora and Red Hat Enterprise Linux.

From outside a containers sosreport seems to do a pretty complete job, looking at the ps and netstat output we can reckon that docker daemon is running, ip_addr tells us that the docker0 interface has been set up correctly… so, we are set to go. sosreport also contains all the syslog (or systemd-journald nowadays) entries so we could inspect potential problems on dockerd startup.

Having a look at more docker specific information is harder. sos plugins for docker specific informations are targeted for milestone 3.2 or are available via github. However these plugins just gather some basic information. I have started opening requests for enhancements with the sos upstream project to get some more detailed docker information.

Taking a peek inside the container is not covered by sosreport by now. In the upcoming days I will explore some options using nsenter and execute sosreport within the container and will followup on this article too.

stay tuned…


Posted at 16° 35' on 2014-08-13 | Filed Under: docker, technology | read on