Most DevOps beginners have problems starting out, the best time to start on DevOps is when you have acquired some experience in backend applications development (REST API using some framework such as flask or tornado will be great). If you have not been making such applications, maybe try out some basic python frameworks to begin with.
So! I assume you already know some basic REST API development. You want to learn or try out some DevOps stuff but don’t know where to start? Lets see if I can help. To me, the most important thing about learning DevOps is the environment. If you have a proper environment set up to learn, then you are halfway there!
In every project there is always a need balance the resources you need to achieve a specific goal versus how much knowledge you currently have. Which means this setup isn’t going to done easy. I plan to set up 3 ubuntu server instances in a single machine which I can bring up and down whenever I want. This, of course, can be achieved using VMware or Virtualbox, but to make it slightly more challenging and create new opportunities for me to learn new skills. I am going to do this using docker instead.
Prerequisites
- Basic Linux Machine (Mine has Ubuntu 18.04 installed)
- Docker installed
- Docker-compose installed
Creating new Docker networks
My first task, will be to create a bridged docker network, this is done so I can assign IP addresses to each ubuntu(docker) instance. In this example, I choose to create my network in the 10.10.x.x subnet.
$ docker network create --subnet 10.10.0.0/24 --gateway 10.10.0.254 mynetname
[OPTIONAL] If you screw up here, you can always list the docker networks and remove those that you don’t want.
$ docker network list
NETWORK ID NAME DRIVER SCOPE
6b52e672b288 bridge bridge local
a47adaebb9ea host host local
60b3086fa33e mynetname bridge local
3d4bf0836766 none null local
[OPTIONAL] In this case, I am going to remove “mynetname”.
$ docker network rm mynetname
mynetname
Creating SSH Ubuntu Docker
You first need to create an Ubuntu image that has SSH installed. Specific instructions can be found on this link.
https://docs.docker.com/engine/examples/running_ssh_service/
For the benefit of others who wants more specific instructions without having to click on the link above, I will walk through the process of creating the Docker image. The first thing you need to do is create a directory with a Dockerfile with your favorite text editor.
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:THEPASSWORDYOUCREATED' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# SSH login fix. Otherwise user is kicked off after login
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
Paste the code above and save the Dockerfile, remember to replace THEPASSWORDYOUCREATED with a password of your choice.
In the same directory where you placed your Dockerfile, run the following command to build the image.
$ docker build -t ubuntu_ssh .
You can now try to run the docker using the following command.
$ docker run -d -P --name test_server ubuntu_ssh
[OPTIONAL] To ensure that your Docker is running properly and you can ssh successfully, you can first retrieve your port number through a port test.
$ docker port test_server 22
0.0.0.0:32772
[OPTIONAL] Note the port numbers, in my case it is 32772. The next step is to retrieve the ip address of your Docker daemon.
$ ip address
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether b8:81:98:bd:2d:a1 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.158/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp2s0
valid_lft 76907sec preferred_lft 76907sec
inet6 fe80::fc44:8201:3675:45a3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:f6:12:8b:b8 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:f6ff:fe12:8bb8/64 scope link
valid_lft forever preferred_lft forever
[OPTIONAL] In my case, the Docker daemon has its ip address on 172.17.0.1. I can now ssh into the Ubuntu Docker instance with the ip address and port numbers.
$ ssh root@172.17.0.1 -p32772
[OPTIONAL] Put in your password and you are good to go!
root@172.17.0.1's password:
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-106-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@523dd9345851:~#
Spinning up the servers
Next, I am going to run 3 instances of ubuntu 16.04 in detached mode on the network I have created. The first server will be created with name “server1” and ip address “10.10.0.1”. The network shall be mynetname which we created earlier
$ docker run --rm -it -d --name server1 --expose 22 --net mynetname --ip 10.10.0.1 ubuntu_ssh:latest
You should now have your first ubuntu instance running, you can normally ssh into it using the following command.
$ ssh root@10.10.0.1
Now repeat the process and set up 2 more servers in detached mode, each given a different name and ip address.
$ docker run --rm -it -d --name server2 --expose 22 --net mynetname --ip 10.10.0.2 ubuntu_ssh:latest
$ docker run --rm -it -d --name server3 --expose 22 --net mynetname --ip 10.10.0.3 ubuntu_ssh:latest
You should now have 3 ubuntu instances running on Docker that you can play with, congratulations and give yourself a small pat on the back! feel the adrenaline rush! Now lets clean up and put our toys back in the box. lets start by inspecting all the containers.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4219ab627bb0 ubuntu_ssh:latest "/usr/sbin/sshd -D" 38 hours ago Up 38 hours 22/tcp server3
adae0537c745 ubuntu_ssh:latest "/usr/sbin/sshd -D" 38 hours ago Up 38 hours 22/tcp server2
aa0073e1b137 ubuntu_ssh:latest "/usr/sbin/sshd -D" 38 hours ago Up 38 hours 22/tcp server1
We can now bring down each container individually.
$ docker stop server1
server1
$ docker stop server2
server2
$ docker stop server3
server3
Now to check if we have successfully freed up all your resources for the next step.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Cleanup complete! I hope you now have a much better understanding of setting up a low-cost, low resource container environment. But we are not done yet, remember the goal is to be able to spin up all servers and bring them down easily. We are going to make use of a docker-compose script to do this.
Making a docker-compose script
To keep this post simple, I am just going to give you the configuration script. In my future posts, I will explain in detail how the docker-compose works.
version: '2'
services:
server1:
image: ubuntu_ssh:latest
networks:
devops_env:
ipv4_address: 172.10.0.1
server2:
image: ubuntu_ssh:latest
networks:
devops_env:
ipv4_address: 172.10.0.2
server3:
image: ubuntu_ssh:latest
networks:
devops_env:
ipv4_address: 172.10.0.3
networks:
devops_env:
driver: bridge
ipam:
config:
- subnet: 172.10.0.0/16
gateway: 172.10.0.254
Once you have saved your docker-compose file, run the following command to bring it up in detached mode in the same Directory.
$ docker-compose up -d
Creating network "dcmpdevopsenv_devops_env" with driver "bridge"
Creating dcmpdevopsenv_server2_1 ...
Creating dcmpdevopsenv_server3_1 ...
Creating dcmpdevopsenv_server1_1 ...
Creating dcmpdevopsenv_server2_1
Creating dcmpdevopsenv_server3_1
Creating dcmpdevopsenv_server1_1 ... done
I have created a new subnet for this docker-compose file so it runs in parallel with what you have already created. You can now ssh into any of the servers with the ip address you specified in the docker-compose file, in my case, the following command will suffice. Enter your password and you are good to go!
$ ssh root@172.10.0.1
The authenticity of host '172.10.0.1 (172.10.0.1)' can't be established.
ECDSA key fingerprint is SHA256:Br0w5T0GWmJyRdjNHUx2JR6nVQ0L3ln1gP4xWT7ao+0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.10.0.1' (ECDSA) to the list of known hosts.
root@172.10.0.1's password:
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-106-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@60a45a4f5105:~#
If at any point of you you need to bring it down again just exit the ssh and use the following command.
$ docker-compose down
Stopping dcmpdevopsenv_server1_1 ... done
Stopping dcmpdevopsenv_server3_1 ... done
Stopping dcmpdevopsenv_server2_1 ... done
Removing dcmpdevopsenv_server1_1 ... done
Removing dcmpdevopsenv_server3_1 ... done
Removing dcmpdevopsenv_server2_1 ... done
Removing network dcmpdevopsenv_devops_env
The docker-compose file makes it really easy whenever you need to spin up a couple of servers to try something new! So do give it a try and feel free to let me know if this post can be improved. The docker-compose and Dockerfiles can be accessed via my github link: https://github.com/snowfoxwoo/dcmp-devops-env
Stay calm and keep coding! – Derrick Woo
