At this point, we have an idea how a couple of things work, you might be feeling the pressure that comes with the diverse skillsets you need in becoming a DevOps engineer. I cannot stress enough that skills in coding are important in DevOps so I am putting together one guide focused on it. If you are currently new to coding, do take your time to perhaps learn more about python. If you already have a preferred programming language, try to adapt this guide or drop me a comment and I shall see if I can help.
“It does not matter how slowly you go as long as you do not stop. ”
– Confucius
Prerequisites
- python with pip installed
Starting out
In this part of the guide I will walk through the development of a simple REST application in python. The application will communicate with a redis docker image.
Setting up a simple database
The inclusion of the database simplifies the design for all of our web services to persist and share common data among each server instance. The database of choice here will be a redis docker image. The first thing to do is to get redis-tools installed on the development machine for testing and verification.
$ sudo apt-get install redis-tools
A simple command will download the image without having to install a redis database instance on the machine. Note that this way of using redis has its strengths and limitations in an architecture perspective.
$ docker pull redis:latest latest: Pulling from library/redis 8559a31e96f4: Pull complete 85a6a5c53ff0: Pull complete b69876b7abed: Pull complete a72d84b9df6a: Pull complete 5ce7b314b19c: Pull complete 04c4bfb0b023: Pull complete Digest: sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5 Status: Downloaded newer image for redis:latest docker.io/library/redis:latest
You can run the following command to check that the latest redis image has been obtained.
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE redis latest 235592615444 3 weeks ago 104MB
You can now run an instance of the database, to simplify the tutorial, I will name the instance as “db”, publish ports 6379 (redis default ports) and in detached mode.
$ docker run --name "db" -e ALLOW_EMPTY_PASSWORD=yes -p 6379:6379 -d redis:latest 8387216a90a24d2bf11580930a5f545927cce3a12bd69a0e579ec1ad22554801
You can now use redis on the command line. Do note that any data on the redis instance will not be persistent if the instance is shut down. You can now connect to redis simply by typing redis-cli.
$ redis-cli 127.0.0.1:6379>
For the benefit of the learners who possibly have an existing redis instance on their machines, the ip and the port to the redis instance can be specified in your command line.
$ redis-cli -h localhost -p 6379 localhost:6379>
If you don’t already know this and have learnt something new, perhaps take some time to play with your new virtual toys! How about initialising a key-value called “counter” and assigning it a value of 0. Fetch the value of counter to verify it is indeed what you set it up to be.
localhost:6379> set counter 0 OK localhost:6379> get counter "0" localhost:6379>
There you have it, a working copy of redis in your machine with which you can persist test data.
Setting up the code environment
Now we get to the exciting part, getting started on building a simple python backend application on your preferred IDE (for me its Visual Studio Code).
Before we start building on python, it is always good practice to prepare a virtual development environment so that packages downloaded for the application can be isolated. This helps to keep your python on the machine clean. My favorite guide is https://docs.python-guide.org/dev/virtualenvs/ but I shall walk you through the process.
We begin by ensuring we have python and pip installed.
$ python --version Python 2.7.17 $ python3 --version Python 3.6.9
So I have python 2.7 and python 3.6 installed by default, next step is to check for pip.
$ pip --version pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7) $ pip3 --version pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
As we can see, we have two versions of pip as well, one for python 2.7 and another for python 3.6.
The following command will install virtualenv on your machine. Just to make sure, test your installation by checking your virtualenv version.
$ pip install virtualenv $ virtualenv --version virtualenv 20.0.21 from /home/work/.local/lib/python2.7/site-packages/virtualenv/__init__.pyc
Different people like to organise their environments differently, my personal preference is to create an environment specifically for the python version I am currently using to code.
$ virtualenv -p /usr/bin/python3.6 venv36 created virtual environment CPython3.6.9.final.0-64 in 2115ms creator CPython3Posix(dest=/home/work/Documents/workspace/venv36, clear=False, global=False) seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/work/.local/share/virtualenv/seed-app-data/v1.0.1) activators PythonActivator,FishActivator,XonshActivator,CShellActivator,PowerShellActivator,BashActivator
You should now be able to activate the virtual environment. your environment name in brackets should appear on your terminal.
$ source venv36/bin/activate (venv36) $
[OPTIONAL] You can deactivate the virtual environment simply by running the command deactivate, the environment name in brackets will cease to exist.
$ deactivate $
Writing the code
Now I am going to build a simple backend REST application using python tornado. Before we do that, a few python packages needs to be installed in the environment we created earlier, create a file in your project folder and name it requirements.txt. The content of the text file shall be the following (version numbers are at the time of writing this entry).
tornado==6.0.4 sockets==1.0.0 redis==3.5.3
The following command will install the required packages into your development environment.
$ pip install -r requirements.txt
Collecting tornado
Downloading tornado-6.0.4.tar.gz (496 kB)
|████████████████████████████████| 496 kB 2.3 MB/s
Collecting sockets
Downloading sockets-1.0.0-py3-none-any.whl (4.5 kB)
Collecting redis
Using cached redis-3.5.3-py2.py3-none-any.whl (72 kB)
Building wheels for collected packages: tornado
Building wheel for tornado (setup.py) ... done
Created wheel for tornado: filename=tornado-6.0.4-cp36-cp36m-linux_x86_64.whl size=427645 sha256=dc57464c4dc13181fbd9d4d60787e50018e86efd80ce4d08b42dfe85da574b9b
Stored in directory: /home/derrick/.cache/pip/wheels/37/a7/db/2d592e44029ef817f3ef63ea991db34191cebaef087a96f505
Successfully built tornado
Installing collected packages: tornado, sockets, redis
Successfully installed redis-3.5.3 sockets-1.0.0 tornado-6.0.4
You should now have all the required packages to run the script. Since the guide is all about learning DevOps and not python programming, I shall give away the python code.
To make the web application a little more interactive rather than just returning hello world, a simple visit counter and ip address check is included to make our lives easier later.
import tornado.ioloop
import tornado.web
import redis
import socket
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.database = redis.Redis(host='localhost', port=6379, db=0)
if(self.database.get('counter') == None):
self.database.set("counter", 0)
visitCount = int(self.database.get('counter'))
visitCount = visitCount + 1
hostName = socket.gethostname()
ipAddr = socket.gethostbyname(hostName)
self.database.set("counter", visitCount)
self.write("Hello! This page is visited %d time(s)<br>current recorded IP is %s" % (visitCount, ipAddr))
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
app = make_app()
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
Start up the python script by running the simple command
(venv36) $ python app.py
Open a web browser and navigate to http://localhost:8888, refreshing the page will increment the visit counter stored in redis.
Conclusion
Now that you have created a simple application, we now have a base application that we can modify or use for deployment in our DevOps environment. Hooray! all of the code for the guide can be found in my github repository through the link. https://github.com/snowfoxwoo/blog_gldp03.git