Continuous Delivery Using Docker And Ansible

Continuous Delivery
Release Often
Release Faster
Great Reliable

Continuous Delivery workflow/pipeline

With which we can Test,Build,Release,& Deploy a simple application.

Application will be of any technology ex:python based.

The work flow will be Based upon using docker & docker-compose which is a emerging technology. With a goal to release our application as docker image.

So that it can be tested, building application artefacts, creating docker-release images, and running acceptance test, by external functionality, and finally publish docker image.

First we can run it locally, & then set in popular jenkins continuous delivery system, we will configure integration with GitHub, allowing continuous delivery work flow to be triggered on each application source code commit.

Assuming application passes all unit,integration & acceptance tests. Our pipeline will release images to docker hub, which will deployed to Aws with ansible using IAAC(S) approach with aws cloud formation and leveraging ec2 container service for running docker container is production.

rmohan@root:~$ mkdir docker-ansible

rmohan@root:~$ cd docker-ansible/

rmohan@root:~/docker-ansible$ django-admin startproject todobackend

rmohan@root:~/docker-ansible$ tree
.
`– todobackend
|– manage.py
`– todobackend
|– __init__.py
|– settings.py
|– urls.py
`– wsgi.py

2 directories, 5 files

rmohan@root:~/docker-ansible$ cd todobackend/

rmohan@root:~/docker-ansible/todobackend$ mkdir src

rmohan@root:~/docker-ansible/todobackend$ mv manage.py src/

rmohan@root:~/docker-ansible/todobackend$ mv todobackend/ src

rmohan@root:~/docker-ansible/todobackend$ tree
.
`– src
|– manage.py
`– todobackend
|– __init__.py
|– settings.py
|– urls.py
`– wsgi.py

2 directories, 5 files

rmohan@root:~/docker-ansible/todobackend$ git init
Initialized empty Git repository in /home/rmohan/docker-ansible/todobackend/.git/

rmohan@root:~/docker-ansible/todobackend$ vim .gitignore
#Ignore the virtual environment
venv

#Ignore compiled python source files
*.pyc

#Ignore SQLite database files
*.sqlite3

rmohan@root:~/docker-ansible/todobackend$ git add -A
rmohan@root:~/docker-ansible/todobackend$ git commit -a -m “Initial commit”
[master (root-commit) edface9] Initial commit
6 files changed, 177 insertions(+)
create mode 100644 .gitignore
create mode 100755 src/manage.py
create mode 100644 src/todobackend/__init__.py
create mode 100644 src/todobackend/settings.py
create mode 100644 src/todobackend/urls.py
create mode 100644 src/todobackend/wsgi.py

rmohan@root:~/docker-ansible/todobackend$ pip install virtualenv
Requirement already satisfied (use –upgrade to upgrade): virtualenv in /usr/local/lib/python2.7/dist-packages
Cleaning up..

rmohan@root:~/docker-ansible/todobackend$ virtualenv venv
New python executable in /home/rmohan/docker-ansible/todobackend/venv/bin/python
Installing setuptools, pip, wheel…done.

rmohan@root:~/docker-ansible/todobackend$ ls
src venv

rmohan@root:~/docker-ansible/todobackend$ ls venv/
bin include lib local pip-selfcheck.json

rmohan@root:~/docker-ansible/todobackend$ source venv/bin/activate
(venv) rmohan@root:~/docker-ansible/todobackend$

(venv) rmohan@root:~/docker-ansible/todobackend$ pwd
/home/rmohan/docker-ansible/todobackend

(venv) rmohan@root:~/docker-ansible/todobackend$ pip install pip –upgrade
Requirement already up-to-date: pip in ./venv/lib/python2.7/site-packages

(venv) rmohan@root:~/docker-ansible/todobackend$ pip install django
Collecting django
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading Django-1.11.5-py2.py3-none-any.whl (6.9MB)
100% |????????????????????????????????| 7.0MB 139kB/s
Collecting pytz (from django)
Downloading pytz-2017.2-py2.py3-none-any.whl (484kB)
100% |????????????????????????????????| 491kB 1.2MB/s
Installing collected packages: pytz, django
Successfully installed django-1.11.5 pytz-2017.2

(venv) rmohan@root:~/docker-ansible/todobackend$ pip install djangorestframework
Collecting djangorestframework
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading djangorestframework-3.6.4-py2.py3-none-any.whl (1.5MB)
100% |????????????????????????????????| 1.5MB 453kB/s
Installing collected packages: djangorestframework
Successfully installed djangorestframework-3.6.4

(venv) rmohan@root:~/docker-ansible/todobackend$ pip install django-cors-headers
Collecting django-cors-headers
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading django_cors_headers-2.1.0-py2.py3-none-any.whl
Installing collected packages: django-cors-headers
Successfully installed django-cors-headers-2.1.0

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py startapp todo
(venv) rmohan@root:~/docker-ansible/todobackend/src$ tree
.
|– manage.py
|– todo
| |– admin.py
| |– apps.py
| |– __init__.py
| |– migrations
| | `– __init__.py
| |– models.py
| |– tests.py
| `– views.py
`– todobackend
|– __init__.py
|– __init__.pyc
|– settings.py
|– settings.pyc
|– urls.py
`– wsgi.py

3 directories, 14 files

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todobackend/settings.py
INSTALLED_APPS = [
.
.

‘rest_framework’,
‘corsheaders’,
‘todo’
]
.
.
MIDDLEWARE_CLASSES = [
‘django.middleware.security.SecurityMiddleware’,
‘corsheaders.middleware.CorsMiddleware’,
]

#Cors Settings do not do it in production
CORS_ORIGIN_ALLOW_ALL = True
Creating Models

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todo/models.py
# Create your models here.
class TodoItem(models.Model):
title = models.CharField(max_length=256, null=True, blank=True)
completed = models.BooleanField(blank=True, default=False)
url = models.CharField(max_length=256, null=True, blank=True)
order = models.IntegerField(null=True, blank=True)

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py makemigrations todo
Migrations for ‘todo’:
todo/migrations/0001_initial.py
– Create model TodoItem

(venv) rmohan@root:~/docker-ansible/todobackend/src$ tree
.
|– db.sqlite3
|– manage.py
|– todo
| |– admin.py
| |– admin.pyc
| |– apps.py
| |– __init__.py
| |– __init__.pyc
| |– migrations
| | |– 0001_initial.py
| | |– __init__.py
| | `– __init__.pyc
| |– models.py
| |– models.pyc
| |– tests.py
| `– views.py
`– todobackend
|– __init__.py
|– __init__.pyc
|– settings.py
|– settings.pyc
|– urls.py
|– urls.pyc
`– wsgi.py

3 directories, 21 files

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, todo
Running migrations:
Applying contenttypes.0001_initial… OK
Applying auth.0001_initial… OK
Applying admin.0001_initial… OK
Applying admin.0002_logentry_remove_auto_add… OK
Applying contenttypes.0002_remove_content_type_name… OK
Applying auth.0002_alter_permission_name_max_length… OK
Applying auth.0003_alter_user_email_max_length… OK
Applying auth.0004_alter_user_username_opts… OK
Applying auth.0005_alter_user_last_login_null… OK
Applying auth.0006_require_contenttypes_0002… OK
Applying auth.0007_alter_validators_add_error_messages… OK
Applying auth.0008_alter_user_username_max_length… OK
Applying sessions.0001_initial… OK
Applying todo.0001_initial… OK

(venv) rmohan@root:~/docker-ansible/todobackend/src$ ls
db.sqlite3 manage.py todo todobackend
Creating Serializers

http://www.django-rest-framework.org/api-guide/serializers/

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todo/serializers.py
from rest_framework import serializers
from todo.models import TodoItem

class TodoItemSerializer(serialisers.HyperlinkedModelSerializer):
url = serializers.ReadOnlyField()
class Meta:
model = TodoItem
fields = (‘url’, ‘title’, ‘completed’, ‘order’)
Create Views

http://www.django-rest-framework.org/api-guide/views/

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todo/views.py
# -*- coding: utf-8 -*-
#from __future__ import unicode_literals
#from django.shortcuts import render
from todo.models import TodoItem
from todo.serializers import TodoItemSerializer
from rest_framework import status
from rest_framework import viewsets
from rest_framework.reverse import reverse
from rest_framework.decorators import list_route
from rest_framework.response import Response
# Create your views here.
class TodoItemViewSet(viewsets.ModelViewSet):
queryset = TodoItem.objects.all()
serializer_class = TodoItemSerializer
Configure Routing

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todobackend/urls.py
from django.conf.urls import url
from django.conf.urls import include
from django.contrib import admin

urlpatterns = [
url(r’^admin/’, admin.site.urls),
url(r’^’, include(‘todo.urls’)),
]

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todo/urls.py
from django.conf.urls import url, include
from todo import views
from rest_framework.routers import DefaultRouter

#create a router and registry our viewsets with it .
router = DefaultRouter(trailing_slash=False)
router.registry(r’todos’, views.TodoItemViewSet)

# the APi urls are now determined automatically by the router
urlpatterns = [
url(r’^’, include(router.urls)) ,
]
Test Driving the application

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py runserver
Performing system checks…

System check identified no issues (0 silenced).
September 14, 2017 – 09:12:44
Django version 1.11.5, using settings ‘todobackend.settings’
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

python1.png
Creating Tests

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todo/tests.py
from django.core.urlresolvers import reverse
from rest_framework import status
from rest_framework.test import APITestCase
from todo.models import TodoItem

# Create your tests here.
def createItem(client):
url = reverse(‘todoitem-list’)
data = {‘title’: ‘Walk the dog’}
return client.post(url, data, format=’json’)

class TestCreateTodoItem(APITestCase):
“””
Ensure we can create a new todo item
“””
def setUp(self):
self.response = createItem(self.client)

def test_received_201_created_status_code(self):
self.assertEqual(self.response.status_code, status.HTTP_201_CREATED)

def test_received_location_header_hyperlink(self):
self.assertRegexpMatches(self.response[‘Location’], ‘^http://.+/todos/[\d]+$’)

def test_item_was_created(self):
self.assertEqual(TodoItem.objects.count(), 1)

def test_item_has_correct_title(self):
self.assertEqual(TodoItem.objects.get().title, ‘Walk the dog’)

class TestUpdateTodoItem(APITestCase):
“””
Ensure we can update an existing todo item using PUT
“””
def setUp(self):
response = createItem(self.client)
self.assertEqual(TodoItem.objects.get().completed, False)
url = response[‘Location’]
data = {‘title’: ‘Walk the dog’, ‘completed’: True}
self.response = self.client.put(url, data, format=’json’)

def test_received_200_created_status_code(self):
self.assertEqual(self.response.status_code, status.HTTP_200_OK)

def test_item_was_updated(self):
self.assertEqual(TodoItem.objects.get().completed, True)

class TestPatchTodoItem(APITestCase):
“””
Ensure we can update an existing todo item using PATCH
“””
def setUp(self):
response = createItem(self.client)
self.assertEqual(TodoItem.objects.get().completed, False)
url = response[‘Location’]
data = {‘title’: ‘Walk the dog’, ‘completed’: True}
self.response = self.client.patch(url, data, format=’json’)

def test_received_200_ok_status_code(self):
self.assertEqual(self.response.status_code, status.HTTP_200_OK)

def test_item_was_updated(self):
self.assertEqual(TodoItem.objects.get().completed, True)

class TestDeleteTodoItem(APITestCase):
“””
Ensure we can delete a todo item
“””
def setUp(self):
response = createItem(self.client)
self.assertEqual(TodoItem.objects.count(), 1)
url = response[‘Location’]
self.response = self.client.delete(url)

def test_received_204_no_content_status_code(self):
self.assertEqual(self.response.status_code, status.HTTP_204_NO_CONTENT)

def test_the_item_was_deleted(self):
self.assertEqual(TodoItem.objects.count(), 0)

class TestDeleteAllItems(APITestCase):
“””
Ensure we can delete all todo items
“””
def setUp(self):
createItem(self.client)
createItem(self.client)
self.assertEqual(TodoItem.objects.count(), 2)
self.response = self.client.delete(reverse(‘todoitem-list’))

def test_received_204_no_content_status_code(self):
self.assertEqual(self.response.status_code, status.HTTP_204_NO_CONTENT)

def test_all_items_were_deleted(self):
self.assertEqual(TodoItem.objects.count(), 0)

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py test
Creating test database for alias ‘default’…
System check identified no issues (0 silenced).
…………
———————————————————————-
Ran 12 tests in 0.201s

OK
Destroying test database for alias ‘default’…

Unit Vs Integration Tests

(venv) rmohan@root:~/docker-ansible/todobackend/src$ mkdir todobackend/settings

(venv) rmohan@root:~/docker-ansible/todobackend/src$ touch todobackend/settings/__init__.py

(venv) rmohan@root:~/docker-ansible/todobackend/src$ cp todobackend/__init__.py todobackend/settings/base.py

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todobackend/settings/base.py
“””
Django settings for todobackend project.

Generated by ‘django-admin startproject’ using Django 1.9.

For more information on this file, see
https://docs.djangoproject.com/en/1.9/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/1.9/ref/settings/
“””

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, …)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Quick-start development settings – unsuitable for production
# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = ‘#%8=y*l-+z*+mot(0+17@lm%!_yt4zb*j69fag-x((vz^zif(l’

# SECURITY WARNING: don’t run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []

# Application definition

INSTALLED_APPS = [
‘django.contrib.admin’,
‘django.contrib.auth’,
‘django.contrib.contenttypes’,
‘django.contrib.sessions’,
‘django.contrib.messages’,
‘django.contrib.staticfiles’,
‘rest_framework’,
‘corsheaders’,
‘todo’
]

MIDDLEWARE_CLASSES = [
‘django.middleware.security.SecurityMiddleware’,
‘corsheaders.middleware.CorsMiddleware’,
‘django.contrib.sessions.middleware.SessionMiddleware’,
‘django.middleware.common.CommonMiddleware’,
‘django.middleware.csrf.CsrfViewMiddleware’,
‘django.contrib.auth.middleware.AuthenticationMiddleware’,
‘django.contrib.auth.middleware.SessionAuthenticationMiddleware’,
‘django.contrib.messages.middleware.MessageMiddleware’,
‘django.middleware.clickjacking.XFrameOptionsMiddleware’,
]

ROOT_URLCONF = ‘todobackend.urls’

TEMPLATES = [
{
‘BACKEND’: ‘django.template.backends.django.DjangoTemplates’,
‘DIRS’: [],
‘APP_DIRS’: True,
‘OPTIONS’: {
‘context_processors’: [
‘django.template.context_processors.debug’,
‘django.template.context_processors.request’,
‘django.contrib.auth.context_processors.auth’,
‘django.contrib.messages.context_processors.messages’,
],
},
},
]

WSGI_APPLICATION = ‘todobackend.wsgi.application’

# Database
# https://docs.djangoproject.com/en/1.9/ref/settings/#databases

DATABASES = {
‘default’: {
‘ENGINE’: ‘django.db.backends.sqlite3’,
‘NAME’: os.path.join(BASE_DIR, ‘db.sqlite3’),
}
}

# Password validation
# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
{
‘NAME’: ‘django.contrib.auth.password_validation.UserAttributeSimilarityValidator’,
},
{
‘NAME’: ‘django.contrib.auth.password_validation.MinimumLengthValidator’,
},
{
‘NAME’: ‘django.contrib.auth.password_validation.CommonPasswordValidator’,
},
{
‘NAME’: ‘django.contrib.auth.password_validation.NumericPasswordValidator’,
},
]

# Internationalization
# https://docs.djangoproject.com/en/1.9/topics/i18n/

LANGUAGE_CODE = ‘en-us’

TIME_ZONE = ‘UTC’

USE_I18N = True

USE_L10N = True

USE_TZ = True

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.9/howto/static-files/

STATIC_URL = ‘/static/’

# CORS Settings

CORS_ORIGIN_ALLOW_ALL = True

(venv) rmohan@root:~/docker-ansible/todobackend/src$ rm -rf todobackend/settings.py

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim manage.py
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “todobackend.settings.base”)

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todobackend/wsgi.py
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “todobackend.settings.base”)

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim todobackend/settings/test.py
from base import *
import os

# Installed Apps
INSTALLED_APPS += (‘django_nose’, )
TEST_RUNNER = ‘django_nose.NoseTestSuiteRunner’
TEST_OUTPUT_DIR = os.environ.get(‘TEST_OUTPUT_DIR’,’.’)
NOSE_ARGS = [
‘–verbosity=2’, # verbose output
‘–nologcapture’, # don’t output log capture
‘–with-coverage’, # activate coverage report
‘–cover-package=todo’, # coverage reports will apply to these packages
‘–with-spec’, # spec style tests
‘–spec-color’,
‘–with-xunit’, # enable xunit plugin
‘–xunit-file=%s/unittests.xml’ % TEST_OUTPUT_DIR,
‘–cover-xml’, # produce XML coverage info
‘–cover-xml-file=%s/coverage.xml’ % TEST_OUTPUT_DIR,
]

# Database
# https://docs.djangoproject.com/en/1.8/ref/settings/#databases
DATABASES = {
‘default’: {
‘ENGINE’: ‘django.db.backends.mysql’,
‘NAME’: os.environ.get(‘MYSQL_DATABASE’,’todobackend’),
‘USER’: os.environ.get(‘MYSQL_USER’,’todo’),
‘PASSWORD’: os.environ.get(‘MYSQL_PASSWORD’,’password’),
‘HOST’: os.environ.get(‘MYSQL_HOST’,’localhost’),
‘PORT’: os.environ.get(‘MYSQL_PORT’,’3306?),
}
}

Installing Mysql

(venv) rmohan@root:~/docker-ansible/todobackend/src$ sudo apt-get install mysql-server

(venv) rmohan@root:~/docker-ansible/todobackend/src$ sudo mysql_secure_installation

(venv) rmohan@root:~/docker-ansible/todobackend/src$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 48
Server version: 5.5.57-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql>
mysql> create database todobackend;
Query OK, 1 row affected (0.00 sec)

mysql> grant all privileges on *.* to ‘todo’@’localhost’ identified by ‘password‘;

Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye

(venv) rmohan@root:~/docker-ansible/todobackend/src$ pip install mysql-python
Collecting mysql-python
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached MySQL-python-1.2.5.zip
Complete output from command python setup.py egg_info:
sh: 1: mysql_config: not found
Traceback (most recent call last):
File “”, line 1, in
File “/tmp/pip-build-dmghKR/mysql-python/setup.py”, line 17, in
metadata, options = get_config()
File “setup_posix.py”, line 43, in get_config
libs = mysql_config(“libs_r”)
File “setup_posix.py”, line 25, in mysql_config
raise EnvironmentError(“%s not found” % (mysql_config.path,))
EnvironmentError: mysql_config not found

—————————————-
Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-build-dmghKR/mysql-python/

(venv) rmohan@root:~/docker-ansible/todobackend/src$ sudo apt-get install python-pip python-dev libmysqlclient-dev

(venv) rmohan@root:~/docker-ansible/todobackend/src$ pip install mysql-python
Collecting mysql-python
Using cached MySQL-python-1.2.5.zip
Building wheels for collected packages: mysql-python
Running setup.py bdist_wheel for mysql-python … done
Stored in directory: /home/rmohan/.cache/pip/wheels/38/a3/89/ec87e092cfb38450fc91a62562055231deb0049a029054dc62
Successfully built mysql-python
Installing collected packages: mysql-python
Successfully installed mysql-python-1.2.5

(venv) rmohan@root:~/docker-ansible/todobackend/src$ export DJANGO_SETTINGS_MODULE=todobackend.settings.test

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py –settings=todobackend.settings.test
Traceback (most recent call last):
File “manage.py”, line 10, in
execute_from_command_line(sys.argv)
File “/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py”, line 364, in execute_from_command_line
utility.execute()
File “/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py”, line 338, in execute
django.setup()
File “/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/django/__init__.py”, line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File “/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/django/apps/registry.py”, line 85, in populate
app_config = AppConfig.create(entry)
File “/home/rmohan/docker-ansible/todobackend/venv/local/lib/python2.7/site-packages/django/apps/config.py”, line 94, in create
module = import_module(entry)
File “/usr/lib/python2.7/importlib/__init__.py”, line 37, in import_module
__import__(name)
ImportError: No module named django_nose

(venv) rmohan@root:~/docker-ansible/todobackend/src$ pip install django_nose

(venv) rmohan@root:~/docker-ansible/todobackend/src$ pip install pinocchio coverage

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py runserver
Performing system checks…

System check identified no issues (0 silenced).

You have 14 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions, todo.
Run ‘python manage.py migrate’ to apply them.

September 14, 2017 – 15:14:06
Django version 1.11.5, using settings ‘todobackend.settings.test’
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py migrate
System check identified some issues:

WARNINGS:
?: (mysql.W002) MySQL Strict Mode is not set for database connection ‘default’
HINT: MySQL’s Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-sql-mode
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, todo
Running migrations:
Applying contenttypes.0001_initial… OK
Applying auth.0001_initial… OK
Applying admin.0001_initial… OK
Applying admin.0002_logentry_remove_auto_add… OK
Applying contenttypes.0002_remove_content_type_name… OK
Applying auth.0002_alter_permission_name_max_length… OK
Applying auth.0003_alter_user_email_max_length… OK
Applying auth.0004_alter_user_username_opts… OK
Applying auth.0005_alter_user_last_login_null… OK
Applying auth.0006_require_contenttypes_0002… OK
Applying auth.0007_alter_validators_add_error_messages… OK
Applying auth.0008_alter_user_username_max_length… OK
Applying sessions.0001_initial… OK
Applying todo.0001_initial… OK
(venv) rmohan@root:~/docker-ansible/todobackend/src$
(venv) rmohan@root:~/docker-ansible/todobackend/src$ python manage.py runserver
Performing system checks…

System check identified no issues (0 silenced).
September 14, 2017 – 15:14:56
Django version 1.11.5, using settings ‘todobackend.settings.test’
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

(venv) rmohan@root:~/docker-ansible/todobackend/src$ deactivate
rmohan@root:~/docker-ansible/todobackend/src$

rmohan@root:~/docker-ansible/todobackend/src$ cd ../../

rmohan@root:~/docker-ansible$ mkdir todobackend-specs

rmohan@root:~/docker-ansible$ cd todobackend-specs/

rmohan@root:~/docker-ansible/todobackend-specs$ git init
Initialized empty Git repository in /home/rmohan/docker-ansible/todobackend-specs/.git/

rmohan@root:~/docker-ansible/todobackend-specs$ touch .gitignore
node_module

rmohan@root:~/docker-ansible/todobackend-specs$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sane defaults.

See `npm help json` for definitive documentation on these fields
and exactly what they do.

Use `npm install –save` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
name: (todobackend-specs)
version: (0.0.0) 0.1.0
description: “Todo Backend Acceptence Test”
entry point: (index.js) app.js
test command: mocha
git repository:
keywords:
author: mshaik
license: (BSD-2-Clause)
About to write to /home/rmohan/docker-ansible/todobackend-specs/package.json:

{
“name”: “todobackend-specs”,
“version”: “0.1.0”,
“description”: “\”Todo Backend Acceptence Test\””,
“main”: “app.js”,
“scripts”: {
“test”: “mocha”
},
“author”: “mshaik”,
“license”: “BSD-2-Clause”
}

Is this ok? (yes)
rmohan@root:~/docker-ansible/todobackend-specs$ ls
package.json
rmohan@root:~/docker-ansible/todobackend-specs$ cat package.json
{
“name”: “todobackend-specs”,
“version”: “0.1.0”,
“description”: “\”Todo Backend Acceptence Test\””,
“main”: “app.js”,
“scripts”: {
“test”: “mocha”
},
“author”: “mshaik”,
“license”: “BSD-2-Clause”
}

rmohan@root:~/docker-ansible/todobackend-specs$ sudo npm install -g n
npm http GET https://registry.npmjs.org/n
npm http 200 https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n/-/n-2.1.8.tgz
npm http 200 https://registry.npmjs.org/n/-/n-2.1.8.tgz
/usr/local/bin/n -> /usr/local/lib/node_modules/n/bin/n
n@2.1.8 /usr/local/lib/node_modules/n

rmohan@root:~/docker-ansible/todobackend-specs$ sudo n stable

install : node-v8.4.0
mkdir : /usr/local/n/versions/node/8.4.0
fetch : https://nodejs.org/dist/v8.4.0/node-v8.4.0-linux-x64.tar.gz
######################################################################## 100.0%
installed : v8.4.0

rmohan@root:~/docker-ansible/todobackend-specs$ sudo npm install bluebird chai chai-as-promised mocha superagent superagent-promise
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN todobackend-specs@0.1.0 No repository field.

+ superagent-promise@1.1.0
+ superagent@3.6.0
+ chai@4.1.2
+ chai-as-promised@7.1.1
+ mocha@3.5.3
+ bluebird@3.5.0
added 66 packages in 9.374s

rmohan@root:~/docker-ansible/todobackend-specs$ cat package.json
{
“name”: “todobackend-specs”,
“version”: “0.1.0”,
“description”: “\”Todo Backend Acceptence Test\””,
“main”: “app.js”,
“scripts”: {
“test”: “mocha”
},
“author”: “mshaik”,
“license”: “BSD-2-Clause”,
“dependencies”: {
“bluebird”: “^3.5.0”,
“chai”: “^4.1.2”,
“chai-as-promised”: “^7.1.1”,
“mocha”: “^3.5.3”,
“superagent”: “^3.6.0”,
“superagent-promise”: “^1.1.0”
}
}

rmohan@root:~/docker-ansible/todobackend-specs$ sudo apt-get install nodejs-legacy

Building Base Image

rmohan@root:~/docker-ansible$ mkdir todobackend-base
rmohan@root:~/docker-ansible$ cd todobackend-base/

rmohan@root:~/docker-ansible/todobackend-base$ git init
Initialized empty Git repository in /home/rmohan/docker-ansible/todobackend-base/.git/

rmohan@root:~/docker-ansible/todobackend-base$ vim Dockerfile
FROM ubuntu:trusty
MAINTAINER rafi494

# Prevent dpkg errors
ENV TERM=xterm-256color

# Set mirrors to NZ
# RUN sed -i “s/http:\/\/archive./http:\/\/nz.archive./g” /etc/apt/sources.list

# Install Python runtime
RUN apt-get update && \
apt-get install -qy \
-o APT::Install-Recommend=false -o APT::Install-Suggests=false \
python python-virtualenv libpython2.7 python-mysqldb

# Create virtual environment
# Upgrade PIP in virtual environment to latest version
RUN virtualenv /appenv && \
. /appenv/bin/activate && \
pip install pip –upgrade

# Add entrypoint script
ADD scripts/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT [“entrypoint.sh”]

LABEL application=todobackend

rmohan@root:~/docker-ansible/todobackend-base$ mkdir scripts
rmohan@root:~/docker-ansible/todobackend-base$ vim scripts/entrypoint.sh
#!/bin/bash
. /appenv/bin/activate
exec $@

rmohan@root:~/docker-ansible/todobackend-base$ docker build -t rafi494/todobackend-base .
Sending build context to Docker daemon 34.82 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:trusty
trusty: Pulling from ubuntu

77ec837caf85: Pull complete
27fb25a22e6b: Pull complete
668a3fc0cef3: Pull complete
ad221bf67cf4: Pull complete
93f6219c4eb0: Pull complete
b44ce450cb60: Pull complete
Digest: sha256:404e4e6352099db4d3fb2781a13c3b3b090d3bcf61fc2b30b009c3987199a3aa
Status: Downloaded newer image for ubuntu:trusty
—> b44ce450cb60
Step 1 : MAINTAINER rafi494
—> Running in e2e87972a04c
—> 6e2f149ea112
Removing intermediate container e2e87972a04c
Step 2 : ENV TERM xterm-256color
—> Running in e22a9e2499db
—> c413f9b3a937
Removing intermediate container e22a9e2499db
Step 3 : RUN apt-get update && apt-get install -qy -o APT::Install-Recommend=false -o APT::Install-Suggests=false python python-virtualenv libpython2.7 python-mysqldb
—> Running in 81df16349ae6
Get:1 http://security.ubuntu.com trusty-security InRelease [65.9 kB]
.
.
Running hooks in /etc/ca-certificates/update.d….done.
—> 38d9bf507eb3
Removing intermediate container 81df16349ae6
Step 4 : RUN virtualenv /appenv && . /appenv/bin/activate && pip install pip –upgrade
—> Running in e454212014dd
New python executable in /appenv/bin/python
Installing setuptools, pip…done.
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: pip
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed pip
Cleaning up…
—> 8bd2596212ea
Removing intermediate container e454212014dd
Step 5 : ADD scripts/entrypoint.sh /usr/local/bin/entrypoint.sh
—> 5090f2d23cbc
Removing intermediate container 79bc20d7085a
Step 6 : RUN chmod +x /usr/local/bin/entrypoint.sh
—> Running in 77a421e12cf2
—> 4a33b5f19bcf
Removing intermediate container 77a421e12cf2
Step 7 : ENTRYPOINT entrypoint.sh
—> Running in f99f99500432
—> c860003f9b59
Removing intermediate container f99f99500432
Step 8 : LABEL application todobackend
—> Running in 5ad48a0bdd6d
—> 00a95d4fd18f
Removing intermediate container 5ad48a0bdd6d
Successfully built 00a95d4fd18f

rmohan@root:~/docker-ansible/todobackend-base$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
rafi494/todobackend-base latest 00a95d4fd18f 2 minutes ago 378.8 MB

rmohan@root:~/docker-ansible/todobackend-base$ docker run -rm rafi494/todobackend-base ps
PID TTY TIME CMD
1 ? 00:00:00 ps

rmohan@root:~/docker-ansible/todobackend-base$ vim scripts/entrypoint.sh
#!/bin/bash
. /appenv/bin/activate
$@

rmohan@root:~/docker-ansible/todobackend-base$ docker build -t rafi494/todobackend-base .
Sending build context to Docker daemon 34.82 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:trusty
—> b44ce450cb60
Step 1 : MAINTAINER rafi494
—> Using cache
—> 6e2f149ea112
Step 2 : ENV TERM xterm-256color
—> Using cache
—> c413f9b3a937
Step 3 : RUN apt-get update && apt-get install -qy -o APT::Install-Recommend=false -o APT::Install-Suggests=false python python-virtualenv libpython2.7 python-mysqldb
—> Using cache
—> 38d9bf507eb3
Step 4 : RUN virtualenv /appenv && . /appenv/bin/activate && pip install pip –upgrade
—> Using cache
—> 8bd2596212ea
Step 5 : ADD scripts/entrypoint.sh /usr/local/bin/entrypoint.sh
—> f0b62f95219f
Removing intermediate container a70509f7c735
Step 6 : RUN chmod +x /usr/local/bin/entrypoint.sh
—> Running in de0ca94753ad
—> 43626306b958
Removing intermediate container de0ca94753ad
Step 7 : ENTRYPOINT entrypoint.sh
—> Running in e82c88a26500
—> 7936a208c626
Removing intermediate container e82c88a26500
Step 8 : LABEL application todobackend
—> Running in e29c5e4599a0
—> 43d184525209
Removing intermediate container e29c5e4599a0
Successfully built 43d184525209

rmohan@root:~/docker-ansible/todobackend-base$ docker run -rm rafi494/todobackend-base ps
PID TTY TIME CMD
1 ? 00:00:00 entrypoint.sh
11 ? 00:00:00 ps

rmohan@root:~/docker-ansible/todobackend-base$ vim scripts/entrypoint.sh
#!/bin/bash
. /appenv/bin/activate
exec $@

rmohan@root:~/docker-ansible/todobackend-base$ docker run -rm rafi494/todobackend-base ps
PID TTY TIME CMD
1 ? 00:00:00 ps

Creating Devlopment Image

rmohan@root:~/docker-ansible/todobackend-base$ cd ../todobackend
rmohan@root:~/docker-ansible/todobackend$ tree -L 1
.
|– src
`– venv

2 directories, 0 files

rmohan@root:~/docker-ansible/todobackend$ mkdir -p docker/dev
rmohan@root:~/docker-ansible/todobackend$ cd docker/dev/

rmohan@root:~/docker-ansible/todobackend/docker/dev$ touch Dockerfile
FROM rafi494/todobackend-base:latest
MAINTAINER rafi

# Install dev/build dependencies
RUN apt-get update && \
apt-get install -qy python-dev libmysqlclient-dev

# Activate virtual environment and install wheel support
RUN . /appenv/bin/activate && \
pip install wheel –upgrade

# PIP environment variables (NOTE: must be set after installing wheel)
ENV WHEELHOUSE=/wheelhouse PIP_WHEEL_DIR=/wheelhouse PIP_FIND_LINKS=/wheelhouse XDG_CACHE_HOME=/cache

# OUTPUT: Build artefacts (Wheels) are output here
VOLUME /wheelhouse

# OUTPUT: Build cache
VOLUME /build

# OUTPUT: Test reports are output here
VOLUME /reports

# Add test entrypoint script
COPY scripts/test.sh /usr/local/bin/test.sh
RUN chmod +x /usr/local/bin/test.sh

# Set defaults for entrypoint and command string
ENTRYPOINT [“test.sh”]
CMD [“python”, “manage.py”, “test”, “–noinput”]

# Add application source
COPY src /application
WORKDIR /application

rmohan@root:~/docker-ansible/todobackend/docker/dev$ vim ../../scripts/test.sh
#!/bin/bash
# Activate virtual environment
. /appenv/bin/activate

# Download requirements to build cache
pip download -d /build -r requirements_test.txt –no-input

# Install application test requirements
pip install –no-index -f /build -r requirements_test.txt

# Run test.sh arguments
exec $@

Creating application Requirement files

rmohan@root:~/docker-ansible/todobackend/docker/dev$ cd ../..rmohan@root:~/docker-ansible/todobackend$ source venv/bin/activate
(venv) rmohan@root:~/docker-ansible/todobackend$ cd src/
(venv) rmohan@root:~/docker-ansible/todobackend/src$ pip freeze > requirements.txt

(venv) rmohan@root:~/docker-ansible/todobackend/src$ cat requirements.txt
colorama==0.3.9
coverage==4.4.1
Django==1.11.5
django-cors-headers==2.1.0
django-nose==1.4.5
djangorestframework==3.6.4
MySQL-python==1.2.5
nose==1.3.7
pinocchio==0.4.2
pytz==2017.2

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim requirements.txt
Django==1.11.5
django-cors-headers==2.1.0
djangorestframework==3.6.4
MySQL-python==1.2.5

(venv) rmohan@root:~/docker-ansible/todobackend/src$ vim requirements_test.txt
-r requirements.txt
colorama==0.3.9
coverage==4.4.1
django-nose==1.4.5
nose==1.3.7
pinocchio==0.4.2

Devlopment Image review
Testing Devlopment Image
(venv) rmohan@root:~/docker-ansible/todobackend/src$ cd ..
(venv) rmohan@root:~/docker-ansible/todobackend$ docker build -t todobackend-dev -f docker/dev/Dockerfile .
Sending build context to Docker daemon 50.65 MB
Sending build context to Docker daemon
Step 0 : FROM rafi494/todobackend-base:latest
—> a2ca1f3870d7
Step 1 : MAINTAINER rafi
—> Running in c684859b3076
—> 8d359fc022e8
Removing intermediate container c684859b3076
Step 2 : RUN apt-get update && apt-get install -qy python-dev libmysqlclient-dev
—> Running in de5056a6328e
.

Step 3 : RUN . /appenv/bin/activate && pip install wheel –upgrade
—> Running in 37e84294c1e6
Collecting wheel

—> b9e7992d4e62
Removing intermediate container 37e84294c1e6
Step 4 : ENV WHEELHOUSE /wheelhouse PIP_WHEEL_DIR /wheelhouse PIP_FIND_LINKS /wheelhouse XDG_CACHE_HOME /cache
—> Running in 5d94907105e3
—> 101e00962323
Removing intermediate container 5d94907105e3
Step 5 : VOLUME /wheelhouse
—> Running in 64488d8f2d97
—> 4607e17f46e1
Removing intermediate container 64488d8f2d97
Step 6 : VOLUME /build
—> Running in b6607d721ded
—> 004347875e10
Removing intermediate container b6607d721ded
Step 7 : VOLUME /reports
—> Running in 5999cc6683b4
—> 9b238fd38525
Removing intermediate container 5999cc6683b4
Step 8 : COPY scripts/test.sh /usr/local/bin/test.sh
—> 07f3369050ee
Removing intermediate container 18bac283237a
Step 9 : RUN chmod +x /usr/local/bin/test.sh
—> Running in ca52d4b7e065
—> ce20f7376ca9
Removing intermediate container ca52d4b7e065
Step 10 : ENTRYPOINT test.sh
—> Running in ae410324d55f
—> 78e8c1392b51
Removing intermediate container ae410324d55f
Step 11 : CMD python manage.py test –noinput
—> Running in a1a4eb631e76
—> 20f2cf563d00
Removing intermediate container a1a4eb631e76
Step 12 : COPY src /application
—> ae7f0b9dad4f
Removing intermediate container 59727c5df809
Step 13 : WORKDIR /application
—> Running in c629a3601c4e
—> d7a363a684d8
Removing intermediate container c629a3601c4e

(venv) rmohan@root:~/docker-ansible/todobackend$ vim .dockerignore
venv

(venv) rmohan@root:~/docker-ansible/todobackend$ docker build -t todobackend-dev -f docker/dev/Dockerfile .
Sending build context to Docker daemon 165.9 kB

(venv) rmohan@root:~/docker-ansible/todobackend$ docker run –rm todobackend-devCollecting Django==1.11.5 (from -r requirements.txt (line 1))
/appenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/appenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading Django-1.11.5-py2.py3-none-any.whl (6.9MB)
Saved /build/Django-1.11.5-py2.py3-none-any.whl
Collecting django-cors-headers==2.1.0 (from -r requirements.txt (line 2))
Downloading django_cors_headers-2.1.0-py2.py3-none-any.whl
Saved /build/django_cors_headers-2.1.0-py2.py3-none-any.whl
Collecting djangorestframework==3.6.4 (from -r requirements.txt (line 3))
Downloading djangorestframework-3.6.4-py2.py3-none-any.whl (1.5MB)
Saved /build/djangorestframework-3.6.4-py2.py3-none-any.whl
Collecting MySQL-python==1.2.5 (from -r requirements.txt (line 4))
Downloading MySQL-python-1.2.5.zip (108kB)
Saved /build/MySQL-python-1.2.5.zip
Collecting colorama==0.3.9 (from -r requirements_test.txt (line 2))
Downloading colorama-0.3.9-py2.py3-none-any.whl
Saved /build/colorama-0.3.9-py2.py3-none-any.whl
Collecting coverage==4.4.1 (from -r requirements_test.txt (line 3))
Downloading coverage-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl (193kB)
Saved /build/coverage-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting django-nose==1.4.5 (from -r requirements_test.txt (line 4))
Downloading django_nose-1.4.5-py2.py3-none-any.whl
Saved /build/django_nose-1.4.5-py2.py3-none-any.whl
Collecting nose==1.3.7 (from -r requirements_test.txt (line 5))
Downloading nose-1.3.7-py2-none-any.whl (154kB)
Saved /build/nose-1.3.7-py2-none-any.whl
Collecting pinocchio==0.4.2 (from -r requirements_test.txt (line 6))
Downloading pinocchio-0.4.2.tar.gz
Saved /build/pinocchio-0.4.2.tar.gz
Collecting pytz (from Django==1.11.5->-r requirements.txt (line 1))
Downloading pytz-2017.2-py2.py3-none-any.whl (484kB)
Saved /build/pytz-2017.2-py2.py3-none-any.whl
Successfully downloaded Django django-cors-headers djangorestframework MySQL-python colorama coverage django-nose nose pinocchio pytz
Collecting Django==1.11.5 (from -r requirements.txt (line 1))
Collecting django-cors-headers==2.1.0 (from -r requirements.txt (line 2))
Collecting djangorestframework==3.6.4 (from -r requirements.txt (line 3))
Collecting MySQL-python==1.2.5 (from -r requirements.txt (line 4))
Collecting colorama==0.3.9 (from -r requirements_test.txt (line 2))
Collecting coverage==4.4.1 (from -r requirements_test.txt (line 3))
Collecting django-nose==1.4.5 (from -r requirements_test.txt (line 4))
Collecting nose==1.3.7 (from -r requirements_test.txt (line 5))
Collecting pinocchio==0.4.2 (from -r requirements_test.txt (line 6))
Collecting pytz (from Django==1.11.5->-r requirements.txt (line 1))
Building wheels for collected packages: MySQL-python, pinocchio
Running setup.py bdist_wheel for MySQL-python: started
Running setup.py bdist_wheel for MySQL-python: finished with status ‘done’
Stored in directory: /cache/pip/wheels/16/ed/55/f27783bb5ab1cb57c9ac00356859d19adf17d76c31230f3f1f
Running setup.py bdist_wheel for pinocchio: started
Running setup.py bdist_wheel for pinocchio: finished with status ‘done’
Stored in directory: /cache/pip/wheels/ab/43/84/ba075171b712e03d94d14b1e264a80678dbca7ebb8bfe4f7b3
Successfully built MySQL-python pinocchio
Installing collected packages: pytz, Django, django-cors-headers, djangorestframework, MySQL-python, colorama, coverage, nose, django-nose, pinocchio
Successfully installed Django-1.11.5 MySQL-python-1.2.5 colorama-0.3.9 coverage-4.4.1 django-cors-headers-2.1.0 django-nose-1.4.5 djangorestframework-3.6.4 nose-1.3.7 pinocchio-0.4.2 pytz-2017.2
…………
———————————————————————-
Ran 12 tests in 0.098s

OK
Creating test database for alias ‘default’…
System check identified no issues (0 silenced).
Destroying test database for alias ‘default’…

How to reduce docker run time

(venv) rmohan@root:~/docker-ansible/todobackend$ docker run -v /tmp/cache:/cache –entrypoint true –name cache todobackend-dev(venv)

(venv) rmohan@root:~/docker-ansible/todobackend$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
664e669f0a1a todobackend-dev:latest “true” 30 seconds ago Exited (0) 29 seconds ago cache

(venv) rmohan@root:~/docker-ansible/todobackend$ time docker run –rm –volumes-from cache todobackend-devCollecting Django==1.11.5 (from -r requirements.txt (line 1))
OK
Creating test database for alias ‘default’…
System check identified no issues (0 silenced).
Destroying test database for alias ‘default’…

real 0m27.803s
user 0m0.028s
sys 0m0.012s

(venv) rmohan@root:~/docker-ansible/todobackend$ time docker run –rm –volumes-from cache todobackend-dev
real 0m11.441s
user 0m0.028s
sys 0m0.008s

Using diffrent test settings

(venv) rmohan@root:~/docker-ansible/todobackend$ time docker run –rm -e DJANGO_SETTINGS=todobackend.settings.test –volumes-from cache todobackend-dev
Collecting Django==1.11.5 (from -r requirements.txt (line 1))
File was already downloaded /build/Django-1.11.5-py2.py3-none-any.whl
Collecting django-cors-headers==2.1.0 (from -r requirements.txt (line 2))
File was already downloaded /build/django_cors_headers-2.1.0-py2.py3-none-any.whl
Collecting djangorestframework==3.6.4 (from -r requirements.txt (line 3))
File was already downloaded /build/djangorestframework-3.6.4-py2.py3-none-any.whl
Collecting MySQL-python==1.2.5 (from -r requirements.txt (line 4))
File was already downloaded /build/MySQL-python-1.2.5.zip
Collecting colorama==0.3.9 (from -r requirements_test.txt (line 2))
File was already downloaded /build/colorama-0.3.9-py2.py3-none-any.whl
Collecting coverage==4.4.1 (from -r requirements_test.txt (line 3))
File was already downloaded /build/coverage-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting django-nose==1.4.5 (from -r requirements_test.txt (line 4))
File was already downloaded /build/django_nose-1.4.5-py2.py3-none-any.whl
Collecting nose==1.3.7 (from -r requirements_test.txt (line 5))
File was already downloaded /build/nose-1.3.7-py2-none-any.whl
Collecting pinocchio==0.4.2 (from -r requirements_test.txt (line 6))
File was already downloaded /build/pinocchio-0.4.2.tar.gz
Collecting pytz (from Django==1.11.5->-r requirements.txt (line 1))
File was already downloaded /build/pytz-2017.2-py2.py3-none-any.whl
Successfully downloaded Django django-cors-headers djangorestframework MySQL-python colorama coverage django-nose nose pinocchio pytz
Collecting Django==1.11.5 (from -r requirements.txt (line 1))
Collecting django-cors-headers==2.1.0 (from -r requirements.txt (line 2))
Collecting djangorestframework==3.6.4 (from -r requirements.txt (line 3))
Collecting MySQL-python==1.2.5 (from -r requirements.txt (line 4))
Collecting colorama==0.3.9 (from -r requirements_test.txt (line 2))
Collecting coverage==4.4.1 (from -r requirements_test.txt (line 3))
Collecting django-nose==1.4.5 (from -r requirements_test.txt (line 4))
Collecting nose==1.3.7 (from -r requirements_test.txt (line 5))
Collecting pinocchio==0.4.2 (from -r requirements_test.txt (line 6))
Collecting pytz (from Django==1.11.5->-r requirements.txt (line 1))
Installing collected packages: pytz, Django, django-cors-headers, djangorestframework, MySQL-python, colorama, coverage, nose, django-nose, pinocchio
Successfully installed Django-1.11.5 MySQL-python-1.2.5 colorama-0.3.9 coverage-4.4.1 django-cors-headers-2.1.0 django-nose-1.4.5 djangorestframework-3.6.4 nose-1.3.7 pinocchio-0.4.2 pytz-2017.2
…………
———————————————————————-
Ran 12 tests in 0.100s

OK
Creating test database for alias ‘default’…
System check identified no issues (0 silenced).
Destroying test database for alias ‘default’…

real 0m20.798s
user 0m0.032s
sys 0m0.032s

Creating multicontainer environment using docker compose

(venv) rmohan@root:~/docker-ansible/todobackend$ vim docker/dev/docker-compose.yml
test:
build: ../../
dockerfile: docker/dev/Dockerfile
volumes_from:
– cache
links:
– db
environment:
DJANGO_SETTINGS_MODULE: todobackend.settings.test
MYSQL_HOST: db
MYSQL_USER: root
MYSQL_PASSWORD: password
TEST_OUTPUT_DIR: /reports

builder:
build: ../../
dockerfile: docker/dev/Dockerfile
volumes_from:
– cache
entrypoint: “entrypoint.sh”
command: [“pip”, “wheel”, “–no-index”, “-f /build”, “.”]

agent:
image: jmenga/ansible
links:
– db
environment:
PROBE_HOST: “db”
PROBE_PORT: “3306”
command: [“probe.yml”]

db:
image: mysql:5.6
hostname: db
expose:
– “3306”
environment:
MYSQL_ROOT_PASSWORD: password

cache:
build: ../../
dockerfile: docker/dev/Dockerfile
volumes:
– /tmp/cache:/cache
– /build
entrypoint: “true”

(venv) rmohan@root:~/docker-ansible/todobackend$ cd docker/dev/

(venv) rmohan@root:~/docker-ansible/todobackend/docker/dev$ docker-compose up test

rmohan@root:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial
rmohan@root:~$ python –version
Python 2.7.12

rmohan@root:~$ sudo apt-get install python-pip
Reading package lists… Done
Building dependency tree
Reading state information… Done
python-pip is already the newest version (8.1.1-2ubuntu0.4).
The following packages were automatically installed and are no longer required:
linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use ‘sudo apt autoremove’ to remove them.
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

rmohan@root:~$ pip install –upgrade pip
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |????????????????????????????????| 1.3MB 478kB/s
Installing collected packages: pip
Successfully installed pip-9.0.1

https://releases.hashicorp.com/terraform/

rmohan@root:~$ wget https://releases.hashicorp.com/terraform/0.11.3/terraform_0.11.3_linux_amd64.zip
–2018-03-08 11:55:25– https://releases.hashicorp.com/terraform/0.11.3/terraform_0.11.3_linux_amd64.zip
Resolving releases.hashicorp.com (releases.hashicorp.com)… 151.101.1.183, 151.101.65.183, 151.101.129.183, …
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.1.183|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 16466291 (16M) [application/zip]
Saving to: ‘terraform_0.11.3_linux_amd64.zip’

terraform_0.11.3_linux_amd64.zip 100%[==================================================================>] 15.70M 5.71MB/s in 2.8s

2018-03-08 11:55:29 (5.71 MB/s) – ‘terraform_0.11.3_linux_amd64.zip’ saved [16466291/16466291]

rmohan@root:~$ ls -l terraform_0.11.3_linux_amd64.zip
-rw-rw-r– 1 rmohan rmohan 16466291 Feb 1 02:23 terraform_0.11.3_linux_amd64.zip

rmohan@root:~$ sudo mkdir /bin/terraform

rmohan@root:~$ sudo unzip terraform_0.11.3_linux_amd64.zip -d /bin/terraform/
Archive: terraform_0.11.3_linux_amd64.zip
inflating: /bin/terraform/terraform

rmohan@root:~$ ls -l /bin/terraform/
total 67612
-rwxrwxr-x 1 root root 69233984 Jan 31 20:50 terraform

rmohan@root:~$ terraform –version
terraform: command not found

rmohan@root:~$ export PATH=$PATH:/bin/terraform/

rmohan@root:~$ terraform –version
Terraform v0.11.3

rmohan@root:~$ terraform
Usage: terraform [–version] [–help] [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you’re just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
apply Builds or changes infrastructure
console Interactive console for Terraform interpolations
destroy Destroy Terraform-managed infrastructure
env Workspace management
fmt Rewrites config files to canonical format
get Download and install modules for the configuration
graph Create a visual graph of Terraform resources
import Import existing infrastructure into Terraform
init Initialize a Terraform working directory
output Read an output from a state file
plan Generate and show an execution plan
providers Prints a tree of the providers used in the configuration
push Upload this Terraform module to Atlas to run
refresh Update local state file against real resources
show Inspect Terraform state or plan
taint Manually mark a resource for recreation
untaint Manually unmark a resource as tainted
validate Validates the Terraform files
version Prints the Terraform version
workspace Workspace management

All other commands:
debug Debug output management (experimental)
force-unlock Manually unlock the terraform state
state Advanced state management

rmohan@root:~$ pip install awscli –upgrade

rmohan@root:~$ aws –version
aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-116-generic botocore/1.4.70

rmohan@root:~$ sudo apt-get install ansible

rmohan@root:~$ ansible –version
ansible 2.0.0.2
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides

rmohan@root:~$ ssh-keygen

rmohan@root:~$ ssh-agent bash

rmohan@root:~$ ssh-add ~/.ssh/id_rsa
Identity added: /home/rmohan/.ssh/id_rsa (/home/rmohan/.ssh/id_rsa)

rmohan@root:~$ ssh-add -l
2048 SHA256:IQ9yajZaWP9O58/5Q8v3y8McEC8PGvjWw4tyX1mGqRY /home/rmohan/.ssh/id_rsa (RSA)

Uncomment the line

rmohan@root:~$ sudo vim /etc/ansible/ansible.cfg
host_key_checking = False

rmohan@root:~$ mkdir terraform_ansible

rmohan@root:~$ cd terraform_ansible/

rmohan@root:~/terraform_ansible$ aws configure
AWS Access Key ID [****************TAKA]:
AWS Secret Access Key [****************BklO]:
Default region name [us-east-1]:
Default output format [None]:

To check aws configured correctly
#############################################
rmohan@root:~/terraform_ansible$ aws ec2 describe-instances
{
“Reservations”: []
}

rmohan@root:~/terraform_ansible$ aws s3 ls
2018-03-07 09:26:12 elasticbeanstalk-us-east-1-536751915275
#############################################

[root@localhost ~]# ansible-galaxy init limits.conf
– limits.conf was created successfully

[root@localhost ~]# ansible-doc pam_limits

[root@localhost ~]# vim limits.conf/tasks/main.yml
—
# tasks file for limits.conf
– pam_limits:
domain: “{{ item.domain }}”
limit_type: “{{ item.limit_type }}”
limit_item: “{{ item.limit_item }}”
value: “{{ item.value }}”
with_items: “{{ limits_conf_settings }}”

[root@localhost ~]# vim limits_conf.yml
—
– hosts: all
roles:
– limits.conf
vars:
limits_conf_settings:
– domain: joe
limit_type: soft
limit_item: nofile
value: 64000

[root@localhost ~]# ansible-playbook limits_conf.yml -C

PLAY [all] *************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************
ok: [localhost]

TASK [limits.conf : pam_limits] ****************************************************************************************************************
skipping: [localhost] => (item={u’domain’: u’joe’, u’limit_item’: u’nofile’, u’limit_type’: u’soft’, u’value’: 64000})

PLAY RECAP *************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0

[root@localhost ~]# ansible-playbook limits_conf.yml

PLAY [all] *************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************
ok: [localhost]

TASK [limits.conf : pam_limits] ****************************************************************************************************************
changed: [localhost] => (item={u’domain’: u’joe’, u’limit_item’: u’nofile’, u’limit_type’: u’soft’, u’value’: 64000})

PLAY RECAP *************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0

[root@localhost ~]# tail -n1 /etc/security/limits.conf
joe soft nofile 64000

[root@localhost ~]# su – joe
Last login: Wed Sep 13 09:05:21 IST 2017 on pts/0
[joe@localhost ~]$ ulimit -Sn
64000

[root@localhost ~]# sysctl -a |grep vm.swappiness
vm.swappiness = 30

[root@localhost ~]# ansible-galaxy init sysctl
– sysctl was created successfully

[root@localhost ~]# ansible-doc sysctl

[root@localhost ~]# vim test.yml
—
– hosts: localhost
roles:
– sysctl
vars:
sysctl_settings:
– name: vm.swappiness
value: 90

[root@localhost ~]# vim sysctl/tasks/main.yml
—
# tasks file for sysctl
– name: sysctl settings
sysctl:
name: “{{ item.name }}”
value: “{{ item.value }}”
reload: true
state: “{{ item.state | default(‘present’) }}”
with_items: “{{ sysctl_settings }}”

[root@localhost ~]# ansible-playbook test.yml

PLAY [localhost] *******************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************
ok: [localhost]

TASK [sysctl : sysctl settings] ****************************************************************************************************************
changed: [localhost] => (item={u’state’: u’present’, u’name’: u’vm.swappiness’, u’value’: 90})

PLAY RECAP *************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0

[root@localhost ~]# sysctl -a |grep vm.swappiness
vm.swappiness = 90