Automate deployment of applications on Amazon EC2 with Cloudify

In this tutorial, you will learn how to use Cloudify to install a web application on Amazone EC2. Cloudify’s DSL (Domain Specific Language) is following OASIS TOSCA (Topology and Orchestration Specification for Cloud Applications). TOSCA has a simplified YAML profile and Cloudify is following this YAML syntax.

Identity and Access Management (IAM) is a web service that helps you securely control access to AWS resources for your users. Cloudify will use this web service to access to the cloud resources. First of all, we need to create a new IAM user.

1. Navigate to Identity & Access Management
   
   
   
2. Click Create New Users.
   
   
   
3. Enter new user name.
   
   
   
4. You will receive access key id and secret access key. Download this information to your machine by simply click Download Credentials, or just save them into a text file.
   
   
   
5. Now, we need to configure permissions to thee user. Select Users.
   
   
   
6. Click Attach Policy.
   
   
   
7. Search for AdminstratorAccess, select and click Attach Policy.
   
   
   
8. Thereafter, you will have this policy attached to your IAM account.
   
   
   

1. Install cloudify 3.2.1 from http://getcloudify.org/downloads/previous_ga_versions.html or follow the instruction from http://getcloudify.org/guide/3.2/installation.html.

1. Only for windows, configure Git on Windows to properly handle line endings:

   $ git config --global core.autocrlf true

2. Create a new working directory:

   $ mkdir cfy
   $ cd cfy

3. Clone sample Cloudify blueprints from the github:

   $ git clone https://github.com/cloudify-cosmo/cloudify-manager-blueprints
   $ cd cloudify-manager-blueprints
   $ git checkout tags/3.2.1
   $ cd ..

4. Initialize Cloudify on your working directory and install required plugins for AWS EC2 blueprint:

   $ cfy init
   $ cfy local install-plugins -p cloudify-manager-blueprints/aws-ec2/aws-ec2-manager-blueprint.yaml

5. Create cloudify-manager-blueprints/aws-ec2/inputs.yaml, change aws_access_key_id and aws_secret_access_key according to your IAM account:

   aws_access_key_id: 'AKIAIV3N3QTUAIFSXIVA'
   aws_secret_access_key: 'gpbh0814p7M/JLKAi/ShyFkyYwCIJ6eTO/rKldO0'
   image_id: 'ami-47a23a30'
   instance_type: 't2.medium'
   manager_keypair_name: 'cloudify-manager-kp'
   agent_keypair_name: 'cloudify-agent-kp'
    
   use_existing_manager_group: false
   use_existing_agent_group: false
   use_existing_manager_keypair: false
   use_existing_agent_keypair: false
   manager_key_pair_file_path: ~/.ssh/cloudify-manager-kp.pem
   agent_key_pair_file_path: ~/.ssh/cloudify-agent-kp.pem
   mananger_security_group_name: cloudify-manager-security-group
   agent_security_group_name: cloudify-agent-security-group
   manager_server_name: cloudify-manager-server
   manager_server_user: ubuntu
   agents_user: ubuntu
   # If no region is provided, the plugin will fallback to the default region for your user.
   ec2_region_name: 'eu-west-1'

6. Bootstrap a Cloudify Manager:

   $ cfy bootstrap --install-plugins -p cloudify-manager-blueprints/aws-ec2/aws-ec2-manager-blueprint.yaml -i cloudify-manager-blueprints/aws-ec2/inputs.yaml

   In case you tried bootstrap before, make sure to delete 2 files, HOME_DIRECTORY\.ssh\cloudify-agent-kp.pem and HOME_DIRECTORY\.ssh\cloudify-manager-kp.pem, before execute the command.

7. To validate this installation, perform the following command:

   $ cfy status
   Getting management services status... [ip=xx.xx.xxx.xxx]
    
   Services:
   +--------------------------------+--------+
   |            service             | status |
   +--------------------------------+--------+
   | Riemann                        |   up   |
   | Celery Management              |   up   |
   | Manager Rest-Service           |   up   |
   | AMQP InfluxDB                  |   up   |
   | RabbitMQ                       |   up   |
   | Elasticsearch                  |   up   |
   | Webserver                      |   up   |
   | Logstash                       |   up   |
   +--------------------------------+--------+

8. Navigate to AWS console, and see your newly created instance, IP address, security groups and key pairs of your Cloudify Manager.
   
   
   
   
   
   

1. Clone a sample nodecellar web application from the Git hub.
   1. For Unix:

      $ git clone https://github.com/cloudify-cosmo/cloudify-nodecellar-example
      $ cd cloudify-nodecellar-example
      $ git checkout tags/3.2.1
      $ cd ..

   2. For Windows, to prevent the bug of git when cloning the UNIX script files, navigate to https://github.com/cloudify-cosmo/cloudify-nodecellar-example select the branch 3.2.1-build and download it as a zip file, extract it to your working directory.
      
      
      
2. Look at cloudify-nodecellar-example/aws-ec2-blueprint.yaml, it contains 2 servers for Node JS Server and Mongo Database.

   node_templates:
    
     mongod_host:
       type: nodecellar.nodes.MonitoredServer
       properties:
         image_id: { get_input: image }
         instance_type: { get_input: size }
       relationships:
         - type: cloudify.aws.relationships.instance_connected_to_security_group
           target: mongod_security_group
    
     nodejs_host:
       type: nodecellar.nodes.MonitoredServer
       properties:
         image_id: { get_input: image }
         instance_type: { get_input: size }
       relationships:
         - type: cloudify.aws.relationships.instance_connected_to_security_group
           target: nodecellar_security_group
         - type: cloudify.aws.relationships.instance_connected_to_elastic_ip
           target: nodecellar_elasticip
    
     mongod:
       type: nodecellar.nodes.MongoDatabase
       properties:
         port: 27017
       interfaces:
         cloudify.interfaces.lifecycle:
   	  
       relationships:
         - type: cloudify.relationships.contained_in
           target: mongod_host
   		
     nodejs:
       type: nodecellar.nodes.NodeJSServer
       relationships:
         - type: cloudify.relationships.contained_in
           target: nodejs_host
    
     nodecellar:
       type: nodecellar.nodes.NodecellarApplicationModule
       properties:
         port: 8080
       relationships:
   	
         ################################
         # Setting the mongo connection
         ################################
    
         - type: node_connected_to_mongo
           target: mongod
    
         ################################
         # Setting the nodejs connection
         ################################
    
         - type: node_contained_in_nodejs
           target: nodejs

   It can be visualize as follow:
   
   

3. Upload this blueprint to the cloudify manager.

   $ cfy blueprints upload -b nodecellar -p cloudify-nodecellar-example/aws-ec2-blueprint.yaml

4. Create a new file cloudify-nodecellar-example/inputs/aws-ec2.yaml with the following content:

   image: 'ami-47a23a30'
   size: 't2.micro'
   agent_user: 'ubuntu'

5. To reduce deployment time, edit cloudify-nodecellar-example/scripts/nodejs/install-nodejs.sh:

   ...
   download https://github.com/KarnYong/cloudify-nodecellar-tars/raw/master/${NODEJS_TARBALL_NAME} ${NODEJS_TARBALL_NAME}
   ...

   And edit cloudify-nodecellar-example/scripts/mongo/install-mongo.sh:

   ...
   download https://github.com/KarnYong/cloudify-nodecellar-tars/raw/master/${MONGO_TARBALL_NAME} ${MONGO_TARBALL_NAME}
   ...

6. Create a deployment by running the command:

   $ cfy deployments create -b nodecellar -d nodecellar --inputs cloudify-nodecellar-example/inputs/aws-ec2.yaml

7. Install a deployment by executing the following command:

   $ cfy executions start -w install -d nodecellar

8. Validate your installation by the following commands:

   $ cfy node-instances list
   $ cfy node-instances get --node-instance-id NODE_INSTANCE_ID
   Getting node instance with ID: 'NODE_INSTANCE_ID' [manager=xx.xx.xx.xxx]
    
   Instances:
   +---------------------------------+---------------+-------------------+---------------------------+---------+
   |                id               | deployment_id |      host_id      |          node_id          |  state  |
   +---------------------------------+---------------+-------------------+---------------------------+---------+
   |           nodejs_d086c          |   nodecellar  | nodejs_host_719ab |           nodejs          | started |
   |   mongod_security_group_0057e   |   nodecellar  |        None       |   mongod_security_group   | started |
   | nodecellar_security_group_98413 |   nodecellar  |        None       | nodecellar_security_group | started |
   |        mongod_host_4ed42        |   nodecellar  | mongod_host_4ed42 |        mongod_host        | started |
   |    nodecellar_elasticip_5e574   |   nodecellar  |        None       |    nodecellar_elasticip   | started |
   |           mongod_1f6a4          |   nodecellar  | mongod_host_4ed42 |           mongod          | started |
   |        nodejs_host_719ab        |   nodecellar  | nodejs_host_719ab |        nodejs_host        | started |
   |         nodecellar_8e0f2        |   nodecellar  | nodejs_host_719ab |         nodecellar        | started |
   +---------------------------------+---------------+-------------------+---------------------------+---------+

9. Navigate to EC2 console, you will see two new instances, one for Node JS server and another one for mongo database. Check the ip address of your Node JS server.
   
   
   
10. Navigate to http://IP_OF_YOUR_NODE_JS_SERVER:8080 to see your web application.