Bosh and Bosh Lite in Cloud Foundry

 Bosh and Bosh Lite in Cloud Foundry

 What is BOSH?  

BOSH is a project that unifies release engineering, deployment, and lifecycle management of small and large-scale cloud software. BOSH can provision and deploy software over hundreds of VMs. It also performs monitoring, failure recovery, and software updates with zero-to-minimal downtime.
While BOSH was developed to deploy Cloud Foundry PaaS, it can also be used to deploy almost any other software (Hadoop, for instance). BOSH is particularly well-suited for large distributed systems. In addition BOSH supports multiple Infrastructure as a Service (IaaS) providers like VMware vSphere, vCloud Director, Amazon Web Services EC2, and OpenStack. There is a Cloud Provider Interface (CPI) that enables users to extend BOSH to support additional IaaS providers such as Google Compute Engine and Apache CloudStack.

Why BOSH?

BOSH allows individual developers and teams to easily version, package and deploy software in a reproducible manner. 

BOSH was purposefully constructed to address the four principles of modern Release Engineering. 
1) Identifiability-  identify all of the source, tools, environment, and other components that make up a particular release.
2) Reproducibility - integrate source, third party components, data, and deployment externals of a software system in order to guarantee operational stability.
3) Consistency - provide a stable framework for development, deployment, audit, and accountability for software components.
4) Agility - integrates well with current best practices of software engineering.

What is a Stemcell?

A stemcell is a versioned Operating System image wrapped with IaaS specific packaging. That means the stemcell has to be selected depending on the IaaS.

A stemcell contains - 1) bare minimum OS skeleton with a few common utilities pre-installed
                                      2) BOSH Agent
                                      3) few configuration files to securely configure the OS by default

Stemcell provides a clear separation between base Operating System and later-installed software. That means Stemcell do not contain any specific information about any software that will be installed once that stemcell becomes a specialized machine in the cluster.

Stemcells for one OS are exactly the same for all infrastructures. That means the BOSH users can quickly and reliably switch between different infrastructures without worrying about the differences between OS images.

The Cloud Foundry BOSH team produces and maintains an official set of stemcells.

What is a Release?  

A release is a versioned collection of configuration properties, configuration templates, start up scripts, source code, binary artifacts, and anything else required to build and deploy software in a reproducible way.
A release is the layer placed on top of a stemcell. 
A release 

• Captures all needed configuration options and scripts for deployment of the software
• Records and keeps track of all dependencies for the software
• can be versioned and track of software releases
• can be IaaS agnostic
• can be self-contained and do not require internet access for deployment

What is a Deployment?
 
A deployment is a collection of VMs, built from a stemcell, that has been populated with specific releases and disks that keep persistent data. These resources are created based on a manifest file in the IaaS and managed by the BOSH Director, a centralized management server.

When a deployment manifest is uploaded to a BOSH Director, requested resources are allocated and stored. Over time, as the deployment manifest changes, VMs are replaced and updated. 

What is BOSH- Lite?

BOSH Lite is a pre-built Vagrant box which includes the Director. It uses Warden containers to emulate VMs which makes it an excellent choice for:

• General BOSH exploration without investing time and resources to configure an IaaS
• Development of releases (including BOSH itself)
• Testing releases locally or in CI  

Bosh-lite is a vagrant VM that comes with pre-installed BOSH server (Director). Once VM with Director is running we are going to use BOSH CLI to send commands to the Director.
 
How to use BOSH- Lite?

Follow the steps in this tutorial 
1) Run virtual machine with bosh- lite 
2) Get the BOSH CLI
3) Make the CLI target at BOSH Lite Director
4) Deploy an application

1. Clone or create a release
2. Name the release
3. Upload the release to the BOSH Lite Director
4. Set the deployment manifest.yml file (manifest file include what services to deploy, properties and resource configuration- ex: set director_uuid)
5. Clone or get a stemcell (stemcell is an OS image used by BOSH to create VMs)
6. upload the stemcell to the director
7. deploy the release

 BOSH Components

1) BOSH init- Tool used to create and update the director
2) CLI- Interact with the director and perform actions on the cloud
3) Director- controls VM creation and deployment                    
               adds tasks (sent by an operator through the CLI) to Task Queue (manage
                                tasks used by director and workers; workers take tasks and act on them) 
4) Cloud Provider Interface (CPI) - Director used to interact with IaaS to create and manage
                                                                                stemcells, VMs and disks
5) Health Monitor- monitor the health of VMs --> if there is a problem with a VM -->
6) ressurrector recreate VMs.
7) DNS Server-Convert unique IP addresses in textual form. ex: en.wikipedia.org/wiki/
                                                    domain-name-system --> 207.142.131.248
8) Database- store information about the desired state of a deployment - stemcells, releases
                                        and deployments
9) Blobstore- operator uploads a release using CLI --> Director insert the release to the
                                        blobstore--> when deploying the release --> compile the packages
                                        and store the result in blobstore
10) Agent - BOSH include an agent on every VM 
               carries out the instructions given by Director
11) Message bus (NATS) - provide instructions to VMs
                                                inform Health Monitor the status of the VM
12) Registry- stores configuration information (used during bootsrapping stage of the VM) for
                                     the VM

References
http://bosh.io/docs/ 
http://mariash.github.io/learn-bosh/#prepare

Introduction to Cloud Foundry

Introduction to Cloud Foundry

What is Cloud Foundry?

Cloud Foundry is an open-source platform as a service (PaaS) that provides you with a choice of clouds, developer frameworks, and application services.

What is a PaaS? 

The latest requirements of developers 

• increase developer productivity - new frameworks and languages
• reduce the time to market 
• run the application on multiple devices and multiple platforms
• scale and maintain the application with exclusively growing data
• access data in real time
• run the application on any infrastructure

If a developer want to have these requirements it is going to add lot of complexity in building, maintaining and deploying the applications. It will need mutiple load balancers, messaging centers, databases, app servers etc. But a developer is not expecting to wire all these middle-ware components. 

This complexity is not what developers except, they want to manage all these complexities in few API calls. i.e. deploy, manage, update, scale up and down in few API calls, the code of the application is not affected by where it is deployed, how it is scaled etc.

That is where Platform as a Service (PaaS) is introduced, as a solution to these complexities for a friction free application deployment and management. 

Characteristics of a PaaS

• Integrated software stack- abstract all these connections between nodes and roles to a simple software stack resolving all the complexities.
• application execution engine - developer just writes the code and push it into the PaaS environment and the PaaS executes it
• self- service application deployment
• automated application infrastructure provisioning- ability to develop locally and deploy to a cloud environment.
• curated, updated and operated as a service

 This provides the opportunity for the developers to just focus on the code of the application.

Why Cloud Foundry?


Most of the PaaS solutions today in the market are incomplete and have significant adoption barriers.

• Cant move between clouds- When you choose the PaaS you have to choose the cloud with it. Choosing the cloud is not decoupled from choosing the PaaS.
• Limited to a single provider- There is only one provider of the service, not having the ability to choose a technology that can be provided by many providers
• No on- premise solutions - not being able to develop the application on local machine and deploy it in PaaS later
• Limited to a single framework- developer have to think where he is going to deploy the application and write the code depending on that
• Require special frameworks

Cloud Foundry is introduced as a complete PaaS addressing all these issues. 
Its all about choice in cloud foundry 

• Choice of clouds for deployment - open PaaS
• Choice of industry- standard frameworks
• Choice of application infrastructure services
• Extensible architecture to "digest" future cloud innovation- bullet proof for future cloud innvoations ex: new frameworks
• Available as open source under apache 2 license and available on github
  
  Cloud Foundry has a significant growing ecosystem
• Have tens of thousands of beta users 
• Thousands of applications deployed
• Multiple distributers, deployers and clouds - cloudfoundry.com is available under pivotal and you can go for other distributors like Canonical, Dell, enStratus etc.
• As Cloud Foundry is open source it is looking for more and more additional frameworks. They already have two charter members : AppFog- PHP and ActiveState- Python
• Strong open source community participation - have hundreds of contributions already

Architecture of Cloud Foundry 

Cloud Foundry has 3 sub systems
1) BOSH

• creates and deploys VMs on top of a physical computing infrastructure
• runs cloud foundry on top of this cloud
• the deployement is done according to a manifest document 

2) Cloud Controller

• runs the applications and other processes on the cloud’s VMs, balancing demand and managing app lifecycles.

3) Go router

• routes incoming traffic from the world to the VMs that are running the applications that the traffic demands, usually working with a customer-provided load balancer.

Cloud Foundry has two types of VMs

1) Component VMs- consitute the platform’s infrastructure
2) Application VMs - host apps for the outside world.

Diego System- distributes the hosted app load over all the Application VMs, and keeps it running and balanced through demand surges, outages, or other changes according to an auction algorithm.


Applications on Cloud Foundry

When an application is deployed to Cloud Foundry

• an image is created for the application
• the image is stored internally. 
• that image is then deployed to a Warden container to run in -
  Cloud Foundry's internal Controller uses BOSH to get the underlying infrastructure to spin up virtual machines to run the Warden containers on.
• for multiple instances, multiple images are started on multiple containers. 
• when an application is terminated, all its VMs can be recycled for another application to use.
• if the application instance crashes, its container is killed and a new Warden container is started automatically.
• a container only ever runs one application ensuring isolation, security and resilience.
•  a load-balancing router sits at the front of Cloud Foundry to route incoming requests to the correct application - essentially to one of the containers where the application is running
• application manifest file tells what to do with the application
  

To meet the demand, multiple application VMs run duplicate instances of the same application. For this the applications must be portable. Cloud Foundry distributes application source code to VMs with everything needed to compile and run the applications locally. This includes
              1) the OS stack that the application runs on
              2) a buildpack containing all languages, libraries, services etc. that the app uses.
Before sending an app to a VM, the Cloud Controller stages it for delivery by combining stack, buildpack, and source code into a droplet that the VM can unpack, compile, and run. (For simple, standalone apps with no dynamic pointers, the droplet can contain a pre-compiled executable instead of source code, language, and libraries.)

Services in Cloud Foundry

Applications deployed to Cloud Foundry use services to access external resources. In a PaaS environment, all external dependencies such as databases, messaging systems, files systems and so on are Services. When an application is pushed to Cloud Foundry, the services it should use also can be specified. Depending on the application language, auto-configuration of services is possible - for example a Java application requiring a MySQL database picks up the MySQL service on Cloud Foundry if it is the only one defined in the current space.
Services have to be deployed to the platform first and then are available to any application using it.

References

https://www.youtube.com/watch?v=KNI2c6yS2Bo&noredirect=1
https://docs.cloudfoundry.org/concepts/overview.html  
https://en.wikipedia.org/wiki/Cloud_Foundry 
https://docs.pivotal.io/pivotalcf/concepts/overview.html