path: root/unit4/unit4.typ

#let title = [
  Unit 4: Virtual Layer
]
#let proverb = [
  Knowledge not shared, remains unknown.
]
#set text(12pt)
#set page(
  header: [ 
    #box()[#proverb]
    #h(1fr)
    #box()[#title]
    ],
)
#align(center, text(20pt)[
  *#title*
])
#show table.cell.where(y: 0): strong
#outline()
#pagebreak()
= Basics
- Deployed on the physical layer.
- Specifies entities like virtualization software, resource poolsm and virtual resources.
- Abstracts the physical resources (compute, storage, network).
- Executing requests from control layer and forwarding them to the physical layer for execution.
= Introduction
_*Virtualization refers to the logical abstraction of physical resources such as compute, network and storage that enables a single hardware resource to support multiple concurrent instances of systems ot multiple hardware resources to support single instance of system.*_
- Allows us to make resources look larger or smaller than they actually are.
- Enables multitenant environment improving utilization of physical resources.
= Benefits of virtualization
1. Optimizes and consolidates utilization of IT resources.
2. Reduces cost and management complexity.
3. Reduces number of people required to maintain the system.
4. Reduces deployment time.
5. Increases flexibility.
= Overview
This layer is formed from virtualized compute, virtualized network, and virtualized storage. This enables 2 characteristics of cloud software: 
1. Resource Pooling
2. Rapid elasticity
This layer also specifies the entities that operate in this layer:
1. Virtualization software
2. Resource Pools
3. Virtual resources 
= Process and Operations
The key steps involved in making resources available to consumers are:
1. Deploying virtualization software.
  - Compute systems.
  - Network devices.
  - Storage devices.
2. Creating resource pools.
  - Processing power and memory
  - Network bandwidth
  - Storage
3. Creating virtual resources.
  - Virtual machines
  - Virtual Networks
  - LUNs
== Virtualization Software
The virtualization software performs the abstraction of the physical resources that ard deployed on computer systems, network devices, and storage devices.\
The key functions of virtualization software is:
1. Create resource pools
2. Create virtual resources.
== Resource Pools
A resource pool is an aggregation of computing resources such as processing power, memory, storage, and network bandwidth which provices an aggregated view of these resources to the control layer.
== Virtual resources
Virtualization software in collaboration with control layer creates virtual resources. Created by allocating physical resources from the resource pool. Share the pooled physical resources.
= Compute Virtualization
== Hypervisor
_*Hypervisor is a piece of software that is installed on a compute system and enables multiple OSs to run concurrently on a physical compute system.*_\
- The hypervisor along with the hypervisor management software (HMS) is the fundamental component for deploying software defined compute environments.\
- Abstracts the physical compute hardware to create multiple *virtual machines*.\
- The hypervisor provides standardized hardware resources, such as processor, memory, network, and disk to all virtual machines.\
Hypervisor has two main components:
1. Kernel
  - Acts as kernel of any normal OS.
  - Performs process creation, file system management, and process scheduling.
  - Optimized for running multiple VMs concurrently.
2. Virtual Machine Manager (VMM)
  - Anstracts hardware and appears as a physical compute system with all components required for running OSs and applications.
  - Each VM is assigned a VMM that gets a share of the physical resources.
== Types of hypervisors
Categorized into two types:
#table(
  columns: 2,
  table.header[Bare Metal Hypervisor][Hosted Hypervisor],
  [Installed directly on hardware.], [Installed as an application on host system.],
  [Direct access to hardware resources.], [No direct access. All access through host OS.],
  [More efficient.], [Less efficient.],
  [Limited device drivers.], [Compatible with all devices supported by host OS.],
  [Designed for data centers and cloud infrastructure.], [Used for development, testing and training.],
)
= Network Virtualization Software
_Network Virtualization Software abstracts physical network resources to create virtual resources like virtual LANs or virtual SANs._\
Built into the network device operating environment, installed on independent compute or built into the hypervisor.
  1. *Operating Environment*
    - Can abstract physical resources.
    - It can divide the physical network into multiple virtual networks.
  2. *External compute*
    - Fundamental component for deploying software defined network environment.
    - It provides a single control point to the entire network.
    - Enables automated policy based network management.
  3. *Hypervisor*
    - Emulates network connectivity among VMs on a physical compute system.
    - Allows the creation of virtual switches that appear to the VM as physical switches.
= Storage Virtualization Software
_The storage virtualization software abstracts physical storage resources into virtual resources like virtual volumes and virtual arrays._\
Built into the network device operating environment, installed on independent compute or built into the hypervisor.
  1. *Operating Environment*
    - Can pool and abstract physical storage drives and present it as local storage.
  2. *External compute*
    - Fundamental component for deploying software defined storage environment.
    - Can pool and abstract existing physical storage and present it as open storage platform.
    - Performs tasks such as virtual volume creation apart from creating virtual arrays.
    - Provides single control point for entire storage infrastructure.
    - Enables automated policy based storage management.
  3. *Hypervisor*
    - Enables creating virtual disk that appears to the operating system as physical disk.
= Resource Pools
#figure(
  image("./assets/resourcepool.png")
)
_A Resource Pool is a logical abstraction of the aggregated computing resources, such as processing power, memory capacity, storage, network bandwidth that are managed collectivly._\
- Cloud services obtain resources from resource pools.
- Resources are dynamically allocated according to consumer demand.
- The allocated resources are returned to the pool when released by the consumers.
- They are designed and sized according to service requirements.
- Cloud administrators can create, remove, exapand, contract resource pools as needed.
- Different pools can be created to provide high-end, low-end, etc. type of service.
== Examples of resource pooling
=== Pooling processing power and memory capacity
#figure(
  image("./assets/pppmc.png")
)
- VMs use processing power and memory capacity from processor and memory pools.
- Processor pool aggregates processing power of multiple machines.
- Memory pool aggregates the memory capacity of multiple machines.
- Each VM is allocated a small amount of both.
- After allocation of resources a certain amount is left in the pool.
=== Pooling storage in a block-based storage system
#figure(
  image("./assets/psbbss.png")
)
- Comprised of aggregated storage space of a set of physical drives.
- Storage space is allocated from the storage pool to logical unit number (LUN).
- LUNs are then provisioned to consumers upon recieving their storage requests.
- Storage of multiple drives is aggregated into a single pool.
- LUNs of smaller amount of storage created and assigned to consumers.
=== Pooling storage across block based storage systems
#figure(
  image("./assets/psabbss.png")
)
- More complex form of strage pooling.
- Higher level storage pool created by pooling multiple storage pools.
- LUNs allocated from these higher level storage pools.
=== Pooling network bandwidth of NICs
#figure(image("./assets/pnbn.png"))
- VMs obtain network bandwidth from network pools.
- VMs allocated appropriate resources to meet required service level.
== Identity pools
#figure(image("./assets/identitypool.png"))
_An identity pool, unlike a resource pool, specifies a range of network identifiers such as virtual network IDs and MAC addresses. These IDs are allocated from the identity pools to the elements of cloud services._\
- May map or allocate IDs to a particular service or to a group of service.
- If it runs out of IDs, then administrators may create an additional pool or add more identities to the existing pool.
- 1-to-1 mapping between identity pool and service simplifies the tracking and use of IDs by a particular service.
- Increases management complexity as multiple identity pools must be created and managed.
= Virtual Resources
== Virtual Machines
=== Virtual Machine
_A logical compute system that, like a physical compute system, runs an OS and applications._\
- Created by hosted or bare-metal hypervisor installed on physical compute.
- Has self-contained OS, applications, virtual hardware.
- 'Guest' OS is installed on a VM the same way it is installed on physical compute.
- From 'guest' perspective, VM appears to be a physical compute.
- Virtual Machine Monitor (VMM) is responsible for execution of VM.
- Each VM has dedicated VMM and configuration for hardware, software, network, etc.
- Does not have direct access to host OS or hardware.
- Hypervisor translates virtual VM requests and maps virtual hardware to physical hardware.
- Compute virtualization software enables creating and managing several VMs on a physical compute or compute cluster.
- Providers provision VMs to consumers to deploy applications.
- VM hardware and software are configured to meet requirements.
- Consumer VMs are isolated from each other so there is no interference.
- Isolation also provides fault tolerance.
=== VM Hardware
_When a VM is created, it is presented with virtual hardware components that appear as physical hardware components to the guest OS._\
- Within vendor's environment, each VM has standardized hardware components, so they are portable.
- Based on requirements, virtual components can be added or removed.
- However some components are part of the virtual motherboard and cannot be removed.
- Can be configured with one more virtual processors. This number can later be changed.
- When a VM starts, it's virtual processors are scheduled by the hypervisor kernal to run on physical processors.
- Each VM is assigned a virtual motherboard with standardized devices necessary for compute to function.
  1. *Virtual RAM* is the amount of physical memory allocated to the VM.
  2. *Virtual Disk* is a large physical file, or a set of files.
  3. *Virtual Network Adapter* provides connectivity to VMs running on same or different compute or VM and compute.
  4. *Virtual optical or floppy drives* can be configured to connect to physical drives or to image files.
  5. *SCSI/IDE virtual controllers* provide a way for VMs to connect to storage devices.
  6. *Virtual USB controller* is used to connect to a physical USB controller to access storage and devices.
=== VM Files
#table(
  columns: (auto, auto),
  table.header([ File ], [ Description ]),
  [ Configuration File ], [ Stores information such as VM name, BIOS information, guest OS type, memory size. ],
  [ Virtual Disk File ], [ Stores contents of VM's disk drive. ],
  [ Memory State File ], [ Stores the memory contents of a VM in a suspended state. ],
  [ Snapshot File ], [ Stores the VM settings and virtual disk of a VM. ],
  [ Log File ], [ Keeps log of the VM's activity and is used for troubleshooting. ]
)
=== File system to manage VM Files
#figure(image("./assets/vmfs.png"))
- File system is configured on storage to manage VM files.
- Capacity of the file system can be dynamically changed without disrupting the VM.
- Hypervisors support two file systems, hypervisor native and shared file system.
==== Hypervisor Native File System
- Usually a clustered file system and optimized for storing VM files.
- Can be deployed on storage provisioned from local storage or external storage.
- Allows multiple hypervisors running on different compute systems, to read and write to same shared storage concurrently.
- This enables high availability capabilities, like migration of VMs between hypervisors in the event of failure.
- Locking mechanism ensures that a VM is not powered on by multiple hypervisors at the same time.
- When hypervisor fails, locking mechanism for each VM on the physical compute is released.
- VMs can them be restarted from other hypervisors.
==== Shared File System
- Allows for VMs to be stored on remote servers or NAS.
- Accessed using file sharing protocols, like NFS and CIFS.
- Hypervisors have builtin NFS/CIFS clients.
=== VM Console
_VM Console is an interface used to view and manage the VMs on a compute system or cluster._\
- VM console can be:
  - Installed locally on compute system.
  - Web-based
  - Accessed over a remote desktop connection.
- Can be used to perform
  - Installing a guest OS and accessing VM BIOS
  - Powering a VM on or off
  - Configuring virtual hardware and troubleshooting.
=== VM Template
_A VM Template is a master copy of a virtual machine with a standardized virtual hardware and softeare configuration that can be used to create and provision new VMs._\
- Typically includes:
  - Guest OS
  - Set of applications
  - hardware and software configurations to deploy a VM
- Can be created in two ways, converting a VM to a template or cloning a VM to a template.
  1. Converting a VM to a template\
    Original VM replaced by the template
  2. Cloning a VM to a template\
    Original VM is retained
- Provides preinstalled and preconfigured software.
- It is faster as it eliminates installation, configuration, and maintenance overheads.
- Ensures consistency and standardization across VMs, making troubleshooting easier.
- Can be updated with new software, OS updates, and software patches.
- After update is complete, VM is converted back into a template.
=== Virtual Appliance
_A virtual appliance is a preconfigured virtual machine preinstalled with a guest OS and application dedicated to a specific function. In a cloud environment they are used to provide different functions, such as provide SaaS, run cloud management software, route packets, and provide security._\
- Simplifies the delivery and operation of applications.
- Faster as VM is preconfigured and has preinstalled software.
- Simplifies installation and eliminates configuration issues.
- Application runs in isolation, so is protected against crashes and other applications.
- Created using Open Virtualization Format, a open, hypervisor-independent packaging and distribution format.
=== VM Network
#figure(image("./assets/vmnet.png"))
_A VM network is a logical network that provides Ethernet connectivity and enables communication between VMs running on a hypervisor within a compute system._\
- Contains logical switches called virtual switches.
- Virtual switches function similarly to physical Ethernet switches but may not have full functionality.
- VM traffic travels over both VM network and physical networks.
=== VM Network Components
#table(
  columns: (auto, auto),
  table.header([ Component ], [ Description ]),
  [ Virtual Switch ], [
    - Logical OSI Layer 2 Ethernet switch created in a compute system.
    - Connects VMs locally can also directs VM traffic to a physical network.
    - Forwards frames to a virtual switch port based on destination address.
    - A distributed virtual switch can function across multiple physical compute systems.
  ], [ Virtual NIC ], [
    - Connects a VM to a virtual switch and functions like a physical NIC.
    - Has unique MAC and IP addresses.
    - Forwards the VM's network I/O in the form of Ethernet frames to a virtual switch.
  ], [ Uplink NIC ], [
    - A physical NIC connected to the uplink port of a virtual switch.
    - Functions as an ISL between virtual and physical Ethernet switches.
    - Not addressable from the network.
  ]
)
== Logical Unit Number (LUN)
=== LUN
_LUN is created by abstracting the identity and internal function of storage systems and appears as physical storage to the compute system._\
- Mapping of virtual to physical layer performed by virtualization layer.
- Assigned to compute system to create file system for storing and managing files.
_LUN Masking is the process that provides data access control by defining which LUNs a compute system can access._\
- Ensure volume access by compute is controlled appropriatly.
- In cloud, LUNs are created and assigned to different services based on the requirements.
=== Creating LUNs form RAID Sets
#figure(image("./assets/lunraid.png"))
- Done by partitioning the available capacity into smaller units.
- Spread across all physical disks that belong to a RAID set.
- Suited for applications that require predictable performance.
- Provide full control for precise data placement and allow an administrator to create LUNs on different RAID groups.
- Organizations concerned with storage space efficiency may use this.
=== Creating LUNs from storage pools
#figure(image("./assets/lunpool.png"))
- Comprises a set of physical drives that provide actual physical storage.
- Storage pool can contain a few to hundreds of drives.
- Two types of LUNs can be created:
  1. *Thick LUN*\
    - Storage capacity is fully allocated on creation.
  2. *Thin LUN*\
    - Do not require physical storage to be completly allocated to them at creation.
    - For OS, thin LUNs appear as traditional LUNs.
    - Thin LUNs consumer storage as needed from storage pool in increments called thin LUN extents.
    - Thin LUN extent defines the minimum amount of physical storage consumed from pool at a time.
    - When destroyed, allocated capacity is reclaimed to the pool.
===== Uses of Thin LUNs
- Provides worse performance compared to thick LUNs.
- Appropriate for applications that can tolerate performance variations.
- In some cases, performance improvements are seen when using thin LUNs.
- However, during contention between two thin LUNs over shared storage space or heavy utilization, performance can degrade.
- Provide the best storage space efficiency.
- Suitable for applicaions where space consumtion is difficult to forecast.
- Reduces storage costs and simplifies storage management.
== Virtual Networks
=== Virtual Network
#figure(image("./assets/virtnet.png"))
_A virtual network is a software based logical network that is either a segment of a physical network or spans across multiple physical networks._\
- Uses the physical network only for simple packet forwarding.
- To the nodes, it appears as a physical network.
- Nodes with common set of requirements can be grouped into virtual network reguardless of geographical location.
- Nodes can communicate without routing frames even if they are in different geographical locations.
- Traffic needs to be routed only when nodes are in different virtual networks.
- They are isolated and independent of each other.
- Each has unque attributes:
  - Routing
  - Switching
  - Independent policies
  - Quality of service
  - Bandwidth
  - Security
- Network management traffic and broadcasts do not propogate from one virtual network to another.
- All types of networks can be virtualized.
- Programmatically created, provisioned and managed from a network management workstation.
- Network and security policies become part of individual VMs in accordance with network and security policies defined for each connected application.
- When the VM is moved, it's networking and security also come with it.
- When new VMs are created to scale the application, network and security policies are applied dynamically.
- Can be scaled without reconfiguration of physical hardware.
_Virtual networks allow cloud providers to create logical networks the span physical boundries, allowing network extension and optimizing resource utilization across clusters and cloud data centers._\
=== Types of virtual networks
==== Virtual LAN (VLAN)
_A VLAN is a virtual network created on a LAN enabling communication between a group of nodes with a common set of functional requirements, independent of their physical location in the network._\
- Allow a network administrator to logically segment a LAN, and the nodes do not have to be physically located in the same LAN.
- Simplifies network configuration and administration.
- Nodes may be members of multiple VLANs, provided OS, hypervisor and storage array OS support it.
- To configure a VLAN
  1. Administrator defines VLANs on physical and virtual switches.
  2. Each VLAN is identified by a 12-bit VLAN ID.
  3. VLAN membership is configured based on techniques such as:
    - *Port based*: Membership is defined by assigning VLAN ID to physical or virtual switch port or port group.
    - *MAC based*: Membership is defined based on MAC address of node.
    - *Protocol based*: Different VLANs are defined different protocols based on protocol types field found in OSI Layer 2 header.
    - *IP-subnet address based*: Membership based on network IP subnet address in OSI Layer 3 header.
    - *Application based*: Specific applications configured to execute on one VLAN.
- If provider assigns one VLAN per customer, this limits the number of customers that can be supported.
- Block addresses assigned to each consumer VLAN which leads to unused IP addresses.
- As the number of nodes increase, the number of assigned addresses may not be large enough to support them.
==== Private VLAN (PVLAN)
#figure(image("./assets/pvlan.png"))
_A PVLAN is an extension of VLAN where VLANs are further divided into sub-VLANS. A PVLAN is made of a primary VLAN and one or more secondary or private VLANs. The primary VLAN is the original VLAN being segregated into smaller groups._\ 
- PVLANs exists only inside the primary VLAN. 
- Have unique VLAN ID and isolates the OSI Layer 2 traffic from other PVLANs. 
- Primary VLANs are promiscuous, ports on PVLAN can communicate with ports on VLAN. 
- Routers attached to promiscuous ports.
- Two types of secondary PVLANs:
  1. *Isolated*: A node attached to a port can only communicate with the promiscuous PVLAN.
  2. *Community*: A node attached to a port can communicate with other ports in the same community PVLAN as well as promiscuous PVLAN.
- To configure PVLAN:
  1. PVLAN feature must be supported and enabled on switch.
  2. Administrator creates a standard VLANs on a switch.
  3. VLANs then configured as primary and secondary.
- Enable providers to support a larger number of customers.
- Addresses issues with scalability with VLAN.
- All members share a common address space, which is allocated to the primary VLAN.
- When nodes are added to the PVLAN, they are assigned addresses allocated to primary VLAN.
- Provide extra security between nodes that do not belong to the same VLAN.
- Simplify administrative overhead of maintaining ACLs on different VLANs.
==== Stretched VLAN
#figure(image("./assets/strvlan.png"))
_A stretched VLAN is a VLAN that spans multiple sites and enables Layer 2 communication between a group of nodes over a Layer 3 WAN infrastructure, independent of their physical location._\
- Extends VLAN across sites and enables nodes to communicate over WAN as if they are in the same network.
- Allow movement of VMs between sites without changing network configurations.
- Enables high-availability clusters, VM migration, and application and workload mobility across sites.
- Created by simply connecting two sites using long distance fibre.
- Configured using different methods depending on underlying WAN technology.
- Use:
  1. Dense wave division multiplexing (DWDM).
  2. Coarse wave division multiplexing (CWDM).
  3. Multi-protocol label switching (MPLS).
  4. IP network.
==== Virtual Extensible LAN (VXLAN)
_A VXLAN is a logical Layer 2 overlay network built on a Layer 3 network, which uses MAC-in-UDP encapsulation to enable communication between a group of nodes, independent of their physical location._\
- VXLAN header is added to Layer 2 (MAC) frame, which is placed inside a UDP-IP packet and tunneled over a Layer 3 network.
- Communication established between two tunnel end-points called Virtual Tunnel Endpoints (VTEPs).
- At transmission, VETP encapsulates traffic into a VXLAN header and at destination, VTEP decapsulates it.
- Enables logical creation of nodes across different networks.
- In case of VMs, VETP is built into the hypervisor.
- Enable seperation of nodes from physical networks.
- Allow VMs to communicate using transparent overlay scheme over physical networks that span Layer 3.
- Extends Layer 2 network across sites.
- VMs are unaware of physical network constraints and only see the virtual Layer 2.
- Nodes identified using a combination of MAC addresses and VXLAN ID.
- VXLAN uses 24-bit ID.
- Make it easier for administrators to scale cloud infrastructure while logically isolating resources of different customers.
- Enable VM migrations across sites and over long distances.
==== Virtual SAN (VSAN)
#figure(image("./assets/vsan.png"))
_A VSAN or virtual fabric is a logical fabric, created on a physical FC or FCoE SAN enabling communication between a group of nodes with a common set of requirements, independent of their physical location in the fabric._\
- Functions conceptually same as a VLAN.
- Each behaves and is managed as an independent fabric.
- Each has it's own fabric services, configurations, FC addresses, etc.
- Fabric related configurations in one VSAN do not affect other VSANs.
- Disruptions in one VSAN do not affect other VSANs.
- VSAN may be extended across sites using long distance fibre, using DWDM, CWDM, FCIP to carry traffic.
- To configure VSANs:
  1. Define VSANs with VSAN IDs.
  2. F_Ports on the switch are assigned VSAN IDs to include them in VSANs.
  3. If N_Port connects to an F_Port that belongs to a VSAN, it becomes a member of that VSAN.
_Both VSANs and Zones enable node ports within a fabric to be logically segmented into groups. But they are not the same and their purposes are different. There is a hierarchial relationship between them. An administrator first assigns physical ports to VSANs and then configures independent zones for each VSAN. A VSAN has it's own fabric services, but fabric services are not available on a per-zone basis._\
=== Mapping VLAN and VSAN in an FCoE SAN
#figure(image("./assets/vfcoesan.png"))
- FCoE protocol enables transmission of FC SAN through a LAN that supports Data Center Bridging (DCB) functionality.
- FC frames remain encapsulated into Ethernet frames during transmission through the LAN.
- If VLANs and VSANs are created, a mapping is required between them.
- Hence, a VLAN must be configured at the FCoE switch for every VSAN.
- VLANs that carry regular LAN traffic should not be used for VSAN.