Welcome back and in this lesson I'm going to be covering EC2, auto scaling groups which is how we can configure EC2 to scale automatically based on demand placed on the system.

Auto scaling groups are generally used together with elastic load balances and launch templates to deliver elastic architectures.

Now we've got a lot to cover so let's jump in and get started.

Auto scaling groups do one thing.

They provide auto scaling for EC2.

Strictly speaking they can also be used to implement a self healing architecture as part of that scaling or in isolation.

Now auto scaling groups make use of configuration defined within launch templates or launch configurations and that's how they know what to provision.

An auto scaling group uses one launch configuration or one specific version of a launch template which is linked to it.

You can change which of those is associated but it's one of them at a time and so all instances launched using the auto scaling group are based on this single configuration definition either defined inside a specific version of a launch template or within a launch configuration.

Now an auto scaling group has three super important values associated with it.

We've got the minimum size, the desired capacity and the maximum size and these are often referred to as min, desired and max and can often be expressed as x, y or z.

For example 1, 2, 4 means 1 minimum, 2 desired and 4 maximum.

Now an auto scaling group has one foundational job which it performs.

It keeps the number of running EC2 instances the same as the desired capacity and it does this by provisioning or terminating instances.

So the desired capacity always has to be more than the minimum size and less than the maximum size.

If you have a desired capacity of 2 but only one running EC2 instance then the auto scaling group provisions a new instance.

If you have a desired capacity of 2 but have three running EC2 instances then the auto scaling group will terminate an instance to make these two values match.

Now you can keep an auto scaling group entirely manual so there's no automation and no intelligence.

You just update values and the auto scaling group performs the necessary scaling actions.

Normally though scaling policies are used together with auto scaling groups.

Scaling policies can update the desired capacity based on certain criteria for example CPU load and if the desired capacity is updated then as I've just mentioned it will provision or terminate instances and visually this is how it looks.

We have an auto scaling group and these run within a VPC across one or more subnets.

The configuration for EC2 instances is provided either using launch templates or launch configurations and then on the auto scaling group we specify a minimum value.

In this case 1 and this means there will always be at least one running EC2 instance.

In this case the cat pictures blog.

We can also set a desired capacity in this example 2 and this will add another instance if a desired capacity is set which is higher than the current number of instances.

If this is the case then instances are added.

Finally we could set the maximum size in this case to 4 which means that two additional instances could be provisioned but they won't immediately be because the desired capacity is only set to 2 and there are currently two running instances.

We could manually adjust the desired capacity up or down to add or remove instances which would automatically be built based on the launch template or launch configuration.

Alternatively we could use scaling policies to automate that process and scale in or out based on sets of criteria.

Architecturally auto scaling groups define where instances are launched.

They're linked to a VPC and subnets within that VPC are configured on the auto scaling group.

Whatever subnets are configured will be used to provision instances into.

When instances are provisioned there's an attempt to keep the number of instances within each availability zone even.

So in this case if the auto scaling group was configured with three subnets and the desired capacity was also set to three then it's probable each subnet would have one EC2 instance running within it but this isn't always the case.

The auto scaling group will try and level capacity where available.

Scaling policies are essentially rules.

Rules which you define which can adjust the values of an auto scaling group and there are three ways that you can scale auto scaling groups.

The first is not really a policy at all it's just to use manual scaling and I just talked about doing that.

This is where you manually adjust the values at any time and the auto scaling group handles any provisioning or termination that's required.

Next there's scheduled scaling which is great for sale periods where you can scale out the group when you know there's going to be additional demand or when you know a system won't be used so you can scale in outside of business hours.

Scheduled scaling adjusts the desired capacity based on schedules and this is useful for any known periods of high or low usage.

For the exam if you have known periods of usage then scheduled scaling is going to be a great potential answer.

Then we have dynamic scaling and there are three subtypes.

What they all have in common is they are rules which react to something and change the values on an auto scaling group.

The first is simple scaling and this well it's simple.

This is most commonly a pair of rules one to provision instances and one to terminate instances.

You define a rule based on a metric and an example of this is CPU utilization.

If the metric for example CPU utilization is above 50% then adjust the desired capacity by adding one and if the metric is below 50% then remove one from the desired capacity.

Using this method you can scale out meaning adding instances or scale in meaning terminating instances based on the value of a metric.

Now this metric isn't limited to CPU it can be many other metrics including memory or disk input output.

Some metrics need the cloud watch agent to be installed.

You can also use some metrics not on the EC2 instances.

For example maybe the length of an SQSQ which will cover elsewhere in the course or a custom performance metric within your application such as response time.

We also have stepped scaling which is similar but you define more detailed rules and this allows you to act depending on how out of normal the metric value is.

So maybe add one instance if the CPU usage is above 50% but if you have a sudden spike of load maybe add three if it's above 80% and the same could happen in reverse.

Step scaling allows you to react quicker the more extreme the change in conditions.

Step scaling is almost always preferable to simple except when your only priority is simplicity.

And then lastly we have target tracking and this takes a slightly different approach.

It lets you define an ideal amount of something say 40% aggregate CPU and then the group will scale as required to stay at that level provisioning or terminating instances to maintain that desired amount or that target amount.

Now not all metrics work for target tracking but some examples of ones that are supported are average CPU utilization, average network in, average network out and the one that's relevant to application load balances request count per target.

Now lastly there's a configuration on an auto scaling group called a cooldown period and this is a value in seconds.

It controls how long to wait at the end of a scaling action before doing another.

It allows auto scaling groups to wait and review chaotic changes to a metric and can avoid costs associated with constantly adding or removing instances.

Because remember there is a minimum billable period.

Since you'll build for at least the minimum time every time an instance is provisioned regardless of how long you use it for.

Now auto scaling groups also monitor the health of instances that they provision.

By default this uses the EC2 status checks.

So if an EC2 instance fails EC2 detects this passes this on to the auto scaling group and then the auto scaling group terminates the EC2 instance then it provisions a new EC2 instance in its place.

This is known as self healing and it will fix most problems isolated to a single instance.

The same would happen if we terminated an instance manually.

The auto scaling group would simply replace it.

Now there's a trick with EC2 and auto scaling groups.

If you create a launch template which can automatically build an instance then create an auto scaling group using that template.

Set the auto scaling group to use multiple subnets in different availability zones.

Then set the auto scaling group to use a minimum of one, a maximum of one and a desired of one.

Then you have simple instance recovery.

The instance will recover if it's terminated or if it fails.

And because auto scaling groups work across availability zones the instance can be reprovisioned in another availability zone if the original one fails.

It's cheap, simple and effective high availability.

Now auto scaling groups are really cool on their own but their real power comes from their ability to integrate with load balancers.

Take this example that Bob is browsing to the cat blog that we've been using so far and he's now connecting through a load balancer.

And the load balancer has a listener configured for the blog and points at a target group.

Instead of statically adding instances or other resources to the target group then you can use an auto scaling group configured to integrate with the target group.

As instances are provisioned within the auto scaling group then they're automatically added to the target group of that load balancer.

And then as instances are terminated by the auto scaling group then they're removed from that target group.

This is an example of elasticity because metrics which measure load on a system can be used to adjust the number of instances.

These instances are effectively added as load balancer targets and any users of the application because they access via the load balancer are abstracted away from the individual instances and they can use the capacity added in a very fluid way.

And what's even more cool is that the auto scaling group can be configured to use the load balancer health checks rather than EC2 status checks.

Application load balancer checks can be much richer.

They can monitor the state of HTTP or HTTPS requests.

And because of this they're application aware which simple status checks which EC2 provides are not.

Be careful though you need to use an appropriate load balancer health check.

If your application has some complex logic within it and you're only testing a static HTML page then the health check could respond as okay even though the application might be in a failed state.

And the inverse of this if your application uses databases and your health check checks a page with some database access requirements well if the database fails then all of your health checks could fail meaning all of your EC2 instances will be terminated and reprovisioned when the problem is with the database not the instances.

And so you have to be really careful when it comes to setting up health checks.

Now the next thing I want to talk about is scaling processes within an auto scaling group.

So you have a number of different processes or functions performed by the auto scaling group.

And these can be set to either be suspended or they can be resumed.

So first we've got launch and terminate and if launch is set to suspend then the auto scaling group won't scale out if any alarms or schedule actions take place.

And the inverse is if terminate is set to suspend then the auto scaling group will not terminate any instances.

We've also got add to load balancer and this controls whether any instances provisioned are added to the load balancer.

Next we've got alarm notification and this controls whether the auto scaling group will react to any cloud watch alarms.

We've also got az rebalance and this controls whether the auto scaling group attempts to redistribute instances across availability zones.

We've got health check and this controls whether instance health checks across the entire group are on or off.

We've also got replace unhealthy which controls whether the auto scaling group will replace any instances marked as unhealthy.

We've got scheduled actions which controls whether the auto scaling group will perform any scheduled actions or not.

And then in addition to those you can set a specific instance to either be standby or in service.

And this allows you to suspend any activities of the auto scaling group on a specific instance.

So this is really useful if you need to perform maintenance on one or more EC2 instances you can set them to standby and that means they won't be affected by anything that the auto scaling group does.

Now before we finish I just want to talk about a few final points and these are really useful for the exam.

Auto scaling groups are free.

The only costs are for the resources created by the auto scaling group and to avoid excessive costs use cooldowns within the auto scaling group to avoid rapid scaling.

To be cost effective you should also think about using more smaller instances because this means you have more granular control over the amount of compute and therefore costs that are incurred by your auto scaling group.

So if you have two larger instances and you need to add one that's going to cost you a lot more than if you have 20 smaller instances and only need to add one.

Smaller instances mean more granularity which means you can adjust the amount of compute in smaller steps and that makes it a more cost effective solution.

Now auto scaling groups are used together with application load balances for elasticity so the load balancer provides the level of abstraction away from the instances provisioned by the auto scaling group so together they're used to provision elastic architectures.

And lastly an auto scaling group controls the when and the where so when instances are launched and which subnets they're launched into.

Launch templates or launch configurations define the what so what instances are launched and what configuration those instances have.

Now at this point that's everything I wanted to cover in this lesson it's been a huge amount of theory for one lesson but these are really essential concepts that you need to understand for the exam.

So go ahead and complete this lesson and when you're ready I look forward to you joining me in the next.

Welcome back and in this lesson I want to cover two features of EC2, launch configurations and launch templates.

Now they both perform a similar thing, but launch templates came after launch configurations and include extra features and capabilities.

Now I want this lesson to be fairly brief, launch configurations and launch templates are actually relatively easy to understand.

What we're going to be covering in the next lesson is auto scaling groups which utilize either launch configurations or launch templates.

So I'll try to keep this lesson as focused as possible, but let's jump in and get started.

Launch configurations and launch templates at a high level perform the same task.

They allow the configuration of EC2 instances to be defined in advance.

Their documents which let you configure things like the AMI to use, the instance type and size, the configuration of the storage which instances use, and the key pair which is used to connect to that instance.

They also let you define the networking configuration and security groups that an instance uses.

They let you configure the user data which is provided to the instance and the IAM role which is attached to the instance used to provide the instance with permissions.

Everything which you usually define at the point of launching an instance, you can define in launch configurations and launch templates.

Now both of these are not editable.

You define them once and that configuration is locked.

Launch templates as the newer of the two allow you to have versions, but for launch configurations versions aren't available.

Launch templates also have additional features or allow you to control features of the newer types of instances.

Things like T2 or T3 unlimited CPU options, placement groups, capacity reservations, and things like elastic graphics.

AWS recommend using launch templates at this point in time because they're a super set of launch configuration.

They provide all of the features that launch configuration provides and more.

Architecturally, launch templates also offer more utility.

Launch configurations have one use.

They're used as part of auto scaling groups which we'll be talking about later in this section.

Auto scaling groups offer automatic scaling for EC2 instances and launch configurations provide the configuration of those EC2 instances which will be launched by auto scaling groups.

And as a reminder, they're not editable nor do they have any versioning capability.

If you need to adjust the configuration inside a launch configuration, you need to create a new one and use that new launch configuration.

Now launch templates, they can also be used for the same thing.

So providing EC2 configuration which is used within auto scaling groups.

But in addition, they can also be used to launch EC2 instances directly from the console or the CLI.

So good old Bob can define his instance configuration in advance and use that when launching EC2 instances.

Now you'll get the opportunity to create and use launch templates in the series of demo lessons later in this section.

For now, I just wanted to cover all of the theory back to back so you can appreciate how it all fits together.

That's everything though that I wanted to cover in this lesson about launch configurations and launch templates.

In the next lesson, I'll be talking about auto scaling groups which are closely related.

Both of them work together to allow EC2 instances to scale in response to the incoming load on a system.

But for now, go ahead and finish this video and when you're ready, I look forward to speaking to you in the next.

Welcome back and in this lesson, I want to cover application and network load balances in a little bit more detail.

It's critical for the exam that you understand when to pick application load balances and when to pick network load balances.

They're both used for massively different situations.

Now we do have a lot to cover, so let's jump in and get started.

I want to start by talking about consolidation of load balances.

Historically, when using classic load balances, you connected instances directly to the load balancer or you integrated an auto scaling group directly with that load balancer, an architecture which looked something like this.

So a single domain name, categor.io using a single classic load balancer and this has attached a single SSL certificate for that domain and then an auto scaling group is attached to that and the classic load balancer distributes connections over those instances.

The problem is that this doesn't scale because classic load balancers don't support SNI and you can't have multiple SSL certificates per load balancer and so every single unique HTTPS application that you have requires its own classic load balancer and this is one of the many reasons that classic load balancers should be avoided.

With this example, we have Catergram and Dogagram and both of these are HTTPS applications and the only way to use these is to have two different classic load balancers.

Compare this to the same application architecture so both of these applications, Catergram and Dogagram, only this time using a single application load balancer.

So this is handling both applications, Catergram and Dogagram.

This time we can use listener based rules and I'll talk about what these do later in the lesson but each of these listener based rules can have an SSL certificate handling HTTPS for both domains.

Then we can have host based rules which direct incoming connections at multiple target groups which forward these on to multiple auto scaling groups.

This is a two to one consolidation so we've halved the number of load balancers required to deliver these two different applications.

But imagine how this would look if we had a hundred legacy applications and each of these used a classic load balancer.

Moving from version one to version two offers significant advantages and one of those is consolidation.

So now I just want to focus on some of the key points about application load balancers.

So these are things which are specific to the version two or application load balancer.

First, it's a true layer seven load balancer and it's configured to listen on either HTTP or HTTPS protocols.

So these are layer seven application protocols and an application load balancer understands both of these and can interpret information carried using both of those protocols.

Now the flip side to this is that the application load balancer can't understand any other layer seven protocols.

So things such as SMTP, SSH or any custom gaming protocols are not supported by a layer seven load balancer such as the application load balancer and that's important to understand.

Now additionally, the application load balancer has to listen using HTTP or HTTPS listeners.

It cannot be configured to directly listen using TCP, UDP or TLS.

And that does have some important limitations and considerations that you need to be aware of.

And I'll talk about that later on in this lesson.

Now because it's a layer seven load balancer, it can understand layer seven content.

So things like the type of the content, any cookies which are used by your application, custom headers, user location and application behavior.

The layer seven load balancer, so the application load balancer is able to inspect all of the layer seven application protocol information and make decisions based on that information.

And that's something that the network load balancer cannot do.

It has to be a layer seven load balancer so the application load balancer to understand all of these individual components.

Now an important consideration about the application load balancer is that any incoming connections, so HTTP or HTTPS, and remember HTTPS is just HTTP, which is transiting using SSL or TLS.

In all of these cases, whichever type of connection is used, it's terminated on the application load balancer.

And this means that you can't have an unbroken SSL connection from your customer through to your application instances.

It's always terminated on the load balancer and then a new connection is made from the load balancer through to the application.

This is important because this is the type of thing that matters to security teams.

And if your business operates in a fairly strict security environment, then this might well be very important.

And in some cases, it can exclude using an application load balancer.

So it can't do end-to-end unbroken SSL encryption between a client and your application instances.

And it also means that all application load balancers which use HTTPS must have SSL certificates installed on that load balancer.

Because the connection has to be terminated on the load balancer and then a new connection made to the instances.

Now application load balancers are also slower than network load balancers because there are additional levels of the networking stack which need to be processed.

So the more levels of the networking stack which are involved, the more complexity the slower the processing.

So if you're facing any exam questions which are really strict on performance, then you might want to look at network load balancers rather than application load balancers.

A benefit though that application load balancers offer is because they're layer seven, then they can evaluate the application health at layer seven.

So in addition to just testing for a successful network connection, they can actually make an application layer request to the application to ensure that it's functioning correctly.

Now application load balancers also have the concept of rules and rules direct connections which arrive at a listener.

So if you make a connection to a load balancer, what the load balancer does with that connection is determined by any rules and rules are processed in priority order.

You can have many rules which might affect a given set of traffic and they're processed in priority order.

And the last one to be processed is the default rule which is a catch all.

But you can add additional rules and each of these can have conditions.

Now things that you can have inside the conditions of a rule include checking for things like host headers, HTTP headers, HTTP request methods, path patterns, query strings and even source IP.

So these rules could take different actions depending on which domain name you're asking for, category or dogogram.

They can perform different actions based on which path you're looking for.

So images or API, they can even take different decisions based on query string and even make different decisions based on the source IP address of any customers connecting to that application load balancer.

Now rules can also have actions.

These are the things that the rules do with the traffic.

So they can forward that traffic through to a target group.

They can redirect traffic at something else.

So maybe another domain name.

They can provide a fixed HTTP response, a certain error code or a certain success code and they can even perform certain types of authentication.

So using open ID or using Cognito.

Now this is how it looks visually.

This is a simple application load balancer deployment, a single domain, category.io.

We've got one host based rule with an attached SSL certificate and the rule is using host header as a condition and forward as an action.

So it's forwarding any connections for Cognito.io to the target group for the Cognito application.

But what if you want additional functionality?

Well, let's take a look.

First, let's imagine that we want to use the same application load balancer for a corporate client who's trying to access Cognito.io.

Maybe users of Bowtie Incorporated who use the 1.3.3.7 IP address are attempting to access our load balancer and we want to present them with an alternative version of the application.

Well, we can easily handle that by defining a listener rule but this time the condition will be the source IP address of 1.3.3.7.

Now this rule would have an action to forward traffic at a separate target group, an auto scaling group which handles a second set of instances dedicated for this corporate client because the application load balancer is a layer seven device.

It can see inside the Htt protocol and make decisions based on anything within that protocol or anything up to layer seven.

Now it's worth pointing out in addition that because this is a layer seven load balancer, the connection from the load balancer to the instances for target group two will be a separate set of connections.

And that's why it's in a slightly different color of purple.

The HTTP connection from our enterprise users are terminated on the load balancer and there's a separate set of connections through to our application instances.

There's no option to pass through the encrypted connection to the instances.

It has to be terminated.

Now this might not matter but it's something that you need to know for the exam.

If you have to forward encrypted connections through to the instances without terminating them on the load balancer, then you need to use a network load balancer.

Now because it's a layer seven load balancer, you can also use rules which work on layer seven elements of the protocol.

You could route based on paths or anything else in the HTTP protocol such as headers.

And you can also redirect traffic from a HTTP level.

An example, let's say that this ALB was also handling traffic for DogoGram.

Well, you could define a rule which matched the DogoGram.io domain name and as an action instead of forwarding, you could configure a redirect towards catagram.io, the obviously superior website.

And these are just a small subset of the features which are available within the application load balancer because it's layer seven, you can pretty much perform routing decisions based on anything which you can observe at layer seven and that makes it a really flexible product.

Before we finish this lesson, let's take a quick look at network load balancers.

Network load balancers function at layer four.

So there are layer four device which means that they can interpret TCP, TLS and UDP protocols as well as TCP and UDP.

But the flip side of this is that they have no visibility or understanding of HTTP or HTTPS.

And this means that they can't interpret headers, they can't see or interpret cookies and they've got no concept of session stickiness from a HTTP perspective because that uses cookies which the network load balancer cannot interpret because that's a layer seven entity.

Now network load balancers are really, really, really fast.

They can handle millions of requests per second and have around 25% of the latency of application load balancers.

And again, this is because they don't have to deal with any of the computationally heavy upper layers of the networking stack.

They only have to deal with layer four.

This also means that they're ideal to deal with any non-HTTP or HTTPS protocols.

So examples might be SMTP email, SSH, game servers which don't use either of the web protocols and any financial applications which are not HTTP or HTTPS.

So if you see any exam questions which talk about things which aren't web or secure web and don't use HTTP or HTTPS, then you should probably default to network load balancers.

One of the downsides of not being aware of layer seven is that health checks which are performed by network load balancers only check ICMP and basic TCP handshaking.

So they're not application aware.

You can't do detailed health checking with network load balancers.

A benefit of network load balancers is that it can be allocated with static IP addresses which is really useful for white listing if you have any corporate clients.

So corporate clients can decide to white list the IPs of network load balancers and allow them to progress straight through their firewall.

And this is great for any strict security environments that you need to operate in.

Another benefit is that they can forward TCP straight through to instances.

Now, if you're familiar with the networking stack, how this works is that upper layers build on layers below them.

So because the network load balancer doesn't understand HTTP or HTTPS, then you can configure a listener to accept TCP only traffic and then forward that through to instances.

And what that means is that any of the layers that are built on top of TCP are not terminated on the load balancer.

And so they're not interrupted.

And this means that you can forward unbroken channels of encryption directly from your clients through to your application instances.

And this is a really important thing to remember for the exam.

So network load balancers and TCP listeners is how you can do unbroken end-to-end encryption.

Network load balancers are also used for private link to provide services to other VPCs.

And this is another really important thing to remember for the exam.

Now, just to finish up this lesson, I want to do a quick comparison of a number of facts that you can use to decide between network load balancing and application load balancing.

And I find it easier to remember the things which you should be using a network load balancer for.

And then if the scenario is none of those, then you can default to using an application load balancer.

So let's step through the reasons why you might choose to use a network load balancer.

Well, the first one is the one we've just discussed.

If you want to perform unbroken encryption between a client and your instances, then use network load balancers.

If you need to use static IPs for white listing, then again, network load balancers.

If you want the absolute best performance, so millions of requests per second and low latency, then again, network load balancers.

If you need to operate on protocols which are not HTTP or not HTTPS, then you need to use network load balancers.

And then finally, if you have any requirement which involves private link, then you need to use network load balancers.

And for anything else, default to using application load balancers because the additional functionality provided by these devices is often really valuable to most scenarios.

Now, with that being said, that's everything I wanted to cover about application load balancers and network load balancers for the exam.

Go ahead and complete this video.

And when you're ready, I'll look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

This time, we have a typical multi-tiered application.

We start with a VPC and inside that two availability zones.

On the left, we also have an Internet Facing Low Balancer.

Then we have a Web Instance Auto Scaling Group providing the front-end capability of the application.

Then we have another Low Balancer, this time an internal Low Balancer, with only private IP addresses allocated to the nodes.

Next, we have an Auto Scaling Group for the application instances.

These are used by the web servers for the application.

Then on the right, we have a pair of database instances.

In this case, let's assume they're both Aurora database instances.

So we have three tiers, Web, Application and Database.

Now, without Load Balancers, everything would be tied to everything else.

Our user, Bob, would have to communicate with a specific instance in the web tier if this failed or scaled, then Bob's experience would be disrupted.

The instance that Bob is connected to would itself connect to a specific instance in the application tier, and if that instance failed or scaled, then again, Bob's experience would be disrupted.

What we can do to improve this architecture is to put Load Balancers between the application tiers to abstract one tier from another.

And how this changes things is that Bob actually communicates with an ELB node, and this ELB node sends this connection through to a particular web server.

But Bob has no knowledge of which web server he's actually connected to because he's communicating via a Load Balancer.

If instances are added or removed, then he would be unaware of this fact because he's abstracted away from the physical infrastructure by the Load Balancer.

Now, the web instance that Bob is using, it would need to communicate with an instance of the application tier, and it would do this via an internal Load Balancer.

And again, this represents an abstraction of communication.

So in this case, the web instance that Bob is connected to isn't aware of the physical deployment of the application tier.

It's not aware of how many instances exist, nor which one it's actually communicating with.

And then at this point, to complete this architecture, the application server that's being used would use the database tier for any persistent data storage needs.

Now, without using Load Balancers with this architecture, all the tiers are tightly coupled together.

They need an awareness of each other.

Bob would be connecting to a specific instance in the web tier.

This would be connecting to a specific instance in the application tier.

And all of these tiers would need to have an awareness of each other.

Load Balancers remove some of this coupling.

They loosen the coupling.

And this allows the tiers to operate independently of each other because of this abstraction.

Crucially, it allows the tiers to scale independently of each other.

In this case, for example, it means that if the load on the application tier increased beyond the ability of two instances to service that load, then the application tier could grow independently of anything else, in this case scaling from two to four instances.

The web tier could continue using it with no disruption or reconfiguration because it's abstracted away from the physical layout of this tier, because it's communicating via a Load Balancer.

It has no awareness of what's happening within the application tier.

Now, we're going to talk about these architectural implications in depth later on in this section of the course.

But for now, I want you to be aware of the architectural fundamentals.

And one other fundamental that I want you to be completely comfortable with is cross zone load balancing.

And this is a really essential feature to understand.

So let's look at an example visually.

Bob accessing a WordPress blog, in this case, The Best Cats.

And we can assume because this is a really popular and well-architected application that it's going to be using a load balancer.

So Bob uses his device and browsers to the DNS name for the application, which is actually the DNS name of the load balancer.

We know now that a load balancer by default has at least one node per availability zone that it's configured for.

So in this example, we have a cut down version of the Animals for Life VPC, which is using two availability zones.

So in this case, an application load balancer will have a minimum of two nodes, one in each availability zone.

And the DNS name for the load balancer will direct any incoming requests equally across all of the nodes of the load balancer.

So in this example, we have two nodes, one in each availability zone.

Each of these nodes will receive a portion of incoming requests based on how many nodes there are.

For two nodes, it means that each node gets 100% divided by two, which represents 50% of the load that's directed at each of the load balancer nodes.

Now, this is a simple example.

In production situations, you might have more availability zones being used, and at higher volume, so higher throughput, you might have more nodes in each availability zone.

But this example keeps things simple.

So however much incoming load is directed at the load balancer DNS name, each of the load balancer nodes will receive 50% of that load.

Now, originally load balancers were restricted in terms of how they could distribute the connections that they received.

Initially, the way that it worked is that each load balancer node could only distribute connections to instances within the same availability zone.

Now, this might sound logical, but consider this architecture where we have four instances in availability zone A and one instance in availability zone B.

This would mean that the load balancer node in availability zone A would split its incoming connections across all instances in that availability zone, which is four ways.

And the node in availability zone B would also split its connections up between all the instances in the same availability zone.

But because there's only one, that would mean 100% of its connections to the single EC2 instance.

Now, with this historic limitation, it means that node A would get 50% of the overall connections and would further split this down four ways, which means each instance would be allocated 12.5% of the overall load.

Node B would also receive 50% of the overall load.

And normally it would split that down across all instances also in that same availability zone.

But because there's only one, that one instance would get 100% of that 50%.

So all of the instances in availability zone A would receive 12.5% of the overall load and the instance in availability zone B would receive 50% of the overall load.

So this represents a substantially uneven distribution of the incoming load because of this historic limitation of how load balancer nodes could distribute traffic.

And the fix for that was a feature known as cross zone load balancing.

Now, the name gives away what this does.

It simply allows every load balancer node to distribute any connections that it receives equally across all registered instances in all availability zones.

So in this case, it would mean that the node in availability zone A could distribute connections to the instance in AZB and the node in AZB could distribute connections to instances in AZA.

And this represents a much more even distribution of incoming load.

And this is known as cross zone load balancing, the ability to distribute or load balance across availability zones.

Now, this is a feature which originally was not enabled by default.

But if you're deploying an application load balancer, this comes enabled as standard.

But you still need to be aware of it for the exam because it's often posed as a question where you have a problem, an uneven distribution of load, and you need to fix it by knowing that this feature exists.

So it's really important that you understand it for the exam.

So before we finish up with this lesson, I just want to reconfirm the most important architectural points about elastic load balancers.

If there are only a few things that you take away from this lesson, these are the really important points.

Firstly, when you provision an elastic load balancer, you see it as one device which runs in two or more availability zones, specifically one subnet in each of those availability zones.

But what you're actually creating is one elastic load balancer node in one subnet in each availability zone that that load balancer is configured in.

You're also creating a DNS record for that load balancer which spreads the incoming requests over all of the active nodes for that load balancer.

Now you start with a certain number of nodes, let's say one node per availability zone, but it will scale automatically if additional load is placed on that load balancer.

Remember by default, cross-own load balancing means that nodes can distribute requests across to other availability zones, but historically this was disabled, meaning connections potentially would be relatively imbalanced.

But for application load balancers, cross-own load balancing is enabled by default.

Now load balancers come in two types.

Internet facing, which just means that the nodes are allocated with public IP version 4 addresses.

That's it.

It doesn't change where the load balancer is placed, it just influences the IP addressing for the nodes of that load balancer.

Internal load balancers are the same, only their nodes are only allocated private IP addresses.

Now one of the most important things to remember about load balancers is that an internet facing load balancer can communicate with public instances or private instances.

EC2 instances don't need public IP addressing to work with an internet facing load balancer.

An internet facing load balancer has public IP addresses on its nodes, it can accept connections from the public internet and balance these across both public and private EC2 instances.

That's really important to understand for the exam, so you don't actually need public instances to utilize an internet facing load balancer.

Now load balancers are configured via listener configuration, which as the name suggests controls what those load balancers listen to.

And again, I'll be covering this in much more detail later on in this section of the course.

And then lastly, remember the confusing part about load balancers.

They require eight or more free IP addresses per subnet that they get deployed into.

Strictly speaking, this means that a /28 subnet would be enough, but the AWS documentation suggests a /27 in order to allow scaling.

For now, that's everything that I wanted to cover, so go ahead and complete this lesson.

And then when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this lesson, I want to talk about the architecture of elastic load balancers.

Now I'm going to be covering load balancers extensively in this part of the course.

So I want to use this lesson as a sort of foundation.

I'm going to cover the high level logical and physical architecture of the product and either refresh your memory on some things or introduce some of the finer points of load balancing for the first time.

And both of these are fine.

Now, before we start, it's the job of a load balancer to accept connections from customers and then to distribute those connections across any registered backend compute.

It means that the user is abstracted away from the physical infrastructure.

It means that the amount of infrastructure can change.

So increase or decrease in number without affecting customers.

And because the physical infrastructure is abstracted, it means that infrastructure can fail and be repaired, all of which is hidden from customers.

So with that quick refresher done, let's jump in and get started covering the architecture of elastic load balancers.

Now I'm going to be stepping through some of the key architectural points visually.

So let's start off with a VPC, which uses two availability zones, AZA and AZB.

And then in those availability zones, we've got a few subnets, two public and some private.

Now let's add a user, Bob, together with a pair of load balancers.

Now, as I just mentioned, it's the job of a load balancer to accept connections from a user base and then distribute those connections to backend services.

For this example, we're going to assume that those services are long running compute or EC2, but as you'll see later in this section, that doesn't have to be the case.

Elastic load balancers, specifically application load balancers, support many different types of compute services.

It's not only EC2.

Now, when you provision a load balancer, you have to decide on a few important configuration items.

The first, you need to pick whether you want to use IP version four only or dual stack.

And dual stack just means using IP version four and the newer IP version six.

You also need to pick the availability zones which the load balancer will use, specifically you're picking one subnet in two or more availability zones.

Now, this is really important because this leads in to the architecture of elastic load balancers, so how they actually work.

Based on the subnets that you pick inside availability zones, when you provision a load balancer, the product places into these subnets one or more load balancer nodes.

So what you see as a single load balancer object is actually made up of multiple nodes and these nodes live within the subnets that you pick.

So when you're provisioning a load balancer, you need to select which availability zones it goes into.

And the way you do this is by picking one and one only, subnet in each of those availability zones.

So in the example that's on screen now, I've picked to use the public subnet in availability zone A and availability zone B and so the product has deployed one or more load balancer nodes into each of those subnets.

Now when a load balancer is created, it actually gets created with a single DNS record.

It's an A record and this A record actually points at all of the elastic load balancer nodes that get created with the product.

So any connections that are made using the DNS name of the load balancer are actually made to the nodes of that load balancer.

The DNS name resolves to all of the individual nodes.

It means that any incoming requests are distributed equally across all of the nodes of the load balancer and these nodes are located in multiple availability zones and they scale within that availability zone.

And so they're highly available.

If one node fails, it's replaced.

If the incoming load to the load balancer increases, then additional nodes are provisioned inside each of the subnets that the load balancer is configured to use.

Now another choice that you need to make when creating a load balancer, and this is really important for the exam, is to decide whether that load balancer should be internet facing or whether it should be internal.

This choice, so whether to use internet facing or internal, controls the IP addressing for the load balancer nodes.

If you pick internet facing, then the nodes of that load balancer are given public addresses and private addresses.

If you pick internal, then the nodes only have private IP addresses.

So that's the only difference.

Otherwise, they're the same architecturally, they have the same nodes and the same load balancer features.

The only difference between internet facing and internal is whether the nodes are allocated public IP addresses.

Now the connections from our customers which arrive at the load balancer nodes, the configuration of how that's handled is done using a listener configuration.

As the name suggests, this configuration controls what the load balancer is listening to.

So what protocols and ports will be accepted at the listener or front side of the load balancer?

Now there's a dedicated lesson coming up later in this section which focuses specifically on the listener configuration.

At this point, I just wanted to introduce it.

So at this point, Bob has initiated connections to the DNS name associated with the load balancer.

And that means that he's made connections to load balancer nodes within our architecture.

Now at this point, the load balancer nodes can then make connections to instances that are registered with this load balancer.

And the load balancer doesn't care about whether those instances are public EC2 instances, so allocated with a public IP address or their private EC2 instances.

So instances which reside in a private subnet and only have private addressing.

I want to keep reiterating this because it's often a point of confusion for students who are new to load balancers.

An internet-facing load balancer, and remember this means that it has nodes that have public addresses so it can be connected to from the public internet, it can connect both to public and private EC2 instances.

Instances that are used do not have to be public.

Now this matters because in the exam when you face certain questions which talk about how many subnets or how many tiers are required for an application, it does test your knowledge that an internet-facing load balancer does not need private or public instances.

It can work with both of those.

The only requirement is that load balancer nodes can communicate with the back-end instances.

And this can happen whether the instances have allocated public addressing or whether they're private only instances.

The important thing is that if you want a load balancer to be reachable from the public internet, it has to be an internet-facing load balancer because logically it needs to be allocated with public addressing.

Now load balancers in order to function need eight or more free IP addresses in the subnets that they're deployed into.

Now strictly speaking, this means a /28 subnet, which provides a total of 16 IP addresses but minus the five reserved by AWS, this leaves 11 free per subnet.

But AWS suggests that you use a /27 or larger subnet to deploy an elastic load balancer in order that it can scale.

Keep in mind that strictly speaking, both a /28 and /27 subnets are both correct in their own ways to represent the minimum subnet size for a load balancer.

AWS do suggest in their documentation that you need a /27, but they also say you need a minimum of eight free IP addresses.

Now logically, a /28, which leaves 11 free, won't give you the room to deploy a load balancer and back end instances.

So in most cases, I try to remember /27 as the correct value for the minimum for a load balancer.

But if you do see any questions which show a /28 and don't show a /27, then /28 is probably the right answer.

Now internal load balancers are architecturally just like internet facing load balancers, except they only have private IPs allocated to their nodes.

And so internal load balancers are generally used to separate different tiers of applications.

So in this example, our user Bob connects via the internet facing load balancer to the web server.

And then this web server can connect to an application server via an internal load balancer.

And this allows us to separate application tiers and allow for independent scaling.

So let's look at this visually.

Okay, so this is the end of part one of this lesson.

It was getting a little bit on the long side and I wanted to give you the opportunity to take a small break, maybe stretch your legs or make a coffee.

Now part two will continue immediately from this point.

So go ahead, complete this video.

And when you're ready, I'll look forward to you joining me in part two.

Welcome back and in this lesson I want to spend a few minutes just covering the evolution at the Elastic Load Balancer product.

It's important for the exam and real world usage that you understand its heritage and its current state.

Now this is going to be a super quick lesson because most of the detail I'm going to be covering in dedicated lessons which are coming up next in this section of the course.

So let's jump in and take a look.

Now there are currently three different types of Elastic Load Balancers available within AWS.

If you see the term ELB or Elastic Load Balancers then it refers to the whole family, all three of them.

Now the load balancers are split between version 1 and version 2.

You should avoid using the version 1 load balancer at this point and aim to migrate off them onto version 2 products which should be preferred for any new deployments.

There are no scenarios at this point where you would choose to use a version 1 load balancer versus one of the version 2 types.

Now the load balancer product started with the classic load balancer known as CLB which is the only version 1 load balancer and this was introduced in 2009.

So it's one of the older AWS products.

Now classic load balancers can load balance HTTP and HTTPS as well as lower level protocols but they aren't really layer 7 devices.

They don't really understand HTTP and they can't make decisions based on HTTP protocol features.

They lack much of the advanced functionality of the version 2 load balancers and they can be significantly more expensive to use.

One common limitation is that classic load balancers only support one SSL certificate per load balancer which means for larger deployments you might need hundreds or thousands of classic load balancers and these could be consolidated down to a single version 2 load balancer.

So I can't stress this enough for any questions or any real world situations you should default to not using classic load balancers.

Now this brings me on to the new version 2 load balancers.

The first is the application load balancer or ALB and these are truly layer 7 devices so application layer devices.

They support HTTP, HTTPS and the web socket protocols.

They're generally the type of load balancer that you'd pick for any scenarios which use any of these protocols.

There's also network load balancers or NLBs which are also version 2 devices but these support TCP, TLS which is a secure form of TCP and UDP protocols.

So network load balancers are the type of load balancer that you would pick for any applications which don't use HTTP or HTTPS.

For example if you wanted to load balance email servers or SSH servers or a game which used a custom protocol so didn't use HTTP or HTTPS then you would use a network load balancer.

In general version 2 load balancers are faster and support target groups and rules which allow you to use a single load balancer for multiple things or handle the load balancing different based on which customers are using it.

Now I'm going to be covering the capabilities of each of the version 2 load balancers separately as well as talking about rules but I wanted to introduce them now as a feature.

Now for the exam you really need to be able to pick between network load balancers or application load balancers for a specific situation so that's what I want to work on over the coming lessons.

For now though this has just been an introduction lesson that talks about the evolution of these products and that's everything that I wanted to cover in this lesson so go ahead complete lesson and when you're ready I'll look forward to you joining me in the next.

Welcome back.

In this lesson, I want to talk about the regional and global AWS architecture.

So let's jump in and get started.

Now throughout this lesson, I want you to think about an application that you're familiar with, which is global.

And for this example, I'll be talking about Netflix, because this is an application that most people have at least heard of.

Now, Netflix can be thought of as a global application, but it's also a collection of smaller regional applications which make up the Netflix global platform.

So these are discrete blocks of infrastructure which operate independently and duplicated across different regions around the world.

As a solutions architect, when we're designing solutions, I find that there are three main types of architectures.

Small scale architectures which will only ever exist in one region or one country.

Then we have systems which also exist in one region or country, but where there's a DR requirement, so if that region fails for some reason, then it fails over to a second region.

And then lastly, we have systems that operate within multiple regions and need to operate through failure in one or more of those regions.

Now, depending on how you architect systems, there are a few major architectural components which will map on to AWS products and services.

So at a global level, first we have global service location and discovery.

So when you type Netflix.com into your browser, what happens?

How does your machine discover where to point at?

Next, we've got content delivery.

So how does the content or data for an application get to users globally?

Are their pockets of storage distributed globally or is it pulled from a central location?

Lastly, we've got global health checks and failover.

So detecting if infrastructure in one location is healthy or not and moving customers to another country as required.

So these are the global components.

Next, we have regional components starting with the regional entry point.

And then we have regional scaling and regional resilience and then the various application services and components.

So as we go through the rest of the course, we're going to be looking at specific architectures.

And as we do, I want you to think about them in terms of global and regional components, which parts can be used for global resilience and which parts are local only.

So let's take a look at this visually starting with the global elements.

So let's keep using Netflix as an example.

And let's say that we have a group of users who are starting to settle down for the evening and want to watch the latest episode of Ozarks.

So the Netflix client will use DNS for the initial service discovery.

Netflix will have configured the DNS to point at one or more service endpoints.

Let's keep things simple at this point and assume that there is a primary location for Netflix in a US region of AWS, maybe US East One.

And this will be used as the primary location.

And if this fails, then Australia will be used as a secondary.

Now, another valid configuration would be to send customers to their nearest location, in this case, sending our TV fans to Australia.

But in this case, let's just assume we have a primary and a secondary region.

So this is the DNS component of this architecture and Route 53 is the implementation within AWS.

Now, because of its flexibility, it can be configured to work in any number of ways.

The key thing for this global architecture, though, is that it has health checks.

So it can determine if the US region is healthy and direct all sessions to the US while this is the case, or direct sessions to Australia if there are problems with the primary region.

Now, regardless of where infrastructure is located, a content delivery network can be used at the global level.

This ensures that content is cached locally as close to customers as possible, and these cache locations are located globally, and they all pull content from the origin location as required.

So just to pause here briefly, this is a global perspective.

The function of the architecture at this level is to get customers through to a suitable infrastructure location, making sure any regional failures are isolated and sessions moved to alternative regions.

It attempts to direct customers at a local region, at least if the business has multiple locations, and lastly, it attempts to improve caching using a content delivery network such as CloudFront.

If this part of our architecture works well, customers will be directed towards a region that has infrastructure for our application, and let's assume this region is one of the US ones.

At this point, the traffic is entering one specific region of the AWS infrastructure.

Depending on the architecture, this might be entering into a VPC or using public space AWS services, but in either case, now we're architecturally zoomed in, and so we have to think about this architecture now in a regional sense.

The most effective way to think about systems architecture is a collection of regions making up a whole.

If you think about AWS products and services, very few of them are actually global.

Most of them run in a region, and many of those regions make up AWS.

Now it's efficient to think in this way, and it makes designing a large platform much easier.

For the remainder of this course, we're going to be covering architecture in depth, so how things work, how things integrate, and what features products provide.

Now the environments that you will design will generally have different tiers, and tiers in this context are high level groupings of functionality or different zones of your application.

Initially, communications from your customers will generally enter at the web tier.

Generally, this will be a regional based AWS service such as an application load balancer or API gateway, depending on the architecture that the application uses.

The purpose of the web tier is to act as an entry point for your regional based applications or application components.

It abstracts your customers away from the underlying infrastructure.

It means that the infrastructure behind it can scale or fail or change without impacting customers.

Now the functionality provided to the customer via the web tier is provided by the compute tier, using services such as EC2, Lambda, or containers which use the elastic container service.

So in this example, the load balancer will use EC2 to provide compute services through to our customers.

Now we'll talk throughout the course about the various different types of compute services which you can and should use for a given situation.

The compute tier though will consume storage services, another part of all AWS architectures, and this tier will use services such as EBS, which is the elastic block store, EFS, which is the elastic file system, and even S3 for things like media storage.

You'll also find that many global architectures utilize CloudFront, the global content delivery network within AWS, and CloudFront is capable of using S3 as an origin for media.

So Netflix might store movies and TV shows on S3 and these will be cached by CloudFront.

Now all of these tiers are separate components of an application and can consume services from each other and so CloudFront can directly access S3 in this case to fetch content for delivery to a global audience.

Now in addition to file storage, most environments require data storage and within AWS this is delivered using products like RDS, Aurora, DynamoDB and Redshift for data warehousing.

But in order to improve performance, most applications don't directly access the database.

Instead, they go via a caching layer, so products like ElastiCache for general caching or DynamoDB Accelerator known as DAX when using DynamoDB.

This way, reads to the database can be minimized.

Applications will instead consult the cache first and only if the data isn't present in the cache will the database be consulted and the contents of the cache updated.

Now caching is generally in memory, so it's cheap and fast.

Databases tend to be expensive based on the volume of data required versus cache and normal data storage.

So where possible, you need to offload reads from the database into the caching layer to improve performance and reduce costs.

Now lastly, AWS have a suite of products designed specifically to provide application services.

So things like Kinesis, Step Functions, SQS and SNS, all of which provide some type of functionality to applications, either simple functionality like email or notifications, or functionality which can change an application's architecture such as when you decouple components using queues.

Now as I mentioned at the start of this lesson, you're going to be learning about all of these components and how you can use them together to build platforms.

For now, just think of this as an introduction lesson.

I want you to get used to thinking of architectures from a global and regional perspective as well as understanding that application architecture is generally built using components from all of these different tiers.

So the web tier, the compute tier, caching, storage, the database tier and application services.

Now at this point, that's all of the theory that I wanted to go through.

Remember, this is just an introduction lesson.

So go ahead, finish this lesson and when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this video, I want to talk at a high level about the AWS Backup product.

Now, this is something that you need to have an awareness of for most of the AWS exams and to get started in the real world.

But it's not something that you need to understand in depth for all of the AWS certifications.

So let's jump in and cover the important points of the product.

So AWS Backup is a fully managed data protection service.

At this level of study, you can think about it as a backup and restore product, but it also includes much more in the way of auditing and management oversight.

The product allows you to consolidate the management and storage of your backups in one place, across multiple accounts and multiple regions if you configure it that way.

So that's important to understand the product is capable of being configured to operate across multiple accounts.

So this utilizes services like Control Tower and organizations to allow this.

And it's also capable of copying data between regions to provide extra data protection.

But the main day-to-day benefit that the product provides is this consolidation of management and storage within one place.

So instead of having to configure backups of RDS in one place, DynamoDB in another, and organize some kind of script to take regular EBS snapshots, AWS Backup can do all this on your behalf.

Now, AWS Backup is capable of interacting with a wide range of AWS products.

So many AWS services are fully supported, various compute services, so EC2 and VMware running within AWS, block storage such as EBS, file storage products such as EFS and the various different types of FSX, and then most of the AWS database products are supported such as Aurora, RDS, Neptune, DynamoDB and DocumentDB.

And then even object storage is supported using S3.

Now, all of these products can be centrally managed by AWS Backup, which means both the storage and the configuration of how the backup and retention operates.

Now, let's step through some of the key concepts and components of the AWS Backup product.

First, we have one of the central components, and that's backup plans.

It's on these where you can configure the frequency of backups, so how often backups are going to occur every hour, every 12 hours, daily, weekly or monthly.

You can also use a chron expression that creates snapshots as frequently as hourly.

Now, if you have any business backup experience, you might recognize this.

If you select weekly, you can specify which days of the week you want backups to be taken, and if you specify monthly, you can choose a specific day of the month.

Now, you can also enable continuous backups for certain supported products, and this allows you to use a point-in-time restore feature.

So if you've enabled continuous backups, then you can restore a supported service to a particular point in time within a window.

Now, you can configure the backup window as well within backup plans, so this controls the time that backups begin and the duration of that backup window.

You can configure life cycles, which define when a backup is transitioned to cold storage and when it expires.

When you transition a backup into cold storage, it needs to be stored there for a minimum of 90 days.

Backup plans also set the vault to use, and more on this in a second, and they allow you to configure region copy, so you can copy backups from one region to another.

Next, we have backup resources, and these are logically what is being backed up.

So whether you want to back up an S3 bucket or an RDS database, that's what a resource is, what resources you want to back up.

Next, we have vaults, and you can think of vaults as the destination for backups.

It's here where all the backup data is stored, and you need to configure at least one of these.

Now, vaults by default are read and write, meaning that backups can be deleted, but you can also enable AWS backup vault lock, and this is not to be confused by glacier or object locking.

AWS backup vault lock enables a write once read many, known as worm mode, for the vault.

Once enabled, you get a 72-hour cool-off period, but once fully active, nobody, including AWS, can delete anything from the vault, and this is designed for compliance-style situations.

Now, any data retention periods that you set still apply, so backups can age out, but setting this means that it's not possible to bypass or delete anything early, and the product is also capable of performing on-demand backups as required, so you're not limited to only using backup plans.

Some services also support a point-in-time recovery method, and examples of this include S3 and RDS, and this means that you can restore to the state of that resource at a specific date and time within the retention window.

Now, with all of these features, the product is constantly evolving, and rather than have this video be out of date the second something changes, I've attached a few links which detail the current state of many of these features, and I'd encourage you to take a look when you want to understand the products up-to-date capabilities when you're watching this video.

Now, this is all you need to understand as a base foundation for AWS Backup for all of the AWS exams.

If you need additional knowledge, so more theory detail in general, perhaps more specialized deep-dive knowledge on the security elements of the product, or maybe some practical knowledge, then there will be additional videos.

These will only be present if you need this additional knowledge for the particular course that you're studying.

If you only see this video, don't worry, it just means that this is all you need to know.

At this point, though, that is everything I wanted to cover, so go ahead and complete this video, and when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this demo lesson you're going to experience the difference that EFS can make to our WordPress application architecture.

Now this demo lesson has three main components.

First we're going to deploy some infrastructure automatically using the one-click deployments.

Then I'm going to step through the CloudFormation template and explain exactly how this architecture is built.

And then right at the end you're going to have the opportunity to see exactly what benefits EFS provides.

So to get started make sure that you're currently logged in to the general AWS account, so the management account of the organization, and as always you need to have the Northern Virginia region selected.

Now this lesson actually has two one-click deployments.

The first deploys the base infrastructure and the second deploys a WordPress EC2 instance, which has been enhanced to utilize EFS.

So you need to apply both of these templates in order and wait for the first one to finish before applying the second.

So we're going to start with the base VPC RDS EFS template first.

So this deploys the base VPC, the Elastic File System and an RDS instance.

Now everything should be pre-populated.

The stack should be called EFS demo -vpc -rds -efs.

Just scroll all the way down to the bottom, check the capabilities box and click on create stack.

While that's going let's switch over to the CloudFormation template and just step through exactly what it does.

So this is the template that you're deploying using the one-click deployment.

It's deploying the Base Animals for Life VPC, an EFS file system as well as mount targets and an Aurora database cluster.

So if we just scroll down we can see all of the VPC and networking resources used by the Base Animals for Life VPC.

Continue scrolling down we'll see the subnets that this VPC contains IP version 6 information.

We'll see an RDS security group, a database subnet group.

We've got the database instance.

Then we've got an instance security group which controls access to all the resources in the VPC that we use that security group on.

Then we have a rule which allows anything with that security group attached to it to communicate with anything else.

We have a rule that the WordPress instance will use and note that this includes permissions on the Elastic File System.

Then we have the instance profile that that instance uses.

Then we have the CloudWatch agent configuration and this is all automated.

And if we just continue scrolling down here we can see the Elastic File System.

So we create an EFS file system and then we create a file system mount target in each application subnet.

So we've got mount target zero which is in application subnet A which is in US East 1A.

We've got mount target one which is an application subnet B which logically is in US East 1B.

And then finally target two which is in subnet app C which is in availability zone 1C.

So we create the VPC, the database and the Elastic File System in this first one click deployment.

Now we need this to be in a create complete state before we continue with the demo lessons.

So go ahead and pause the video, wait for this to move into a create complete status and then we can use the second one click deployment.

Okay that stacks now finished creating which means we can move on to the second one click deployment.

Now there are actually two WordPress one click deployments which are attached to this lesson.

We're going to use them both but for now I want you to use the WordPress one one click deployment.

So go ahead and click on that link this will create a stack called EFS demo hyphen WordPress one.

Everything should be pre-populated just go ahead and click on create stack.

Now this is going to use the infrastructure provided by that first one click deployment.

So it's going to use EFS demo hyphen VPC hyphen RDS hyphen EFS and let's quickly step through exactly what this is doing while it's provisioning.

So this is the cloud formation template that is being used and we can skip past most of this.

What I want to focus on is the resource that's being created so that's WordPress EC2.

So this is using cross stack references to import a lot of the resources created in that first cloud formation stack.

So it's importing the instance profile to use it's importing the web a subnet so it knows where to place this instance.

And it's importing the instance security group that's created in that previous cloud formation stack.

Now in addition to this if we look through the user data for this WordPress instance one major difference is that it's mounting the EFS file system into this folder.

So forward slash var forward slash w w w forward slash HTML forward slash WP hyphen content.

Now if you remember from earlier demo lessons this is the folder which WordPress users to store its media.

So now instead of this folder being on the local EC2 file system this is now the EFS file system.

The EFS file system is mapped into this folder on this WordPress instance.

Other than that everything else is the same WordPress is installed.

It's configured to you as the RDS instance the cow say custom login banner is displayed.

It automatically configures the cloud watch agent and then it signals cloud formation that it's finished provisioning this instance.

Now what we'll end up with when this stack has finished creating is an EC2 instance which will use the services provided by this original stack.

So let's just refresh this.

It's still in progress so go ahead and pause the video and wait for this stack to move into a create complete state and then we good to continue.

So this stacks now finished creating and if we move across to the EC2 console so click on services locate EC2 right click and open that in a new tab.

Then click on instances running and you'll see that we have this A4L WordPress instance.

Now if we select that copy the IP address into your clipboard and then open that in a new tab we need to perform the WordPress installation.

So go ahead and enter the site title the best cats and add some exclamation points.

For username we need to use admin then for the password go back to the cloud formation stack and click on parameters and we're going to use the DB password.

So copy that into your clipboard then go back paste it into the password box and then put test at test.com for the email address and click install WordPress.

Then as before we need to log in so click on login admin for username reenter that password and click on login.

Then we need to go to posts we need to click on trash below hello world to delete that post then click on add new close down this dialogue.

For title put the best cats ever and some exclamation points then click on the plus click gallery click upload.

There's a link attached to this lesson with four cat images so go ahead and download that link and extract it locate those four images select them and click on open.

And then once you've done that click on publish and publish again and then click on view post.

Now what that's doing in the background is it's adding these images to the WP hyphen content folder which is on the EC two instance but now we have that folder mounted using EFS and so the images are being stored on the elastic file system rather than the local instance file system.

The cat pictures are there but what we're going to do to validate this is to go back to instances right click on this a four L hyphen WordPress instance and click on connect and then connect to this instance using EC two instance connect.

Now once we connected to the instance you CD space forward slash VAR forward slash WWW forward slash HTML and then do an LS space hyphen LA to do a full listing you'll see that we have this WP hyphen content folder.

So type CD space WP hyphen content and press enter then we'll clear the screen and do an LS space hyphen LA and then inside this folder we have plugins themes and uploads go into the uploads folder do an LS space hyphen LA depending on when you do this demo lesson you should see a folder representing the year so move into that folder then a folder representing the month again this will vary depending on when you do the demo lesson.

Move into that folder and then you should see all four of my cat images and if you do a DF space hyphen K you'll be able to see that this folder so forward slash VAR forward slash WWW forward slash HTML WP hyphen content this is actually mounted using EFS so this is an EFS file system.

Now this means the local instance file system is no longer critical it no longer stores the actual media that we upload to these posts so what we can do is we can go back to cloud formation go to stacks select the EFS demo hyphen WordPress one stack and then click on delete and delete that stack so that's going to terminate the EC two instance that we've just used to upload that media.

We need to wait for that stack to fully delete before continuing so go ahead and pause the video and wait for this stack to disappear so that stacks disappeared and now there's a second WordPress one click deployment link attached to this lesson remember there are two so now go ahead and click on the second one this one should create a stack called EFS demo hyphen WordPress two scroll to the bottom and click on create stack that's going to create a new stack and a new EC two instance.

So while we're doing this just close down all of these additional tabs at the top of the screen close them all down apart from the cloud formation one.

We're going to need to wait for this to finish provisioning and move into the create complete state so again pause the video wait for this to change into create complete and then we go to to continue.

After a few minutes the WordPress two stack has moved into a create complete state click on services open the EC two console in a new tab click on instances running you'll see a new A4L hyphen WordPress instance this is a brand new instance which has been provisioned using the one click deployment link that you've just used so the WordPress two one click deployment link.

If we select this copy the public IP address into your clipboard and open that in a new tab it again loads our WordPress blog if we open the blog post.

Now we can see these images because they're being loaded from EFS from the file system that EFS provides so no longer are we limited to only operating from a single EC two instance for our WordPress application because now there's nothing which gets stored specifically on that EC two instance.

Instead everything stored on EFS and accessible from any EC two instance that we decide to give permissions to know what we can do to demonstrate this if we go back to cloud formation.

Now remember attached to this lesson are two WordPress one click deployments we initially applied number one then we deleted that and applied number two so now I want you to reapply number one.

So again click on the WordPress one one click deployment this again will create a new stack this time called EFS demo hyphen WordPress one click on create stack you need to wait for this to move into a create complete state so pause the video and resume it once the stack changes to create complete after a few minutes this stack also moves into create complete.

Let's click on resources we can see it's provisioned a single EC two instance so let's click on this to move directly to this new instance select it copy this instance is IP address into your clipboard and open that in a new tab and again we have our WordPress blog and if we click on the post it loads those images so now we have a number of EC two instances we have to EC two instances both with WordPress installed both using the same RDS data.

And both using the shared file system provided by EFS and it means that if any posts are edited or any images uploaded on either of these two EC two instances then those updates will be reflected on all other EC two instances and this means that we've now implemented this architecture that's on screen now and this is what's going to support us when we evolve this architecture more and add scalability in an upcoming section of the core.

For now though we've just been focused on the shared file system now all that remains at this point is for us to tidy up the infrastructure that we've used in this demo lesson so close down all of these tabs we need to be at the cloud formation console we need to start by deleting EFS demo WordPress one and WordPress two so pick either of those click delete and then delete stack then select the other delete and then delete stack.

Now we need both of these to finish deleting and then we can delete this last stack so go ahead and pause the video wait for both of these to disappear and then we can resume both of those have deleted so now we can click the final stack EFS demo hyphen VPC hyphen RDS hyphen EFS so select that delete and then delete stack and that's everything that you need to do in this demo lesson and once that stacks finished deleting the account will be in the same state as it was at the start of this.

Now I hope you've enjoyed this demo lesson and that it's been useful what you've implemented in this demo is one more supportive step towards us moving this architecture from being a monolith through to being fully elastic.

Now the application is in this state where we have a single shared RDS database for all of our application instances and we're also using a shared file system provided by EFS and this means that we can have one single EC2 instance we could have two EC2 instances or even 200 all of them sharing the same database and the same shared file system provided by EFS.

Now in an upcoming section of this course we're going to extend this further by creating a launch template which automatically builds EC2 instances as part of this application architecture.

We're going to utilize auto scaling groups together with application load balancers to implement an architecture which is fully elastic and resilient and this has been one more supportive step towards that objective.

At this point though that's everything that you needed to do in this demo lesson so go ahead complete this video and when you're ready I look forward to you joining me in the next.

Welcome back.

In this lesson, I'm going to be covering a really useful product within AWS, the Elastic File System, or EFS.

It's a product which can prove useful for most AWS projects because it provides network-based file systems which can be mounted within Linux EC2 instances and used by multiple instances at once.

For the Animals for Life WordPress example that we've been using throughout the course so far, it will allow us to store the media for posts outside of the individual EC2 instances, which means that the media isn't lost when instances are added and removed, and that provides significant benefits in terms of scaling as well as self-healing architecture.

In summary, we're moving the EC2 instances to a point where they're closer to being stateless.

So let's jump in and step through the EFS architecture.

The EFS service is an AWS implementation of a fairly common shared storage standard called NFS, the Network File System, specifically version 4 of the Network File System.

With EFS, you create file systems which are the base entity of the product, and these file systems can be mounted within EC2 Linux instances.

Linux uses a tree structure for its file system.

Devices can be mounted into folders in that hierarchy, and EFS file system, for example, could be mounted into a folder called forward slash NFS forward slash media.

What's more impressive is that EFS file systems can be mounted on many EC2 instances, so the data on those file systems can be shared between lots of EC2 instances.

Now keep this in mind as we talk about evolving the architecture of the Animals for Life WordPress platform.

Remember, it has a limitation that the media for posts, so images, movies, audio, they're all stored on the local instance itself.

If the instance is lost, the media is also lost.

EFS storage exists separately from an EC2 instance, just like EBS exists separately from EC2.

Now EBS is block storage, whereas EFS is file storage, but Linux instances can mount EFS file systems as though they are connected directly to the instance.

EFS is a private service by default.

It's isolated to the VPC that it's provisioned into.

Architecturally, access to EFS file systems is via mount targets, which are things inside a VPC, but more on this next when we step through the architecture visually.

Now even though EFS is a private service, you can access EFS file systems via hybrid networking methods that we haven't covered yet, so if your VPC is connected to other networks, then EFS can be accessed over those.

So using VPC peering, VPN connections, or AWS Direct Connect, which is a physical private networking connection between a VPC and your existing on-premises networks.

Now don't worry about those hybrid products, I'll be covering all of them in detail later in the course.

For now though, just understand that EFS is accessible outside of a VPC using these hybrid networking products as long as you configure this access.

So let's look at the architecture of EFS visually.

Architecturally, this is how it looks.

EFS runs inside a VPC, in this case the Animals for Life VPC.

Inside EFS, you create file systems and these use POSIX permissions.

If you don't know what this is, I've included a link attached to the lesson which provides more information.

Super summarized though, it's a standard for interoperability which is used in Linux.

So a POSIX permissions file system is something that all Linux distributions will understand.

Now the EFS file system is made available inside a VPC via mount targets and these run from subnets inside the VPC.

The mount targets have IP addresses taken from the IP address range of the subnet that they're inside and to ensure high availability, you need to make sure that you put mount targets in multiple availability zones, just like NAT gateways for a fully highly available system, you need to have a mount target in every availability zone that a VPC uses.

Now it's these mount targets that instances use to connect to the EFS file systems.

Now it's also possible, as I touched on on the previous screen, that you might have an on-premises network and this generally would be connected to a VPC using hybrid networking products such as VPNs or Direct Connect and any Linux-based server that's running on this on-premises environment can use this hybrid networking to connect through to the same mount targets and access EFS file systems.

Now before we move on to a demo where you'll get the practical experience of creating a file system and accessing it from multiple EC2 instances, there are a few things about EFS which you should know for the exam.

First, EFS is for Linux-only instances.

From an official AWS perspective, it's only officially supported using Linux instances.

EFS offers two performance modes, general purpose and max IO.

General purpose is ideal for latency-sensitive use cases, web servers, content management systems, it can be used for home directories or even general file serving as long as you're using Linux instances.

Now general purpose is the default and that's what we'll be using in this section of the course within the demos.

Max IO that can scale to higher levels of aggregate throughput and operations per second but it does have a trade-off of increased latencies.

So max IO mode suits applications that are highly parallel.

So if you've got any applications or any generic workloads such as big data, media processing, scientific analysis, anything that's highly parallel then it can benefit from using max IO but for most use cases just go with general purpose.

There are also two different throughput modes, bursting and provisioned.

Bursting mode works like GP2 volumes inside EBS so it has a burst pool but the throughput of this type scales with the size of the file systems.

So the more data you store in the file system the better performance that you get.

With provisioned you can specify throughput requirements separately from size.

So this is like the comparison between GP2 and IO1.

With provisioned you can specify throughput requirements separate from the amount of data you store so that's more flexible but it's not the thing that's used by default.

Generally you should pick bursting.

Now for the exam you don't need to remember the raw numbers but I have linked some in the lesson description if you want additional information.

So you can see the different performance characteristics of all of these different options.

Now Amazon EFS file systems have two storage classes available.

We've got infrequent access or IA and that storage class is a lower cost storage class which is designed for storing things that are infrequently accessed.

So if you need to store data in a cost effective way but you don't intend to access it often then you can use infrequent access.

Next we've got standard and the standard storage class is used to store frequently accessed files.

It's also the default and you should consider it the default when picking between the different storage classes.

Conceptually these mirror the trade-offs of the S3 object storage classes.

Use standard for data which is used day to day and infrequent access for anything which isn't used on a consistent basis.

And just like S3 you have the ability to use life cycle policies which can be used to move data between classes.

Okay so that's the theory of EFS.

It's not all that difficult a product to understand but you do need to understand it architecturally for the exam and so to help with that it's now time for a demo.

I want you to really understand how EFS works.

It's something that you probably will use if you use AWS for any real world projects.

Now the best way to understand it is to use it and so that's what we're going to do in the next lesson which is a demo.

You're going to have the opportunity to create an EFS file system, provision some EC2 instances and then mount that file system within both EC2 instances, create a test file and see that that's accessible from both of those instances.

Proving that EFS is a shared network file system.

But at this point that's all of the theory that I wanted to cover so go ahead finish up this video and when you're ready I look forward to you joining me in the demo lesson.

Welcome back, this is part two of this lesson.

We're going to continue immediately from the end of part one, so let's get started.

Okay, so all three of these mount targets are now in an available state and that means we can connect into this EFS file system from any of the availability zones within the Animals for Life VPC.

So what we need to do is test out this process and we're going to interact with this file system from our EC2 instances.

So move back to the tab where we have the EC2 console open.

And at this point I want you to either, and this depends on your browser, I'll either want you to right click and duplicate this tab to open another identical copy.

If you can't do this in your browser then just open a new tab and copy and paste this URL into that tab.

You'll end up with two separate tabs open to the same EC2 screen.

So on the first tab we're going to connect to A4L-EFS instance A.

So right click and then select connect.

We're going to use instance connect.

So make sure the username is EC2-user and then click on connect.

Now right now this instance is not connected to this EFS file system and we can verify that by running a DF space-k and press enter.

You'll see that nowhere here is listed this EFS file system.

These are all volumes directly attached to the EC2 instance and of course the boot volume is provided by EBS.

Now within Linux all devices or all file systems are mounted into a folder.

So the first thing that we need to do to interact with EFS is to create a folder for the EFS file system to be mounted into.

And we can do that using this command so shudu space mkdir space-p space/efs/wp-content.

Now the hyphen p option just means that everything in this path will be created if it's not already.

So this will create forward/EFS if it doesn't already exist.

So press enter to create that folder.

So I'm going to clear the screen to keep this easy to see.

And the next thing I need to do is to install a package of tools which allows this instance or specifically the operating system to interact with the EFS product.

Now the command I'm going to use to install these tools is shudu to give us admin permissions and then DNF which is the package manager for this operating system.

And then a space hyphen y to automatically acknowledge any prompts and then a space and then install because I want to install a package and then a space.

And then the name of the tools that I want to install is amazon hyphen EFS hyphen utils.

So this is a set of tools which allows this operating system to interact with EFS.

So go ahead and press enter and that will install these tools and then we can configure the interaction between this operating system and EFS.

Again I'm going to clear the screen to keep this easy to see and I want to mount this EFS file system in that folder that we've just created.

But specifically I want it to mount every time the instance is restarted.

So of course that means we need to add it to the FSTAB file.

Now if you remember this file from elsewhere in the course it's contained within the forward/ETC folder.

So we need to move into that folder cd///ETC and then the file is called FSTAB.

So we need to run shudu to give us admin permissions and then nano which is a text editor and then the name of the file which is FSTAB.

So press enter and the file will likely have only one or two lines which is the root and/or boot volume of this instance.

So let's just move to the end because we're going to add a new line and this is contained within the lesson commands document but we're going to paste in this line.

So this line tells us that we want to mount this file system ID so file system ID colon forward/.

We want to mount that into this folder so forward/efs forward/wp-content.

We tell it that the file system type is EFS.

Remember EFS is actually based on NFS which is the network file system but this is one provided by AWS as a service and so we use a specific AWS file system which is EFS.

And the support for this has been installed by that tools package which we just installed.

Now the exact functionality of this is beyond the scope of this course but if you do want to research further then go ahead and investigate exactly what these options do.

What we need to do though is to point it at our specific EFS file system.

So this is this component of the line all the way from the start here to this forward/.

So to get the file system ID we need to go back to the EFS console and we need to copy down this full file system ID and yours will be different so make sure you copy your own file system ID into the clipboard.

Then go back here and select the colon and then delete all the way through to the start of this line.

And once you've done that paste in your file system ID what it should look like is the file system ID then a colon and then a forward/.

So at this point we need to save this file so control O to save and then enter and control X to exit.

Again I'm going to clear the screen to make it easier to see.

Then I'll run a DF space -K and this is what the file systems currently attached to this instance look like.

Then we're going to mount the EFS file system into the folder that we've created and the way that we do this is with this command.

So shudu mount and then we specify the name of the folder that we want to mount.

Now the way that this works is that this uses what we've just defined in the FSTAB file.

So we're going to mount into this folder whatever file system is defined in that file.

So that's the EFS file system and if we press enter after a few moments it should return back to the prompt and that's mounted that file system.

There we go we back at the prompt and if we do a DF space -K again we'll see that now we've got this extra line at the bottom.

So this is the EFS file system mounted into this folder.

Now to show you that this is in fact a network file system let's go ahead and move into that folder using this command.

And now that we're in that folder we're going to create a file.

So we're going to use shudu so that you have admin privileges and then we're going to use the command touch which if you remember from earlier in the course just creates an empty file.

And we're going to call this file amazing test file dot txt.

Go ahead and press enter and then do an LS space -LA and you'll see that we now have this file created within this folder.

And while we're creating it on this EC2 instance it's actually put this file on a network file system.

Now to verify that let's move back to the other tab that we have open to the EC2 console the one that's still on this running instances screen.

And now let's go ahead and connect to instance B.

So right click on instance B select connect again instance connect verify the username is as it should be and click on connect.

So now we're on instance B.

Let's do a DF space -K to verify that we don't currently have any EFS file system mounted.

Next we need to install the EFS tools package so that we can mount this file system.

So let's go ahead and install that package clear the screen to make it easier to see then we need to create the folder that we're going to be mounting this file system into.

We'll use the same command as on instance A.

Then we need to edit the FSTAB file to add this file system configuration.

So we'll do that using this command so shudu space nano space forward slash ETC forward slash FSTAB press enter.

Remember this is instance B so it won't have the line that we added on instance A.

So we need to go down to the bottom paste in this placeholder and then we need to replace the file system ID at the start with the actual file system ID.

So delete this leaving the colon and forward slash go back to the EFS console copy the file system ID into your clipboard.

Move back to this instance paste that in everything looks good.

Save that file with control O press enter exit with control X then we back at the prompt clear the screen.

We'll use the shudu mount forward slash EFS forward slash WP hyphen content command again to mount the EFS file system onto this instance and again it's using the configuration that we've just defined in the FSTAB file press enter.

After a few moments you'll be placed back at the prompt we can verify whether this is mounted with DF space hyphen K.

It has mounted by the looks of things it's at the bottom.

So now if we move into that folder so CD forward slash EFS forward slash WP hyphen content forward slash and press enter.

We now in that folder and if we do a listing so LS space hyphen LA what we'll see is the amazing test file dot txt which was created on instance A.

So this proves that this is a shared network file system where any files added on one instance are visible to all other instances.

So EFS is a multi user network based file system that can be mounted on both EC2 Linux instances as well as on premises physical or virtual servers running Linux.

Now this is a simple example of how to use EFS for now we've done everything that we need to do in this demo lesson so we just need to clean up all of the infrastructure that we've used to do that.

Go back to the EFS console we're going to go ahead and delete this file system so we should already have it selected just select delete you'll need to confirm that process by pasting in the file system ID.

So go ahead and put your file system ID and then select confirm.

Now that can take some time to delete and you'll need to wait for this process to complete.

Once it has completed we're going to go ahead and move across to the cloud formation console.

You should still have this open in a tab if you don't just type cloud formation in the search box at the top and then move to the cloud formation console.

You should still have the stack name of implementing EFS which is the stack you created at the start with the one click deployment.

Go ahead and select this stack then click on delete and confirm that deletion and once that finishes deleting that's all of the infrastructure gone that we've created in this demo lesson.

So I hope this has been a fun and enjoyable demo lesson where you've gained some practical experience of working with EFS at this point though that is everything that you need to do in this demo lesson.

So go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this demo lesson I want to give you some abstract practical experience of using the Elastic File System or EFS.

Now we're going to need some infrastructure.

Before we apply that as always make sure that you're logged into the general AWS account, so the management account of the organization and you'll need the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment link so go ahead and click that.

This is going to provision some infrastructure.

It's going to take you to the quick create stack screen and everything should be pre-populated.

You'll just need to scroll to the bottom, check the box beneath capabilities and then click on create stack.

You're also going to be typing some commands within this demo lesson so also attached to this lesson is a lesson commands document.

Go ahead and open that in a new tab.

So this is just a list of the commands that we're going to be using during the demo lesson and there are some placeholders such as file system ID that you'll need to replace as we go but make sure you've got this open for reference.

Now we're going to need this stack to be in a create complete state before we continue with the demo lesson so go ahead pause the video and resume it once your stack moves into a create complete state.

Okay so the stacks now moved into a create complete status and what this has actually done is create the animals for life base VPC as well as a number of EC2 instances.

So if we go to the EC2 console and click on instances running you'll note that we've created a for L - EFS instance A and a for L - EFS instance B and we're going to be creating an EFS file system and mount points and then mounting that on both of these instances and interacting with the data stored on that file system.

We're going to get you the experience of working with a network shared file system so let's go ahead and do that.

So to get started we need to move to the EFS console so in the search box at the top just type EFS and then open that in a brand new tab.

We're going to leave this tab open to the instances part of the EC2 console because we're going to come back to this very shortly.

So let's move across to the EFS console that we have open in a separate tab and the first step is to create a file system so a file system is the base entity of the elastic file system product and that's what we're going to create.

Now you've got two options for setting up an EFS file system you can use this simple dialogue or you can click on customize to customize it further.

So if we're using the simple dialogue we'd start by naming the file system so let's say we use A4L - EFS and then you'd need to pick a VPC for this file system to be provisioned into and of course we'd want to select the animals for life VPC.

Now we want to customize this further we don't want to just accept these high-level defaults so we need to click on customize.

This is going to move us to this user interface which has many more options so we've still got the A4L - EFS name for this file system.

Now for the storage class we're going to pick standard which means the data is replicated across multiple availability zones.

If you're doing this in a test or development environment or you're storing data which is not important then you can choose to use one zone which stores data redundantly but only within a single AZ.

Now again in this demonstration we are going to be using multiple availability zones so make sure that you pick standard for storage class.

You're able to configure automatic backups of this file system using AWS backup and if you're taking an appropriate certification course this is something which I'll be covering in much more detail.

You can either enable this or disable it obviously for a production usage you'd want to enable it but for this demonstration we're going to disable it.

Now EFS as I mentioned in the theory lesson comes with different classes of storage and you can configure lifecycle management to move files between those different storage classes so if you want to configure lifecycle management to move any files not accessed for 30 days you can move those into the infrequent access storage class and you can also transition out of infrequent access when anything is accessed so go ahead and select on first access for transition out of IA.

So in many ways this is like S3 with the different classes of storage for different use cases.

When you're creating a file system you're able to set different performance and throughput modes.

For throughput mode you can choose between bursting and enhanced.

If you pick enhanced you're able to select between elastic and provisioned.

I've talked more about these in the theory lesson.

We're going to pick bursting.

Now for performance you can choose between general purpose and max I/O.

General purpose is the default and rightfully so and you should use this for almost all situations.

Only use max I/O if you want to scale to really high levels of aggregate throughput and input output operations per second so only select it if you absolutely know that you need this option.

You've also got the ability to encrypt the data on the file system and if you do encrypt it it uses KMS and you need to pick a KMS key to use.

Of course this means that in order to interact with objects on this file system permissions are needed both on the EFS service itself as well as the KMS key that's used for the encryption operation.

Now this is something that you will absolutely need to use for production usage but for this demonstration we're going to switch it off.

We won't be setting any tags for this file system so let's go ahead and click on next.

You need to configure the network settings for this file system so specifically the mount targets that will be created to access this file system.

Now best practice is that any availability zones within a VPC where you're consuming the services provided by EFS you should be creating a mount target so in our case that's US - East - 1A, 1B and 1C.

So we're going to go through and configure this so first let's delete all of these default security group assignments.

Every mount target that you create will have an associated security group so we'll be setting these specifically.

For now though we need to choose the application subnet in each of these availability zones so in the top drop-down which is US - East - 1A I'm looking for app A so go ahead and do the same.

In US - East - 1B I want to select the app B subnet and then in US - East - 1C logically I'll be selecting the app C subnet so that's app A, app B and app C.

Now for security groups the CloudFormation 1 click deployment has provisioned this instance security group and by default this security group allows all connections from any entities which have this attached so this is a really easy way that we can allow our instances to connect to these mount targets so for each of these lines go ahead and select the instance security group you'll need to do that for each of the mount targets so we'll do the second one and then we'll do the third one and that's all of the network configuration options that we need to worry about so click on next it's here where you can define any policies on the file system so you can prevent root access by default you can enforce read only access by default you can prevent anonymous access or you can enforce encryption in transit for all clients connected to this EFS file system so any clients that connect to the mount targets to access the file system you can ensure that that uses encryption in transit and if you're using this in production you might want to select at least this last option to improve security for this demo lesson we're not going to use any of these policy options nor are we going to define a custom policy in the policy editor instead we'll just click on next at this point we just need to review everything's to our satisfaction everything looks good so we're going to scroll down to the bottom and just click on create now in order to continue with this demo lesson we're going to need both the file system and all of its mount targets so go into the file system click on network and you'll see three mount targets being created all three of these need to be ready before we can continue the demo lesson so this seems like a great time to end part one of this demo lesson go ahead and finish this video and then when all of these mount targets are ready to go you can start part two.

Welcome back and in this lesson I want to cover a service which starts to feature more and more on the exam the database migration service known as DMS.

Now this lesson is an extension of my lesson from the Associate Architect course so even if you've taken that course and watched that lesson you should still watch this lesson fully.

Now this product is something which as well as being on the exam if you're working as a Solutions Architect in the AWS space and if your projects involve databases you will extensively use this product it's something that you need to be aware of regardless so let's jump in and get started.

Database migrations are complex things to perform normally if we exclude the vendor tooling which is available it's a manual process end to end it usually involves setting up replication which is pretty complex or it means taking a point in time back up and restoring this to the destination database but how do you handle changes which occur between taking that back up and when the new database is live how do you handle migrations between different databases these are all things where DMS comes in handy it's essentially a managed database migration service the concept is simple enough it starts with a replication instance which runs on EC2 this instance runs one or more replication tasks you need to define a source and destination endpoints which point at the source and target databases and the only real restriction with the service is that one of the endpoints must be running within AWS you can't use the product for migrations between two on-premises databases now you don't actually need to have any experience using the product but there will be a demo lesson elsewhere in this section which gives you some practical exposure for this theory lesson though we need to focus on the architecture so let's continue by reviewing that visually using DMS is simple enough architecturally you start with a source and target database and one of those needs to be within AWS the databases themselves can use a range of compatible engines such as MySQL Aurora Microsoft SQL MariaDB MongoDB PostgreSQL Oracle Azure SQL and many more now in between these conceptually is the database migration service known as DMS which uses a replication instance essentially an EC2 instance with migration software and the ability to communicate with the DMS service now on this instance you can define replication tasks and each of these replication instances can run multiple replication tasks tasks define all of the options relating to the migration but architecturally two of the most important things are the source and destination endpoints which store the replication information so that the replication instance and task can access the source and target databases so a task essentially moves data from the source database using the details in the source endpoint to the target database using the details stored in the destination endpoint configuration and the value from DMS comes in how it handles those migrations now jobs can be one of three types we have full load migrations and these are used to migrate existing data so if you can afford an outage long enough to copy your existing data then this is a good one to choose this option simply migrates the data from your source database to your target database and it creates the tables as required next we have full load plus CDC and this stands for change data capture and this migrates existing data and replicates any ongoing changes this option performs a full load migration and at the same time it captures any changes occurring on the source after the full load migration is complete then captured changes are also applied to the target eventually the application of changes reaches a steady state and at this point you can shut down your applications let the remaining changes flow through to the target and then restart your applications and point them at the new target database finally we've got CDC only and this is designed to replicate only data changes in some situations it might be more efficient to copy existing data using a method other than AWS DMS also certain databases such as Oracle have their own export and import tools and in these cases it might be more efficient to use those tools to migrate the initial data and then use DMS simply to replicate the changes starting at the point when you do that initial bulk load so CDC only migrations are actually really effective if you need to bulk transfer the data in some way outside of DMS now lastly DMS doesn't natively support any form of schema conversion but there is a dedicated tool in AWS known as the schema conversion tool or SCT and the sole purpose of this tool is to perform schema modifications or schema conversions between different database versions or different database engines so this is a really powerful tool that often goes hand-in-hand with migrations which are being performed by DMS now DMS is a great tool for migrating databases from on-premises to AWS it's a tool that you will get to use for most larger database migrations so as a solutions architect it's another tool which you need to understand end-to-end in the exam if you see any form of database migration scenario as long as one of the databases is within AWS and as long as there are no weird databases involved which aren't supported by the product then you can default to using DMS it's always a safe default option for any database migration questions if the question talks about a no downtime migration then you absolutely should default to DMS now at this point let's talk in a little bit more detail about a few aspects of DMS which are important first I want to talk about the schema conversion tool or SCT in a little bit more detail so this is actually a standalone application which is only used when converting from one database engine to another it can be used as part of migrations where the engines being migrated from and to aren't compatible and another use case is that it can be used for larger migrations when you need to have an alternative way of moving data between on-premises and AWS rather than using a data link now SCT is not used and this is really important it's not used for movements of data between compatible database engines for example if you're performing a migration from an on-premises MySQL server to an AWS based RDS MySQL server then the engines are the same even though the products are different the engines are the same and so SCT would not be used SCT works with OLTP databases such as MySQL, Microsoft SQL and Oracle and also OLAP databases such as Teradata, Oracle, Vertica and even Green Plum now examples of the types of situations where the schema conversion tool would be used include things like on-premises Microsoft SQL through to AWS RDS MySQL migrations because the engine changes from Microsoft SQL to MySQL and then we could also use SCT for an on-premises Oracle to AWS based Aurora database migration again because the engines are changing now there is another type of situation where DMS can be used in combination with SCT and that's for larger migrations so DMS can often be involved with large-scale database migrations so things which are multi terabytes in size and for those types of projects it's often not optimal to transfer the data over the network it takes time and it consumes network capacity that might be used heavily for normal business operations so DMS is able to utilize the snowball range of products which are available for bulk transfer of data into and out of AWS so you can use DMS in combination with snowball and this actually uses the schema conversion tool so this is how it works so step one you use the schema conversion tool to extract the data from the database and store it locally and then move this data to a snowball device which you've ordered from AWS step two is that you ship that device back to AWS they load that data into an S3 bucket and then DMS migrates from S3 into the target store so the target database if you decide to use change data capture then you can also migrate changes since the initial bulk transfer these also use S3 as an intermediary before being written to the target database by DMS so DMS normally will transfer the data over the network it can transfer over direct connect or a VPN or even a VPC peer but if the data volumes that you're migrating are bigger than you can practically transfer over your network link then you can order a snowball and use DMS together with SCT to make that transfer much quicker and more effective now the rule to remember for the exam is that SCT is only used for migrations when the engine is changing and the reason why SCT is used here is because you're actually migrating a database into a generic file format which can be moved using snowballs and so this doesn't break the rule of only doing it when the database engine changes because you are essentially changing the database you're changing it from whatever engine the source uses and you're storing it in a generic file format for transfer through to AWS on a snowball device now that's everything that I wanted to cover in this lesson and this has been an extension of the coverage which I did at the associate architect level you are going to get the chance to experience this product practically in a demo but in this lesson I just wanted to cover the theory so thanks for watching go ahead and complete this lesson and when you're ready I look forward to you joining me in the next.

Welcome back and in this video I want to talk about a feature of RDS called RDS proxy.

This is something which is important to know in its own right but it also supports many other architectures involving RDS.

Now we've got a lot to cover so let's jump in and get started.

Before we talk about how RDS proxy works let's step through why you might want to use the product.

First, opening and closing connections to databases takes time and consumes resources.

It's often the bulk of many smaller database operations.

If you only want to read and write a tiny amount, the overhead of establishing a connection can be significant.

This can be especially obvious when using serverless because if you have a lot of lambda functions invoking or accessing an RDS database for example then that's a lot of connections to constantly open and close especially when you're only built for the time that you're using compute as with lambda.

Now another important element is that handling failure of database instances is hard.

How long should you wait for the connection to work?

What should your application do while waiting?

When should it consider it a failure?

How should it react?

And then how should it handle the failover to the standby instance in the case of RDS?

And doing all of this within your application adds significant overhead and risk.

A database proxy is something that can help but maybe you don't have any database proxy experience and even if you do can you manage them at scale?

Well that's where RDS proxy adds value.

At a high level what RDS proxy does or indeed any database proxy is change your architecture.

Instead of your application connecting to a database every time they use it instead they connect to a proxy and the proxy maintains a pool of connections to the database which are open for the long term.

Then any connections to the proxy can use this already established pool of database connections.

It can actually do multiplexing where it can maintain a smaller number of connections to a database versus the connections to the proxy.

A multiplex requests over the connection pool between the proxy and the database.

So you can have a smaller number of actual connections to the database versus the connections to the database proxy.

And this is especially useful for smaller database instances where resources are at a premium.

So in terms of how an architecture might look using RDS proxy let's start with this.

A VPC in US East one with three availability zones and three subnets in each of those availability zones.

In AZB we have a primary RDS instance replicating to a standby running in AZC.

Then we have Categoram our application running in the web subnets in the middle here and the application makes use of some lambda functions which are configured to use VPC networking and run from the subnet in availability zone B and so there's a lambda ENI in that subnet.

Without RDS proxy the Categoram application servers will be connecting directly to the database every time they needed to access data.

Additionally every time one of those lambda functions invoked they would need to directly connect to the database which would significantly increase their running time.

With RDS proxy though things change.

So the proxy is a managed service and it runs only from within a VPC in this case across all availability zones A, B and C.

Now the proxy maintains a long term connection pool in this case to the primary node of the database running in AZB.

These are created and maintained over the long term.

They're not created and terminated based on individual application needs or lambda function invocations.

Our clients in this case the Categoram EC2 instances and lambda functions connect to the RDS proxy rather than directly to the database instances.

Now these connections are quick to establish and place no load on the database server because they're between the clients and the proxy.

Now at this point the connections between the RDS proxy and database instances can be reused.

This means that even if we have constant lambda function invocation they can reuse the same set of long running connections to the database instances.

More so multiplexing is used so that a smaller number of database connections can be used for a larger number of client connections and this helps reduce the load placed on the database server even more.

RDS proxy even helps with database failure or failover events because it abstracts these away from the application.

The clients we have can connect to the RDS proxy instances and wait even if the connection to the back end database isn't operational and this is a situation which might occur during failover events from the primary to the standby.

In the event that there is a failure the RDS proxy can establish new connections to the new primary in the background.

The clients stay connected to the same endpoint, the RDS proxy and they just wait for this to occur.

So that's a high level example architecture.

Let's look at when you might want to use RDS proxy and this is more for the exam but you need to have an appreciation for the types of scenarios where RDS proxy will be useful.

So you might decide to use it when you have errors such as too many connection errors because RDS proxy helps reduce the number of connections to a database and this is especially important if you're using smaller database instances such as T2 or T3.

So anything small or anything burst related.

Additionally, it's useful when using AWS Lambda because you're not having the per invocation database connection setup usage and termination.

It can reuse a long running pool of connections maintained by the RDS proxy and it can also use existing IAM authentication which the Lambda functions have access to via their execution role.

Now RDS proxy is also useful for long running applications such as SAS apps where low latency is critical.

So rather than having to establish database connections every time a user interaction occurs they can use this existing long running connection pool.

RDS proxy is also really useful where resilience to database failure is a priority.

Remember your clients connect to the proxy and the proxy connects to the backend databases so it can significantly reduce the time for a failover event and make it completely transparent to your application.

So this is a really important concept to grasp because your clients are connected to the single RDS proxy endpoint even if a failover event happens in the background instead of having to wait for the database C name to move from the primary to the standby your applications are transparently connected to the proxy and they don't realize it's a proxy they think they're connecting to a database.

The proxy though is handling all of the interaction between them and the backend database instances.

Now before we finish up I want to cover some key facts about RDS proxy think of these as the key things that you need to remember for the exam.

So RDS proxy is a fully managed database proxy that's usable with RDS and Aurora.

It's auto scaling and highly available by default so you don't need to worry about it and this represents a much lower admin overhead versus managing a database proxy yourself.

Now it provides connection pooling which significantly reduces database load.

Now this is for two main reasons.

Firstly we don't have the constant opening and closing of database connections which does put unnecessary stress on the database but in addition we can also multiplex to use a lower number of connections between the proxy and the database relative to the number of connections between the clients and the proxy.

So this is really important.

Now RDS proxy is only accessible from within a VPC so you can't access this from the public internet it needs to occur from a VPC or from private VPC connected networks.

Accesses to the RDS proxy use a proxy endpoint and this is just like a normal database endpoint it's completely transparent to the application.

An RDS proxy can also enforce SSL TLS connections so it can enforce these to ensure the security of your applications and it can reduce fail over time by over 60% in the case of Aurora.

This is somewhere in the region of 66 to 67% improvement versus connecting to Aurora directly.

Critically it abstracts the failure of a database away from your application so the application connected to an RDS proxy will just wait until the proxy makes a connection to the other database instance.

So during a failover event where we're failing over from the primary to the standby the RDS proxy will wait until it can connect to the standby and then just continue fulfilling requests from client connections and so it abstracts away from underlying database failure.

Now at this point that is everything I wanted to cover in this high level lesson on RDS proxy so go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this lesson I want to talk about an advanced feature of Amazon Aurora, multi master rights.

This feature allows an Aurora cluster to have multiple instances which are capable of performing both reads and writes.

This is in contrast with the default mode for Aurora which only allows one writer and many readers.

So let's get started and look at the architecture.

So just to refresh where we are the default Aurora mode is known as single master and this equates to one read write instance so one database instance that can perform read and write operations and then in addition it can also have zero or more read only replicas.

Now an Aurora cluster that's running in the default mode of single master has a number of endpoints which use to interact with the database.

We've got the cluster endpoint which can be used for read or write operations and then we've got another endpoint, a read endpoint that's used for load balancing reads across any of the read only replicas inside the cluster.

An important consideration with an Aurora cluster running in single master mode is that failover takes time for a failover to occur.

A replica needs to be promoted from read only mode to read write mode.

In multi master mode all of the instances by default are capable of both read and write operations so there isn't this concept of a lengthy failover if one of the instances fails in a multi master cluster.

At a high level a multi master Aurora cluster might seem similar to a single master one.

The same cluster structure exists the same shared storage.

Multiple Aurora provisioned instances also exist in the cluster.

The differences start though with the fact that there is no cluster endpoint to use an application is responsible for connecting to instances within the cluster.

There's no load balancing across instances with a multi master cluster the application connects to one or all of the instances in the cluster and initiates operations directly.

So that's important to understand there is no concept of a load balanced endpoint for the cluster an application can initiate connections to one or both of the instances inside a multi master cluster.

Now the way that this architecture works is that when one of the read write nodes inside a multi master cluster receives a write request from the application it immediately proposes that data be committed to all of the storage nodes in that cluster.

So it's proposing that the data that it receives to write is committed to storage.

Now at this point each node that makes up a cluster either confirms or rejects the proposed change.

It rejects it if it conflicts with something that's already in flight for example another change from another application writing to another read write instance inside the cluster.

What the writing instance is looking for is a quorum of nodes to agree a quorum of nodes that allow it to write that data at which point it can commit the change to the shared storage.

If the quorum rejects it then it cancels the change with the application it generates an error.

Now assuming that it can get a quorum to agree to the write then that write is committed to storage and it's replicated across every storage node in the cluster just as it is with a single master cluster but and this is the major difference with a multi master cluster that change is then replicated to other nodes in the cluster.

This means that those other writers can add the updated data into their in-memory caches and this means that any reads from any other instances in the cluster will be consistent with the data that's stored on shared storage because instances cached data we need to make sure in addition to committing it to disk it's also updated inside any in-memory caches of any other instances within the multi master cluster.

So that's what this replication does once the instance on the right has got agreement to be able to commit that change to the cluster shared storage it replicates that change to the instance on the left the instance on the left updates it's in memory cache and then if that instance is used for any read operations it's always got access to the up-to-date data.

Now to understand some of the benefits of multi master mode let's look at a single master failover situation in this scenario we have an Aurora single master cluster with one primary instance performing reads and writes and one replica which is only performing read operations.

Now Bob is using an application and this application connects to this Aurora cluster using the cluster endpoint and the cluster endpoint at this stage points to the primary instance the cluster endpoint the one that's used for read and write operations always points at the primary instance.

If the primary instance fails then access to the cluster is interrupted so immediately we know that this application cannot be fault tolerant because access to the database is now disrupted.

At this point though the cluster will realize that there is a failure event and it will change the cluster endpoint to point at the replica which the cluster decides will be the new primary instance but this failover process takes time it's quicker than normal rds because each replica shares the cluster storage and they can be more replicas but it can take time.

The configuration change to make one of the other replicas the new primary instance inside the cluster is not an immediate change it causes disruption.

Now let's contrast this with multi master.

With multi master both instances are able to write to the shared storage they're both writers the application can connect with one or both of them and let's assume at this stage that it connects to both.

Both instances are capable of read and write operations the application could maintain connections to both and be ready to act if one of them fails but when that writer fails it could immediately just send a hundred percent of any future data operations to the writer which is working perfectly there would be little if any disruption.

If the application is designed in this way it's designed to operate through this failure the application could almost be described as fault tolerant so an Aurora multi master cluster is one component that is required in order to build a fault tolerant application it's not a guarantee and it's not always a thousand percent fault tolerant but it is the foundation of being able to build a fault tolerant application because the application can maintain connections to multiple writers at the same time.

Now in terms of the high level benefits it offers better and much faster availability the failover events can be performed inside the application and it doesn't even need to disrupt traffic between the application and the database because it can immediately start sending any write operations at another writer.

It can be used to implement fault tolerance but the application logic needs to manually load balance across the instances it's not something that's handled by the cluster with that being said though that's everything I wanted to cover in this lesson it's not something I expect to immediately feature in detail on the exam so we can keep it relatively brief go ahead complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this lesson I want to quickly cover the Aurora Global Database Product.

Now the name probably gives away the function, but to avoid any confusion, global databases allow you to create global level replication using Aurora from a master region to up to five secondary AWS regions.

Now this is one of the things which you just need an awareness of for the exam.

I don't expect it to feature heavily, but I want you to be aware of exactly what functionality Aurora Global Database provides.

So on to keep this lesson as brief as possible, let's quickly jump in and look at the architecture first.

So this is a common architecture that you might find when using Aurora Global Databases.

We've got an environment here which operates from two or more regions.

We've got a primary region, US East One on this example on the left.

This primary region offers similar functionality to a normal Aurora cluster.

It has one read and write instance and up to 15 read only replicas in that cluster.

Global databases introduce the concept of secondary regions and the example that's on screen is AP Southeast 2, which is the Sydney region on the right of your screen.

And these can have up to 16 replicas.

The entire secondary cluster is read only.

So in this example, all 16 replicas would be read only replicas.

The entire secondary cluster during normal operations is read only.

Now the replication from the primary region to secondary regions, that occurs at the storage layer.

And replication is typically within one second from the primary to all of the secondries.

Applications can use the primary instance in the primary region for write operations.

And then the replicas in the primary or the replicas in the secondary regions for read operations.

So that's the architecture.

But what's perhaps more important for the exam is when you would use global databases.

So let's have a look at that next.

Aurora Global Databases are great for cross region disaster recovery and business continuity.

So you can basically create a global database, set up multiple secondary regions.

And then if you do have a disaster, which affects an entire AWS region, then you can make these secondary clusters act as primary clusters so they can do read write operations.

So it offers a great solution for cross region disaster recovery and business continuity.

And because of the one second replication time between the primary region and secondary regions, it makes sure that both the RPO and RTO values are going to be really low.

If you do perform a cross region failover, they're also great for global read scaling.

So if you want to offer low latency to any international areas where you have customers, remember low latency generally equates to really good performance.

So if you want to offer low latency performance improvements to international customers, then you can create lots of secondary regions replicated from a primary region.

And then the application can sit in those secondary regions and just perform read operations against the secondary clusters.

And you provide your customers with great performance.

Again, it's important to understand that Aurora Global Databases, the replication occurs at the storage layer.

And it's generally around one second or even less between regions.

So from the primary region to all secondary regions.

It's also important to understand that this is one way replication from the primary to the secondary regions.

It is not bidirectional replication and replication has no impact on database performance because it occurs at the storage layer.

So no additional CPU usage is required to perform the replication tasks.

It happens at the storage layer.

Secondary regions can have 16 replicas.

If you think about Aurora, normally it can have one read and write primary instance and then up to 15 read replicas for a total of 16.

So it makes sense that secondary regions, because they don't have this read write primary instance, all of the replicas inside a secondary can be read replicas.

So it can have a total of 16 replicas per secondary region.

And all of these can be promoted to read write if you do have any disaster situations.

And currently there is a maximum of five secondary regions.

Though just like most things in AWS, this is likely to change.

Now again, for the exam, I don't expect this particular product to feature extensively, but I do want you to have an awareness so that when it does begin to be mentioned in the exam, or if you need to use it in production, you have a starting point by understanding the architecture.

With that being said, though, that is everything that I wanted to cover in this theory lesson.

So go ahead, complete the video, and when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this demo lesson you're going to get some experience of migrating a snapshot which you've previously taken from Aurora in provisioned mode and migrate this into Aurora running in serverless mode.

Now before we begin as always make sure that you're logged in to the general AWS account, so the management account of the organization and you'll need to have Northern Virginia selected.

Now attached to this lesson is a one-click deployment link and I'll need you to go ahead and open that to start the process.

Now once you've got this open we're going to need the Aurora snapshot name that you created in a previous demo.

So click on the services drop down and locate RDS.

It's probably going to be in the recently visited section if not you can search in the box at the top but go ahead and open that in a new tab.

Go to that tab and once it's loaded click on snapshots and you should have these two snapshots in your account.

The first snapshot is a4l wordpress -with -cat - post -mySQL57.

The other snapshot the one that we're going to use is this one so a4l wordpress -aura -with -cat -post.

Go ahead and select that entire snapshot name and copy that into your clipboard because we're restoring an Aurora provisioned snapshot into Aurora serverless.

We're not performing a migration we're performing a restore and so we don't need the snapshot ARN we need the snapshot name.

So go back to the cloud formation stack everything should be pre-populated but there's a box that you need to paste in the snapshot name that you'll be restoring so that's this one paste that in check the acknowledgement box at the bottom under capabilities and then create stack.

Now that process can take up to 45 minutes to complete sometimes it can be a little bit quicker and while that's working we're going to follow the same process through manually but we're going to stop before provisioning the Aurora serverless cluster.

So go back to the RDS tab make sure that you still have this snapshot selected then click on actions and then restore snapshot and I want to step through the options available when restoring an Aurora provisioned snapshot into Aurora serverless.

So these are the options you'll have when you're restoring an Aurora provisioned snapshot you'll see a list of compatible engines so anything compatible with the snapshot that you're restoring in our case it's only my SQL compatibility then you'll have to select your capacity type now it defaults to provisioned but we want to restore to a serverless cluster so we'll select serverless.

You need to select the version of Aurora serverless that you're restoring to and again it's only going to show you compatible versions in this case only 2.07.1 and that's why I was so precise with the version numbers when doing the demos earlier in this section.

Now under database identifier it's here where we would need to provide a unique identifier within this region inside this account for what we're restoring so we might use a4l wordpress-serveless we then need to provide connectivity information so we'd click in the VPC drop-down and make sure we select the animals for life VPC we'd still need to provide a database subnet group to use now currently there isn't one that exists in the account because the cloud formation template is still provisioning but we'd need to choose a relevant subnet group in this box we'd also need to choose a VPC security group which controls access to this database cluster then we have additional configuration and this is a feature which I'm going to be talking about in a dedicated lesson if you're doing the developer or sysops associate courses and this is an API which can be provisioned to give access to the data within this Aurora serverless cluster and it can do so in a way which is very lightweight and this makes it ideal for use with things like serverless applications which prefer a connectionless architecture so this is something that you will use if you want to use for example Aurora serverless with a serverless application based on lambda now something unique to Aurora serverless is the concept of capacity units and I've talked about these in the theory lesson where I talk about Aurora serverless these are the units of database service which the Aurora serverless cluster can make use of and you're able to set a minimum capacity unit and a maximum capacity unit and this provides a range of resources that this cluster can move between based on load on the cluster so as I've talked about in the theory lesson it will automatically provision more capacity or less capacity between these two values based on load now you have additional options for scaling and one that I'll be demonstrating a little bit later on in this demo lesson is how you can actually pause the compute capacity after a consecutive number of minutes of inactivity and this as long as your application supports it can actually reduce the amount of cost that you have running a database platform down to almost zero so you won't have any compute capacity build when the Aurora serverless cluster isn't in use and again I'll be demonstrating that very shortly in this demo lesson you're able to set encryption options just like with other forms of RDS and then under additional configuration you can also configure backup options now these options are obviously based on restoring a snapshot and you have a similar yet more extensive set of options if you're creating an Aurora serverless cluster from scratch so if we select Amazon Aurora and then we go down and select the serverless capacity type then obviously we can select from different versions and we have a wider range of options that we can set so the cluster identifier the admin username and password we've still got the capacity settings we still need to define connectivity options we've got additional configuration options around creating a database controlling the parameter group customizing backup options encryption and enabling deletion protection so whether you're restoring a snapshot or creating an Aurora serverless cluster from scratch these options are similar but you have access to slightly more configuration if you're creating a brand new cluster because when you're restoring a snapshot many of these configuration items are taken from that snapshot at this point we're not going to actually create the cluster manual so I'm going to cancel out of that and I'm going to refresh and as you can see we already have our Aurora serverless DB cluster and it's in an available state so let's go back to our cloud formation stack and refresh it's still in a create in progress state for the stack itself and in order to continue with this demo lesson we're going to need this to be in a create complete state so go ahead pause the video wait for your stack to move into a create complete state and then we can continue so this stacks now moved into a create complete state and we're good to continue so the first thing that I want to draw your attention to if we move back to the RDS console and then let's just refresh you'll see that this cluster is currently using two Aurora capacity units let's go inside the cluster we'll be able to see that it's available it's currently using two capacity units but otherwise it looks very similar to a provisioned Aurora cluster now what we're going to do is click on services open the EC2 console in a new tab go to instances running you should see a single WordPress instance so select that copy the public IP version 4 address into your clipboard making sure not to use this open address and open that in a new tab you'll see that it loads up the WordPress application and it still has the post within it that you created in the previous demo lesson the best cats ever and if you open this post you'll see that it doesn't have any of the attached images because remember they're not stored in the database they're stored on the local instance file system and that's something that we're going to rectify in an upcoming section of the course either called advanced storage or network storage depending on what course you're currently taking but I just wanted to demonstrate that all we've done is restore an Aurora provision snapshot into an Aurora serverless cluster and it still operates in the same way as Aurora provisioned but this is where things change if we go back to the RDS console we know that this Aurora serverless cluster makes use of Aurora capacity units or ACUs and currently it's set to move between one and two Aurora capacity units and the reason it's currently set to two is because we've just used it we've just restored an existing snapshot into this cluster and that operation comes with a relatively high amount of overhead so it needs to go to the two capacity units maximum in order to give us the best performance now what we're going to see over the next few minutes if we just sit here and keep refreshing this screen what should happen is that because we're not using our application first we're going to see it drop down from two capacity units to one capacity unit and that will of course reduce the costs for running this Aurora serverless cluster after a certain amount of time it's going to go from one capacity unit to zero capacity units because it's going to pause the cluster because of no usage we've got this configured if I click on the configuration tab to pause the compute capacity after a number of consecutive minutes of inactivity and it's set to five minutes so after five minutes of no usage on this database it's actually going to pause the compute capacity and we won't be incurring any costs for the compute side of this Aurora serverless cluster so that's one of the real benefits of Aurora serverless versus all of the other types of RDS database engine so let's just go ahead and refresh this and see if it's already changed from two capacity units it's currently still on two so let's select logs and events and refresh we don't see any events currently so this means that we've had no scaling events on this database but if we click on monitoring you'll see how the CPU utilization has decreased from around 25% to just over 5% and the database connection count has reduced from the one when we just accessed the application back down to zero after a few refreshers we'll see that it either decreases from two capacity units down to one or it will go straight to zero if we reach this five minute timer before it performs that scaling event to reduce from two to one so in our case we've skipped the point of having one capacity unit we've reached that five minute threshold where it pauses the compute capacity and so it's gone straight down to zero so your experience might vary it might go from two down to one and then pause or it might go from two straight down to zero but in a case for me my database is currently running at zero capacity units because this time frame has been reached with no activity and the compute has been paused so this means I have no costs for the compute side of Aurora serverless now if I go back to the application and do a refresh you'll see that we don't get a refresh straight away there's a pause and this is because now that the database cluster experiences some incoming load it's unpausing that compute it's resuming the compute part of the cluster and this isn't an immediate process so it's important to understand that when you implement an application and use this functionality the application does need to be able to tolerate lengthier connection times now sometimes in the case of WordPress you will see an error page when you attempt to do a refresh because a timeout value within WordPress is reached before the cluster can resume in the case of this demo lesson that didn't happen it was able to resume the cluster straight away and if we go back to the RDS console and then refresh this page we'll be able to see just how many capacity units this cluster is now operating with and it's operating with two capacity units now in production usage you could be a lot more granular and customize this based on the needs of your application in my case my minimum is one and my maximum is two and my pause time frame is a relatively low five minutes because I wanted to keep it simple for this demo lesson in production usage you might have a larger range between minimum and maximum you might have a higher minimum to be able to cope with a certain level of base load and the time frame between the last access and the pausing of the compute might be significantly longer than five minutes but this demonstration lesson is just that a demo and it's just designed to highlight this at a really high level so that when it comes to you using this in production you understand the architecture now that's everything that I wanted to cover in this demo lesson it's just been a brief bit of experience of using Aurora serverless now to tidy up to return the account into the same state as it was at the start of the demo lesson just go ahead and close down all of these tabs we need to go back to the cloud formation console make sure the Aurora serverless stack is selected and then just go ahead and click on delete and then delete stack and that will remove all of those resources returning the account into the same state as it was at the start of the demo now this whole section of the course has been around trying to improve the database part of our application so we've moved from having a database running on the same server as the application we've split that off we've moved it into rds and we've evolved that from my sequel rds through to Aurora provisioned and now to Aurora serverless we still have one major limitation with our application and that's that for any posts you make on the blog the media for those posts are stored locally on the instance file system and that's something that we're going to start tackling next in the course and we're going to be using the elastic file system product or EFS at this point though that's everything that I wanted to cover in this demo lesson go ahead and complete this video and when you're ready I look forward to you joining me in the next.

Welcome back and in this lesson I want to cover Aurora Serverless.

Aurora Serverless is a service which is to Aurora what Fargate is to ECS.

It provides a version of the Aurora database product where you don't need to statically provision database instances of a certain size or worry about managing those database instances.

It's another step closer to a database as a service product.

It removes one more piece of admin overhead, the admin overhead of managing individual database instances.

From now on when you're referring to the Aurora product that we've covered so far in the course you should refer to it as Aurora provisioned versus Aurora Serverless which is what we'll cover in this lesson.

With Aurora Serverless you don't need to provision resources in the same way as you did with Aurora provisioned.

You still create a cluster but Aurora Serverless uses the concept of ACUs or Aurora capacity units.

Capacity units represent a certain amount of compute and a corresponding amount of memory.

For a cluster you can set minimum and maximum values and Aurora Serverless will scale between those values adding or removing capacity based on the load placed on the cluster.

It can even go down to zero and be paused meaning that you're only billed for the storage that the cluster consumes.

Now billing is based on the resources that you use on a per second basis and Aurora Serverless provides the same levels of resilience as you're used to with Aurora provisioned.

So you get cluster storage that's replicated across six storage nodes across multiple availability zones.

Now some of the high-level benefits of Aurora Serverless it's much simpler, it removes much of the complexity of managing database instances and capacity, it's easier to scale, it seamlessly scales the compute and memory capacity in the form of ACUs as needed with no disruption to client connections and you'll see how that works architecturally on the next screen.

It's also cost effective when you use Aurora Serverless you only pay for the database resources that you consume on a per second basis.

Unlike with Aurora provisioned where you have to provision database instances in advance and you charge for the resources that they consume whether you're utilizing them or not.

The architecture of Aurora Serverless has many similarities with Aurora provisioned but it also has crucial differences so let's review both of those the similarities and the differences.

The Aurora cluster architecture still exists but it's in the form of an Aurora Serverless cluster.

Now this has the same cluster volume architecture which Aurora provisioned uses.

In an Aurora Serverless cluster though instead of using provisioned servers we have ACUs which are Aurora capacity units.

These capacity units are actually allocated from a warm pool of Aurora capacity units which are managed by AWS.

The ACUs are stateless, they're shared across many AWS customers and they have no local storage so they can be allocated to your Aurora Serverless cluster rapidly when required.

Now once these ACUs are allocated to an Aurora Serverless cluster they have access to the cluster storage in the same way that a provisioned Aurora instance would have access to the storage in a provisioned Aurora cluster.

It's the same thing it's just that these ACUs are allocated from a shared pool managed by AWS.

Now if the load on an Aurora Serverless cluster increases beyond the capacity units which are being used and assuming the maximum capacity setting of the cluster allows it then more ACUs will be allocated to the cluster.

And once the compute resource which represents this new potentially bigger ACU is active then any old compute resources representing unused capacity can be deallocated from your Aurora Serverless cluster.

Now because of the ACU architecture because the number of ACUs are dynamically increased and decreased based on load the way that connections are managed within an Aurora Serverless cluster has to be slightly more complex versus a provisioned cluster.

In an Aurora Serverless cluster we have a shared proxy fleet which is managed by AWS.

Now this happens transparently to you as a user of an Aurora Serverless cluster but if a user interacts with the cluster via an application it actually goes via this proxy fleet.

Any of the proxy fleet instances can be used and they will broker a connection between the application and the Aurora capacity units.

Now this means that because the client application is never directly connecting to the compute resource that provides an ACU.

It means that the scaling can be fluid and it can scale in or out without causing any disruptions to applications while it's occurring because you're not directly connecting with an ACU.

You're connecting via an instance in this proxy fleet.

So the proxy fleet is managed by AWS on your behalf.

The only thing you need to worry about for an Aurora Serverless cluster is picking the minimum and maximum values for the ACU and you only have a build for the amount of ACU that you're using at a particular point in time as well as the cluster storage.

So that makes Aurora Serverless really flexible for certain types of use cases.

Now a couple of examples of types of applications which really do suit Aurora Serverless.

The first is infrequently used applications.

Maybe a low volume blog site such as the best cats where connections are only attempted for a few minutes several times per day or maybe on really popular days of the week.

With Aurora Serverless if you were using the product to run the best cat pics blog which you'll experience in the demo lesson then you'd only pay for resources for the Aurora Serverless cluster as you consume them on a per second basis.

Another really good use case is new applications if you're deploying an application where you're unsure about the levels of load that will be placed on the application so you're going to be unsure about the size of database instance that you'll need.

With Aurora provisioned you would still need to provision that in advance and potentially change it which could cause disruption.

If you use Aurora Serverless you can create the Aurora Serverless cluster and have the database autoscale based on the incoming load.

It's also really good for variable workloads if you're running a normally lightly used application which has peaks maybe 30 minutes out of an hour or on certain days of the week during sale periods then you can use Aurora Serverless and have it scale in and out based on that demand.

You don't need to provision static capacity based on the peak or average as you would do with Aurora provisioned.

It's also really good for applications with unpredictable workloads so if you're really not sure about the level of workload at a given time of day you can't predict it you don't have enough data then you can provision an Aurora Serverless cluster and initially set a fairly large range of ACUs so the minimum is fairly low and the maximum is fairly high and then over the initial period of using the application you can monitor the workload and if it really does stay unpredictable then potentially Aurora Serverless is the perfect database product to use because if you're using anything else say an Aurora provisioned cluster then you always have to have a certain amount of capacity statically provisioned.

With Aurora Serverless you can in theory leave an unpredictable application inside Aurora Serverless constantly and just allow the database to scale in and out based on that unpredictable workload.

It's also great for development and test databases because Aurora Serverless can be configured to pause itself during periods of no load and during the database pause you only build for the storage so if you do have systems which are only used as part of your development and test processes then they can scale back to zero and only incur storage charges during periods when it's not in use and that's really cost effective for this type of workload and it's also great for multi-tenant applications if you've got an application where you're billing a user a set dollar amount per month per license to the application if your incoming load is directly aligned to your incoming revenue then it makes perfect sense to use Aurora Serverless.

You don't mind if a database supporting your product scales up and costs you more if you also get more customer revenue so it makes perfect sense to use Aurora Serverless for multi-tenant applications where the scaling is fairly aligned between infrastructure size and incoming revenue.

So these are some classic examples of when Aurora Serverless makes perfect sense.

Now this is a product I don't yet expect to feature extensively on the exam it will feature more and more as time goes on and so by learning the architecture at this point you get a head start and you can answer any questions which might feature on the exam about Aurora Serverless and comparing it to the other RDS products which is often just as important but at this point that's all of the theory that I wanted to cover all of the architecture so go ahead finish up this video and when you're ready I look forward to joining you in the next lesson.

Welcome back and in this lesson I'm going to be covering the architecture of the Amazon Aurora managed database product from AWS.

I mentioned earlier that Aurora is officially part of RDS but from my perspective I've always viewed it as its own distinct product.

The features that it provides and the architecture it uses to deliver those features are so radically different than normal RDS, it needs to be treated as its own product.

So we've got a lot to cover so let's jump in and get started.

As I just mentioned the Aurora architecture is very different from normal RDS.

At its very foundation it uses the base entity of a cluster which is something that other engines within RDS don't have and a cluster is made up of a number of important things.

Firstly from a compute perspective it's made up of a single primary instance and then zero or more replicas.

Now this might seem similar to how RDS works with the primary and the standby replica but it's actually very different.

The replicas within Aurora can be used for reads during normal operations so it's not like the standby replica inside RDS.

The replicas inside Aurora can actually provide the benefits of both RDS multi AZ and RDS read replicas.

So they can be inside a cluster and they can be used to improve availability but also they can be used for read operations during the normal operation of a cluster.

Now that alone would be worth the move to Aurora since you don't have to choose between read scaling and availability.

Replicas inside Aurora can provide both of those benefits.

Now the second major difference in the Aurora architecture is its storage.

Aurora doesn't use local storage for the compute instances.

Instead an Aurora cluster has a shared cluster volume.

This is storage which is shared and available to all compute instances within a cluster.

This provides a few benefits such as faster provisioning, improved availability and better performance.

A typical Aurora cluster looks something like this.

It functions across a number of availability zones in this example A, B and C.

Inside the cluster is a primary instance and optionally a number of replicas.

And again these function as failover options if the primary instance fails.

But they can also be used during normal functioning of the cluster for read operations from applications.

Now the cluster has shared storage which is SSD based and it has a maximum size of 128TIB.

And it also has six replicas across multiple availability zones.

When data is written to the primary DB instance Aurora synchronously replicates that data across all of these six storage nodes spread across the availability zones which are associated with your cluster.

All instances inside your cluster so the primary and all of the replicas have access to all of these storage nodes.

The important thing to understand though from a storage perspective is that this replication happens at the storage level.

So no extra resources are consumed on the instances or the replicas during this replication process.

By default the primary instance is the only instance able to write to the storage and the replicas and the primary can perform read operations.

Because Aurora maintains multiple copies of your data in three availability zones the chances of losing data as a result of any disk related failure is greatly minimized.

Aurora automatically detects failures in the disk volumes that make up the cluster shared storage.

When a segment or a part of a disk volume fails Aurora immediately repairs that area of disk.

When Aurora repairs that area of disk it uses the data inside the other storage nodes that make up the cluster volume and it automatically recreates that data.

It ensures that the data is brought back into an operational state with no corruption.

As a result Aurora avoids data loss and it reduces any need to perform pointing time restores or snapshot restores to recover from disk failures.

So the storage subsystem inside Aurora is much more resilient than that which is used by the normal RDS database engines.

Another powerful difference between Aurora and the normal RDS database engines is that with Aurora you can have up to 15 replicas and any of them can be the failover target for a failover operation.

So rather than just having the one primary instance and the one standby replica of the non Aurora engines with Aurora you've got 15 different replicas that you can choose to fail over to.

And that failover operation will be much quicker because it doesn't have to make any storage modifications.

Now as well as the resiliency that the cluster volume provides there are a few other key elements that you should be aware of.

The cluster shared volume is based on SSD storage by default.

So it provides high IOPS and low latency.

It's high performance storage by default.

You don't get the option of using magnetic storage.

Now the billing for that storage is very different than with the normal RDS engines.

With Aurora you don't have to allocate the storage that the cluster uses.

When you create an Aurora cluster you don't specify the amount of storage that's needed.

Storage is simply based on what you consume.

As you store data up to the 128 TIB limit you'll build on consumption.

Now the way that this consumption works is that it's based on a high watermark.

So if you consume 50 GIB of storage you'll build for 50 GIB of storage.

If you free up 10 GIB of data so move down to 40 GIB of consumed data you'll still build for that high watermark of 50 GIB.

But you can reuse any storage that you free up.

What you'll build for is a high watermark, the maximum storage that you've consumed in a cluster.

And if you go through a process of significantly reducing storage and you need to reduce storage costs then you need to create a brand new cluster and migrate data from the old cluster to the new cluster.

Now it is worth mentioning that this high watermark architecture is being changed by AWS and this no longer is applicable for the more recent versions of Aurora.

Now I'm going to update this lesson once this feature becomes more widespread but for now you do still need to assume that this high watermark architecture is being used.

Now because the storage is for the cluster and not for the instances it means replicas can be added and removed without requiring storage provisioning or removal which massively improves the speed and efficiency of any replica changes within the cluster.

Having this cluster architecture also changes the access method versus RDS.

Aurora clusters like RDS clusters use endpoints.

So these are DNS addresses which are used to connect to the cluster.

Unlike RDS, Aurora clusters have multiple endpoints that are available for an application.

As a minimum you have the cluster endpoint and the reader endpoint.

The cluster endpoint always points at the primary instance and that's the endpoint that can be used for read and write operations.

The reader endpoint will also point at the primary instance if that's all that there is but if there are replicas then the reader endpoint will load balance across all of the available replicas and this can be used for read operations.

Now this makes it much easier to manage read scaling using Aurora versus RDS because as you add additional replicas which can be used for reads this reader endpoint is automatically updated to load balance across these new replicas.

You can also create custom endpoints and in addition to that each instance so the primary and any of the replicas have their own unique endpoint.

So Aurora allows for a much more custom and complex architecture versus RDS.

So let's move on and talk about costs.

With Aurora one of the biggest downsides is that there isn't actually a free tier option.

You can't use Aurora within the free tier and that's because Aurora doesn't support the micro instances that are available inside the free tier but for any instances beyond an RDS single AZ micro sized instance Aurora offers much better value.

For any compute that you use there's an hourly charge and you'll build per second with a 10 minute minimum.

For storage you'll build based on a gigabyte month consumed metric of course taking into account the high watermark so this is based on the maximum amount of storage that you've consumed during the lifetime of that cluster and as well there is an I/O cost per request made to the cluster shared storage.

Now in terms of backups you're given 100% of the storage consumption for the cluster in free backup allocation.

So if your database cluster is 100GIB then you're given 100GIB of storage for backups as part of what you pay for that cluster.

So for most situations for anything low usage or medium usage unless you've got high turnover in data unless you keep the data for long retention periods in most cases you'll find that the backup costs are often included in the charge that you pay for the database cluster itself.

Now Aurora provides some other really exciting features.

In general though backups in Aurora work in much the same way as they do in RDS.

So for normal backup features, for automatic backups, for manual snapshot backups this all works in the same way as any other RDS engine and restores will create a brand new cluster.

So you've experienced this in the previous demo lesson where you created a brand new RDS instance from a snapshot and this architecture by default doesn't change when you use Aurora.

But you've also got some advanced features which can change the way that you do things.

One of those is backtrack and this is something that needs to be enabled on a per cluster basis and it will allow you to roll back your database to a previous point in time.

So consider the scenario where you've got major corruption inside an Aurora cluster and you can identify the point at which that corruption occurred.

Well rather than having to do a restore to a brand new database at a point in time before that corruption if you enable backtrack you can simply roll back in place your existing Aurora cluster to a point before that corruption occurred.

And that means you don't have to reconfigure your applications you simply allow them to carry on using the same cluster it's just the data is rolled back to a previous state before the corruption occurred.

You need to enable this on a per cluster basis and you can adjust the window that backtrack will work for but this is a really powerful feature that's exclusive at the time of creating this lesson to Aurora.

You also have the ability to create what's known as a fast clone and a fast clone allows you to create a brand new database from an existing database but crucially it doesn't make a one-for-one copy of the storage for that database.

What it does is it references the original storage and it only stores any differences between those two.

Now differences can be either you update the storage in your cloned database or it can also be that data is updated in the original database which means that your clone needs a copy of that data before it was changed on the source.

So essentially your cloned database only uses a tiny amount of storage it only stores data that's changed in the clone or changed in the original after you make the clone and that means that you can create clones much faster than if you had to copy all of the data bit by bit and it also means that these clones don't consume anywhere near the full amount of data they only store the changes between the source data and the clone.

So I know that's a lot of architecture to remember.

I've tried to quickly step through all of the differences between Aurora and the other RDS engines.

You'll have lessons upcoming later in this section which deep dive into a little bit more depth of specific Aurora features that I think you will need for the exam but in this lesson I just wanted to provide a broad level overview of the differences between Aurora and the other RDS engines.

So in the next demo lesson you're going to get the opportunity to migrate the data for our WordPress application stack from the RDS MariaDB engine into the Aurora engine.

So you'll get some experience of creating an Aurora cluster and interacting with it with some data that you've migrated but at this point that's all of the theory that I wanted to cover.

So go ahead complete this video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this video I want to talk about a specific feature of RDS called RDS Custom.

Now this is a really niche topic.

I've yet to see it used in the real world and for the exams you really only need to have the most surface level understanding so I'm going to keep this really brief.

So RDS Custom fills the gap between the main RDS product and then EC2 running a database engine.

RDS is a fully managed database server as a service product.

Essentially it gives you access to databases running on a database server which is fully managed by AWS and so any OS or engine access is limited using the main RDS product.

Now databases running on EC2 they're self managed but this has significant overhead because done in this way you're responsible for everything from the operating system upwards.

So RDS Custom bridges this gap it gives you the ability to occupy a middle ground where you can utilize RDS but still get access to some of the customizations that you have access to when running your own DB engine on EC2.

Now currently RDS Custom works for MS SQL and Oracle and when you're using RDS Custom you can actually connect using SSH, RDP and session manager and actually get access to the operating system and database engine.

Now RDS Custom unlike RDS is actually running within your AWS account.

If you're utilizing normal RDS then if you look in your account you won't see any EC2 instances or EBS volumes or any backups within S3.

That's because they're all occurring within an AWS managed environment.

With RDS the networking works by injecting elastic network interfaces into your VPC.

That's how you get access to the RDS instance from a networking perspective but with RDS Custom everything is running within your AWS account so you will see an EC2 instance, you will see EBS volumes and you will see backups inside your AWS account.

Now if you do need to perform any type of customization of RDS Custom then you need to look at the database automation settings to ensure that you have no disruptions caused by the RDS automation while you're performing customizations.

You need to pause database automation, perform your customizations and then resume the automation so re-enable full automation and this makes sure that the database is ready for production usage.

Now again I'm skipping through a lot of these facts and talking only at a high level because realistically you're probably never going to encounter this in production and if you do have any exposure to it on the exam just knowing that it exists will be enough.

Now from a service model perspective this is how using RDS Custom changes things.

So on this screen anything that you see in blue is customer managed, anything that you see in orange is AWS managed and then anything that has a gradient is a shared responsibility.

So if you're using a database engine running on-premises then you're responsible for everything as the customer.

So application optimization, scaling, high availability, backups, any DB patches, operating system patches, operating system install and management of the hardware.

End-to-end that's your responsibility.

Now if you migrate to using RDS this is how it looks where AWS have responsibility for everything but application optimization.

Now if for whatever reason you can't use RDS then historically your only other option was to use a database engine running on EC2 and this was the model in that configuration.

So AWS handled the hardware but from an operating system installation perspective, operating system patches, database patches, backups, HA, scaling and application optimization they were still the responsibility at the customer.

So you only gained a tiny amount of benefit versus using an on-premises system.

With RDS custom we have this extra option where the hardware is AWS's responsibility, the application optimization is the customer responsibility but everything else is shared between the customer and AWS.

So this gives you some of the benefits of both.

It gives you the ability to use the RDS product and benefit from the automation while at the same time allowing you an increased level of customization and the ability to connect into the instance using SSH, session manager or RDP.

Now once again for the exam this is everything that you'll need to understand it only currently works for Oracle and MS SQL and for the real world you probably won't encounter this outside of very niche scenarios.

With that being said though that is everything I wanted to cover in this video so go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this lesson I want to talk about data security within the RDS product.

I want to focus on four different things.

Authentication, so how users can log into RDS.

Authorization, how access is controlled.

Encryption in transit between clients and RDS.

And then encryption at rest, so how data is protected when it's written to disk.

Now we've got a lot to cover so let's jump in and get started.

With all of the different engines within RDS you can use encryption in transit which means data between the client and the RDS instance is encrypted via SSL or TLS and this can actually be set to mandatory on a per user basis.

Encryption at rest is supported in a few different ways depending on the database engine.

By default it's supported using KMS and EBS encryption.

So this is handled by the RDS host and the underlying EBS based storage.

As far as the RDS database engine knows it's just writing unencrypted data to storage.

The data is encrypted by the host that the RDS instance is running on.

KMS is used and so you select a customer master key or CMK to use.

Either a customer managed CMK or an AWS managed CMK and then this CMK is used to generate data encryption keys or DECs which are used for the actual encryption operations.

Now when using this type of encryption then the storage, the logs, the snapshots and any replicas are all encrypted using the same customer master key and importantly encryption cannot be removed once it's added.

Now these are features supported as standard with RDS.

In addition to KMS EBS based encryption Microsoft SQL and Oracle support TDE.

Now TDE stands for transparent data encryption and this is encryption which is supported and handled within the database engine.

So data is encrypted and decrypted within the database engine itself not by the host that the instance is running on and this means that there's less trust.

It means that you know data is secure from the moment it's written out to disk by the database engine.

In addition to this RDS Oracle supports TDE using cloud HSM and with this architecture the encryption process is even more secure with even stronger key controls because cloud HSM is managed by you with no key exposure to AWS.

It means that you can implement encryption where there is no trust chain which involves AWS and for many demanding regulatory situations this is really valuable.

Visually this is how the encryption architecture looks.

Architecturally let's say that we have a VPC and inside this a few RDS instances running on a pair of underlying hosts and these instances use EBS for underlying storage.

Now we'll start off with Oracle on the left which uses TDE and so cloud HSM is used for key services because TDE is native and handled by the database engine.

The data is encrypted from the engine all the way through to the storage with AWS having no exposure and outside of the RDS instance to the encryption keys which are used.

With KMS based encryption KMS generates and allows usage of CMKs which themselves can be used to generate data encryption keys known as DECays.

These data encryption keys are loaded onto the RDS hosts as needed and are used by the host to perform the encryption or decryption operations.

This means the database engine doesn't need to natively support encryption or decryption it has no encryption awareness.

From its perspective it's writing data as normal and it's encrypted by the host before sending it on to EBS in its final encrypted format.

Data that's transferred between replicas as with MySQL in this example is also encrypted as are any snapshots of the RDS EBS volumes and these use the same encryption key.

So that's at rest encryption and there's one more thing that I want to cover before we finish this lesson and that's IAM authentication for RDS.

Normally logins to RDS are controlled using local database users.

These have their own usernames and passwords they're not IAM users and are outside of the control of AWS.

One gets created when you provision an RDS instance but that's it.

Now you can configure RDS to allow IAM user authentication against a database and this is how.

We start with an RDS instance on which we create a local database user account configured to allow authentication using an AWS authentication token.

How this works is that we have IAM users and roles in this case an instance role and attached to those roles and users are policies.

These policies contain a mapping between that IAM entity so the user or role and a local RDS database user.

This allows those identities to run a generate DB auth token operation which works with RDS and IAM and based on the policies attached to the IAM identities it generates a token with a 15-minute validity.

This token can then be used to log in to the database user within RDS without requiring a password.

So this is really important to understand by associating a policy with an IAM user or an IAM role.

It allows either of those two identities to generate an authentication token which can be used to log into RDS instead of a password.

Now one really important thing to understand going into the exam is that this is only authentication.

This is not authorization.

The permissions over the RDS database inside the instance are still controlled by the permissions on the local database user.

So authorization is still handled internally.

This process is only for authentication which involves IAM and only if you specifically enable it on the RDS instance.

Now that's everything I wanted to cover about encryption in transit, encryption at rest as well as RDS IAM based authentication.

So thanks for watching go ahead and complete this video and when you're ready I look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

Now the next thing that I want to demonstrate is how we can restore RDS if we have data corruption.

The way that we're going to simulate this is to go back to our WordPress blog and we're going to corrupt part of this data.

So we're going to change the title of this blog post from the best cats ever to not the best cats ever, which is clearly untrue.

But we're going to change this and this is going to be our simulation of data corruption of this application.

So go ahead and click on update to update the blog post with this new obviously incorrect data.

Now let's assume that we need to restore this database from an earlier snapshot.

Now let's ignore the automatic backup feature of RDS and just look at manual snapshots.

Well, let's move back to the RDS console and click on snapshots and we'll be able to see the snapshot that we created at the start of this demo lesson.

Remember, this does have the blog post contained within it in its original correct form.

Now to do a restore, we need to select this snapshot, click on actions and then restore snapshot.

Now I mentioned this in the theory lesson about backups and restores within RDS.

Restoring a snapshot actually creates a brand new database instance.

It doesn't restore to the existing one using normal RDS.

So we have to restore a snapshot.

Obviously the engine set to MySQL community and we're provided with an entry box for a brand new database identifier.

And we're going to use a4lwordpress-restore.

So this allows us to more easily distinguish between this and the original database instance.

We also need to decide on the deployment option.

So go ahead and select single DB instance.

This is only a demo, so we don't need to select multi-AZ DB instance.

We need to pick the type of instance that we're going to restore to.

And again, because this is a new instance, we're not limited to the previous free tier restrictions.

So we're able to select from any of the available instance types.

So go ahead and select burstable classes and then pick either t2 or t3.micro.

We'll leave storage as default.

We'll need to provide the VPC to provision this new database instance into.

So we'll make sure that a4l-vpc1 is selected and we'll use the same subnet group that was created by the one-click deployment, which you used at the start of this demo.

You're allowed to choose between public access yes or no.

We'll choose no.

You'll have to pick a VPC security group to use for this RDS instance.

Now the one-click deployment did create one, so click in the drop-down and select the RDS multi-AZ snap RDS security group.

So not the instance security group, but the RDS security group.

Once you've selected that, then delete default, scroll down.

You can specify database authentication and encryption settings.

And again, if applicable in the course that you're studying, I'll be covering these in a separate lesson.

We'll leave all of that as default and click on restore DB instance.

Now this is going to begin the process of restoring a brand new database instance from that snapshot.

Now the important thing that you need to understand is this is a brand new instance.

We're not restoring the snapshot to the same database instance.

Instead, it's creating a brand new one.

Now when this finishes restoring, when it's available for use, if we want our application to make use of it, and the restored non-corrupted data, then we're going to need to change the application to point at this newly restored database.

So at this point, go ahead and pause the video because for the next step, which is to adjust the WordPress configuration, we need this database to be in an available state.

So pause the video, wait for the status to change from creating all the way through to available, and then we're good to continue.

Okay, so the snapshot restore is now completed and we have a brand new database instance, A4LWordPress-Restore.

And in my case, it took about 10 minutes to perform that restoration.

Now just to reiterate this concept, because it's really important, it features all the time in the exams, and you'll need this if you operate in the real world using AWS.

If we go into the original RDS instance, just pay attention to this endpoint DNS name.

So we have a standard part, which is the region, and then .rds, and then .amazonaws.com.

Before this, though, we have this random part.

Now this represents the name of the database instance as well as some random.

If we go back to the databases list and then go into the restored version, now we can see that we have A4LWordPress-Restore.

And this is different than that original database endpoint name for the original database.

So the really important, the critical thing to understand is that a restore with a normal RDS will create a brand new database instance.

It will have a brand new database endpoint DNS name, the CNAME, and you will need to update any application configuration to use this brand new database.

So go ahead and just leave this open in this tab because we'll be needing it very shortly.

Click on Services, find EC2, and open that in a new tab.

So as a reminder, if we go back to the WordPress tab and just hit Refresh, we can see that we still have the corrupt data.

Now what we want to do is point WordPress at the restored correct database.

So to do that, go to the EC2 tab that you just opened, right click on the A4LWordPress instance, select Connect.

We're going to use Instance Connect, so choose that to make sure the username is EC2-user and then connect to the instance.

This process should be familiar by now because we're going to edit the WordPress configuration file.

So cd/var/www/html, then we'll do a listing ls-la, and we want to edit the configuration file which is wp-config.php, so shudu, space, nano which is the text editor, space, wp-config.php.

Once we're in this file, just scroll down and again we're looking for the dbhost configuration which is here.

Now this DNS name you'll recognize is pointing at the existing database with the corrupt data.

So we need to delete all of this just to leave the two single quotes.

Make sure your cursor's over the second quote.

Go back to the RDS console and we need to locate the DNS name for the A4LWordPress-Restore instance.

Remember this is the one with the correct data.

So copy that into your clipboard, go back to EC2 and paste that in, and then Ctrl+O and Enter to save, and Ctrl+X to exit.

That's all of the configuration changes that we need.

If we go back to the WordPress application and hit refresh, we'll see that it's now showing the correct post, the best cats ever, because we're now pointing at this restored database instance.

So the key part about this demo lesson really is to understand that when you're restoring a normal RDS snapshot, you're restoring it to a brand new database instance, its own instance with its own data and its own DNS endpoint name.

So you have to update your application configuration to point at this new database instance.

With normal RDS, it's not possible to restore in place.

You have to restore to a brand new database instance.

Now this is different with a feature of Aurora which I'll be covering later in this section, but for normal RDS, you have to restore to a brand new instance.

So those are the features which I wanted to demonstrate in this demo lesson.

I wanted to give you a practical understanding of the types of recovery options and resilience options that you have available using the normal RDS version, so MySQL.

Now different versions of RDS such as Microsoft SQL, PostgreSQL, Oracle, and even AWS specific versions such as Aurora and Aurora Serverless, they all have their own collections of features.

For the exam and for most production usage, you just need to be familiar with a small subset of those.

Generally, you'll either be using Oracle, MSSQL, or one of the open source or community versions, so you'll only have to know the feature set of a small subset of the wider RDS product.

So I do recommend experimenting with all of the different features and depending on the course that you're taking, I will be going into much more depth on those specific features elsewhere in this section.

For now though, that is everything that I wanted to talk about, so all that remains is for us to tidy up the infrastructure that we've used in this demo lesson.

So go to databases.

I want you to select the A4L WordPress -Restore instance because we're going to delete this fully.

We're not going to be using this anymore in this section of the course, so select it, click on the Actions drop down, and then select Delete.

Don't create a final snapshot.

We don't need that.

Don't retain automated backups and because we don't choose either of these, we need to acknowledge our understanding of this and type Delete Me into this box.

So do that and then click on Delete.

Now that's going to delete that instance as well as any snapshots created as part of that instance.

So if we go to Snapshots, we only have the one manual snapshot.

If we go to System Snapshots, we can see that we have one snapshot for this Restore database, and if you're deleting a database instance, then any system created snapshots for that database instance will also be deleted either immediately or after the retention period expires.

So those will be automatically cleared up as part of this deletion process.

We're not going to delete the manual snapshot that we created at the very start of this lesson with the catpost in because we're going to be using this elsewhere in the course.

So leave this in place.

Click on Databases again.

We're going to need to wait for this Restored Database instance to finish deleting before we can continue.

So go ahead and pause the video, wait for this to disappear from the list, and then we can continue.

Okay, so that Restored Database instance has completed deleting.

So now all that remains is to move back to the CloudFormation console.

You should still have a tab open.

Select the stack deployed as part of the one-click deployment.

It should be called RDS Multi-AZ Snap.

Select Delete and then confirm that deletion, and that will clear up all of the infrastructure that we've used in this demo lesson.

It will return the account into the same state as it was at the start of this demo with one exception.

And that one exception is the snapshot that we created of the RDS instance as part of this deployment.

So that's everything you need to do in this demo lesson.

I hope you've enjoyed it.

I know it's been a fairly long one where you've been waiting a lot of the time in the demo for things to happen, but it's important for the exam and real-world usage that you get the practical experience of working with all of these different features.

So you should leave this demo lesson with some good experience of the resilience and recovery features available as part of the normal RDS product.

Now at this point, that's everything you need to do, so go ahead and complete this video, and when you're ready, I look forward to you joining me in the next.

Welcome back and in this demo lesson we're going to continue implementing this architecture.

So in the previous demo lesson you migrated a database from a self-managed MariaDB running on EC2 into RDS.

In this demo lesson you're going to get the experience working with RDS's multi-availability zone mode as well as creating snapshots, restoring those snapshots and experimenting with RDS failover.

Now in order to complete this demo lesson you're going to need some infrastructure.

So let's move across to our AWS console.

You need to be logged in to the general AWS account.

So that's the management account of the organization and as always make sure that you have the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment link so go ahead and open that.

This will take you to a quick create stack page and everything should be pre-populated and ready to go.

So the stack name is RDS multi-AZ snap.

All of the parameters have default values.

Multi-AZ is currently set to false so leave that at false, check the capabilities box at the bottom and then click on create stack.

Now this infrastructure will take about 15 minutes to apply and we need it to be in a create complete state before we continue.

So go ahead, pause the video and resume it once CloudFormation has moved into a create complete state.

Okay so now that this stack has moved into a create complete state we need to complete the installation of WordPress and add our test blog post because we're going to be using those throughout this demo lesson.

Now this is something that you've done a number of times before so we can speed through this.

So click on the services drop down, move to the EC2 console.

We need to go to running instances and we'll need the public IP version 4 address of A4L-WordPress.

So go ahead and copy the public IP version 4 address into your clipboard.

Don't use this open address.

Open that in a new tab.

We'll be calling the site as always the best cats for username put admin for the password.

We'll be using the animals for life strong password and then as always test at test.com for the email address.

Enter all of that and click on install WordPress.

Then you'll need to log in admin for the username and to the password click on login.

Once we logged in go to posts click on trash under hello world to delete the existing post and then add a new post.

Close down this dialogue for the title of the post the best cats ever click on the plus select gallery.

At this point go ahead and click the link that's attached to this lesson to download the blog images.

Once downloaded extract that zip file and you'll get four images.

Once you've got those images ready click on upload locate those images select them and click on open and that will add those to the post.

Once they're fully loaded in we can go ahead and click on publish and then publish again and that will publish this post to our blog.

And as a reminder that stores these images on the local instance file system and adds the post metadata to the database and that's now running within RDS.

Now I want to step through a few pieces of functionality of RDS and I want you for a second to imagine that this blog post is actually a production enterprise application.

Maybe a content management system and I want to view all of the actions that we perform in this demo lesson through the lens of this being a production application.

So go ahead and return to the AWS console click on services and we're going to move back to RDS.

The first thing that we're going to do is to take a snapshot of this RDS instance.

So just close down any additional dialogues that you see go to databases.

Then I want you to select the database that's been created by the one click deployment link that you used at the start of this demo lesson.

Then select actions and then we're going to take a snapshot.

Now a snapshot is a point in time copy of the database.

When you first do a snapshot it takes a full copy of that database so it consumes all of the capacity of the data that's being used by the RDS instance.

So this initial snapshot is a full snapshot containing all of the data within that database instance.

Now we're going to take a snapshot and we're going to call it a four L wordpress hyphen with hyphen cat hyphen post hyphen mySQL hyphen and then the version number without any dots or spaces.

Now depending on when you're watching this video doing this lesson you might have been using a different version of SQL.

And so in the lesson description for this lesson I've included the name of the snapshot that you need to use.

So go ahead and check that now and include that in this box.

So that informs us what it is, what it contains and the version number that this snapshot refers to.

So go ahead and enter that and then click on take snapshot and that's going to begin the process of creating this snapshot.

Now the process takes a variable amount of time.

It depends on the speed of AWS on that particular day.

It depends on the amount of data contained within the database and it also depends on whether this is the first snapshot or a subsequent snapshot.

Now the way that snapshots work within AWS is the first snapshot contains a full copy of all of the data of the thing being snapshotted and any subsequent snapshot only contains the blocks of data which have changed from that last previous successful snapshot.

So of course the first snapshot always takes the longest and everything else only takes the amount of time required to copy the changed data.

So if we just give this a few minutes let's keep refreshing.

Mine's still reporting at 0% complete so we need to allow this to complete before we move on.

So go ahead and pause the video and resume it once your snapshot has completed.

And there we go our snapshots now moved into an available status and the progress has completed.

And in my case that took about five minutes to complete from start to finish.

So again just to reiterate this snapshot has been taken.

It's a copy of an RDS MySQL database of a particular version and it contains our WordPress database together with the cat post that we just added.

And that's important to keep in mind as we move on with the demo lesson.

Now you could go ahead and take another snapshot and this one would be much quicker to complete.

It would only contain any data changed between the point that you take it and when you took this previous snapshot.

I'm not going to demonstrate that in this video but you can do that.

And for production usage you may use snapshots in addition to the normal automated backups provided by RDS.

Snapshots that you take manually live past the life cycle of the RDS instance.

And if you want to tidy them up you have to do that manually or by using scripts that you create.

So snapshots that are taken manually are not managed by RDS in any way.

And that's important to understand from a DR and the cost management perspective.

Now the next thing that I want to demonstrate is the multi AZ mode of RDS.

So if we go back to the RDS console just expand this menu and go to databases.

Currently this database is using a single RDS instance.

So this RDS instance is not resilient to the failure of an availability zone within this region.

Now to change that process we can provision a standby replica in another availability zone and that's known as multi AZ.

Now it's worth noting that this is not included within the AWS free tier.

So there will be a small charge to do this optional step to enable multi AZ mode.

Make sure that you have the database instance selected and then click on modify.

Now it's on this screen that we can change a lot of the options which relate to this entire RDS instance.

We've got the option to adjust the database identifier, provide a new database admin password.

We can change the DB instance size or type if we want.

We can adjust the amount of storage available to the database instance, even enable storage auto scaling.

But what we're looking for specifically is adjusting the availability and durability settings.

Currently this is set to do not create a standby instance and we're going to modify this.

We're going to change it to create a standby instance and this is something that's recommended for any production usage.

This creates a standby replica in a different availability zone.

So it picks another availability zone, specifically another subnet that's available within the database subnet group that was created by the one click deployment.

So we're going to set that option and scroll down and then select continue.

Now because we have a maintenance window defined on this RDS instance, we have two different options of when to apply this change.

We can either apply the change during the next scheduled maintenance window.

Remember, this is a definable value that you can set when you create an RDS instance or you modify its settings.

Or we can specify that we want to apply immediately the change that we're making.

And for this demo lesson, that's what we're going to do.

Now it does warn you that any changes could cause a performance impact and even an outage.

So it's really important that if you are applying changes immediately, you understand the impact of those changes.

So make sure that you have apply immediately selected and then click on modify DB instance.

Now a multi AZ deployment is essentially an automatic standby replica in a separate availability zone.

What happens behind the scenes is that the primary database instance is synchronously replicated into this standby replica inside a different availability zone.

Now this provides a few benefits.

It provides data redundancy benefits.

It means that any operations which can interrupt IO such as system backups will occur from the standby replica.

So won't impact the primary database and that provides a real advantage for production RDS deployments.

But the main reason beyond performance is that it helps protect any databases in the primary instance against failure of an availability zone.

So if the availability zone of the primary instance fails and then the C name of the database will be changed to point at the standby replica.

And that will minimize any disruption to your application and its users.

Now if we just hit refresh, we can see the status is modifying and what's happening behind the scenes is AWS are taking a snapshot of the primary DB instance.

It's restoring that snapshot into the standby replica, which is in a different availability zone.

And then it's setting up synchronous replication between the primary and the standby replica.

So this is a process which happens behind the scenes.

But it does mean that we need to wait for this process to be complete until the process is complete.

This is not a multi AZ deployment.

So go ahead and pause the video and wait for the status to change away from modifying.

We need this to be in an available state in order to continue with the demo.

So go ahead and pause the video and resume it once this modification has completed.

Okay, so the status has now changed to available.

And in my case, it took about 10 minutes to enable multi AZ mode.

So that's the provisioning of a standby replica in another availability zone.

Now, the likelihood of an AZ failure happening while I'm recording this demo lesson is relatively small, but we can simulate a failure to do that.

If we have the database instance selected and then select the actions drop down and then reboot, we can use the option reboot with failover.

If we choose this option, then part of the process is that a simulated failover occurs.

So the C name, the database endpoint, that's moved so that it now points at the standby replica and then the old primary instance is restarted.

So that's what we're going to do to simulate this process.

So go ahead and select to reboot the database instance.

Make sure that you have reboot with failover selected and then click on confirm.

And this will begin the process of rebooting the database instance.

Now, if we go back to the WordPress blog and we click on view post, you'll see that right away it's not immediately loading.

And that's because the failover from the primary to the standby isn't immediate.

Failover times are typically 60 to 120 seconds.

So that's important to keep in mind if you're deploying RDS in a business critical situation.

It doesn't offer immediate failover.

So let's just stop this and hit reload again.

And now we can see that the page is starting to load because the C name for the database has been moved from pointing at the primary to pointing at the standby replica, which is the new primary.

Okay, so this is the end of part one of this lesson.

It was getting a little bit on the long side and so I wanted to add a break.

It's an opportunity just to take a rest or grab a coffee.

Part two will be continuing immediately from the end of part one.

So go ahead, complete the video and when you're ready, join me in part two.

Welcome back.

In this video I want to talk about RDS read replicas.

Now read replicas provide a few main benefits to us as solutions architects or operational engineers.

They provide performance benefits for read operations, they help us create cross-region failover capability, and they provide a way for RDS to meet really low recovery time objectives, just as long as data corruption isn't involved in a disaster scenario.

Now let's step through the key concepts and architectures because they're going to be useful for both the exam and the real world.

Read replicas, as the name suggests, are read-only replicas of an RDS instance.

Unlike MultiAZ, where you can't by default use the standby replica for anything, you can use read replicas but only for read operations.

Now MultiAZ running in cluster mode, which is the newer version of MultiAZ, is like a combination of the old MultiAZ instance mode together with read replicas.

But, and this is really important, you have to think of read replicas as separate things.

They aren't part of the main database instance in any way.

They have their own database endpoint address and so applications need to be adjusted to use them.

An application, say WordPress, using an RDS instance will have zero knowledge of any read replicas by default.

Without application support, read replicas do nothing.

They aren't functional from a usage perspective.

There's no automatic failover, they just exist off to one side.

Now they're kept in sync using a synchronous replication.

Remember MultiAZ uses synchronous replication and that means that when data is written to the primary instance, at the same time as storing that data on disk on the primary, it's replicated to the standby.

And conceptually think of this as a single write operation, both on the primary and on the standby.

With asynchronous, data is written to the primary first at which point it's viewed as committed.

Then after that it's replicated to the read replicas and this means in theory there could be a small lag, maybe seconds, but it depends on network conditions and how many writes occur on the database.

For the exam for any RDS questions and exclude Aurora for now, remember that synchronous means MultiAZ and asynchronous means read replicas.

Read replicas can be created in the same region as the primary database instance or they can be created in other AWS regions known as cross region read replicas.

If you create a cross region read replica, then AWS handle all of the networking between regions and this occurs transparently to you and it's fully encrypted in transit.

Now why do read replicas matter?

Well there are two main areas of importance that I want you to think about.

First is read performance and read scaling for a database instance.

So you can create five direct read replicas per database instance and each of these provides an additional instance of read performance.

So this offers a simple way of scaling out your read performance on a database.

Now read replicas themselves can also have their own read replicas but this means that lag starts to become a problem because asynchronous replication is used.

There can be a lag between the main database instance and any read replicas and if you then create read replicas of read replicas then this lag becomes more of a problem.

So while you can use multiple levels of read replicas to scale read performance even more lag does start to become even more of a problem.

So you need to take that into consideration.

Additionally read replicas can help you with global performance improvements for read workloads.

So if you have read workloads in other AWS regions then these workloads can directly connect to read replicas and not impact the performance at the primary instance in any way.

In addition read replicas benefit us in terms of recovery point objectives and recovery time objectives.

So snapshots and backups improve RPOs the more frequent snapshots occur and the better backups are this offers improved recovery point objectives because it limits the amount of data which can be lost but it doesn't really help us for recovery time objectives because restoring snapshots takes a long time especially for large databases.

Now read replicas offer a near zero RPO and that's because the data that's on the read replica is synced from the main database instance.

So there's very little potential for data loss assuming we're not dealing with data corruption.

Read replicas can be promoted quickly they offer a near zero RPO.

So in a disaster scenario where you have a major problem with your RDS instance you can promote a read replica and this is a really quick process but and this is really important you should only look at using read replicas during disaster recovery scenarios when you're recovering from failure.

If you're recovering from data corruption then logically the read replica will probably have a replica of that corrupted data.

So read replicas are great for achieving low RTOs but only for failure and not for data corruption.

Now read replicas are read only until they're promoted and when they're promoted you're able to use them as a normal RDS instance.

There's also a really simple way to achieve global availability improvements and global resilience because you can create a cross region read replica in another AWS region and use this as a failover region if AWS ever have a major regional issue.

Now at this point that's everything I wanted to cover about read replicas.

If appropriate for the exam that you're studying I might have another lesson which goes into more technical depth or a demo lesson which allows you to experience this practically.

If you don't see either of these then don't worry they're not required for the exam that you're studying.

At this point though that's everything I'm going to cover so go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this video I want to talk about how RDS can be backed up and restored, as well as covering the different methods of backup that we have available.

Now we do have a lot to cover, so let's jump in and get started.

Within RDS there are two types of backup-like functionality.

We have automated backups and we have snapshots.

Now both of these are stored in S3, but they use AWS managed buckets, so they won't be visible to you within your AWS console.

You can see backups in the RDS console, but you can't move to S3 and see any form of RDS bucket, which exists for backups.

Keep this in mind because I've seen questions on it in the exam.

Now the benefits of using S3 is that any data contained in backups is now regionally resilient, because it's stored in S3, which replicates data across multiple AWS availability zones within that region.

Now RDS backups, when they do occur, are taken in most cases from the standby instance, if you have multi-AZ enabled.

So while they do cause an I/O pause, this occurs from the standby instance, and so there won't be any application performance issues.

If you don't use multi-AZ, for example with test and development instances, then the backups are taken from the only available instance, so you may have pauses in performance.

Now I want to step through how backups work in a little bit more detail, and I'm going to start with snapshots.

Snapshots aren't automatic.

They're things that you run explicitly or via a script or custom application.

You have to run them against an RDS database instance.

They're stored in S3, which is managed by AWS, and they function like the EBS snapshots that you've covered elsewhere in the course.

Snapshots and automated backups are taken of the instance, which means all the databases within it, rather than just a single database.

The first snapshot is a full copy of the data stored within the instance, and from then on, snapshots only store data which has changed since the last snapshot.

When any snapshot occurs, there is a brief interruption to the flow of data between the compute resource and the storage.

If you're using single AZ, this can impact your application.

If you're using multi AZ, this occurs on the standby, and so won't have any noticeable effect.

Time-wise, the initial snapshot might take a while.

After all, it's a full copy of the data.

From then on, snapshots will be much quicker because only changed data is being stored.

Now the exception to this are instances where there's a lot of data change.

In this type of scenario, snapshots after the initial one can also take significant amounts of time.

Snapshots don't expire.

You have to clear them up yourself.

It means that snapshots live on past when you delete the RDS instance.

Again, they're only deleted when you delete them manually or via some external process.

Remember that one because it matters for the exam.

Now you can run one snapshot per month, one per week, one per day, one per hour.

The choice is yours because they're manual.

And one way that lower recovery point objectives can be met is by taking more frequent snapshots.

The lower the time frame between snapshots, the lower the maximum data loss that can occur when you have a failure.

Now this is assuming we only have snapshots available, but there is another part to RDS backups, and that's automated backups.

These occur once per day, but the architecture is the same.

The first one is a full, and any ones which follow only store changed data.

So far you can think of them as though they're automated snapshots, because that's what they are.

They occur during a backup window which is defined on the instance.

You can allow AWS to pick one at random or use a window which fits your business.

If you're using single AZ, you should make sure that this happens during periods of little to no use, as again there will be an IO pause.

If you're using multi AZ, this isn't a concern as the backup occurs from the standby.

In addition to this automated snapshot, every five minutes, database transaction logs are also written to S3.

Transaction logs store the actual operations which change the data, so operations which are executed on the database.

And together with the snapshots created from the automated backups, this means a database can be restored to a point in time with a five minute granularity.

In theory, this means a five minute recovery point objective can be reached.

Now automated backups aren't retained indefinitely.

They're automatically cleared up by AWS, and for a given RDS instance, you can set a retention period from zero to 35 days.

Zero means automated backups are disabled and the maximum is 35 days.

If you use a value of 35 days, it means that you can restore to any point in time over that 35 day period using the snapshots and transaction logs, but it means that any data older than 35 days is automatically removed.

When you delete the database, you can choose to retain any automated backups, but, and this is critical, they still expire based on the retention period.

The way to maintain the contents of an RDS instance past this 35 day max retention period is that if you delete an RDS instance, you need to create a final snapshot, and this snapshot is fully under your control and has to be manually deleted as required.

Now RDS also allows you to replicate backups to another AWS region, and by backups, I mean both snapshots and transaction logs.

Now charges apply for both the cross region data copy and any storage used in the destination region, and I want to stress this really strongly.

This is not the default.

This has to be configured within automated backups.

You have to explicitly enable it.

Now let's talk a little bit about restores.

The way RDS handles restores is really important, and it's not immediately intuitive.

It creates a new RDS instance when you restore an automated backup or a manual snapshot.

Why this matters is that you will need to update applications to use the new database end point address because it will be different than the existing one.

When you restore a manual snapshot, you're restoring the database to a single point in time.

It's fixed to the time that the snapshot was created, which means it influences the RPO.

Unless you created a snapshot right before a failure, then chances are the RPO is going to be suboptimal.

Automated backups are different.

With these, you can choose a specific point to restore the database to, and this offers substantial improvements to RPO.

You can choose to restore to a time which was minutes before a failure.

The way that it works is that backups are restored from the closest snapshot, and then transaction logs are replayed from that point onwards, all the way through to your chosen time.

What's important to understand though is that restoring snapshots isn't a fast process.

If appropriate for the exam that you're studying, I'm going to include a demo where you'll get the chance to experience this yourself practically.

It can take a significant amount of time to restore a large database, so keep this in mind when you think about disaster recovery and business continuity.

The RDS restore time has to be taken into consideration.

Now in another video elsewhere in this course, I'm going to be covering read replicas, and these offer a way to significantly improve RPO if you want to recover from failure.

So RDS automated backups are great as a recovery to failure, or as a restoration method for any data corruption, but they take time to perform a restore, so account for this within your RTO planning.

Now once again, if appropriate for the exam that you're studying, you're going to get the chance to experience a restore in a demo lesson elsewhere in the course, which should reinforce the knowledge that you've gained within this theory video.

If you don't see this then don't worry, it's not required for the exam that you're studying.

At this point though, that is everything I wanted to cover in this video, so go ahead and complete the video, And when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this video I want to talk through the ways in which RDS offers high availability.

Historically there was one way, multi-AZ.

Over time RDS has been improved and now there's multi-AZ instance deployments and multi-AZ cluster deployments.

And these offer different benefits and trade-offs and so in this video I want to step through the architecture and functionality of both.

Now we do have a lot to cover so let's jump in and get started straight away.

Historically the only method of providing high availability which RDS had was multi-AZ.

So again this is now called multi-AZ instance deployment.

With this architecture RDS has a primary database instance containing any databases that you create and when you enable multi-AZ mode this primary instance is configured to replicate its data synchronously to a standby replica which is running in another availability zone.

And this means that this standby also has a copy of your databases.

Now in multi-AZ instance mode this replication is at the storage level.

This is actually less efficient than the cluster multi-AZ architecture but more on this later in this video.

The exact method that RDS uses to do this replication depends on the database engine that you pick.

MariaDB, MySQL, Oracle and PostgreSQL use Amazon failover technology whereas Microsoft's SQL instances use SQL server database mirroring or always on availability groups.

In any case this is abstracted away all you need to understand is that it's a synchronous replica.

Now architecturally how this works is that all accesses to the databases are via the database CNAME.

This is a DNS name which by default points at the primary database instance.

With multi-AZ instance architecture you always access the primary database instance.

There's no access to the standby even for things like reads.

Its job is to simply sit there until you have a failure scenario with the primary instance.

Other things though such as backups can occur from the standby so data is moved into S3 and then replicated across multiple availability zones in that region.

Now this places no extra load on the primary because it's occurring from the standby.

And remember this is important because all accesses so reads and writes from this multi-AZ architecture will occur to and from the primary instance.

Now in the event that anything happens to the primary instance this will be detected by RDS and a failover will occur.

This can be done manually if you're testing or if you need to perform maintenance but generally this will be an automatic process.

What happens in this scenario is the database CNAME changes instead of pointing at the primary it points at the standby which becomes the new primary.

Because this is a DNS change it generally takes between 60 to 120 seconds for this to occur so there can be brief outages.

This can be reduced by removing any DNS caching in your application for this specific DNS name.

If you do remove this caching it means that the second RDS has finished the failover and the DNS name has been updated.

Your application will use this name which is now pointing at the new primary instance.

So this is the architecture when you're using the older multi-AZ instance architecture.

I want to cover a few key points of this architecture before we look at how multi-AZ cluster architecture works.

So let's move on.

So just to summarize replication between primary and standby is synchronous and what this means is that data is written to the primary and then immediately replicated to the standby before being viewed as committed.

Now multi-AZ does not come within the free tier because of the extra cost for the standby replica that's required.

And multi-AZ with the instance architecture means that you only have one standby replica and that's important.

It's only one standby replica and this standby replica cannot be used for reads or writes.

Its job is to simply sit there and wait for failover events.

A failover event can take anywhere from 60 to 120 seconds to occur and multi-AZ mode can only be within the same region.

So different availability zones within the same AWS region.

Backups can be taken from the standby replica to improve performance and failovers will occur for various different reasons such as availability zone outage, the failure of the primary instance, manual failover, instance type change so when you change the type of the RDS instance and even when you're patching software.

So you can use failover to move any consumers of your database onto a different instance, patch the instance which has no consumers and then flip it back.

So it does offer some great features which can help you maintain application availability.

Now next I want to talk about multi-AZ using a cluster architecture.

And when you watch the Aurora video you might be confused between this architecture and Amazon Aurora.

So I'm going to stress the differences between multi-AZ cluster for RDS and Aurora in this video.

And this is to prepare you for when you watch the Aurora video.

It's really critical for you to understand the differences between multi-AZ cluster mode for RDS and Amazon Aurora.

So we start with a similar VPC architecture only now in addition to the single client device on the left.

I'm adding two more.

In this mode RDS is capable of having one writer replicate to two reader instances.

And this is a key difference between this and Aurora.

With this mode of RDS multi-AZ you can have two readers only.

These are in different availability zones than the writer instance but there will only be two whereas with Aurora you can have more.

The difference between this mode of multi-AZ and the instance mode is that these readers are usable.

You can think of the writer like the primary instance within multi-AZ instance mode in that it can be used for writes and read operations.

The reader instances unlike multi-AZ instance mode these can be utilized while they're in this state.

They can be used only for read operations.

This will need application support since your application needs to understand that it can't use the same instance for reads and writes.

But it means that you can use this multi-AZ mode to scale your read workloads unlike multi-AZ instance mode.

Now in terms of replications between the writer and the readers while data is sent to the writer and it's viewed as being committed when at least one of the readers confirms that it's been written.

It's resilient at that point across multiple availability zones within that region.

Now the cluster that RDS creates to support this architecture is different in some ways and similar in others versus Aurora.

In RDS multi-AZ mode each instance still has its own local storage which as you'll see elsewhere in this course is different than Aurora.

Like Aurora though you access the cluster using a few endpoint types.

First is the cluster endpoint and you can think of this like the database C name in the previous multi-AZ architecture.

It points at the writer and can be used for reads and writes against the database or administration functions.

Then there's a reader endpoint and this points at any available reader within the cluster.

And in some cases this does include the writer instance.

Remember the writer can also be used for reads.

In general operation though this reader endpoint will be pointing at the dedicated reader instances and this is how reads within the cluster scale.

So applications can use the reader endpoint to balance their read operations across readers within the cluster.

Finally there are instance endpoints and each instance in the cluster gets one of these.

Generally it's not recommended to use them directly as it means any operations won't be able to tolerate the failure of an instance because they don't switch over to anything if there's an instance failure.

So you generally only use these for testing and fault finding.

So this is the multi-AZ cluster architecture.

Before I finish up with this video I just want to cover a few key points about this specific type of multi-AZ implementation.

And don't worry you're going to get the chance to experience RDS practically in other videos in this part of the course.

So first RDS using multi-AZ in cluster mode means one writer and two reader DB instances in different availability zones.

So this gives you a higher level of availability versus instance mode because you have this additional reader instance versus the single standby instance in multi-AZ.

In instance mode.

In addition multi-AZ cluster mode runs on much faster hardware.

So this is Graviton architecture and uses local NVMe SSD storage.

So any writes are written first to local superfast storage and then flushed through to EBS.

So this gives you the benefit of the local superfast storage in addition to the availability and resilience benefits of EBS.

In addition when multi-AZ uses cluster mode then readers can be used to scale read operations against the database.

So if your application support it it means you can set read operations to use the reader endpoint which frees up capacity on the writer instance and allows your RDS implementation to scale to high levels of performance versus any other mode of RDS.

And again you'll see when you're watching the Aurora video, Aurora as a database platform can scale even more.

And I'll detail exactly how in that separate video.

Now when using multi-AZ in cluster mode replication is done using transaction logs and this is much more efficient.

This also allows a faster failover.

In this mode failover rather than taking 60 to 120 seconds can occur in as little as 35 seconds plus any time required to apply the transaction logs to the reader instances.

But in any case this will occur much faster than the 60 to 120 seconds which is needed when using multi-AZ instance mode.

And again just to confirm when running in this mode writes a viewed as committed when they've been sent to the writer instance and stored and replicated to at least one reader which has confirmed that it's written that data.

So as you can see these are completely different architectures and in my opinion multi-AZ in cluster mode adds some significant benefits over instance mode.

And you'll see how this functionality is extended again when I talk about Amazon Aurora.

But for now that's everything I wanted to cover in this video so thanks for watching.

Go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

Okay, so the instance is now in an available state.

Let's just close down this informational dialogue at the top.

And let's just minimize this menu on the left.

Let's maximize the amount of screen space that we have for this specific purpose.

So I just want us to go inside this database instance and explore together the information that we have available.

So I talked in the theory lesson how every RDS instance is given an endpoint name and an endpoint port.

So this is the information that we'll use to connect to this RDS instance.

Networking wise, this instance has been provisioned in US-EAST-1A.

It's in the Animals for Life VPC and it's used our A4L subnet group that we created at the start of this demo.

And that means that it's currently utilizing all three database subnets in the Animals for Life VPC.

But it's chosen because we only deployed one instance to use US-EAST-1A.

Now this is the VPC security group that we're going to need to configure.

So right click on this and open it in a new tab and move to that tab.

This is the security group which controls access to this RDS instance.

So let's expand this at the bottom.

So currently it has my IP address being the only source allowed to connect into this RDS instance.

So the only inbound rule on the security group protecting this RDS instance is allowing my IP address.

So we're going to click on Edit and then click on Add Rule.

And we're going to add a rule which allows our other instances to connect to this RDS instance.

So first in the type drop down click and then type mySQL to get the same option as the line above and then click to select.

Next go ahead and type instance into the source box and then select the migrate to RDS-instance security group.

Now this is the security group that's used by any instances deployed by our one click deployment.

And this allows those instances to connect to our RDS instance and that's what we want.

So go ahead and select that and then click on Save Rules.

And this means now that our WordPress instance can communicate with RDS.

So now let's move back to the RDS tab and then make sure we're inside the A4L WordPress database instance.

So that's the connectivity and the security tab.

We also have the monitoring tab and it's here where you can see various CloudWatch provided metrics about the database instance.

You also have logs and events related to this instance.

So if we go and have a look at recent events we can see all of the events such as when the database instance was created, when its first backup was created.

And you can explore those because they might be different in your environment.

You can click on the Configuration tab and see the current configuration of the RDS instance.

The Maintenance and Backups tab is where you can configure the maintenance and backup processes and then of course you can tag the RDS instance.

Now in other lessons in this section of the course and depending on what course you're taking I will be talking about many of these options, what you can modify and which actions you can perform on RDS instances.

But for now we're just going to move on with this demo.

So the next step is that we need to migrate our existing data into this RDS instance.

So what we're going to do is to click on the Connectivity and Security tab and we're going to leave this open.

We're going to need this endpoint name and port very shortly.

You should still have a tab open to the EC2 console.

If you don't you can reach that by going on Services and then opening EC2 in a new tab.

But I want you to select the A4L-WordPress instance and then right click and connect to it using Instance Connect.

So go ahead and do that.

Once you've done that we're going to start referring to the lesson commands document.

So make sure you've got that open.

We're going to use this command to take a backup of the existing MariaDB database.

So we need to replace a placeholder.

What we need to do is delete this and replace it with the private IP address of the MariaDB EC2 instance.

So go back to the EC2 console, select the DB-WordPress instance and copy the private IP version 4 address into your clipboard.

And then let's move back to the WordPress instance and paste that in.

Go ahead and press Enter and it will prompt you for the password.

And the password is the same Animals for Life strong password that we've been using everywhere.

Copy that into your clipboard.

So this is the password for the A4L WordPress user on the MariaDB EC2 instance.

So paste that in and press Enter and then LS-LA to confirm that we now have this A4L WordPress.SQL database backup file.

And we do, so that's good.

So as we did in the previous demo lesson, we're going to take this backup file and we're going to import it into the new destination database, which is going to be the RDS instance.

To do that, we'll use this command, but we're going to need to replace the placeholder hostname with the CNAME of the RDS instance.

So go ahead and delete this placeholder, then go back to the RDS console and I'll want you to copy the endpoint name into your clipboard.

So select it, right click and then copy.

We won't need the port number because this is the standard MySQL port and if you don't specify it, most applications will assume this default.

So just make sure that you have the endpoint DNS name or endpoint CNAME in your clipboard.

And then back on the WordPress EC2 instance, go ahead and paste this database name into this command and press Enter.

And again, you'll be asked for the password and that's the same Animals for Life strong password.

So copy that into your clipboard, paste that in and press Enter.

And that's imported this A4LWordPress.SQL file into the RDS instance.

So now we need to follow the same process and change WordPress so that it points at the RDS instance.

And we do that by moving to where the WordPress configuration file is.

So cd space forward slash var forward slash ww w forward slash html and press Enter.

And then shudu.

So we have admin privileges, nano, which is a text editor and then wp-config.php and press Enter.

Then we need to scroll down and we're looking for where it says DB host and currently it has a host name here.

Now if you go back to the EC2 console and you look at the A4L-DB-WordPress instance, you'll see that its private IP version for DNS name is what's listed inside this configuration item.

So it's currently pointing at this dedicated database instance.

What we need to do is replace that and we're going to replace it with the RDS database DNS name or the CNAME of this RDS instance.

So copy that into your clipboard and then go ahead and delete this private DNS name for the MariaDB EC2 instance and then paste in the RDS endpoint name, also known as the RDS CNAME.

Once you've done that, control O and Enter to save and control X to exit.

And now our WordPress instance is pointing at the RDS instance for its database.

Now we can verify that by checking WordPress, move back to instances, select the WordPress instance, copy the public IP version for addressing to your clipboard.

Don't use this open address link.

Open that in a new tab.

Go ahead and just click on the best cats ever to verify the functionality and it does look as though it's working.

And to verify that, if we go back to the EC2 console, select the A4L-DB-WordPress instance and right click and then stop that instance.

Now the original database that was providing database services to WordPress is going to move into a stopped state.

And if our WordPress blog continues functioning, we know that it's using the RDS instance.

So let's keep refreshing and wait for this to change into a stopped state.

There we go.

It's stopped.

And if we go back to our WordPress page and refresh, it still loads.

And so we know that it's now using RDS for its database services.

So at this point, that's everything that I wanted you to do in this demo lesson.

You've stepped through the process of provisioning an RDS instance.

So you've created a subnet group, provisioned the instance itself, explored the functionality of the instance, including how to provide access to it by selecting a security group.

And then editing that security group to allow access.

You've performed a database migration and you've explored how the RDS instance is presented in the console.

So that's everything that you need to do within this demo lesson.

And don't worry, we're going to be exploring much more of the advanced functionality of RDS as we move through this section of the course.

For now, though, I want us to clear up the infrastructure that we've created as part of this demo lesson.

Now, because we've provisioned RDS manually outside of CloudFormation, unfortunately, there is a little bit more manual work involved in the cleanup.

So I want you to go to the RDS console, move to databases, select this database, click on actions, and then select delete.

Now it will prompt you to create a final snapshot and we're not going to do that.

We're not going to retain automated backups and so you'll need to acknowledge that upon instance deletion, automated backups including any system snapshots and pointing time recoveries will no longer be available.

And don't worry, I'll be talking about backups and recovery in another lesson in this section of the course.

For now, just acknowledge that and then type delete me into this box and confirm the deletion.

Now this deletion is going to take a few minutes.

It's not an immediate process.

It will start in a deleting state and we need to wait for this process to be completed before we continue the cleanup.

So go ahead and pause this video and wait for this instance to fully delete before continuing.

Now that the instance has been deleted, it vanishes from this list.

Next, we need to delete the subnet group that we created earlier.

So click on subnet groups, select the subnet group and then delete it.

You'll need to confirm that deletion.

Once done, it too should vanish from that list.

Next, go to the tab you've got open to the VPC console, scroll down and select security groups.

Now look through this list and locate the security group that you created as part of provisioning the RDS instance.

It should be called a4LVPC-RDS-SG.

Select that, click on actions and then delete security group and you'll need to confirm that process as well.

Once that's deleted, the final step is to go to the cloud formation console and then you'll need to delete the cloud formation stack that was created using the one-click deployment at the start of the demo.

It should be called migrate to RDS.

Select it, click on delete and confirm that deletion.

And once deleted, the account will be returned into the same state as it was at the start of the demo lesson.

So all of the infrastructure that we've used will be removed from the account and the account will be in the same state as at the start of the demo.

Now I hope you've enjoyed this demo and that we're repeating the same WordPress installation and then the creation of the blog post over and over again.

But I want you to get used to different parts of this process over and over again.

You need to know why not to use a database on EC2.

You need to know why not to perform a lot of these processes manually.

From this point onward in the course, we're going to be using RDS to evolve our WordPress design into something that is truly elastic.

And so all of these processes, the things I'm having you repeat are really useful to aid in your understanding of all of these different components.

So from this point onward, we're going to be automating the creation of RDS and focusing on the specific pieces of functionality that you need to understand.

But at this point, that's everything that you need to do in this demo.

So go ahead, complete the video and when you're ready, I look forward to you joining me in the next.

Welcome back and in this demo lesson you're going to get some experience of how to provision an RDS instance and how to migrate a database from an existing self-managed MariaDB database instance through to RDS.

So over the next few demo lessons in this section of the course, you're going to be evolving your database architecture.

We're going to start with a single database instance, then we're going to add multi-AZ capability as well as talking about backups and restores.

But in this demo lesson specifically, we're going to focus on provisioning an RDS instance and migrating data into it.

Now in order to get started with this demo lesson, as always make sure that you're logged into the general AWS account, so the management account of the organization and you need to have the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment link that you'll need to use to provision this demo lesson's infrastructure.

So go ahead and click on that link now.

That's going to move you to a quick create stack screen.

The stack name should be pre-populated with migrate to RDS.

Scrolling down all of the parameter values will be pre-populated.

All you need to do is to click on the capabilities checkbox and then create stack.

There's also a lesson commands document linked to this lesson and I'd suggest you go ahead and open that in a new tab because you'll be referencing it as you move through this demo lesson.

Now you'll notice that this will look similar to the previous demo lesson's lesson commands document, but it has one small difference.

The initial command way of doing the backup of the source database, because that source database is going to be stored on a separate MariaDB database running on a separate EC2 instance, instead of taking the backup from the local instance, in this case it's connecting to a separate EC2 instance.

Otherwise, most of these commands are similar to the ones you used in the previous demo lesson.

Now you're going to need to wait for this stack to move into a create complete state before you continue the demo.

So go ahead and pause the video, wait for your stack to change to create complete and then you're good to continue.

Okay, so that cloud formation stack has now moved into a create complete state and it's created a familiar set of infrastructure.

Let's go ahead and click on the services drop down and then move to the EC2 console and just take a look.

So if we click on instances, you'll see that we have the same two instances as you saw in the previous demo lesson.

So we have A4L-WordPress, which is running the Apache web server and the WordPress application.

And then we have A4L-DB-WordPress and this is running the separate MariaDB database instance.

So what we need to do in order to perform this migration is first create the WordPress blog itself and the sample blog post.

And this is the same thing that we did in the previous demo.

So we should be able to go through this pretty quickly.

So go ahead and select the A4L-WordPress instance and copy its public IP version for address into your clipboard and then open that in a new tab.

And again, make sure not to use the open address because this uses HTTPS.

So copy the public IP version for address and then open that in a new tab.

Again, we're going to call the site the best cats.

We're going to use admin for the username.

And then for the password, let's go back to the CloudFormation tab.

Make sure you've got the migrate to RDS stack selected and then click on parameters.

We're going to use the same database password.

So copy that into your clipboard and replace the automatically generated one with the animals for live complex password.

And then enter test@test.com into the email box and click on install WordPress.

Once installed, click on login.

You'll need to use the admin username and the same password.

Click on login.

Then we're going to go to posts.

We're going to select the existing Hello World post.

Select trash this time.

Then click on add new.

Close down this dialog for title.

We're going to use the best cats ever.

Click on the plus.

Select gallery.

At this point, go ahead and click the link that's attached to this lesson to download the blog images.

Once downloaded, extract that zip file and you'll get four images.

Once you've got those images ready, click on upload, locate those images, select them and click on open.

Wait for them to load in.

Select publish and publish again.

And that saved the images onto the application instance and added the data for this post onto the separate MariaDB database.

So now we have this simple working blog.

Let's go ahead and look at how we can provision an RDS instance and how we can migrate the data into that RDS instance.

So move back to the AWS console.

Click on the services drop down and type RDS into the search box and open that in a new tab.

Now, as I've mentioned in the theory parts of this section, RDS is a managed database server as a service product from AWS.

It allows you to create database instances and those instances can contain databases that your applications can make use of.

Now to provision an RDS instance, the first thing that we need to do is to create a subnet group.

Now a subnet group is how we inform RDS which subnets within a VPC we want to use for our database instance.

So first we need to create a subnet group.

So select subnet groups on the menu on the left and then create a DB subnet group.

Now we're going to use a4lsn group, so animals for life subnet group for both the name and for the description.

And then select the VPC drop down and we're going to select the a4l-vpc1vpc.

So this is the animals for life VPC which has been created by the one click deployment that you used at the start of this demo.

Now once we've selected a name and a description and a VPC for this subnet group, then what we need to do is select the subnets that this database will be going into.

So we're going to select the database subnets in US East 1A, US East 1B and US East 1C.

So click on the availability zone drop down and pick those three availability zones.

So 1A, 1B and 1C.

Once we've selected the availability zones that this subnet group is going to use, next we pick the subnets.

So click on the drop down.

Now we want to pick the database subnets within the animals for life VPC and all we can see here are the IP address ranges.

So to help us with this click on the services drop down, type VPC and then open that in another new tab.

Once that loads, go ahead and click on subnets, sort the subnets by name and then locate sn-dba, dbb and dbc.

And just move your cursor across to the right hand side and note what the IP address ranges are for those different database subnets.

So 16, 80 and 144.

Go back to the RDS console, click on the subnets drop down and we need to pick each of those three subnets.

So 16, 80 and 144.

So these represent the database subnets in availability zone 1A, 1B and 1C.

And then once we've configured all of that information, we can go ahead and click on create to create this subnet group.

So this subnet group is something that we use when we're provisioning an RDS instance.

And as I mentioned moments ago, it's how RDS determines which subnets to place database instances into.

Now when we're only using a single database instance, then that decision is fairly easy.

But RDS deployments can scale up to use multiple replicas in multiple different availability zones.

You can have multi-AZ instances, read replicas.

Aurora has a cluster architecture which we'll talk about later in this section.

And so subnet groups are essential to inform RDS which subnets to place things into.

So now that we've configured that subnet group, let's go ahead and provision our RDS database instance.

So to do that, click on databases and then we're going to create a database.

So click on create database.

Now when you're creating a database, you have the option of using standard create where you have visibility of all of the different options and then easy create which applies some best practice configurations.

Now I want you to get the maximum experience possible, so we're going to use standard create.

Now when you're creating an RDS database instance, you have the ability to pick from many different engines.

So some of these are commercial like Oracle or Microsoft SQL Server.

And with some of these, you have the option of either paying for a license included with RDS or you can bring your own license.

For other database engines, there isn't a commercial price to pay for their usage and so they're much cheaper to use.

But you should select the engine type which is compatible with your application.

Now we're going to be talking about Amazon Aurora in dedicated lessons later in this section of the course.

Amazon Aurora is an AWS designed database product which has compatibility with MySQL and PostgreSQL.

For this demo lesson, we're going to use MySQL.

So go ahead and select MySQL and it's going to be using MySQL Community Edition.

So now let's just scroll down and step through some of the other options that we get to select when provisioning an RDS instance.

Now for all of these database engines, you have the ability to pick different versions of that engine.

And this is fairly critical because there are different major and minor versions that you can select from.

And different versions of these have different limitations.

So for example, we're going to be talking about snapshots later in this section.

And if you want to take a snapshot of an RDS database and then import that into an Aurora cluster, you need to pick a compatible version.

And then Aurora Serverless which we'll be talking about later on in this section has even more restrictions.

Now to keep things simple, I want you to ignore what version I pick in this video and instead look in this lesson's description and pick the version that I indicate in the lesson description because I'll keep this updated if AWS make any changes.

Now you can choose to use a template.

These templates give you access to only the options which are relevant for the type of deployment that you're trying to use.

So in production, you would pick the production template.

If you have any smaller or less critical dev or test workloads, then you could pick this template.

If you want to ensure that you can only select free tier options, then you should pick this template.

And that's what we're going to do in this demo because we want this demo to fall under the free tier.

So click on the free tier template.

I'll be talking about availability and durability later in this section because we've selected free tier only.

We don't have the ability to create a multi AZ RDS deployment.

And now we need to provide some configuration information about the database instance specifically.

So the first thing that we need to do is to provide a database instance identifier.

So this is the way that you can identify one particular instance from any other instances in the AWS account in the current region.

So this needs to be unique.

So we're going to use a four L WordPress for this database instance.

Then we need to pick a username which will be given admin privileges on this database instance.

And we're going to replace admin with a four L WordPress.

So we're going to use this for both the database identifier and the admin user of this database.

Now for the password for this admin user, we're going to move back to the cloud formation console and we're going to use this same animals for life complex password.

So copy that into your clipboard and paste it in for the password and the confirm password box.

And this just keeps things consistent between the self banished database and the RDS database.

Scroll down further still and it's here where you can select the database instance class to use.

Now because we've selected free tier only, we're limited as to what database size and type we can pick.

If we'd have selected production or dev test from the templates above, we would have access to a much wider range of database instance classes, both standard, memory optimized and burstable.

But because we've selected the free tier template, we're limited as to what we can select.

Now this might change depending on when you're watching this demonstration, but at the point I'm recording this video, it's db.t3.micro.

So don't be concerned if you see something different in this box.

Just make sure that you select the type of instance which falls under the free tier.

Then continue scrolling down and we need to pick the size of storage and the type of storage to use for this RDS instance.

Now whether you need to select this is dependent on what engine type you pick.

If you select Aurora, which we'll be talking about later on in this section, then you don't need to pre-allocate storage.

If you're using the MySQL version of RDS, then you do need to set a type of storage and a size of storage.

Now we're going to use the minimum which is 20GIB because our requirements for this database are relatively small.

And if we wanted to, if this was production, we could set storage autoscaling.

And this allows RDS to automatically increase the storage when a particular threshold is met.

But again, because this is a demo and it's only using a very small blog, we don't need storage autoscaling.

So go ahead and uncheck that option.

Now we need to select a VPC for this RDS instance to go into.

So click in the drop down and select the Animals for Life VPC.

So that's A4L-VPC1.

And then we need to pick a subnet group.

Now this is the thing that we've just created.

We only have one in this account, so there's nothing else to select.

But this is how we can advise RDS on which subnets to use inside the VPC.

Scroll down further still and we can specify whether we want this database to be available.

We want this database to be publicly accessible.

So this is whether we want instances and devices outside the VPC to be able to connect to this database.

This obviously comes with some security trade-offs.

And because we don't need that in this demonstration, because the only thing that we want to connect to this RDS instance is our WordPress instance, which is in the same VPC, then we can select Not to Use Public Access.

So make sure the No option is selected.

Now the way that you control access to RDS is you allocate a VPC security group to that instance.

So we could either choose an existing security group or we could create a new one.

So it's this security group which surrounds the network interfaces of the database and controls access to what can go into that database.

So we want to create a new VPC security group.

So we want to make that option.

We're going to call the security group A4LVPC-RDS-SG.

And we need to remember to update this so that our WordPress instance can communicate with our RDS instance.

And we'll do that in the next step.

If we wanted to pick a specific availability zone for this instance to go into, then we could select one here or we can leave it up to RDS to pick the most suitable.

So we can select No Preference.

Continue scrolling down.

We won't change the Database Authentication option because we want to allow password authentication.

Continue scrolling down and we're going to expand Additional Configuration.

By default, an RDS instance is created with no database on that instance.

In this case, because we're migrating an existing WordPress database into RDS, we're going to go ahead and create an initial database.

And to keep things easy and consistent, we're going to use the same name, so A4L WordPress.

Now you can enable automatic backups for RDS instances.

And I'll be talking about these in a separate theory lesson.

If you do select automatic backups, then you can also pick a backup retention period as well as a backup window.

So we've got Advanced Monitoring, various log exports.

We don't need to use any of those.

You can also set the Maintenance window for an RDS instance.

So when Maintenance will be performed, you can enable Deletion Protection if you want.

If this is a production database, we don't need to do that.

What we're going to do is scroll all the way down to the bottom and then click on Create Database.

Now this process can take some time.

I've seen it take anywhere from five to 45 minutes.

And we're going to need this to be finished before we move on to the next step.

So this seems like a great time to end this video.

It gives you the opportunity to grab a coffee or stretch your legs.

Wait for this database creation to finish.

And then when you're ready, I'll look forward to you joining me in part two of this video.

Welcome back and in this video which is the first of this series I'm going to step through the architecture of the relational database service known as RDS.

Now this video will focus on the architecture of the product with upcoming videos going into specific features in more depth.

Now we do have a lot to cover so let's jump in and get started.

Now I've heard many people refer to RDS as a database as a service or DB AAS product.

Now details are important and you need to understand why this is not the case.

A database as a service product is where you pay money and in return you get a database.

This isn't what RDS does.

With RDS you pay for and receive a database server so it would be more accurate to call it a database server as a service product.

Now this matters because it means that on this database server or instance which RDS provides you can have multiple databases.

RDS provides a managed version of a database server that you might have on-premises only with RDS you don't have to manage the hardware, the operating system or the installation as well as much of the maintenance of the DB engine and RDS of course runs within AWS.

Now with RDS you have a range of database engines to use including MySQL, Maria DB, PostgreSQL and then commercial databases such as Oracle and Microsoft SQL.

Some of these are open source and some are commercial and so there will be licensing implications and if appropriate for the exam that you're working towards there will be a separate video on this topic.

Now there's one specific term that I want you to disassociate from RDS and that's Amazon Aurora.

You might see Amazon Aurora discussed commonly along with RDS but this is actually a different product.

Amazon Aurora is a custom database engine and product created by AWS which has compatibility with some of the above engines but it was designed entirely by AWS.

Many of the features I'll step through while talking about RDS are different for Aurora and most of these are improvements so in your mind separate Aurora from RDS.

So in summary RDS is a managed database server as a service product.

It provides you with a database instance so a database server which is largely managed by AWS.

Now you don't have access to the operating system or SSH access.

Now I have a little asterisk here because there is a variant of RDS called RDS custom where you do have some more low level access but I'll be covering that in a different video if required.

In general when you think about RDS think no SSH access and no operating system access.

Now what I think might help you at this point is to look at a typical RDS architecture visually and then over the remaining videos in this series I'll go into more depth on certain elements of the product.

So RDS is a service which runs within a VPC so it's not a public service like S3 or DynamoDB.

It needs to operate in subnets within a VPC in a specific AWS region and for this example let's use US East 1 and to illustrate some cross region bits of this architecture our second region will be AP Southeast 2 and then we're going to have within US East 1 a VPC and let's use three availability zones here A, B and C.

Now the first component of RDS which I want to introduce is an RDS subnet group.

This is something that you create and you can think of this as a list of subnets which RDS can use for a given database instance or instances.

So in this case let's say that we create one which uses all three of the availability zones.

In reality this means adding any subnets in those three availability zones which you want RDS to use and in this example I'm going to actually create another one.

We going to have two database subnet groups and you'll see why in a second.

In the top database subnet group let's say I add two public subnets and in the bottom database subnet group let's say three private subnets.

So when launching an RDS instance whether you pick to have it highly available or not and I'll talk about how this works in an upcoming video you need to pick a DB subnet group to use.

So let's say that I picked the bottom database subnet group and launched an RDS instance and I chose to pick one with high availability.

So it would pick one subnet for the primary instance and another for the standby.

It picks at random unless you indicate a specific preference but it will put the primary and standby within different availability zones.

Now because these database instances are within private subnets it means that they would be accessible from inside the VPC or from any connected networks such as on-premises networks connected using VPNs or Direct Connect or any other VPCs which appeared with this one and I'll cover all of those topics elsewhere in the course if I haven't already done so.

Now I could also launch another set of RDS instances using the top database subnet group and the same process would be followed assuming that I picked to use multi AZ.

RDS would pick two different subnets in two different availability zones to use.

Now because these are public subnets we could also if we really wanted to elect to make these instances accessible from the public internet by giving them public addressing and this is something which is really frowned upon from a security perspective but it's something that you need to know is an option when deploying RDS instances into public subnets.

Now you can use a single DB subnet group for multiple instances but then you're limited to using the same defined subnets.

If you want to split databases between different sets of subnets as with this example then you need multiple DB subnet groups and generally as a best practice I like to have one DB subnet group for one RDS deployment.

I find it gives me the best overall flexibility.

Okay so another few important aspects of RDS which I want to cover.

First RDS instances can have multiple databases on them.

Second every RDS instance has its own dedicated storage provided by EBS so if you have a multi AZ pair primary and standby each has their own dedicated storage.

Now this is different than how Amazon Aurora handles storage so try to remember this architecture for RDS each instance has its own dedicated EBS provided storage.

Now if you choose to use multi AZ as in this architecture then the primary instances replicate to the standby using synchronous replication.

Now this means that the data is replicated to the standby as soon as it's received by the primary.

It means the standby will have the same set of data as the primary so the same databases and the same data within those databases.

Now you can also decide to have read replicas.

I'll be covering what these are and how they work in another dedicated video but in summary read replicas use asynchronous replication and they can be in the same region but also other AWS regions.

These can be used to scale read load or to add layers of resilience if you ever need to recover in a different AWS region.

Now lastly we also have backups of RDS.

There is a dedicated video covering backups later on in this section of the course but just know that backups occur to S3.

It's to an AWS managed S3 bucket so you don't see the bucket within your account but it does mean that data is replicated across multiple availability zones in that region.

So if you have an AZ failure backups will ensure that your data is safe.

If you use multi AZ mode then backups occur from the standby instance which means no negative performance impact.

Now this is the basic product architecture.

I'll be expanding on all of these key areas in dedicated videos as well as giving you the chance to get practical experience via some demos and mini projects if appropriate.

For now let's cover one final thing before we finish this video and that's the cost architecture of RDS.

So before I finish the video I want to talk about RDS costs because it's a database server as a service product you're not really build based on your usage.

Instead like EC2 which RDS is loosely based on you'll build for resource allocation and there are a few different components to RDS's cost architecture.

First you've got the instance size and type.

Logically the bigger and more feature rich the instance the greater the cost and this follows a similar model to how EC2 is built.

The fee that you see is an hourly rate but it's billed per second.

Next we have the choice of whether multi AZ is used or not because multi AZ means more than one instance there's going to be additional cost.

Now how much more cost depends on the multi AZ architecture which I'll be covering in detail in another video.

Next is a per gig monthly fee for storage which means the more storage you use the higher the cost and certain types of storage such as provisioned IOPS cost more and again this is aligned to how EBS works because the storage is based on EBS.

Next is the data transfer costs and this is a cost per gig of data transfer in and out of your DB instance from or to the internet and other AWS regions.

Next we have backups and snapshots so you get the amount of storage that you pay for for the database instance in snapshot storage for free.

So if you have 2 TB of storage then that means 2 TB of snapshots for free.

Beyond that there is a cost and this cost is gig per month of storage so the more data is stored the more it costs the longer it's stored the more it costs.

One TB for one month is the same cost as 500 GB for two months so it's a per GB month cost and then finally we have any extra costs based on using commercial DB engine types and again I'll be covering this if appropriate in a dedicated video elsewhere in the course.

Okay so at this point that is everything I wanted to cover in this video as I mentioned at the start this is just an introduction to RDS architecture.

We're going to be going into more detail on specific key points in upcoming videos but for now that's everything I wanted to cover.

So go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome to this demo lesson where you're going to migrate from the monolithic architecture on the left of your screen towards a tiered architecture on the right.

Essentially you're going to split the WordPress application architecture, you're going to move the database from being on the same server as the application to being on a different server and this will form step one of moving this architecture from being a monolith through to being a fully elastic architecture.

Now this is the first stage of many but it is a necessary one.

Now in order to perform this demonstration you're going to need some infrastructure.

Before we apply the infrastructure just make sure that you're logged in to the general AWS account, so the management account of the organization and as always you need to have the Northern Virginia region selected.

Now once you've got both of those set there's a one-click deployment link attached to this lesson so go ahead and click on that link.

What this is going to do is deploy the Animals for Life base infrastructure, it's going to deploy the monolithic WordPress application instance and it's also going to deploy a separate MariaDB database instance that you're going to use as part of the migration.

Now everything set, the stack name should be set to a suitable default, all you need to do is to scroll all the way down to the bottom, check this capabilities box and click on create stack.

Now also attached to this lesson is a lesson commands document which contains all the commands you'll be using throughout this demo.

So go ahead and open that in a new tab, you'll be referencing it constantly as you're making the adjustments to the WordPress architecture.

Now we're going to need this CloudFormation stack to be fully complete before we can continue so go ahead and pause the video and resume once the CloudFormation stack moves into a create complete state.

So now the stacks moved into a create complete state, we're good to continue.

Now this has created the base Animals for Life infrastructure which includes a number of EC2 instances so let's take a look at those, let's click on services and then locate and open EC2 in a brand new tab.

Once you're at the EC2 console if you do see any dialogues around user interface updates then just go ahead and close those down and then click on instances running.

Once you're here you'll see two EC2 instances, one will be called A4L-WordPress and this is the monolith so this is the EC2 instance which contains the WordPress application and the built-in database.

So this is the WordPress installation that we're going to migrate from and then this instance A4L-DB-WordPress this contains a standalone MariaDB installation so we're going to migrate the database for WordPress from this instance onto the DB instance and this will create a tiered application architecture rather than the monolith which we currently have.

So step number one is to perform the WordPress installation so to do that I want you to go ahead and copy the public IP version for address of the WordPress EC2 instance into your clipboard and then open it in a new tab.

Now be careful not to use the open address link that will use HTTPS which we're not currently using so copy the IP address into your clipboard and open that in a new tab.

Now when you do that you'll see a familiar WordPress installation dialog we're going to create a simple blog for site title go ahead and call it the best cats for username pick admin and then for the password instead of using the randomly selected one go ahead and use this same complex password that we've used for the CloudFormation template so this is animals for life but with number substitution.

So if you go back to your CloudFormation tab and go to the parameters tab this is the same password that we use for the DB password and the DB root password.

Now of course in production this is incredibly bad practice we're just doing it in this demo to keep things simple and avoid any mistakes.

So back to the WordPress installation screen site title the best cats username admin this for the password and then just go ahead and type a fake email so I don't want to use my real email for this I'm going to type test at test.com you can do the same and then go ahead and click on install WordPress so this is installed the WordPress application and it's using the Maria DB server that's on the same EC2 instance so part of the same monolith.

So we're going to log in we'll need to type admin and then use the animals for life strong password and click on login and once we logged in we're going to create a simple blog post so click on posts we're going to select the existing hello world post select trash this time then click on add new then we're going to add a new post we can close down this introduction dialogue and for title go ahead and type the best cats ever and then some exclamation points next click on this plus sign and we're going to add a gallery now at this point you're going to need some images to upload to this blog post I've attached an images link to this lesson so if you go ahead and click that link it will download a zip file if you extract that zip file it's going to contain four image files all four of my cats so at this point once you've downloaded and extracted that file go ahead and click on upload locate those images there should be four select them all and click on open that will add these images to this blog post and once you've added them all you can go ahead and click on publish and then publish again and this will publish this blog post so it will add data to the database that's running on the monolithic application instance as well as store these images on the local instance file system now making a point of mentioning that these images are stored on the file system because as you'll see later in the course this is one of the things that we need to migrate when we're moving to a fully elastic architecture we can't have images stored on the instances themselves we need to move that to a shared file system for now though we're focusing on the database so at this point we have the working blog the images for this blog are stored on the local file system of a4l-wordpress and the data for that blog post is stored on the MariaDB database that's also running on this EC2 instance so the next step of this demo lesson is that you're going to migrate the data from a4l-wordpress onto a4l-db-wordpress and this is an isolated MariaDB instance this is dedicated for the database so to do this migration select a4l-wordpress right click we're going to connect to this instance we'll be using EC2 instance connect so just make sure that the username is set to EC2-user and then click on connect now this is where you're going to be using the commands that are stored within the lesson commands document so you need to make sure that you have this ready to reference because it's far easier to copy and paste these commands and then adjust any placeholders rather than type them out manually because that's prone to errors the first step is to get the data from the database that's running on this monolithic application instance and store it in a file on disk so that's the first thing we need to do we need to do a backup of the database into a .sql file now to do that we use this command so it's a utility called my sql-dump it uses the -u to specify the user that we're going to be using to connect to the database then we use -p to specify that we want to provide a password and we could either provide the password on the command line or we could have it prompt us now if we supply the password with no spaces next to this -p then it will accept it as input on this command if we don't specify anything so there's a space here then it's going to ask us for the password the next thing we specify is the database name that we want to do the dump of in this case it's a4l WordPress which is the database for the animals for life WordPress instance now if we just run this command on its own it would output the dump so all of the data in the database to standard output which in this case is our screen we don't want it to do that we want it to store the results in a file called a4l WordPress dot sql and so we use this symbol which means that it's going to take the output of this component of the command and it's going to redirect it into this file so let's go ahead and run this command and it's going to prompt us for the password for this database now to get that go back to cloud for information make sure parameters are selected and it's this password that we need which is the DB password so copy that into your clipboard go back to the instance paste that in press enter and that will output all the data in the database to this file now you won't see any indication of success or failure but if you do an LS space -la and press enter one of the files that you'll see is a4l WordPress dot sql so now we have a copy of the WordPress database containing our blog post the next thing that we need to do is to take this file this backup of the database and inject it into the new database that we want to use so the dedicated Maria DB EC2 instance and to do that we're going to use this command so this command has two components the first component is this which connects to the Maria DB database instance the second component is this which takes the backup that we've just made and it feeds it into this command so this backup contains all the necessary definitions to create a new database and inject the data required this component of the command just allows us to connect to this new dedicated Maria DB instance now there are some place holders that we need to change the database name that we're going to use is the same so a4l WordPress we're still going to want to be prompted for a password so -p is what we use this time though we're going to connect using a user called a4l WordPress so we're not using the root user we're going to connect to this separate Maria DB database instance using a user a4l WordPress the only thing that we need to change is that we need to connect to a non-local host so when we used the mysql dump command we didn't specify a host to connect to and this defaulted to local host so the current machine in the case of this command we're operating with a separate server this dedicated EC2 instance which is running the Maria DB database server so a4l -db -wordpress we need to connect to this so what we'll need to connect to this is the private IP version for address of this separate database instance so select it look for private IP version for addresses and then click on the icon next to this to copy the private IP version for address of this separate database server into your clipboard then return to the application instance and we need to replace the placeholder here with that value so make sure that you're one space after the end of this placeholder and just delete this leave a space between -h and where the cursor is and then paste in that IP address so this is going to connect to this separate EC2 instance using its private IP it's going to use the a4l WordPress user it will prompt us for a password it will perform the operation on the a4l WordPress database and it's going to use the contents of this backup file to perform those tasks so go ahead and press enter and you'll be prompted for a password now again it's the same password this has all been set up as part of the cloud formation one-click deployment this lesson is about the migration process not setting up a database server so I've automated this component of the infrastructure so copy the DB password into your clipboard go back to the instance paste it in and press enter so now we've uploaded our WordPress application database into this separate MariaDB database server the next step is to configure WordPress to point at this new database server so to do that cd space forward slash var forward slash www forward slash html and press enter and then we're going to run a shudu space nano which is a text editor space wp-config.php and this is the WordPress configuration file so press enter now what we're looking for if we scroll down is we're looking for the line which says define and then a space and then DB host so this is the database host that WordPress attempts to connect to and currently it's set to local host which means it will use the database on the same EC2 instance as the application we're going to delete this local host so delete until we have two single quotes and then make sure that you still have the private IP version for address of this separate database instance in your clipboard if you don't just go ahead and copy it again from the EC2 console and then paste that in place of local host so now you should see DB underscore host and this now represents this private IP address and now the private IP address that you should use here will be different you need to use your private IP address of your a4l - DB - WordPress EC2 instance so now that you've updated this configuration file press control o and enter to save and then control x to exit out of editing this file now this now means that the WordPress instance is going to be communicating with the separate MariaDB database instance let's verify that let's go back to the tab that we have to our WordPress application and let's just go ahead and do a refresh if everything's working as expected we should see that the blog reloads successfully now this means that this blog is now pointing at this separate MariaDB database instance to be doubly sure of this though let's go back to the WordPress instance and let's shut down the MariaDB database server and we do that using this command so shudu space service space MariaDB space and then stop so type or copy and paste that command in and press enter and that's going to stop the MariaDB database service which is running on a4l WordPress so now the only MariaDB database that we have running is on the a4l - DB - WordPress EC2 instance now we can go back to the WordPress tab and hit refresh and assuming it loads in as it does in my case this now confirms that WordPress is communicating with this dedicated MariaDB EC2 instance now the reason why I wanted to step you through all these tasks in this demo lesson is the time a firm believer that in order to understand best practice architecture you need to understand bad architecture and as I mentioned in the theory lesson there is almost no justification for running your own self-managed database server on an EC2 instance in almost all situations it's preferable to use the RDS service but I need you to understand exactly how the architecture works when you're self managing a database and how to migrate from a monolithic all-in-one architecture through to having a separate self managed database in the demo lesson that's coming up next in the course you're going to migrate from this through to an RDS instance so that's step two but at this point you've done everything that I wanted you to do in this demo lesson you've implemented the architecture that's on screen now on the right all we need to do is to tidy up all of the infrastructure that we've used within this lesson so to do that it's nice and easy just go back to the cloud formation console make sure that you have the monolith to EC2 DB stack selected click on the delete button and then confirm that deletion and that stack deleting will clean up all of the infrastructure that we've used throughout this demo lesson and it will return the account into the same state as it was at the start of the lesson at this point you've completed all of the tasks that I want you to do so I hope you've enjoyed this demo lesson go ahead and complete this video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this lesson I want to cover something which can be argued is bad practice to do inside AWS and that's running databases directly on EC2.

As you'll find out in this section of the course there are lots of AWS products which provide database services so running any database on EC2 at best requires some justification.

In this lesson I want to step through why you might want to directly run databases on EC2 and why it's also a bad idea.

It's actually always a bad idea to run databases on EC2.

The argument really is whether the benefits to you or your business outweigh the fact that it is a bad idea.

So let's jump in and take a look at some of the reasons why you should and shouldn't run databases on EC2.

Generally when people think about running databases on EC2 they picture one of two things.

First, a single instance and on this instance you're going to be running a database platform, an application of some kind and perhaps a web server such as Apache.

Or you might picture a simple split architecture where the database is separated from the web server and application.

So you'll have two instances, probably smaller instances than the single large one.

And architecturally I hope this makes sense so far.

So far in the course with the Animals for Life WordPress application stack example we've used the architecture on the left.

A single EC2 instance with all of the application tiers or components on one single instance.

Crucially one single instance running within a single availability zone.

Now if you have a split architecture like on the right you can either have both EC2 instances inside the same availability zone or you could split the instances across two.

So AZA and AZB.

Now when you change the architecture in this way, when you split up the components into separate instances, whether you decide to put those both in the same availability zone or split them, you need to understand that you've introduced a dependency into the architecture.

The dependency that you've introduced is that there needs to be reliable communication between the instance running the application and the database instance.

If not the application won't work.

And if you do decide to split these instances across multiple availability zones then you should also be aware that there is a cost for data when it's transiting between different availability zones in the same region.

It's small but it does exist.

Now that's in contrast to where communications between instances using private IPs in the same availability zone is free.

So that's a lot to think about from an architectural perspective.

But that's what we mean when we talk about running databases on EC2.

This is the architecture.

Generally one or more EC2 instances with at least one of them running the database platform.

Now there are some reasons why you might want to run databases on EC2 in your own environment.

You might need access to the operating system of the database and the only way that you can have this level of access is to run on EC2 because other adbs products don't give you OS level access.

This is one of those things though that you should really question if a client requests it because there aren't many situations where OS level access is really a requirement.

Do they need it?

Do they want it?

Or do they only think that they want it?

So if you have a client or if your business states that they do need OS level access the first thing that you should do is question that statement.

Now there are some database tuning things which can only be done with root level access and because you don't have this level of access with managed database products then these values or these configuration options won't be tuneable.

But in many cases and you'll see this later on in this section AWS does allow you to control a lot of these parameters that historically you would need root access for without having root access.

So again this is one of those situations where you need to question any situation where it's presented to you that you need database root access.

It's worth noting often that it's an application vendor demanding this level of access not the business themselves.

But again it's often the case that you need to delve into the justifications.

This level of access is often not required and a lot of software vendors now explicitly support AWS's managed database products.

So again verify any suggestion of this level of access.

Now something that is often justified is that you might need to run a database or a database version which AWS don't provide.

This is certainly possible and more so with emerging types of databases or databases with really niche use cases.

You might actually need to implement an application with a particular database that is not supported by AWS and any of its managed database products.

And in that case the only way of servicing that demand is to install that database on EC2.

So that's one often justified reason for running databases on EC2.

Or it might be that a particular project that you're working on has really, really detailed and specific requirements and you need a very specific version of an OS and a very specific version of a DB in combination which AWS don't provide.

Or you might need or want to implement an architecture which AWS also don't provide.

Certain types of replication done in certain ways or at certain times.

Or it could be something as simple as the decision makers in your organization just want a database running on EC2.

You could argue that they're being unreasonable to just demand a database running on EC2 but in many cases you might not have a choice.

So it can always be done.

You can run databases on EC2 as long as you're willing to accept the negatives.

So these are all valid.

Some of them I would question or fight or ask for justification but situations certainly do exist which require you to use databases on EC2.

And I'm stressing these because I've seen tricky exam questions where the right answer is to use a database on EC2.

So I want to make sure that you've got fresh in your mind some of the styles of situations where you might actually want to run a database on EC2.

But now let's talk about why you really shouldn't put a database product on EC2.

Even with the previous screen in mind, even with all of those justifications, you need to be aware of the negatives.

And the first one is the admin overhead.

The admin overhead of managing the EC2 instance as well as the database host, the database server.

Both of these require significant management effort.

Don't underestimate the effort required to keep an EC2 instance patched or keep a database host running at a certain compatible level with your application.

You might not be able to upgrade or you might have to upgrade and keep the database version running in a very narrow range in order to be compatible with the application.

And whenever you perform upgrades or whenever you're fault finding, you need to do it out of core usage hours, which could mean additional time, stress and cost for staff to maintain both of these components.

Also don't forget about backups and disaster recovery management.

So if your business has any disaster recovery planning, running databases on EC2 adds a lot of additional complexity.

And in this area, when you're thinking about backups and DR, many of AWS's managed database products we'll talk about throughout this section include a lot of automation to remove a lot of this admin overhead.

Perhaps one of the most serious limitations though is that you have to keep in mind that EC2 is running in a single availability zone.

So if you're running on an EC2 instance, keep in mind you're running on an EBS volume in an EC2 instance.

Both of those are within a single availability zone.

If that zone fails, access to the database could fail and you need to worry about taking EBS snapshots or taking backups of the database inside the database server and putting those on storage somewhere, maybe S3.

Again, it's all admin overhead and risk that your business needs to be aware of.

Another issue is features.

Some of AWS's database products genuinely are amazing.

A lot of time and effort and money have been put in on your behalf by AWS to make these products actually better than what you can achieve by installing database software on EC2.

So by limiting yourself to running databases on EC2, you're actually missing out on some of the advanced features and we'll be talking about all of those throughout this section of the course.

Another aspect is that EC2 is on or off.

EC2 does not have any concept of serverless because explicitly it is a server.

You're not going to be able to scale down easily or keep up with bursty style demand.

There are some AWS managed database products we'll talk about in this section which can scale up or down rapidly based on load.

And by running a database product on EC2, you do limit your ability to scale and you do set a base minimum cost of whatever the hourly rate is for that particular size of EC2 instance.

So keep that in mind.

So again, if you're being asked to implement this by your business, you should definitely fight this fight and get the business to justify why they want the database product on EC2 because they're missing out on some features and they're committing themselves to costs that they might not need to.

There's also replication.

So if you've got an application that does need replication, there are the skills to set this up, the setup time, the monitoring and checking for its effectiveness and all of this tends to be handled by a lot of AWS's managed database products.

So again, there's a lot of additional admin overhead that you need to keep in mind.

And lastly, we've got performance.

This relates in some way to when I talked about features moments ago.

AWS do invest a considerable amount of time into optimization of their database products and implementing performance based features.

And if you simply take an off the shelf database product and implement it on EC2, you're not going to be able to take advantage of these advanced performance features.

So keep that in mind.

If you do run database software directly on EC2, you're limiting the performance that you can achieve.

But with that out of the way, that's all of the theory and logic that I wanted to cover in this lesson.

So now you have an idea about why you should and why you shouldn't run your own database on an EC2 instance.

In the next lesson, which is a demo, we're going to take the single instance WordPress deployment that we've been using so far in the course, and we're going to evolve it into two separate EC2 instances.

One of these is going to be running Apache and WordPress.

So it's going to be the application server.

And the other is going to be running a database server MariaDB.

Now this kind of evolution is best practice, at least as much as it can ever be best practice to run a self-managed database platform.

Now the reason we're doing this is we want to split up our single monolithic application stack.

We want to get it to the point so the database is not running on the same instance as the application itself.

Because once we've done that, we can move that database into one of AWS's managed database products later in this section.

And that will allow us to take advantage of these features and performance that these products deliver.

It's never a good idea to have a single monolithic application stack when you can avoid it.

So the way that we're running WordPress at the moment is not best practice for an enterprise application.

So by splitting up the application from the database, as we go through the course, it will allow us to scale each of these independently and take advantage independently of different AWS products and services, which can help us improve each component of our application.

So with that being said, go ahead and finish up this video.

And then when you're ready, you can join me in the next lesson, which is going to be a demo where we're going to split up this monolithic WordPress architecture into two separate compute instances.

Welcome to this lesson where I want to provide a really quick theoretical introduction to Acid and Base, which are two database transaction models that you might encounter in the exam and in the real world.

Now this might seem a little abstract, but it does feature on the exam, and I promise in real world usage knowing this is a database superpower.

So let's jump in and get started.

Acid and Base are both acronyms and I'll explain what they stand for in a moment.

But they are both database transaction models.

They define a few things about transactions to and from a database, and this governs how the database system itself is architected.

At a real foundational level, there's a computer science theorem called the CAP theorem, and it stands for consistency, availability, and partition tolerance.

Now let's explore each of these quickly because they really matter.

Consistency means that every read to a database will receive the most recent write or it will get an error.

On the other hand, availability means that every request will receive a non-error response, but without the guarantee that it contains the most recent write, and that's important.

Partition tolerance means that the system can be made of multiple network partitions, and the system continues to operate even if there are a number of dropped messages or errors between these network nodes.

Now the CAP theorem states that any database product is only capable of delivering a maximum of two of these different factors.

One reason for this is that if you imagine that you have a database with many different nodes, all of these are on a network.

Imagine if communication fails between some of the nodes or if any of the nodes fail.

Well you have two choices if somebody reads from that database.

You can cancel the operation and thus decrease the availability but ensure the consistency, or you can proceed with the operation and improve the availability but risk the consistency.

So as I just mentioned, it's widely regarded as impossible to deliver a database platform which provides more than two of these three different elements.

So if you have a database system which has multiple nodes and if a network is involved, then you generally have a choice to provide either consistency or availability, and the transaction models of ACID and BASE choose different trade-offs.

ACID focuses on consistency and BASE focuses on availability.

Now there is some nuance here and some additional detail but this is a high-level introduction.

I'm only covering what's essential to know for the exam.

So let's quickly step through the trade-offs which each of these makes and we're going to start off with ACID.

ACID means that transactions are atomic, transactions are also consistent, transactions are also isolated, and then finally, transactions are durable.

And let's get the exam power-up out of the way.

Generally if you see ACID mentioned, then it's probably referring to any of the RDS databases.

These are generally ACID-based and ACID limits the ability of a database to scale and I want to step through some of the reasons why.

Now I'm going to keep this high-level but I've included some links attached to this lesson if you want to read about this in additional detail.

In this lesson though I'm going to keep it to what is absolutely critical for the exam.

So let's step through each of these individually.

Atomic means that for a transaction either all parts of a transaction are successful or none of the parts of a transaction are successful.

Consider if you run a bank and you want to transfer $10 from account A to account B.

That transaction will have two parts.

Part one will remove $10 from account A and part two will add $10 to account B.

Now you don't want a situation where the first part or the second part of that transaction can succeed on its own and the other part can fail.

Either both parts of a transaction should be successful or no parts of the transaction should be applied and that's what atomic means.

Now consistent means that transactions applied to the database move the database from one valid state to another.

Nothing in between is allowed.

In databases such as relational databases there may well be links between tables where an item in one table must have a corresponding item in another where values might need to be in certain ranges and this element just means that all transactions need to move the database from one valid state to another as per the rules of that database.

Isolated means that because transactions to a database are often executed in parallel they need not to interfere with each other.

Isolation ensures that concurrent executions of transactions leave the database in the same state that would have been obtained if transactions were executed sequentially.

So this is essential for a database to be able to run lots of different transactions at the same time maybe from different applications or different users.

Each of them need to execute in full as they would do if they were the only transaction running on that database.

They need not to interfere with each other and then finally we have durable which means that once a transaction has been committed it will remain committed even in the case of a system failure.

Once the database tells the application that the transaction is complete and committed once it's succeeded that data restored somewhere that system failure or power failure or the restart of a database server or node won't impact the data.

Now most relational database platforms use acid-based transactions it's why financial institutions generally use them because it implements a very rigid form of managing data and transactions on that data but because of these rigid rules it does limit scalability.

Now next we have base and base stands for basically available it also stands for soft state and then lastly it stands for eventually consistent and again this is super high level and I've included some links attached to this lesson with more information.

Now it's also going to sound like I'm making fun of this transaction model because some of these things seem fairly odd but just stick with me and I'll explain all of the different components.

Basically available means that read and write operations are available as much as possible but without any consistency guarantees.

So reads and writes are kinder or maybe.

Essentially rather than enforcing immediate consistency base modeled no-sequal databases will ensure availability of data by spreading and replicating that data across all of the different nodes of that database.

There isn't really an aim within the database to guarantee anything to do with consistency it does its best to be consistent but there's no guarantee.

Now soft state is another one which is a tiny bit laughable in a way it means that base breaks off with the concept of a database which enforces its own consistency instead it delegates that responsibility to developers.

Your application needs to be aware of consistency and state and work around the database if you need immediate consistency so if you need a read operation to always have access to all of the writes which occurred before it immediately and if the database optionally allows it then your application needs to specifically ask for it otherwise your application has to tolerate the fact that what it reads might not be what another instance of that application has previously written.

So with soft state databases your application needs to deal with the possibility that the data that you're reading isn't the same data that was written moments ago.

Now all of these are fairly fuzzy and do overlap but lastly we have the fact that base does not enforce immediate consistency it means that it might happen eventually if we wait long enough then what we read will match what has been previously written eventually.

Now this is important to understand because generally by default a base transaction model means that any reads to a database are eventually consistent so applications do need to tolerate the fact that reads might not always have the data for previous writes.

Many databases are capable of providing both eventually consistent and immediately consistent reads but again the application has to have an awareness of this and explicitly ask the database for consistent reads.

Now it sounds like base transactions are pretty bad right?

Well not really databases which use base are actually highly scalable and can deliver really high performance because they don't have to worry about all the pesky annoying things like consistency within the database they offload that to the applications.

Now DynamoDB within AWS is an example of a database which normally works in a base like way it offers both eventually and immediately consistent reads but your application has to be aware of that.

Now DynamoDB also offers some additional features which offer acid functionality such as DynamoDB transactions so that's something else to keep in mind.

Now for the exam specifically I have a number of useful defaults.

If you see the term base mentioned then you can safely assume that it means a NoSQL style database.

If you see the term acid mentioned then you can safely assume as a default that it means an RDS database but if you see NoSQL or DynamoDB mentioned together with acid then it might be referring to DynamoDB transactions and that's something to keep in mind.

Now that's everything I wanted to cover in this high-level lesson about the different transaction models.

This topic is relatively theoretical and pretty deep and there's a lot of extra reading but I just wanted to cover the essentials of what you need for the exam so I've covered all of those facts in this lesson and at this point it is the end of the lesson so thanks for watching go ahead and complete the video and then when you're ready I look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

Now, there are other types of database platforms, no SQL platforms, and this doesn't represent one single way of doing things.

So I want to quickly step through some of the common examples of no SQL databases or non-relational databases.

The first type of database in the no SQL category that I want to introduce is key value databases.

The title gives away the structure.

Key value databases consist of sets of keys and values.

There's generally no concept of structure.

It's just a list of keys and value pairs.

In this case, it's a key value database for one of the animals for life rescue centers.

It stores the date and time and a sensor reading from a feeding sensor, recording the number of cookies removed from the feeder during the previous 60 minutes.

So essentially, the key on the left stores the date and time, and on the right is the number of cookies eaten as detected by the sensor during the previous 60 minutes.

So that's it for this type of database.

It's nothing more complex than that.

It's just a list of key value pairs.

As long as every single key is unique, then the value doesn't matter.

It has no real schema nor does it have any real structure because there are no tables or table relationships.

Some key value databases allow you to create separate lists of keys and values and present them as tables, but they're only really used to divide data.

There are no links between them.

This makes key value databases really scalable because sections of this data could be split onto different servers.

In general, key value databases are just really fast.

It's simple data with no structure.

There isn't much that gets in the way between giving the data to the database and it being written to disk.

For key value databases, only the key matters.

You write a value to a key and you read a value from a key.

The value is opaque to the database.

It could be text, it could be JSON, it could be a cat picture, it doesn't matter.

In the exam, look out for any question scenarios which present simple requirements or mention data which is just names and values or pairs or keys and values.

Look out for questions which suggest no structure.

If you see any of these type of scenarios, then key value stores are generally really appropriate.

Key value stores are also used for in-memory caching.

So if you see any questions in the exam that talk about in-memory caching, then key value stores are often the right way to go.

And I'll be introducing some products later in the course which do provide in-memory key value storage.

Okay, so let's move on.

And the next type of database that I want to talk about is actually a variation of the previous model, so a variation on key value.

And it's called a wide column store.

Now, this might look familiar to start with.

Each row or item has one or more keys.

Generally, one of them is called the partition key.

And then optionally, you can have additional keys as well as the partition key.

Now, DynamoDB, which is an AWS example of this type of database, this secondary key is called the sort or the range key.

It differs depending on the database, but most examples of wide column stores generally have one key as a minimum, which is the partition key.

And then optionally, every single row or item in that database can have additional keys.

Now, that's really the only rigid part of a wide column store.

Every item in a table has to have the same key layout.

So that's one key or more keys.

And they just need to be unique to that table.

Wide column stores offer groupings of items called tables.

But they're still not the same type of tables as in relational database products.

They're just groupings of data.

Every item in a table can also have attributes.

But, and this is really important, they don't have to be the same between items.

Remember how in relational database management systems, every table had attributes and then every row in that table had to have a value for every one of those attributes.

That is not the case for most no SQL databases and specifically wide column stores because that's what we're talking about now.

In fact, every item can have any attribute.

It could have all of the attributes, so all of the same attributes between all of the items.

It could have a mixture, so mix and matching attributes on different items.

Or an item could even have no attributes.

There is no schema, no fixed structure on the attribute side.

It's normally partially opaque for most database operations.

The only thing that matters in a wide column store is that every item inside a table has to use the same key structure and it has to have a unique key.

So whether that's a single partition key or whether it's a composite key, so a partition key and something else.

If it's a single key, it has to be unique.

If it's a composite key, the combination of both of those values has to be unique.

That's the only rule for placing data into a table using wide column stores.

Now DynamoDB inside AWS is an example of this type of database.

So DynamoDB is a wide column store.

Now this type of database has many users.

It's very fast.

It's super scalable.

And as long as you don't need to run relational operations such as SQL commands on the database, it often makes the perfect database product to take advantage of, which is one of the reasons why DynamoDB features so heavily amongst many web scale or large scale projects.

Okay, so let's move on.

And next I want to talk about a document database.

And this is a type of no-SQL database that's designed to store and query data as documents.

Documents are generally formatted using a structure such as JSON or XML.

But often the structure can be different between documents in the same database.

You can think of a document database almost like an extension of a key value store where each document is interacted with via an ID that's unique to that document.

But the value, the document contents, are exposed to the database allowing you to interact with it.

Document databases work best for scenarios like order databases or collections or contact style databases, situations where you generally interact with the data as a document.

Document databases are also great when you need to interact with deep attributes, so nested data items within a document structure.

The document model works well with use cases such as catalogs, user profiles, and lots of different content management systems where each document is unique but it changes over time.

So it might have different versions.

Documents might be linked together in hierarchical structures or when you're linking different pieces of content in a content management system.

For any use cases like this, document style databases are perfect.

Each document has a unique ID and the database has access to the structure inside the document.

Document databases provide flexible indexing so you can run really powerful queries against the data that could be nested deep inside a document.

Now let's move on.

Column databases are the next type of database type that I want to discuss.

And understanding the power of these databases requires knowing the limitations of their counterpart, row-based databases, which is what most SQL-based databases use.

Row-based databases are where you interact with data based on rows.

So in this example we have an orders table.

It has order ID, the product ordered, color, size, and price.

For every order we have a row and those rows are stored on disk together.

If you needed to read the price of one order from the database, you read the whole row from disk.

If you don't have indexes or shortcuts, you'll have to find that row first and that could mean scanning through rows and rows of data before you reach the one that you want to query.

Now if you want to do a query which operates over lots of rows, for example you wanted to query all the sizes of every order, then you need to go through all of the rows, finding the size of each.

Row-based databases are ideal when you operate on rows, creating a row, updating a row, or deleting rows.

Row-based databases are often called OLTP or Online Transaction Processing Databases and they are ideal as the name suggests for systems which are performing transactions.

So order databases, contact databases, stock databases, things which deal in rows and items where these rows and items are constantly accessed, modified, and removed.

Now column-based databases handle things very differently.

Instead of storing data in rows on disk, they store it based on columns.

The data is the same but it's grouped together on disk based on column.

So every order value is stored together, every product item, every color, size, and price, all grouped by the column that the data is in.

Now this means two things.

First, it makes it very, very inefficient for transaction style processing which is generally operating on whole rows at a time.

But this very same aspect makes column databases really good for reporting.

So if your queries relate to just one particular column because that whole column is stored on disk grouped together, then that's really efficient.

You could perform a query to retrieve all products sold during a period or perform a query which looks for all sizes sold in total ever and looks to build up some intelligence around which are sold most and which are sold least.

With column store databases, it's really efficient to do this style of querying, reporting style querying.

An example of a column based database in AWS is Redshift which is a data warehousing product and that name gives it away.

Generally what you'll do is take the data from an OLTP database, a row based database, and you'll shift that into a column based database when you're wanting to perform reporting or analytics.

So generally column store databases are really well suited to reporting and analytics.

Now lastly I want to talk about graph style databases.

Earlier in the lesson I talked about tables and keys and how relational database systems handle the relationships by linking the keys of different tables.

Well with graph databases, relationships between things are formally defined and stored in the database itself along with the data.

They're not calculated each and every time you run a query.

And this makes them great for relationship driven data.

For example social media or HR systems.

Consider this data, three people, two companies and a city.

These are known as nodes inside a graph database, nodes and nouns, so objects.

Nodes can have properties which are simple key value pairs of data and these are attached to the nodes.

So far this looks very much like a normal database, nothing is new so far.

But with graph databases there are also relationships between the nodes which are known as edges.

Now these edges have a name and a direction.

So Natalie works for XYZ corp and Greg works for both XYZ corp and Acme widgets.

Relationships themselves can also have attached data, so name value pairs.

In this particular example we might want to store the start date of any employment relationship.

A graph database can store a massive amount of complex relationships between data or between nodes inside a database and that's what's key.

These relationships are actually stored inside the database as well as the data.

A query to pull up details on all employees of XYZ corp would run much quicker than on a standard SQL database because that data of those relationships is just being pulled out of the database just like the actual data.

With a relational style database you'd have to retrieve the data and the relationships between the tables is computed when you execute the query.

So it's a really inefficient process with relational database systems.

These relationships are fixed and computed each and every time a query is run.

With a graph based database those relationships are fluid, dynamic, they're stored in the database along with the data and it means when you're interacting with data and looking to take advantage of these fluid relationships it's much more efficient to use a graph style database.

Now using graph databases it's very much beyond the scope of this course but I want you to be aware of it because you might see questions in the exam which mention the technology and you need to be able to identify or eliminate answers based on the scenario, based on the type of database that the question is looking to implement.

So if you see mention of social media in an exam or systems with complex relationships then you should think about graph databases first.

Now that's all I wanted to cover in this lesson.

I know it's been abstract and high level.

I wanted to try and make it as brief as possible.

I know I didn't really succeed because we had a lot to cover but I want this to be a foundational set of theory that you can use throughout the databases section and it will help you in the exam.

For now though that's everything I wanted to cover in this lesson so go ahead complete the video and when you're ready you can join me in the next.

Welcome back and in this first technical lesson of this section of the course, I wanted to provide a quick fundamentals lesson on databases.

If you already have database experience then you can play me on super fast speed and think of this lesson as a good confirmation of the skills that you already have.

If you don't have database experience though, that's okay.

This lesson will introduce just enough knowledge to get you through the course and I'll include additional reading material to get you up to speed with databases in general.

Now we do have a fair amount to get through so let's jump in and get started.

Databases are systems which store and manage data.

But there are a number of different types of database systems and crucial differences between how data is physically stored on disk and how it's managed on disk and in memory, as well as how the systems retrieve data and present it to the user.

Database systems are very broadly split into relational and non-relational.

Relational systems are often referred to as SQL or SQL.

Now this is actually wrong because SQL is a language which is used to store, update and retrieve data.

It's known as the structured query language and it's a feature of most relational database platforms.

Strictly speaking, it's different than the term relational database management system but most people use the two interchangeably.

So if you see or hear the term SQL or RDBMS which is relational database management system, they're all referring to relational database platforms.

Most people use them interchangeably.

Now one of the key identifiable characteristics of relational database systems is that they have a structure to their data.

So that's inside and between database tables and I'll cover that in a moment.

The structure of a database table is known as a schema and with relational database systems it's fixed or rigid.

That means it's defined in advance before you put any data into the system.

A schema defines the names of things, valid values of things and the types of data which are stored and where.

More importantly, with relational database systems there's also a fixed relationship between tables.

So that's fixed and also defined in advance before any data is entered into the system.

Now no SQL on the other hand.

Well let's start by making something clear.

No SQL isn't one single thing.

No SQL as the name suggests is everything which doesn't fit into the SQL mold.

Everything which isn't relational.

But that represents a large set of alternative database models which I'll cover in this lesson.

One common major difference which applies to most no SQL database models is that generally there is a much more relaxed concept of a schema.

Generally they all have weak schemers or no schemers and relationships between tables are also handled very differently.

Both of these impact the situations that a particular model is right for and that's something that you need to understand at a high level for the exam and also when you're picking a database model for use in the real world.

Before I talk about the different database models I wanted to visually give you an idea of how relational database management systems known as RDBMSs or SQL systems conceptualize the data that you store within them.

Consider an example of a simple PET database.

You have three humans and for those three humans you want to record the PETs that those humans are owned by.

The key component of any SQL based database system is a table.

Now every table has columns and these are known as attributes.

The column has a name, its attribute name and then within each row of that table each column has to have a value and this is known as the attribute value.

So in this table for example the columns are f name, first name, l name, last name and age and then for each of the rows 1, 2 and 3 the row has an attribute value for each of the columns.

So each of the attributes which are the columns have an attribute value in each row.

Now generally the way that data is modeled in a relational database management system or a SQL database system is that data which relates together is stored within a table.

So in this case all of the data on the humans is stored within one table.

Every row in the table has to be uniquely identifiable and so we define something that's known as a primary key.

This is unique in the table and every row of that table has to have a unique value for this attribute.

So note in this table how every row has a unique value 1, 2 and 3 for this primary key.

Now with this database model we've also got a similar table for the animals.

So we've got whiskers and woofy and they also have a primary key that's been defined which is the animal or AID.

And this primary key on this table also has to have a unique value in every row on the table.

So in this case whiskers is animal ID 1 and woofy is animal ID 2.

Each table in a relational database management system can have different attributes but for a particular table every row in that table needs to have a value stored for every attribute in that table.

So see how the animals table has name and types whereas the human table has first name, last name and age.

But note how in both tables for every row every attribute has to have a value.

Because SQL systems are relational we generally define relationships between the tables.

Now this is a join table.

It makes it easy to have many to many relationships.

So a human could have many animals and each animal can have many human minions.

A join table has what's known as a composite key which is a key formed of two parts.

And for composite keys together they have to be unique.

So notice how the second and third rows have the same animal ID.

That's fine because the human ID is different.

As long as the composite key in its entirety is unique that's also fine.

Now the keys in different tables are how the relationships between the tables are defined.

So in this example the human table has a relationship with the join table.

It allows each human to have multiple animals and each animal to have multiple humans.

In this example the animal ID of two which is woofy is linked to human ID two and three which is Julie and James.

They're both woofies minions because that doggo needs double the amount of tasty treats.

Now all these keys and the relationships are defined in advance.

This is done using the schema.

It's fixed and it's very difficult to change after the first piece of data goes in.

The fact that this schema is so fixed and has to be declared in advance makes it difficult for a sequel or a relational system to store any data which has rapidly changing relationships.

And a good example of this is a social network such as Facebook where relationships change all the time.

So this is a simple example of a relational database system.

It generally has multiple tables, a table stores data which is related so humans and animals.

Tables have fixed schemas.

They have attributes.

They have rows.

Each row has a unique primary key value and has to contain some value for all of the attributes in the table.

And in those tables they have relationships between each other which are also fixed and defined in advance.

So this is sequel.

This is relational database modelling.

Okay so this is the end of part one of this lesson.

It was getting a little bit on the long side and so I wanted to add a break.

It's an opportunity just to take a rest or grab a coffee.

Part two will be continuing immediately from the end of part one.

So go ahead complete the video and when you're ready join me in part two.

Welcome back and in this lesson I want to talk through how we implement DNSSEC using Route 53.

Now if you haven't already watched my DNS and DNSSEC fundamentals video series you should pause this video and watch those before continuing.

Assuming that you have let's jump in and get started.

Now you should be familiar with this architecture.

This is how Route 53 works normally.

In this example I'm using the animals for live.org domain so a query against this would start with our laptop, go to a DNS resolver then to the root servers looking for details of the .org top level domain and then it would go to the .org top level domain name servers looking for animals for live.org and then it would proceed to the four name servers which are hosting the animals for live.org zone using Route 53.

On the right hand side here we have an AWS VPC using the plus two address which is the Route 53 resolver and those instances can query the animals for live.org domain from inside the VPC.

Now enabling DNSSEC on a Route 53 hosted zone is done from either the Route 53 console UI or the CLI and once initiated the process starts with KMS.

This part can either be done separately or as part of enabling DNSSEC signing for the hosted zone but in either case an asymmetric key pair is created within KMS meaning a public part and a private part.

Now you can think of these conceptually as the key signing keys or KSKs but in actual fact the KSK is created from these keys.

These aren't the actual keys but this is a nuance which isn't required at this level.

So these keys are used to create the public and private key signing keys which Route 53 uses and these keys need to be in the US East 1 region that's really important so keep that in mind.

Next Route 53 creates the zone signing keys internally.

This is really important to understand both the creation and the management of the zone signing keys is handled internally within Route 53.

KMS isn't involved.

Next Route 53 adds the key signing key and the zone signing key public parts into a DNS key record within the hosted zone.

This tells any DNSSEC resolvers which public keys to use to verify the signatures on any other records in this zone.

Next the private key signing key is used to sign those DNS key records and create the RRSIG DNS key record and these signatures mean that any DNSSEC resolver can verify that the DNS key records are valid and unchanged.

Now at this point that's signing within the zone configured which is step one.

Next Route 53 has to establish the chain of trust with the parent zone.

The parent zone needs to add a DS record or delegated signer record which is a hash of the public part of the key signing key for this zone and so we need to make this happen.

Now how we do this depends on if the domain is registered via Route 53.

If so the registered domains area of the Route 53 console or the equivalent CLI command can be used to make this change.

Route 53 will liaise with the appropriate top level domain and add the delegated signer record.

Now if we didn't register the domain using Route 53 and are instead just using it to host the zone then we're going to need to perform this step manually.

Once done the top level domain in this case.org will trust this domain via the delegated signer record which as I mentioned is a hash of the domains public key signing key and the domain zone will sign all records within it either using the key signing or zone signing keys.

As part of enabling this you should also make sure to configure cloud watch alarms.

Specifically create alarms for DNSSEC internal failure and DNSSEC key signing keys needing action.

Both of these indicate a DNSSEC issue with the zone which needs to be resolved urgently.

Either an issue with the key signing key itself or a problem interacting with KMS.

Lastly you might want to consider enabling DNSSEC validation for VPCs.

This means for any DNSSEC enabled zones if any records fail validation due to a mismatch signature or otherwise not being trusted they won't be returned.

This doesn't impact non-DNSSEC enabled zones which will always return results and this is how to work with the Route 53 implementation of DNSSEC.

What I wanted to do now is step you through an actual implementation of DNSSEC for a hosted zone within Route 53 and to do that we're going to need to move across to my AWS console.

Okay so now we're at the AWS console and I'm going to step you through an example of enabling DNSSEC on a Route 53 domain and to get started I'm going to make sure I'm in an AWS account where I have admin permissions.

In this case I'm logged in as the I am admin user which is an I am identity with admin permissions.

As always I'm going to make sure that I have the Northern Virginia region selected and once I've done that I'm going to go ahead and open Route 53 in a new tab.

In my case it's already inside recently visited services if it's not you can just search for it in the search box at the top but I'm going to go ahead and open Route 53.

So I'm going to go to the Route 53 console and click on hosted zones and in my case I've got two hosted zones animals for life.org and animals for life 1337.org.

I'm going to go ahead and DNSSEC enable animals for life.org so I'm going to go ahead and go inside this hosted zone.

Now if I just go ahead and move across to my command prompt and if I run this command so dig animals for life.org and then DNS key with plus DNSSEC this will query this domain attempting to look for any DNS key records using DNSSEC and as you can see there are no DNSSEC results returned which is logical because this domain is not enabled for DNSSEC.

So moving back to this console I'm going to click on DNSSEC signing under the domain and then click on enable DNSSEC signing.

Now if this were a production domain the order of these steps really matters and you need to make sure that you wait for certain time periods before conducting each of these steps.

Specifically you need to make sure that you're making changes taking into consideration the TTL values within your domain.

I'll include a link attached to this video which details all of these time critical prerequisites that you need to make sure you consider before enabling DNS signing.

In my case I don't need to worry about that because this is a fresh empty domain.

The first thing we're going to do is create a key signing key and as I mentioned earlier in this video this is done using a KMS key.

So the first thing I'm going to do is to specify a KSK name so a key signing key name and I'm going to call it A4L KSK for Animals for Life key signing key.

Next you'll need to decide on which key to use within KMS to create this key signing key.

Now unfortunately the user interface is a little bit inconsistent.

AWS have decided to rename CMKs to KMS keys so you might see the interface looking slightly different when you're doing this video.

Regardless you need to create a KMS key so check the box saying create customer managed CMK or create KMS key depending on what state the user interface is in and you'll need to give a name to this key and again this is creating an asymmetric KMS key.

So I'm going to call it A4L KSK and then KMS key and once I've done that I can go ahead and click on create KSK and enable signing.

Now behind the scenes this is creating an asymmetric KMS key and using this to create the key signing key pair that this hosted zone is going to use.

Now this part of the process can take a few minutes and so I'm going to skip ahead until this part has completed.

Okay so that's completed and that means that we now have an active key signing key within this hosted zone and that means it's also created a zone signing key within this hosted zone.

If I go back to my terminal and I rerun this same command and press enter you'll see that we still get the same empty results and this can be because of caching so I need to wait a few minutes before this will update.

If I run it again now we can see for the same query it now returns DNS key records so one for 256 which represents the zone signing key so the public part of that key pair and one for 257 which represents the key signing key again the public part of that key pair and then we have the corresponding RRSIG DNS key record which is a signature of these using the private key signing key so now internally we've got DNSSEC signing enabled for this hosted zone and what we need to do now is create the chain of trust with the parent zone in this case the .org top level domain.

Now to do that because I've also registered this domain using Route 53 I can do that from the registered domains area of the console so I'll open that in a brand new tab.

I'm going to go there and then I'm going to go to the animals for life dot org registered domain and this is the area of the console where I can make changes and Route 53 will liaise with the .org top level domain and enter those changes into the .org zone.

Now the area that I'm specifically interested in is the DNSSEC status currently this is set to disabled.

What I'm going to do is to click on manage keys and it's here where I can enter the public key specifically it's going to be the public key signing key of the animals for life dot org zone and I'm going to enter it so that it creates the delegated signer record in the .org domain which establishes this chain of trust.

So first I'm going to change the key type to KSK then I'm going to go back to our hosted zone and I'm going to click on view information to create DS record then I'm going to expand establish a chain of trust and depending on what type of registrar you used you either need to follow the bottom instructions or these if you used Route 53 and I did so I can go ahead and use the Route 53 registrar details.

Now the first thing you need to do is to make sure that you're using the correct signing algorithm so it's ecds ap 256 char 256 so I'm going to go ahead and move back to the registered domains console click on the algorithm drop down and select the matching signing algorithm so ecds ap 256 char 256.

Next I'll go back and I'll need to copy the public key into my clipboard so remember a delegated signer record is just a hash of the public part of the key signing key so this is what I'm copying into my clipboard this is the public key of this key signing key so I'm going to go back and paste this in and then click on add now this is going to initiate a process where Route 53 are going to make the changes to the animals for life dot org part of the .org top level domain zone so specifically in the .org top level domain zone there's going to be an entry for animals for life dot org by default for normal DNS this is going to contain name server records which delegate through to Route 53 what this process is going to do is also add a DS record which is a delegated signer record and it is going to contain a hash of this public key now that will take a few minutes to take effect that's not a process which only involves Route 53 it also involves the .org top level domain so it can take a few minutes it can actually take anywhere up to a few hours and depends somewhat on the top level domain as well as the relationship with that entity which Route 53 has so I'm just going to go ahead and refresh this we can see that the DNS sec status has changed and we've now got this entry so now if I move back to my terminal I'm going to clear the screen to make it easier to see and now I'm going to run this command so dig space org because I want to do a query on the org top level domain space NS and then a space plus short and this will give me a listing at the authoritative name servers for the .org top level domain and I'm going to pick one of those servers so I'm gonna go ahead and pick the top one then I'm going to run this command so dig animals for live.org which is my hosted zone then a space I want to query DS records so delegated signers then an at sign and then the host name of the .org top level domain name server so this is going to mean that we're querying one specific name server and then press enter now if you don't see any DS record returned again it's because of DNS caching and again there is a delay between when you make the changes within Route 53 and when this takes effect within the DNS hierarchy so I'll clear the screen again rerun that command again I'm not getting any DS record returned so at this point I'm going to skip ahead to when the .org top level domain have updated with the changes that we've just made now in my case after about five more minutes this was added so note off from the same command so dig space animals for live.org and then DS for delegated signer and then at and then one of the .org TLD name servers so I'm querying for the delegated signer record for my domain and we can see here in the answers section we've got it here so animals for life DS for delegated signer and then here we've got the record and this record contains a hash of my public key signing key that's used for the animals for live.org domain so now I've established the chain of trust from the .org top level domain through to my hosted zone inside AWS and then because I've enabled the NSX signing it means if I create any records within this domain then they too will be signed so for test that if I go ahead and click on create record I'm going to use simple routing define a simple record going to call this test so test dot animals for live.org it's going to be an a record type I'm going to choose IP address or another value and then I'm just going to enter a test IP address so 1.1.1.1 and a TTL of one minute and then I'm going to define that simple record and create the record now for just refresh this inside the hosted zone for the UI you won't see anything which looks different however if I move back to my terminal clear the screen to make it easy to see and do a dig space test dot animals for live.org and then a space and then a and press enter that's going to do a normal DNS query for this a record so we can see it test animals for live dot org it's an a record and then it points at 1.1.1.1 if I run the same command only now put plus DNS sec and press enter now we can see in addition to the normal DNS query result now we have the DNS sec RR sick and this is a signature of the record above using the private part of the zone signing key and this signature can be verified using the DNS key record which contains the public zone signing key so now we have an end-to-end chain of trust from the DNS route all the way through to this resource record now that's everything I wanted to cover in this video I just wanted to give you an overview of how to implement a DNS sec within route 53 both from a theory and practical perspective at this point that's the end of the video so go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back and in this video I want to cover Route 53 interoperability.

What I mean by that is using Route 53 to register domains or to host zone files when the other part of that is not with Route 53.

Generally both of these things are performed together by Route 53 but it's possible for Route 53 just to do one or the other.

So let's start by stepping through exactly what I mean.

When you register a domain using Route 53 it actually does two jobs at the same time.

While these two jobs are done together conceptually they're two different things.

Route 53 acts as a domain registrar and it provides domain hosting so it can do both which is what happens initially when you register a domain or it can be a domain registrar alone or it can host domains alone.

It might only do one of them if for example you register a domain elsewhere and you want to use it with Route 53 and I want to take the time in this video to explain those edge case scenarios.

So let's quickly step through what happens when you register a domain using Route 53.

So first it accepts your money the domain registration fee.

This is a one-off fee or more specifically a once a year or once every three-year fee for actually registering the domain.

Next it allocates for Route 53 DNS servers called name servers then it creates a zone file which it hosts on the four name servers that I've just talked about.

So that's the domain hosting part allocating those servers and creating and hosting the zone file.

If you hear me mention domain hosting that's what it means.

Then once the domain hosting is sorted Route 53 communicates with the registry for the specific top level domain that you're registering your domain within so they have a relationship with the registry.

So Route 53 is acting as the domain registrar the company registering the domain on your behalf with the domain registry and the domain registry is the company or entity responsible for the specific top level domain.

So Route 53 gets the registry to add an entry for the domain say for example animalsforlife.org.

Inside this entry it adds four name server records and it points these records at the four name servers that I've just been talking about.

This is the domain registrar part.

So the registrar registers the domain on your behalf that's one duty and then another entity provides DNS or domain hosting and that's another duty.

Often these are both provided by Route 53 but they don't have to be so fix in your mind these two different parts the registrar which is the company who registers the domain on your behalf and the domain hosting which is how you add and manage records within hosted zones.

So let's step through this visually looking at a few different options.

First we have a traditional architecture where you register and host a domain using Route 53.

So on the left conceptually we have the registrar role and this is within the registered domains area of the Route 53 console.

On the right we have the DNS hosting role and this is managed in the public hosted zone part of the Route 53 console.

So step one is to register a domain within Route 53.

For now let's assume that it's the animalsforlife.org domain.

So you liaise with Route 53 and you pay the fee required to register a domain which is a per year or per three year fee.

Now assuming nobody else has registered the domain before the process continues.

First the Route 53 registrar liaisers with the Route 53 DNS hosting entity and it creates a public hosted zone which allocates for Route 53 name servers to host that zone which are then returned to the registrar.

I want to keep driving home that conceptually the registrar and the hosting are separate functions of Route 53 because it makes everything easier to understand.

Once the registrar has these four name servers it passes all of this along through to the .org top level domain registry.

The registry is the manager of the .org top level domain zone file and it's via this entity that records are created in the top level domain zone for the animalsforlife.org domain.

So entries are added for our domain which point at the four name servers which are created and managed by Route 53 and that's how the domain becomes active on the public DNS system.

At this point we've paid once for the domain registration to the registrar which is Route 53 and we also have to pay a monthly fee to host the domain so the hosted zone and with this architecture this is also paid to Route 53.

So this is a traditional architecture and this is what you get if you register and host a domain using Route 53 and this is a default configuration.

So when you register a domain while you might see it as one step it's actually two different steps done by two different conceptual entities the registrar and the domain hoster and it's important to distinguish between these two whenever you think about DNS.

But now let's have a look at two different configurations where we're not using Route 53 for both of these different components.

This time Route 53 is acting as a registrar only so we still pay Route 53 for the domain they still liaise on our behalf with the registry for the top level domain but this time a different entity is hosting the domain so the zone file and the name servers and let's assume for this example it's a company called Hover.

This architecture involves more manual steps because the registrar and the DNS hosting entity are separate so as the DNS admin you would need to create a hosted zone.

The third party provider would generally charge a monthly fee to host that zone on name servers that they manage.

You would need to get the details of those servers once it's been created and pass those details on to Route 53 and Route 53 would then liaise with the .org top level domain registry and set those name server records within the domain to point at the name servers managed in this case by Hover.

With this configuration which I'll admit I don't see all that often in the wild the domain is managed by Route 53 but the zone file and any records within it are managed by the third party domain hosting provider in this case Hover.

Now the reason why I don't think we see this all that often in the wild is the domain registrar functionality that Route 53 provides it's nothing special.

With this architecture you're not actually using Route 53 for domain hosting and domain hosting is the part of Route 53 which adds most of the value.

If anything this is the worst way to manage domains let's look at another more popular architecture which I see fairly often in the wild and that's using Route 53 for domain hosting only.

Now you might see this either when a business needs to register domains via a third party provider maybe they have an existing business deal or business discount or you might have domains which have already been historically registered with another provider and where you want to get the benefit that Route 53 DNS hosting provides.

With this architecture the domain is registered via a third party domain registrar in this case Hover so it's the registrar in this example who liais with the top level domain registry but we use Route 53 to host the domain.

So at some point either when the domain is being created or afterwards we have to create a public hosted zone within Route 53.

This creates the zone and the name servers to host the zone obviously for a monthly fee.

So once this has been created we pass those details through to the registrar who liais with the registry for the top level domain and then those name server records are added to the top level domain meaning the hosted zone is now active on the public internet.

Now it's possible to do this when registering the domain so you could register the domain with Hover and immediately provide Route 53 name servers or you might have a domain that's been registered years ago and you now want to use Route 53 for hosting and record management.

So you can use this architecture either while registering a domain or after the fact by creating the public hosted zone and then updating the name server records in the domain via the third party registrar and then the dot org registry.

Now I know that this might seem complex but if you just keep going back to basics and thinking about Route 53 as two things then it's much easier.

Route 53 offers a component which registers the domain so this is the registrar and it also offers a component which hosts the zone files and provides managed DNS name servers.

If you understand that both of those are different things and when you normally register a domain using Route 53 both of them are being used.

A hosted zone is created for you and then via the registrar part it's added to the domain record by the top level domain registry.

If you understand that so see these as two completely different components then it's easy to understand how you can use Route 53 for only one of them and a separate third-party company for the other.

Now generally I think Route 53 is one of the better DNS providers on the market and so generally for my own domains I will use Route 53 for both the registrar and the domain hosting components but depending on your architecture, depending on any legacy configuration, you might have a requirement to use different entities for these different parts and that's especially important if you're a developer looking at writing applications that take advantage of DNS or if you're an engineer looking to implement or fault find these type of architectures.

Now with that being said that's everything I wanted to cover in this theory video I just wanted to give you a brief overview of some of the different types of scenarios that you might find in more complex Route 53 situations.

At this point go ahead and complete this video and when you're ready.

I'll look forward to you joining me in the next.

Welcome back and in this video I want to talk about Geoproximity routing which is another routing policy available within Route 53.

So let's just jump in and get started.

Geoproximity aims to provide records which are as close to your customers as possible.

If you recall latency based routing provides the record which has the lowest estimated latency between your customer and the region that the record is in.

Geoproximity aims to calculate the distance between a customer and a record and answer with the lowest distance.

Now it might seem similar to latency but this routing policy works on distance and also provides a few key benefits which I'll talk about in this video.

When using Geoproximity you define rules so you define the region that a resource is created in if it's an AWS resource or provide the latitude and longitude coordinates if it's an external resource.

You also define a bias but more on that in a second.

Let's say that you have three resources one in America, one in the UK and one in Australia.

Well we can define rules which means that requests are routed to those resources.

If these were resources in AWS we could define the region that the resources were located in so maybe US East 1 or AP South East 2.

If the resources were external so non-AWS resources we could define their location based on coordinates but in either case Route 53 knows the location of these resources.

It also knows the location of the customers making the requests and so it will direct those requests at the closest resource.

Now we're always going to have some situations where customers in countries without any resources are using our systems.

In this case Saudi Arabia which is over 10,000 kilometers away from Australia and about 6,700 kilometers away from the UK.

Under normal circumstances this would mean that the UK resource would be returned for any users in Saudi Arabia.

What geo proximity allows us to do though is to define a bias.

So rather than just using the actual physical distance we can adjust how Route 53 handles the calculation.

We can define a plus or minus bias.

So for example with the UK we might define a plus bias meaning the effective area of service for the UK resource is increased larger than it otherwise would be.

And we could do the same for the Australian resource but maybe providing a much larger plus bias.

Now routing is distance based but it includes this bias.

So in this case we can influence Route 53 so that customers from Saudi Arabia are routed to the Australian resource rather than the UK one.

Geo proximity routing lets Route 53 route traffic to your resources based on the geographic location of your users and your resources.

But you can optionally choose to route more traffic or less traffic to a given resource by specifying a value.

The value is called a bias.

A bias expands or shrinks the size of a geographic region that is used for traffic to be routed to.

So even in the example of the UK where it's just a single relatively small country by adding a plus bias we can effectively make the size larger.

So that more surrounding countries route towards that resource.

In the case of Australia by adding an even larger bias we can make it so that countries even in the Middle East route towards Australia rather than the closer resource in the UK.

So geo proximity routing is a really flexible routing type that not only allows you to control routing decisions based on the locations of your users and resources.

It also allows you to place a bias on these rules to influence those routing decisions.

So this is a really important one to understand and it will come in really handy for a certain set of use cases.

Now thanks for watching.

That's everything that I wanted to cover in this video.

Go ahead and complete the video and when you're ready I look forward to you joining me in the next.

Welcome back and in this video I want to talk about geolocation routing which is another routing policy available within Route 53.

Now this is going to be a pretty brief video so let's jump in and get started.

In many ways geolocation routing is similar to latency.

Only instead of latency the location of customers and the location of resources are used to influence resolution decisions.

With geolocation routing when you create records you tag the records with the location.

Now this location is generally a country so using ISO standard country codes it can be continents again using ISO continent codes such as SA for South America in this case or records can be tagged with default.

Now there's a fourth type which is known as a subdivision.

In America you can tag records with the state that the record belongs to.

Now when a user is making a resolution request an IP check verifies the location of the user.

Depending on the DNS system this can be the user directly or the resolver server but in most cases these are one and the same in terms of the user's location.

So we have the location of the user and we have the location of the records.

What happens next is important because geolocation doesn't return the closest record it only returns relevant records.

When a resolution request happens Route 53 takes the location of the user and it starts checking for any matching records.

First if the user doing the resolution request is based in the US then it checks the state of the user and it tries to match any records which have a state allocated to them.

If any records match they're returned and the process stops.

If no state records match then it checks the country of the user.

If any records are tagged with that country then they're returned and the process stops.

Then it checks the continent.

If any records match the continent that the user is based in then they're returned and the process stops.

Now you can also define a default record which is returned if no record is relevant for that user.

If nothing matches though so there are no records that match the user's location and there's no default record then a no answer is returned.

So to stress again this type of routing policy does not return the closest record it only returns any which are applicable or the default or it returns no answer.

So geolocation is ideal if you want to restrict content.

For example providing content for the US market only.

If you want to do that then you can create a US record and only people located in the US will receive that record as a response for any queries.

You can also use this policy type to provide language specific content or to load balance across regional endpoints based on customer location.

Now one last time because this is really important for the exam and for real world usage.

This routing policy type is not about the closest record geolocation returns relevant locations only.

You will not get a Canadian record returned if you're based in the UK and no closer records exist.

The smallest type of record is a subdivision which is a US state then you have country then you have continent and finally optionally a default record.

Use the geolocation routing policy if you want to route traffic based on the location of your customers.

Now it's important that you understand which is why I've stressed this so much that geolocation isn't about proximity.

It's about location.

You only have records returned if the location is relevant.

So if you're based in the US but are based in a different state than a record you won't get that record.

If you're based in the US and there is a record which is tagged as the US as a country then you will get that record returned.

If there isn't a country specific record but there is one for the continent that you're in you'll get that record returned and then the default is a catchall.

It's optional if you choose to add it then it's returned if your user is in a location where you don't have a specific record tagged to that location.

Now that's everything that I wanted to cover in this video.

Thanks for watching.

Go ahead and complete the video and when you're ready I look forward to you joining me in the next.

Welcome back and in this video I want to talk about latency based routing which is yet another routing policy available within Route 53.

So let's jump in and get started.

Latency based routing should be used when you're trying to optimize for performance and user experience.

When you want Route 53 to return records which can provide better performance.

So how does it work?

Well it starts with a hosted zone within Route 53 and some records with the same name.

So in this case www, three of those records, they're A records and so they point at IP addresses.

In addition for each of the records you can specify a record region.

So US East 1, US West 1 and AP Southeast 2 in this example.

Latency based routing supports one record with the same name for each AWS region.

The idea is that you're specifying the region where the infrastructure for that record is located.

Now in the background AWS maintains a database of latencies between different regions of the world.

So when a user makes a resolution request it will know that that user is in Australia in this example.

It does this by using an IP lookup service and because it has a database of latencies it will know that a user in Australia will have a certain latency to US East 1, a certain latency to US West 1 and hopefully the lowest latency to a record which is tagged to be in the Asia Pacific region.

So AP Southeast 2.

So that record is selected and it's returned to the user and used to connect to resources.

Latency based routing can also be combined with health checks.

If a record is unhealthy then the next lowest latency is returned to the client making the resolution request.

This type of routing policy is designed to improve performance for global applications by directing traffic towards infrastructure with the best, so lowest latency for users accessing that application.

It's worth noting though that the database which AWS maintain isn't real time.

It's updated in the background and doesn't really account for any local networking issues but it's better than nothing and can significantly help with performance of your applications.

Now that's all of the theory that I wanted to cover about latency based routing.

So go ahead and complete the video and when you're ready I look forward to you joining me in the next.

Welcome back.

In this video I want to talk about weighted routing, which is another routing policy available within Route 53.

So let's jump in and get started straight away.

Weighted routing can be used when you're looking for a simple form of load balancing or when you want to test new versions of software.

Like all other types of routing policy it starts with a hosted zone and in this hosted zone you guessed it records.

In this case three www records.

Now these are all a records and so they point at IP addresses and let's assume that these are three EC2 instances.

With weighted routing you're able to specify a weight for each record and this is called the record weight.

Let's assume 40 for the top record, 40 for the middle and 20 for the record at the bottom.

Now how this record weight works is that for a given name www in this case the total weight is calculated.

So 40 plus 40 plus 20 for a total of 100.

Each record then gets returned based on its weighting versus the total weight.

So in this example it means that the top record is returned 40% of the time, the middle also 40% of the time and the bottom record gets returned 20% of the time.

Setting a record weight to zero means that it's never returned so you can do this if temporarily you don't want a particular record to be returned unless all of the records are set to zero in which case they're all returned.

So any of the records with the same name are returned based on its weight versus the total weight.

Now I've kept this example simple by using record weights that total 100 so it makes it easy to view them as percentages but the same formula is used regardless.

An individual record is returned based on its weight versus the total weight.

Now you can combine weighted routing with health checks and if you do so when a record is selected based on the above weight calculation if that record is unhealthy then the process repeats.

It's skipped over until a healthy record is selected and then that one's returned.

Health checks don't remove records from the calculation and so don't adjust the total weight.

The process is followed normally but if an unhealthy record is selected to be returned it's just skipped over and the process repeats until a healthy record is selected.

Now weighted routing as I mentioned at the start is great for very simple load balancing or when you want to test new software versions.

If you want to have 5% of resolution requests go to a particular server which is running a new version of Catergram then you have that option.

So weighted routing is really useful when you have a group of records with the same name and want to control the distribution so the amount of time that each of them is returned in response to queries.

Now that's everything I wanted to cover in this video so go ahead finish the video and when you're ready I'll look forward to you joining me in the next.

Welcome back.

In this video, I want to talk about multivalu routing, which is another routing policy available within Route 53.

So let's jump in and get started.

Multivalu routing in many ways is like a mixture between simple and failover, taking the benefits of each and merging them into one routing policy.

With multivalu routing, we start with a hosted zone, and with multivalu routing, you can actually create many records all with the same name.

In this case, we have three www records, and each of those records in this example is an a record, which maps onto an IP address.

Each of the records when using this routing type can have an associated health check, and when queried, up to eight healthy records are returned to the client.

If you have more than eight records, then eight are selected at random.

Now at this point, the client picks one of those values and uses it to connect to the resource.

Because each of the records is health checked, any of the records which fail the check, such as the bottom record in this example, won't be returned to the client, and won't be selected by the client when connecting to resources.

So multivalu routing, it aims to improve availability by allowing a more active, active approach to DNS.

You can use it if you have multiple resources, which can all service requests, and you want to select one at random.

Now it's not a substitute for a load balancer, which handles the actual connection process from a network perspective, but the ability to return multiple health checkable IP addresses is a way to use DNS to improve availability of an application.

So simple routing has no health checks and is generally used for a single resource, such as a web server.

Failover is used for active backup architectures, commonly with an S3 bucket as a backup, whereas multivalu is used when you have many resources which can all service requests, and you want them all health checked and then returned at random.

So any healthy records will be returned to the client.

If you have more than eight, they'll be returned randomly.

OK, so that's everything for this type of routing policy.

Go ahead and complete the video when you're ready, and I'll look forward to you joining me in the next video.

Welcome back.

In this video I want to quickly step through a topic which confuses people who are new to DNS and Route 53 and that's the difference between C names and alias records.

Now I've seen exam questions which test your understanding of when to use one versus the other, so let's quickly go through the key things which you need to know.

Now let's start by describing the problem that we have if we only use C names.

So in DNS an A record maps a name to an IP address, for example the name Categor.io to the IP address 1.3.3.7.

By now that should make sense.

A C name on the other hand maps a name to another name, so if you had the above A record for Categor.io then you could create a C name record for Categor.io, pointing at Categor.io.

It's a way to create another alternative name for something within DNS.

The problem is that you can't use a C name for the apex of a domain, also known as the naked domain.

So you couldn't have a C name record for Categor.io pointing at something else, it just isn't supported within the DNS standard.

Now this is a problem because many AWS services such as Elastic Load Balancers, they don't give you an IP address to use, they give you a DNS name.

And this means that if you only use C names, pointing the naked Categor.io at an Elastic Load Balancer wouldn't be supported.

You could point www.categor.io at an Elastic Load Balancer because using a C name for a normal DNS record is fine, but you can't use a C name for the domain Apex, also known as the naked domain.

Now this is a problem which alias records fix.

So for anything that's not the naked domain where you want to point a name at another name, C name records are fine.

They might not be optimal as I'll talk about in a second, but they will work.

For the naked domain known as the Apex of a domain, if you need to point at another name such as Elastic Load Balancers, you can't use C names.

But let's go through the solution, alias records.

An alias record generally maps a name onto an AWS resource.

Now it has other functions, but at this level let's focus on the AWS resource part.

Alias records can be used for both the naked domain known as the domain Apex or for normal records.

For normal records such as www.categor.io, you could use C names or alias records in most cases.

But for naked domains known as the domain Apex, you have to use alias records if you want to point at AWS resources.

For AWS resources, AWS try to encourage you to use alias records and they do this by making it free for requests made where an alias record points at an AWS resource.

So generally in most production situations and for the exam default to picking alias records for anything in a domain where you're pointing at AWS resources.

Now an alias is actually a subtype.

You can have an A record alias and a C name record alias and this is confusing at first.

But the way I think about this is both of them are alias records, but you need to match the record type with the type of the record you're pointing at.

So take the example of an elastic load balancer.

With an ELB, you're given an A record for the elastic load balancer.

It's a name which points at an IP address.

So you have to create an A record alias if you want to point at the DNS name provided by the elastic load balancer.

If the record that the resource provides is an A record, then you need to use an A record alias.

So you're going to use alias records when you're pointing at AWS services such as the API gateway, cloud front, elastic beanstalk, elastic load balancers, global accelerator and even S3 buckets.

And you're going to experience this last one in a demo lesson which is coming up very soon.

Now it's going to make a lot more sense when you see it in action elsewhere in the course.

For now, I just want to make sure that you understand the theory of both the limitations of C name records and the benefits that alias records provide.

Now the alias is a type of record that's been implemented by AWS and it's outside of the usual DNS standard.

So it's something that in this form you can only use if Route 53 is hosting your domains.

Keep that in mind as I talk about more of the features of Route 53 as we move through this section of the course.

But at this point, that's everything that I wanted to cover.

So go ahead, complete this video and when you're ready, I look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

Now one final thing before we finish with this demo lesson, and I want to talk about private hosted zones.

So move back to the Route 53 console.

I'm going to go to Hosted Zones, and I'm going to create a private hosted zone.

So click "Create Hosted Zone" because it's a private hosted zone, it doesn't even need to be one that I actually own.

So I'm going to call my hosted zone "IlikeDogsReally.com".

It's going to be a private hosted zone.

And for now, I'm going to associate it with the default VPC in US-East-1.

So I'm going to pick the region, US-East, and then select Northern Virginia, and then click in the VPC ID box, and we should see two VPCs listed.

One is the Animals for Life VPC, it's tagged A4L-VPC1, but I'm not going to pick this one, I'm going to pick the one without any text after it, which is the default VPC.

So once that's set, I'm going to create the hosted zone.

Then inside the hosted zone, I'm going to create a record.

The record's going to use the simple routing policy.

Click on "Next".

I'm going to define a simple record.

I'm going to call it "www".

The record type is going to be "a routes traffic to an IP version 4 address and some resources".

I'm going to click in this endpoint box and select IP address or another value, depending on record type.

And then into this box, I'm just going to put a test IP address of 1.1.1.1.

And then down at the bottom, I'm going to click "1M" to change this TTL to 60 seconds.

And I'm going to click "Define simple record".

And then finally, "Create records".

So now we have a record called "www.ilikedogsrealy.com".

So copy that into the clipboard.

Move back to the EC2 console.

Click on "Dashboard".

Click on "Instances running".

Right click, "Connect".

We're going to use EC2 "Instance connect".

And then just click on "Connect".

Now once connected, I'm going to try pinging the record which I just created.

So "Ping Space" and then paste in "www.ilikedogsrealy.com" and press "Enter".

What you should see is "Name or service not found".

The reason for this is the private hosted zone which we created is currently associated with the default VPC.

And this instance is not in the default VPC.

To enable this instance to resolve records inside this private hosted zone, we need to associate it with the "Animals for Life" VPC.

So go back to the Route 53 console.

Expand "Hoster Zone Details" and then "Edit hosted zone".

Scroll down and we're going to add another VPC.

In the region drop down, "US-East-1" and then in the "Choose VPC" box select "A4L-VPC-1".

Scroll down and save changes.

Now this might take a few seconds to take effect, but if we go back to the EC2 instance and try to run this ping again, and we still get "Name or service not found".

So what I want you to do is go ahead and pause this video, wait for 4 or 5 minutes and then resume and try this command again.

Now in my case it took about 5 minutes, but after a while I can now ping www.ilikedogsreally.com because I've now associated this private hosted zone with the VPC that this instance is running from.

Now that's everything that I wanted to cover in this demo lesson, so all that remains is for us to clean up all of the infrastructure which we've created in this demo lesson.

So if we go back to the Route 53 console and select "Health Checks", first we're going to delete the health check.

So select "A4L Health" and click on "Delete Health Check" and confirm.

Click on "Hostered Zones".

Go inside the private hosted zone that you created.

Select the www.ilikedogsreally.com record and then click on "Delete Record".

Confirm that deletion.

Go back to "Hostered Zones".

Select the entire private hosted zone and click on "Delete".

Type "Delete" and then click to confirm.

And that will delete the entire private hosted zone.

Then go inside the public hosted zone that you have.

Select the two www records that you created earlier in this lesson.

Click on "Delete Records".

Click "Delete" to confirm.

Then go to the S3 console.

Click on the bucket that you created earlier in this lesson.

Click "Empty".

Copy and paste or type "Permanently Delete" and click on "Empty".

Once that bucket is emptied click on "Exit".

With it still selected click on "Delete".

Copy and paste or type the full name into the box and click on "Delete Bucket".

Then go to the EC2 console.

Click the hamburger menu.

Scroll down.

Click "Elastic IPs".

Select the elastic IP that you associated with the EC2 instance.

Click on the actions drop down.

Disassociate and then click to disassociate.

With it still selected click on "Actions".

Release elastic IP addresses and click on "Release".

At that point all of the manually created infrastructure has been removed.

Go back to the cloud formation console.

Go to "Stacks".

Select the stack that you created at the start of this lesson using the one click deployment.

It should be called DNS and failover demo.

Select it.

Click on "Delete".

Then click on "Delete Stack" to confirm that deletion.

Once that's deleted the account will be back in the same state as it was at the start of the lesson.

At this point that's everything I wanted to cover in this demo.

I hope it's been enjoyable and it's given you some good practical experience of how to use failover routing and private hosted zones.

That will be useful both for the exam and real world usage.

At this point that's everything so go ahead and complete this video.

When you're ready I'll look forward to you joining me in the next.

Welcome to this demo lesson where you're going to get experience configuring fail-over routing as well as private hosted zones.

Now with this demo lesson you have the choice of either following along in your own environment or watching me perform the steps.

If you do wish to follow along in your own environment you will need a domain name that's registered within Route 53.

Remember that was an optional step at the start of this course so if you did register a domain of your own then you can do this demo lesson.

In my case I registered animals for life 1337.org.

If you registered a domain it will be different and so wherever you see me use animals for life.org you need to replace it with your registered domain.

If you didn't register one then you'll have to watch me perform all of these steps because you can't do this lesson without your own registered domain.

In order to get started you need to make sure that you're logged in as the I am admin user of the general AWS account which is the management account of the organization and you'll need to have the Northern Virginia region selected.

Now we're going to need to create some infrastructure in order to perform this demo lesson so attached to this lesson is a one-click deployment link and you should go ahead and click that link now.

That's going to take you to a quick create stack screen.

Everything should be pre-populated the stack name is DNS and failover demo all you'll need to do is scroll down to the bottom check this capabilities box and then click on create stack.

That's going to take a few minutes and it's going to create infrastructure that we're going to need to continue with the demo lesson so go ahead and pause the video wait for your stack to move into a create complete state and then we're good to continue.

Okay so the stacks now in a create complete state and it's created a number of resources the most important one being a public EC2 instance so we just need to test this first so if you just click in the search box and type EC2 and then right click to open that in a new tab.

Once you're there click on instances running and you should see a4l-web just select that under public IP version 4 just click on this symbol to copy the IP address into your clipboard make sure you don't click open address because that's going to try and use HTTPS which we don't want so copy this IP address into your clipboard and then open that in a new tab and you should see the animals for life super minimal homepage and if you see that it means everything's working as intended so go ahead and close down that tab.

Now we also need to give this instance an elastic IP address so that it has a static public IP version 4 address now to give it an elastic IP on the menu on the left scroll down to the bottom under network and security select elastic IPs and then we need to allocate an elastic IP make sure us - east - 1 is in this box scroll down and click on allocate once the elastic IP address is allocated to this account select it click on actions and then associate elastic IP once we're at this screen make sure instance is selected click in this search box and then select a4l-web once selected click in the private IP address box and select the private IP address of this instance and then check the box to say allow this elastic IP address to be re-associated once all that's complete click on associate and that now means that our EC2 instance has been allocated with a static IP version 4 address now we're configuring failover DNS and so the EC2 instance is going to be our primary record so we're going to assume that this is the animals for life main website and we want to configure an S3 bucket which is running as the backup in case this EC2 instance fails so the next thing we need to do is to create the S3 bucket so click in the search box type S3 and open that in a new tab and then go to the S3 console and at this point we're going to create an S3 bucket and configure it as a static website now the naming of the S3 bucket is important earlier in the course you should have registered a domain name in my case I registered animals for life 1337.org so I'm going to create a bucket with the name www.animalsforlife1337.org you need to create one which is called www.and then the domain name that you registered so I'm going to click and create bucket the bucket name is www.animalsforlife1337.org and it's going to be in the US east northern Virginia region which is US-East-1 then we're going to scroll down and we're going to need to uncheck block all public access because this bucket is going to be used to host a static website I'll need to acknowledge that I'm okay with that so I'll do that and then scroll all the way down to the bottom and then I'm going to click on create bucket then I'm going to go inside the bucket click on upload and then add files now attach to this lesson is an assets file I want you to go ahead and download that file then extract it and wave extracted it it should create a folder called R53 underscore zones underscore and underscore failover go inside that folder and there'll be two more folders one which is 01 underscore A4L website and another which is 02 underscore A4L failover we're interested in the A4L failover so go into that folder select both these files so index.html and minimal.jpeg click on open and then upload those files so we'll scroll down and click on upload once that's completed click on close then we go into enable static website hosting so click on properties and to enable this it's all the way down towards the bottom click on edit next to static website hosting and enable it make sure that host a static website is selected and then for the index document and the error document we're going to type index.html and once both of those are entered scroll down to the bottom and save changes now we've one final thing to do on this bucket we need to add a bucket policy so that this bucket is public so we need to click on permissions scroll down and then under bucket policy click on edit and this bucket currently does not have a bucket policy now also inside the assets folder that you extracted earlier in this lesson there's a file called bucket underscore policy.json this is the file so you'll need to copy the contents of that file into your clipboard and then inside this policy box paste that in and then click on the icon next to the bucket ARN to copy that into your clipboard and then we need to replace this placeholder with what you've just copied so I want you to select from just to the right of the first speech mark all the way through to before the forward slash so you should have ARN colon AWS colon S3 colon colon colon colon and then example bucket and then go ahead and paste the text from your clipboard which will overwrite that with the actual bucket ARN so it should look like this once you've got that scroll down and save the changes so now we have the failover website configured the static website running from the S3 bucket so now we need to go ahead and move to the route 53 console where we going to create a health check and configure the failover record so click in the search box type route 53 right click and open that in a new tab then click on health checks we're going to create a health check for the health check name type a4l health and it's going to be an end point health check scroll down we're going to specify the endpoint by IP address the protocols going to be HTTP and we need the IP address of the EC2 instance so if we go back to the EC2 console the EC2 instance is now using the elastic IP so if we scroll down and click on elastic IPs and copy the elastic IP into our clipboard then go back to the route 53 console and paste that in and the health check is going to be configured to health check the index.html document so in path we need to click and type index.html then we're going to expand advanced configuration and by default a health check is checking every 30 seconds so this is a standard health check we need to change this to fast because we want our health check to react as fast as possible if our primary website fails so select fast scroll down to the bottom click on next we don't want to create an alarm because we don't want to take any action if this health check fails we're just going to use it as part of our fail-over routing so go ahead and make sure no is selected and then click create health check now the health check is going to start off with an unknown status because it hasn't gathered enough information about the health of the primary website it's going to take a few minutes to move from this status to either healthy or unhealthy what we can do though is if we check this we can click on the health checkers tab and start to see the results of the globally distributed set of health check endpoints so we can see that we're already getting success HTTP status code 200 and this is telling us our primary website is already passing these individual checks and after a couple of minutes if we hit refresh we should see that the status changes from unknown to healthy so next we need to create the failover record so click on hosted zones locate the hosted zone for the domain that you registered at the start of the course and click it then click on create record now you can switch between two different modes either the quick create record mode or the wizard mode we're going to keep this demo simple so click on switch to wizard we're going to choose a failover record so select failover and click next we're going to call the record www we're going to set a TTL of one minute so click 1m and that will change the TTL seconds to 60 scroll down and we're going to define some failover records so click define failover record first we need to create the primary record so click in this first drop down and we're going to pick IP address or another value depending on record type so click that and then we need the elastic IP address so go back to the EC2 console and copy the elastic IP into your clip board and paste that into this box and then for failover record type this is the primary record so click on primary we need to associate it with a health check so click in that drop down and choose a for L health now once we do that it means that this primary record will only be returned if this health check is healthy otherwise the secondary record will be returned which we're going to define in a second under record ID just go ahead and type EC2 this needs to be unique within this set of records with the same name so going to call one EC2 and the other S3 so this one's EC2 so define that failover record and then we're going to define a new failover record so click that box again this time in this drop down we need to scroll down and we're going to select alias to an S3 website endpoint so select that choose the region and it needs to be US - East - 1 once selected you should be able to click in this box and see the S3 bucket that you just created so click on this to select that S3 bucket and we're going to set this as the secondary record so click on secondary we won't be associating this with a health check and we won't be evaluating the target health this record will only ever be used if the primary fails its health check and so we want this record to take effect whenever the health check associated with the primary fails and we're going to test that by shutting down the EC2 instance so this record should then take over so finally we need to enter S3 in the record ID and click on define failover record once we've done both of those we can go ahead and click on create records so now that we have both of those records in place the primary pointing at EC2 and the secondary at S3 if we copy down this full DNS name into our clipboard and open that in a new tab that should direct towards the animals for life.org super minimal homepage remember this is the website running on EC2 now what we need to do is to simulate a failure so go back to the EC2 console scroll to the top click on EC2 dashboard then instances running right click on this instance select stop instance and confirm that by clicking stop so now we've stopped this instance it should begin failing the health check so let's go back to the route 53 console click on health checks select this a4 health health check click on the health checkers tab and then click on refresh and over the coming seconds we should start to see some failure responses in this status column there we go we're getting connection timed out and over the next minute or so we should see that the status of the health check overall should move from healthy to unhealthy let's click on refresh it might take a minute or so for that to take effect so let's just give it a minute or so and now we can see that it's moved into an unhealthy state now this means that our failover record will detect this and then it's going to start returning the secondary record rather than the primary now DNS does have a cache remember we set the TTL value to 60 seconds so one minute but what we should find after that cache expires if we go back to this tab which we have open to the www.animalsforlife.org website and if we now hit refresh we should see that it changes to the animals for life.org super minimal failover page and this is the website that's running on s3 so the failover record has used a health check detected the failure of the EC2 instance and redirected us towards the backup s3 site so now we can go ahead and reverse that process if we go back to the EC2 console we can right click on this instance and start the instance that will take a few minutes to move from the stopped state through the pending state and then finally to running and once it's in a running state if we go back to the route 53 console and select this health check and then refresh on the health checkers initially we'll see a number of different messages if we keep hitting refresh over the next few minutes we should see this change to an okay message there we can see the first HTTP status code 200 if we keep refreshing we'll see more of those again more 200 statuses which means okay now that all of these are coming back okay let's click refresh on the health check itself it's still showing us unhealthy let's give it a few more seconds now it's reporting as healthy again if we go back to the tab that we have open to the website and click on refresh now it should change back to the original EC2 based website and it does so that means our failover record has worked in both directions it's failed over to s3 and failed back to EC2 okay so this is the end of part one of this lesson it was getting a little bit on the long side and so I wanted to add a break it's an opportunity just to take a rest or grab a coffee part 2 will be continuing immediately from the end of part one so go ahead complete the video and when you're ready join me in part 2.

Welcome back.

In this video I want to cover the Health Check feature within Route 53.

Health checks support many of the advanced architectures of Route 53 and so it's essential that you understand how they work as an architect, developer or engineer.

So let's jump in and get started.

First let's quickly step through some high level concepts of Health checks.

Health checks are separate from but are used by records inside Route 53.

You don't create the checks within records.

Health checks exist separately.

You configure them separately.

They evaluate something's health and they can be used by records within Route 53.

Health checks are performed by a fleet of health checkers which are distributed globally.

This means that if you're checking the health of systems which are hosted on the public internet then you need to allow these checks to occur from the health checkers.

If you think they're bots or exploit attempts and block them then it will cause false alarms.

Health checks as I just indicated they're not just limited to just AWS targets.

You can check anything which is accessible over the public internet.

It just needs an IP address.

The checks occur every 30 seconds by default or this can be increased to every 10 seconds at an additional cost.

The checks can be TCP checks where Route 53 tries to establish a TCP connection with the endpoint and this needs to be successful within 10 seconds.

You can have HTTP checks where Route 53 must be able to establish a TCP connection with the endpoint within 4 seconds and in addition the endpoint must respond with a HTTP status code in the 200 range or 300 range within 2 seconds after connecting.

And this is more accurate for web applications than a simple TCP check.

And finally with HTTP and HTTPS checks you can also perform string matching.

Route 53 must be able to establish a TCP connection with the endpoint within 4 seconds and the endpoint must respond with a HTTP status code in the 200 or 300 range within 2 seconds and Route 53 health checker when it receives the status code it must also receive the response body from the endpoint within the next 2 seconds.

Route 53 searches the response body for the string that you specify.

The string must appear entirely in the first 5,120 bytes of the response body or the endpoint fails the health check.

This is the most accurate because not only do you check that the application is responding using HTTP or HTTPS but you can also check the content of that response versus what the application should do in normal circumstances.

Based on these health checks an endpoint is either healthy or unhealthy.

It moves between those states based on its health based on the checks conducted.

Now lastly the checks themselves can be one of 3 types.

You can have endpoint checks and these are checks which assess the health of an actual endpoint that you specify.

You can use cloud watch alarm checks which react to cloud watch alarms which can be configured separately and can involve some detailed in OS or in app tests if you use the cloud watch agent which we cover elsewhere in the course.

Finally checks can be what's known as calculated checks so checks of other checks.

So you can create health checks which check application wide health with lots of individual components.

Now you're going to get the opportunity to actually implement a health check in a demo lesson which is coming up very shortly in this section of the course.

But what I want to do before that is to just give you an overview of exactly how the console looks when you're creating a health check.

So let's move across to the console.

Okay so we're at the AWS console logged in to the general account in the northern Virginia region.

So to create a health check we need to move to the route 53 console so I'm going to go ahead and do that.

Remember how earlier in the theory component of this lesson I mentioned how health checks are created externally from records.

So rather than going into a hosted zone selecting a record and configuring a health check there to create a health check we go to the menu on the left and click on health checks.

Then we'll click on create health check and this is where we enter the information required to create the health check.

First we need to give it a name so let's just say that we use the example of test health check.

I mentioned that there are three different types of health checks.

We've got an endpoint health check and this checks the health of the particular endpoint.

We can use status of other health checks so this is a calculated health check and as I mentioned this allows you to create a health check which monitors the application as a whole and involves the health status of individual application components and then finally we can use the status of a cloud watch alarm to form the basis of this health check.

If we select endpoint for now then you're able to pick either IP address or domain name.

So you can specify the domain name of an application endpoint or you can use IP address.

If you pick domain name then what this configures is that all of the Route 53 health checkers will resolve this domain name first and then perform a health check on the resulting IP address.

Now in either case you've got the option of either picking TCP which does a simple TCP check in which case you need to specify either the IP version 4 or IP version 6 address together with a port number.

If you choose to use the more extensive HTTP or HTTPS health check then you're asked to specify the same IP address and port number so that will be used to establish the TCP connection.

You can also specify the host name and if you specify that it will pass this value to the endpoint as a host header so if you've got lots of different virtual hosts configured then this is how you can specify a particular host that the website should deliver.

You're also able because this is HTTP you can specify a path to use for this health check.

You can either specify the route path or you can specify a particular path to check.

If you change this to HTTPS then all of this information is the same only this time it will use secure HTTP rather than normal HTTP.

Now if we scroll down and expand advanced configuration it's here where you can select the request interval so the default is every 30 seconds or you can specify fast and have the checks occur every 10 seconds.

Now this is a check every 10 seconds from every health checker involved within this health check so the actual frequency of the health checks occurring on the endpoint will be much more frequent.

This is one check every 10 seconds from every health checker.

You can specify the failure threshold so this is the number of consecutive health checks that an endpoint must pass or fail for out 53 to change the current status.

So if you want to allocate a buffer and allow for the opportunity of the odd fail check not to influence the health state then you can specify a suitable value in this box.

It's here where you can specify a simple check so HTTP or HTTPS or you can elect to use string matching to do more rich checks of application health.

So if you know that your application should deliver a certain string and the request body then you can specify that here.

Now you can also configure a number of advanced options one of them is latency graph so you can show the latency of the checks against this endpoint.

You can invert the health check status so if the health check of an application is unhealthy you can invert it to healthy and vice versa.

So this is a fairly situational option that I haven't found much use for.

You also have the option of disabling the health check this might be useful if you're performing temporary maintenance on an application and if you check this box then even if the application endpoint reports as unhealthy it's considered healthy.

You also get the option of specifying the health checker regions you can use the recommended suggestion and the health checkers will come from these locations or you can select customize and pick the particular regions that you want to use.

In most cases you would use the recommended options.

Now if we just go ahead and enter some sample values here so I'm going to use 1.1.1.1.

I'm going to leave the host name blank I'm going to set the port number to 80 and then I'll scroll down and just enter a search string again we're not going to create this or just enter a placeholder click on next and it's here where you can configure what happens when the health check fails.

Now this is completely optional we can use health checks within resource records only we don't have to configure any notification but if we do want to configure a notification then we can create an alarm and we can send this to either an existing or new s and s topic and this is a method of how we can integrate this with other systems so we can have other aws services configured to respond to notifications on this topic or we could integrate external systems so that when a health check fails external action is taken but this is what I wanted to show you I just wanted to give you an overview of how it looks creating a health check within the console UI now don't worry you're actually going to be doing this in a demo lesson which is coming up elsewhere in this section but I wanted to give you that initial exposure to how the console looks when creating a health check at this point let's go ahead and finish up the theory component of this lesson by returning to the architecture now you've seen how a health check is created architecturally health checks look something like this let's assume that somewhere near the UK we have an application Catergram and we point a route 53 record at this application so let's assume that this is Catergram dot IO what we can do is to associate a health check with this resource record and doing so means that our application will be health checked by a globally distributed set of health checkers so each of these health checkers performs a periodic check of our application based on this check they report the resource as healthy or unhealthy if more than 18 percent of the health checkers report as healthy then the health check overall is healthy otherwise it's reported as unhealthy and in most cases records which are unhealthy are not returned in response to queries now you're going to see throughout this section of the course and the wider course itself how health checks can be used to influence how DNS response to queries and how applications can react to component failure so route 53 is an essential design and operational tool that you can use to influence how resolution requests occur and how they're routed through to your various different application components and so understanding health checks is essential to be able to design route 53 infrastructure integrate this with your applications and then manage it day to day as an operational engineer so it's really important that you understand this topic end to end no matter which stream of the AWS certifications that you're currently studying for now that's everything that I wanted to cover in this video go ahead and complete the video and when you're ready I'll I'll look forward to you joining me in the next.

Welcome back and in this video I want to talk about the second Route 53 routing policy that I'm going to be covering in this series of videos and that's fail over routing.

Now let's just jump in and get started straight away.

With fail over routing we start with a hosted zone and inside this hosted zone a www.record.

However with fail over routing we can add multiple records of the same name, a primary and a secondary.

Each of these records points at a resource and a common example is an out of band failure architecture where you have a primary application endpoint such as an EC2 instance and a backup or fail over resource using a different service such as an S3 bucket.

The key element to fail over routing is the inclusion of a health check.

The health check generally occurs on the primary record.

If the primary record is healthy then any queries to www in this case resolve to the value of the primary record which is the EC2 instance running category in this example.

If the primary record fails its health check then the secondary value of the same name is returned in this case the S3 bucket.

The use case for fail over routing is simple.

Use it when you need to configure active passive fail over where you want to route traffic to a resource when that resource is healthy or to a different resource when the original resource is failing its health check.

Now this is a fairly simple concept that you'll be experiencing yourself in a demo video which is coming up very soon but at this point that's everything that I wanted to cover in this video.

So go ahead complete the video and when you're ready I look forward to you joining me in the next.

Welcome back and in this video I want to cover the first of a range of routing policies available within Route 53.

We're going to start with the default and as the name suggests it's the simplest.

This video is going to be pretty quick so let's jump in and get started straight away.

Simple routing starts with a hosted zone.

Let's assume it's a public hosted zone called animalsforlife.org.

With simple routing you can create one record per name.

In this example, WWW which is an A record type.

Each record using simple routing can have multiple values which are part of that same record.

When a client makes a request to resolve WWW and simple routing is used all of the values are returned in the same query in a random order.

The client chooses one of the values and then connects to that server based on the value in this case 1.2.3.4.

Simple routing is simple and you should use it when you want to route requests towards one single service.

In this example a web server.

The limitation of simple routing is that it doesn't support health checks and I'll be covering what health checks are in the next video.

But just remember with simple routing there are no checks that the resource being pointed at by the record is actually operational and that's important to understand because all of the other routing types within Route 53 offer some form of health checking and routing intelligence based on those health checks.

Simple routing is the one type of routing policy which doesn't support health checks and so it is fairly limited but it is simple to implement and manage.

So that's simple routing again as the name suggests it's simple it's not all that flexible and it doesn't really offer any exciting features but don't worry I'll be covering some advanced routing types over the coming videos.

For now just go ahead and complete this video and then when you're ready I look forward to you joining me in the next.

Welcome back and in this video I want to talk about the other type of hosted zone available within Route 53 and that's private hosted zone.

So let's jump in and get started straight away.

A private hosted zone is just like a public hosted zone in terms of how it operates only it's not public.

Instead of being public it's associated with VPCs within AWS and it's only accessible within VPCs that it's associated with.

You can associate a private hosted zone with VPCs in your account using the console UI, CLI and API and even in different accounts if you use the CLI and API only.

Everything else is the same you can use them to create resource records and these are resolvable within VPCs.

It's even possible to use a technique called split view or split horizon DNS which is where you have public and private hosted zones of the same name meaning that you can have a different variant of a zone for private users versus public.

You might do this if you want your company intranet to run on the same address as your website and have your users be presented with your intranet when internal but the public website when anyone accesses from outside of your corporate network or if you wanted certain systems to be accessible via your business's DNS but only within your environment.

Now let's quickly step through how private hosted zones work visually so that you have more of an idea of the end-to-end architecture.

So we start with a private hosted zone and as with public zones we can create records within this zone.

Now from the public internet our users can do normal DNS queries so for things like Netflix.com and Categor.io but the private hosted zone is inaccessible from the public internet.

It can be made accessible though from VPCs.

Let's assume all three of these VPCs have services inside them and use the Route 53 resolver so the VPC +2 address.

Well any VPCs which we associate with the private hosted zone will be able to access that zone via the resolver.

Any VPCs which aren't associated will face the same problem as the user on the public internet on the left.

Access isn't available so private hosted zones are great when you need to provide records using DNS but maybe they're sensitive and need to be accessible only from internal VPCs.

Just remember to be able to access a private hosted zone the service needs to be running inside a VPC and that VPC needs to be associated with the private hosted zone.

Now before I finish up this short lesson let's talk about split view or split horizon DNS.

Consider this scenario you have a VPC running an Amazon workspace and to support some business applications a private hosted zone with some records inside it.

The private hosted zone is associated with VPC 1 on the right meaning the workspace could use the Route 53 resolver to access the private hosted zone.

For example to access the accounting records stored within the private hosted zone.

Now the private hosted zone is not accessible from the public internet but what split view allows us to do is to create a public hosted zone with the same name.

This public hosted zone might only have a subset of records that the private hosted zone has so from the public internet access to the public hosted zone would work in the same way as you would expect so via the ISP resolver server then through to the DNS root servers from there to the .org TLD servers and from there through to the animals for life name servers provided by Route 53.

Any records inside the public hosted zone would be accessible but records in the private hosted zone which are not in the public hosted zone so accounting in this example would be inaccessible from the public internet and this is a common architecture where you want to use the same domain name for public access and internal access but with a different set of records available to each.

It's something that you'll need to be comfortable with as an architect designing solutions, a developer integrating DNS into your applications or an engineer implementing this within AWS.

Now that's everything I want to cover on the theory of private hosted zone so go ahead and complete this video and when you're ready I look forward to you joining me in the next.

Welcome back.

In this video, I want to talk about Route 53 public hosted zones.

There are two types of DNS zones in Route 53, public and private.

To start with, let's cover off some general facts and then we can talk specifically about public hosted zones.

A hosted zone is a DNS database for a given section of the global DNS database, specifically for a domain such as AnimalsForLife.org.

Route 53 is a globally resilient service.

These name servers are distributed globally and have the same dataset, so whole regions can be affected by outages and Route 53 will still function.

Hosted zones are created automatically when you register a domain using Route 53 and you saw that earlier in the course when I registered the AnimalsForLife.org domain.

They can also be created separately if you want to register a domain elsewhere and use Route 53 to host it.

There's a monthly fee to host each hosted zone and a fee for the query is made against that hosted zone.

A zone where the public or private hosts DNS records.

Examples of these being A records or the IP version 6 equivalent, MX records, NS records and text records and I've covered these at an introductory level earlier in the course.

In summary, hosted zones are databases which are referenced via delegation using name server records.

A hosted zone when referenced in this way is authoritative for a domain such as AnimalsForLife.org.

When you register a domain, name server records for that domain are entered into the top level domain zone.

These point at your name servers and then your name servers and the zone that they host become authoritative for that domain.

A public hosted zone is a DNS database, so a zone file which is hosted by Route 53 on public name servers.

This means it's accessible from the public internet and within VPCs using the Route 53 resolver.

Architecturally, when you create a public hosted zone, Route 53 allocates four public name servers.

It's on those name servers that the zone file is hosted.

To integrate it with the public DNS system, you change the name server records for that domain to point at those four Route 53 name servers.

Inside a public hosted zone, you create resource records which are the actual items of data which DNS uses.

You can, and I'll cover this in an upcoming video, use Route 53 to host zone files for externally registered domains.

So for example, you can use hover or goad adi to register a domain.

You can create the public hosted zone in Route 53, get the four name servers which are allocated to that hosted zone, and then via the hover or goad adi interface, you can add those name servers into the DNS system for your domain.

And I'll cover how this works in detail in a future video.

Visually, this is how a public hosted zone looks and functions.

We start by creating a public hosted zone, and for this example, it's animalsforlife.org.

Creating this allocates four Route 53 name servers for this zone, and those name servers are all accessible from the public internet.

They're also accessible from AWS VPCs using the Route 53 resolver, which assuming DNS is enabled for the VPC is directly accessible from an internal IP address of that VPC.

Inside this hosted zone, we can create some resource records, in this case, a www.wrecord, two MX records for email, and a text record.

Within the VPC, the access method is direct, the VPC resolver using the VPC plus two address, and this is accessible from any instances inside the VPC, which use this as their DNS resolver.

So they can query the hosted zone as they can any public DNS zone using the Route 53 resolver.

From a public DNS perspective, the architecture is the same in that the same zone file is used, but the mechanics are slightly different.

DNS starts with the DNS Route servers, and these are the first servers queried by our users resolver server.

So Bob is using a laptop talking to his ISP DNS resolver server, which queries the Route servers.

The Route servers have information on the .org top level domain, and so the ISP resolver server can then query the .org servers.

These servers host the .org zone file, and this zone file has an entry for AnimalsForLife.org, which has four name servers and these all point at the Route 53 public name servers for the public hosted zone for Animals For Life.

This process is called "walking the tree", and this is how any public internet host can access the records inside a public hosted zone using DNS.

And that's how public hosted zones work.

They're just a zone file which is hosted on four name servers provided by Route 53.

This public hosted zone can be accessed from the public internet or any VPCs which are configured to allow DNS resolution.

There's a monthly cost for hosting this public hosted zone and a tiny charge for any queries made against it.

Almost nothing in the grand scheme of things, but for larger volume sites, it's something to keep in mind.

So that's public hosted zones, that's everything I wanted to cover in this video on the theory side of things.

So go ahead and complete this video and then when you're ready, I'll look forward to you joining me in the next.

Welcome back and in this lesson I want to cover two important EC2 optimisation topics, Enhanced Networking and EBS Optimised Instances.

Both of these are important on their own, both provide massive benefits to the way EC2 performs and they support other performance features within EC2 such as placement groups.

As a solutions architect understanding their architecture and benefits is essential.

So let's get started.

Now let's start with Enhanced Networking.

Enhanced Networking is a feature which is designed to improve the overall performance of EC2 networking.

It's a feature which is required for any high-end performance features such as cluster placement groups.

Enhanced Networking uses a technique called SRIOV or Single Route IO Virtualisation.

And I've mentioned this earlier in the course.

At a high level it makes it so that a physical network interface inside an EC2 host is aware of virtualisation.

Without Enhanced Networking this is how networking looks on an EC2 host architecturally.

In this example we have two EC2 instances, each of them using one virtual network interface.

And both of these virtual network interfaces talk back to the EC2 host and each of them use the host's single physical network interface.

The crucial thing to understand here is that the physical network interface card isn't aware of virtualisation.

And so the host has to sit in the middle controlling which instance has access to the physical card at one time.

It's a process taking place in software so it's slower and it consumes a lot of host CPU.

When the host is under heavy load so CPU or IO it can cause drops in performance, spikes in latency and changes in bandwidth.

It's not an efficient system.

Enhanced Networking or SRIOV changes things.

Using this model the host has network interface cards which are aware of virtualisation.

Instead of presenting themselves as single physical network interface cards which the host needs to manage, it offers what you can think of as logical cards, multiple logical cards per physical card.

Each instance is given exclusive access to one of these logical cards and it sends data to this the same as it would do if it did have its own dedicated physical card.

The physical network interface card handles this process end to end without consuming mass amounts of host CPU.

And this means a few things which matter to us as solutions architects.

First in general it allows for higher IO across all instances on the host and lower host CPU as a result because the host CPU doesn't have the same level of involvement as when no enhanced networking is used.

What this translates into directly is more bandwidth.

It allows for much faster networking speeds because it can scale and it doesn't impact the host CPU.

Also because the process occurs directly between the virtual interface that the instance has and the logical interface that the physical card offers, you can achieve higher packets per second or PPS.

And this is great for applications which rely on networking performance, specifically those which need to shift lots of small packets around the small isolated network.

And lastly because the host CPU isn't really involved because it's offloaded to the physical network interface card, you get low latency and perhaps more importantly consistent low latency.

Enhanced networking is a feature which is either enabled by default or available for no charge on most modern EC2 instance types.

There's a lot of detail in making sure that you have it enabled, but for the solutions architect stream none of that is important.

As always though, I'll include some links attached to the lesson if you do want to know how to implement it operationally.

Okay, so that's enhanced networking.

Let's move on to EBS optimized instances.

Whether an instance is EBS optimized or not depends on an option that's set on a per instance basis.

It's either on or it's off.

To understand what it does, it's useful to appreciate the context.

What we know already is that EBS is block storage for EC2, which is delivered over the network.

Historically, networking on EC2 instances was actually shared with the same network stack being used for both data networking and EBS storage networking.

And this resulted in contention and limited performance for both types of networking.

Simply put, an instance being EBS optimized means that some stack optimizations have taken place and dedicated capacity has been provided for that instance for EBS as usage.

It means that faster speeds are possible with EBS and the storage side of things doesn't impact the data performance and vice versa.

Now, on most instances that you'll use at this point in time, it's supported and enabled by default at no extra charge.

Disabling it has no effect because the hardware now comes with the capability built in.

On some older instances, it's supported but enabling it costs extra.

EBS optimization is something that's required on instance types and sizes which offer higher levels of performance.

So things which offer high levels of throughput and IOPS, especially when using the GP2 and IO1 volume types, which promise low and consistent latency as well as high input output operations per second.

So that's EBS optimization.

It's nothing complex.

It essentially just means adding dedicated capacity for storage networking to an EC2 instance.

And at this point in time, it's generally enabled and comes with all modern types of instances.

So it's something you don't have to worry about, but you do need to know that it exists.

Now, that's the theory that I wanted to cover.

I wanted to keep it brief.

There's a lot more involved in using both of these and understanding the effects that they can have.

But this is an architecture lesson for this stream.

You just need to know that both features exist and what they enable you to do what features they provide at a high level.

So thanks for watching.

Go ahead, finish this video. video and when you're ready you can join me in the next.

Welcome back.

In this lesson I want to cover EC2 dedicated hosts, a feature of EC2 which allows you to gain access to hosts dedicated for your use which you can then use to run EC2 instances.

Now I want to keep it brief because for the exam you just need to know that the feature exists and it tends to have a fairly narrow use case in the real world.

So let's just cover the really high level points and exactly how it works architecturally.

So let's jump in and get started.

An EC2 dedicated host as the name suggests is an EC2 host which is allocated to you in its entirety.

So allocated to your AWS account for you to use.

You pay for the host itself which is designed for a specific family of instances.

For example A1, C5, M5 and so on.

Because you're paying for the host there are no charges for any instances which are running on the host.

The host has a capacity and you're paying for that capacity in its entirety so you don't pay for instances running within that capacity.

Now you can pay for a host in a number of ways either on demand which is good for short term or uncertain requirements or once you understand long term requirements and patterns of usage you can purchase reservations with the same one or three year terms as the instances themselves.

And this uses the same payment method architecture so all upfront, partial upfront or no upfront.

The host hardware itself comes with a certain number of physical sockets and cores and this is important for two reasons.

Number one it dictates how many instances can be run on that host.

And number two software which is licensed based on physical sockets or cores can utilize this visibility of the hardware.

Some enterprise software is licensed based on the number of physical sockets or cores in the server.

Imagine if you're running some software on a small EC2 instance but you have to pay for the software licensing based on the total hardware in the host that that instance runs on.

Even though you can't use any of that extra hardware without paying for more instance fees.

With dedicated hosts you pay for the entire host so you can license based on that host which is available and dedicated to you.

And then you can use instances on that host free of charge after you've paid the dedicated host fees.

So the important thing to realize is you pay for the host.

Once you've paid for that host you don't have any extra EC2 instance charges.

You're covered for the consumption of the capacity on that host.

Now the default way that dedicated hosts work is that the hosts are designed for a specific family and size of instance.

So for example an A1 dedicated host comes with one socket and 16 cores.

All but a few types of dedicated hosts are designed to operate with one specific size at a time.

So you can get an A1 host which can run 16 A1 medium instances or 8 large or 4 extra large or 2 extra large or 1 4 extra large.

All of these options consume the 16 cores available.

And all but a few types of dedicated hosts require you to set that in advance.

So they require you to set that one particular host can only run 8 large instances or 4 extra large or 2 extra large and you can't mix and match.

Newer types of dedicated hosts, so those running the Nitro virtualization platform, they offer more flexibility.

An example of this is an R5 dedicated host which offers 2 sockets and 48 cores.

Because this is Nitro based, you can use different sizes of instances at the same time up to your core limit of that dedicated host.

So one host might be running 1 12 extra large, 1 4 extra large and 4 2 extra large which consumes 48 cores of that dedicated host.

Another host might use a different configuration, maybe 4 4 extra large and 4 2 extra large which also consumes 48 cores.

With Nitro based dedicated hosts, there's a lot more flexibility allowing a business to maximize the value of that host, especially if they have varying requirements for different sizes of instances.

Now this is a great link which I've included in the lesson text which details the different dedicated host options available.

So you've got different dedicated hosts for different families of instance, for example the A1 instance family.

This offers 1 physical socket and 16 physical cores and offers different configurations for different sizes of instances.

Now if you scroll all the way down, it also gives an overview of some of the Nitro based dedicated hosts which support this mix and match capability.

So we've got the R5 dedicated host that I just talked about on the previous screen.

We've also got the C5 dedicated host and this gives 2 example scenarios.

In scenario 1 you've got 1 instance of a C5 9 extra large, 2 instances of C5 4 extra large and 1 instance of C5 extra large.

And that's a total cores consumed of 36.

There's also another scenario though where you've got 4 times 4 extra large, 1 times extra large and 2 times large.

Same core consumption but a different configuration of instances.

And again, I'll make sure this is included in the lesson description.

It also gives the on-demand pricing for all of the different types of dedicated host.

Now there are some limitations that you do need to keep in mind for dedicated host.

The first one is AMI limits.

You can't use REL, Seuss Linux or Windows AMIs with dedicated host.

They are simply not supported.

You cannot use Amazon RDS instances.

Again, they're not supported.

You can't utilize placement groups.

They're not supported on dedicated hosts.

And there's a lesson in this section which talks in depth about placement groups.

But in this context, as it relates to dedicated hosts, you cannot use placement groups with dedicated hosts.

It's not supported.

Now with dedicated hosts, they can be shared with other accounts inside your organization using the RAM product, which is the resource access manager.

It's a way that you can share certain AWS products and services between accounts.

We haven't covered it yet, but we will do later in the course.

You're able to share a dedicated host with other accounts in your organization.

And other AWS accounts in your organization can then create instances on that host.

Those other accounts which have a dedicated host shared into them can only see instances that they create on that dedicated host.

They can't see any other instances.

And you, as the person who owns the dedicated host, you can see all of the instances running on that host.

But you can't control any of the instances running on your host created by any accounts you share that host with.

So there is a separation.

You can see all of the instances on your host.

You can only control the ones that you create.

And then other accounts who get that host shared with them, they can only see instances that they create.

So there's a nice security and visibility separation.

Now that's all of the theory that I wanted to cover around the topic of dedicated hosts.

You don't need to know anything else for the exam.

And if you do utilize dedicated hosts for any production usage in the real world, it is generally going to be around software licensing.

Generally using dedicated hosts, there are restrictions.

Obviously they are specific to a family of instance.

So it gives you less customizability.

It gives you less flexibility on sizing.

And you generally do it if you've got licensing issues that you need solved by this product.

In most cases, in most situations, it's not the approach you would take if you just want to run EC2 instances.

But with that being said, go ahead, complete this video.

And when you're ready, I'll look forward to you joining me in the next one.

Welcome back and in this lesson I want to talk about an important feature of EC2 known as placement groups.

Normally when you launch an EC2 instance its physical location is selected by AWS placing it on whatever EC2 host makes the most sense within the availability zone that it's launched in.

Placement groups allow you to influence placement ensuring that instances are either physically close together or not.

As a Solutions Architect understanding how placement groups work and why you would use them is essential so let's jump in and get started.

There are currently three types of placement groups for EC2.

All of them influence how instances are arranged on physical hardware but each of them do it for different underlying reasons.

At a high level we have cluster placement groups and these are designed to ensure that any instances in a single cluster placement group are physically close together.

We've got spread placement groups which are the inverse ensuring that instances are all using different underlying hardware and then we've got partition placement groups and these are designed for distributed and replicated applications which have infrastructure awareness.

So where you want groups of instances but where each group is on different hardware.

So I'm going to cover each of them in detail in this lesson once we talk about each of them they'll all make sense.

Now cluster and spread tend to be pretty easy to understand.

Partition is less obvious if you haven't used the type of application which they support but it will be clear once I've explained it and once you've finished with this lesson.

Now let's start with cluster placement groups.

Cluster placement groups are used when you want to achieve the absolute highest level of performance possible within EC2.

With cluster placement groups you create the group and best practice is that you launch all of the instances which will be in the group all at the same time.

This ensures that AWS allocate capacity for everything that you require.

So for example if you launch with nine instances imagine that AWS place you in a location with the capacity for 12.

If you want to double the number of instances you might have issues.

Best practice is to use the same type of instance as well as launching them all at the same time because then AWS will place all of them in a suitable location with capacity for everything that you need.

Now cluster placement groups because of their performance focus have to be launched into a single availability zone.

Now how this works is that when you create the placement group you don't specify an availability zone.

Instead when you launch the first instance or instances into that placement group it will lock that placement group to whichever availability zone that instance is also launched into.

The idea with cluster placement groups is that all of the instances within the same cluster placement group generally use the same rack but often the same EC2 host.

All of the instances within a placement group have fast direct bandwidth to all other instances inside the same placement group.

And when transferring data between instances within that cluster placement group they can achieve single stream transfer rates of 10 GB per second versus the usual 5 GB per second which is achievable normally.

Now this is single stream transfer rates while some instances do offer significantly faster networking you're always going to be limited to the speed that a single stream of data a single connection can achieve.

And inside a cluster placement group this is 10 GB per second versus 5 GB per second which is achievable normally.

Now the connections between these instances because of the physical placement they're the lowest latency possible and the maximum packets per second possible within AWS.

Now obviously to achieve these levels of performance you need to be using instances with high performance networking so IE more bandwidth than the 10 GB per second single stream and you should also use enhanced networking on all instances so definitely to achieve the low latency and max packets per second you do need also to use enhanced networking.

So cluster placement groups are used when you really need performance.

They're needed to achieve the highest levels of throughput and the lowest consistent latencies within AWS but the trade-off is because of the physical location if the hardware that they're running on fails logically it could take down all of the instances within that cluster placement group.

So cluster placement groups offer little to no resilience.

Now some key points which you need to be aware of for the exam you cannot span availability zones with cluster placement groups this is locked when launching the first instance.

You can span VPC peers but this does significantly impact performance in a negative way.

Cluster placement groups are not supported on every type of instance it requires a supported instance type and generally you should use the same type of instance to get the best results though this is not mandatory and you should also launch them at the same time.

Also this is not mandatory but it is very recommended and ideally you should always launch all of the instances as the same type and at the same time when using cluster placement groups.

Now cluster placement groups offer 10 GB per second of single stream performance and the type of use cases where you would use them are any type of workloads which demand performance so fast speeds and low latency.

So this might be things like high performance compute or other scientific analysis which demand fast node-to-node speed and low consistent latency.

Now the next type of placement group I want to talk about is spread placement groups and these are designed to ensure the maximum amount of availability and resilience for an application.

So spread placement groups can span multiple availability zones in this case availability zone A and availability zone B.

Instances which are placed into a spread placement group are located on separate isolated infrastructure racks within each availability zone so each instance has its own isolated networking and power supply separate from any of the other instances also within that same spread placement group.

This means if a single rack fails either from a networking or power perspective the fault can be isolated to one of those racks.

Now with spread placement groups there is a limit to seven instances per availability zone because each instance is in a completely separate infrastructure rack and because there are limits on the number of these within each availability zone you do have that limit of seven instances per availability zone for spread placement groups.

Now the more availability zones in a region logically the more instances can be a part of each spread placement group but remember the seven instances per availability zone in that region.

Now again just some points that you should know for the exam spread placement groups provides infrastructure isolation so you're guaranteed that every instance launched into a spread placement group will be entirely separated from every other instance that's also in that spread placement group.

Each instance runs from a different rack each rack has its own network and power source and then just to stress again there is this hard limit of seven instances per availability zone.

Now with spread placement groups you can't use dedicated instances or hosts they're not supported and in terms of use cases spread placement groups are used when you have a small number of critical instances that need to be kept separated from each other so maybe mirrors of a file server or maybe different domain controllers within an organization anywhere where you've got a specific application and you need to ensure as high availability for each member of that application as possible where you want to create separate blast radiuses for each of the servers within that particular application and ensure that if one fails there is a smaller chance as possible that any of the other instances will fail.

You have to keep in mind these limits it's seven instances per availability zone but if you want to maximize the availability of your application this is the type of placement group to choose.

Now lastly we've got partition placement groups and these have a similar architecture to spread placement groups which is why they're often so difficult to understand fully and why it's often so difficult to pick between partition placement groups and spread placement groups.

Partition placement groups are designed for when you have infrastructure where you have more than seven instances per availability zone but you still need the ability to separate those instances into separate fault domains.

Now a partition placement group can be created across multiple availability zones in a region in this example az a and az b and when you're creating the partition placement group you specify a number of partitions with a maximum of seven per availability zone in that region.

Now each partition inside the placement group has its own racks with isolated power and networking and there is a guarantee of no sharing of infrastructure between those partitions.

Now so far this sounds like spread placement groups except with partition placement groups you can launch as many instances as you need into the group and you can either select the partition explicitly or have EC2 make that decision on your behalf.

With spread placement groups remember you had a maximum of seven instances per availability zone and you knew 100% that each instance within that spread placement group was separated from every other instance in terms of hardware.

With partition placement groups each partition is isolated but you get to control which partition to launch instances into.

If you launch 10 instances into one partition and it fails you lose all 10 instances.

If you launch seven instances and put one into each separate partition then it behaves very much like a spread placement group.

Now the key to understanding the difference is that partition placement groups are designed for huge scale parallel processing systems where you need to create groupings of instances and have them separated.

You as the designer of a system can have control over which instances are in the same and different partitions so you can design your own resilient architecture.

Partition placement groups offer visibility into the partitions.

You can see which instances are in which partitions and you can share this information with topology aware applications such as HDFS, HBase and Cassandra.

Now these applications use this information to make intelligent data replication decisions.

Imagine that you had an application which used 75 EC2 instances.

Each of those instances had its own storage and that application replicated data three times across that 75 instances.

So each piece of data was replicated on three instances and so essentially you had three replication groups each with 25 instances.

If you didn't have the ability to use partition placement groups then in theory all of those 75 instances could be in the same hardware and so you wouldn't have that resiliency.

With partition placement groups if the application is topology aware then it becomes possible to replicate data across different EC2 instances knowing that those instances are in separate partitions and so it allows more complex applications to achieve the same types of resilience as you get with spread placement groups.

Only that it has an awareness of that topology and it can cope with more than seven instances.

So the difference between spread and partition placement is that with spread placement it's all handled for you but you have that seven instance per availability zone limit but with partition placement groups you can have more instances but you or your application which is topology aware needs to administer the partition placement.

For larger scale applications that support this type of topology awareness this can significantly improve your resilience.

Now some key points for the exam around partition placement groups again seven partitions per availability zone instances can be placed into a specific partition or you can allow EC2 to automatically control that placement.

Partition placement groups are great for topology aware applications such as HDFS, HBase and Cassandra and partition placement groups can help a topology aware application to contain the impact of a failure to a specific part of that application.

So by the application and AWS working together using partition placement groups it becomes possible for large-scale systems to achieve significant levels of resilience and effective replication between different components of the application.

Now it's essential that you understand the difference between all three for the exam so make sure before moving on in the course you are entirely comfortable about the differences between spread placement groups and partition placement groups and then the different situations where you would choose to use cluster, spread and partition.

With that being said though that's everything I wanted to cover so go ahead and complete this lesson and when you're ready I look forward to you joining me in the next.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

So let's go back to the instance now.

Just press enter a few times to make sure it hasn't timed out.

That's good.

Now there's a small bug fix that we need to do before we move on.

The CloudWatch agent expects a piece of system software to be installed called CollectD.

And on Amazon Linux that is not installed.

So we need to do two things.

The first is to create a directory that the agent expects to exist.

So run that command.

And the second is to create a database file that the agent also expects to exist.

And we can do that by running this command.

Now at this point we're ready to move on to the final step.

So we've installed the agent and we've run the configuration wizard to generate the agent configuration.

And we're now safely stored inside the parameter store.

The final step is to start up the CloudWatch agent and provide it with the configuration that's stored inside the parameter store.

And by doing that the agent can access the configuration.

It can download it.

It can configure itself as per that configuration.

And then because we've got an attached instance role that has the permissions required, it can also inject all of the logging data for the web server and the system into CloudWatch logs.

So the final step is to run this command.

So this essentially runs the Amazon hyphen CloudWatch hyphen agent hyphen CTL.

And it specifies a command line option to fetch the configuration.

And it uses hyphen C and specifies SSM colon and then the parameter store parameter name.

Essentially what this command does is to start up the agent, pull the config from the parameter store, make sure the agent is running and then it will start capturing that logging data and start injecting it into CloudWatch logs.

So at this point if it's functioning correctly, what you should be able to do is go back to the AWS console, go to services, type CloudWatch and then select CloudWatch to move to the CloudWatch console.

Then if we go to log groups, now you might see a lot of log groups here.

That's fine.

Every time you apply the animals for life VPC templates, it's actually using a Lambda function which we haven't covered yet to apply an IP version six workaround, which I'll explain later in the course when we cover Lambda.

What you should find though is if you just scroll down all the way to the bottom, you should see either one, two or three of the log groups that we've created.

In this example on screen now, you can see that I have /var/log/httbd/error_log.

Now these logs will start to appear when they start getting new entries and those entries are sent into CloudWatch.

So right now you can see that I only have the error log.

Now if you don't see access underscore log, what you can do is go back to the EC2 consoles, select the WordPress instance that you've created using the one click deployment and then copy the public IP version four address into your clipboard.

Don't use this link, just copy the IP address and then open that in a new tab.

Now by doing that, it will generate some activity within the Apache web server and that will put some log items into the access log and that will mean that that logging information will then be injected into CloudWatch logs using the CloudWatch agent.

So if we move back to CloudWatch logs and then refresh, scroll down to the bottom.

Now we can see the access underscore log file.

Open the log stream for the EC2 instance.

This log file details any accesses to the web server on the EC2 instance.

You won't have a lot of entries in this.

Likewise, if you go back to log groups and look for the error log, that will detail any errors, any accesses which weren't successfully served.

So if you try to access a web page which doesn't exist, if there's a server error or any module errors, these will show inside this log group.

Now also, because we're using the CloudWatch agent, we also have access to some metrics inside the EC2 instance that we otherwise would not have had.

If we click on metrics and just drag this up slightly so we can see it, you'll see the AWS namespaces.

So these are namespaces with metrics inside that you would have had access to before, but there'll also be the CWAgent namespace and inside here, just maybe select the image ID, instance ID, instance type name.

Inside there, you'll see all of the metrics that you now have access to because you have the CloudWatch agent installed on this EC2 instance.

So these are detailed operating system level metrics such as disk, IO, read and write, and you would have not had access to these before installing the agent.

If we select another one, image ID, instance ID, instance type CPU, we'll be able to see the CPU cores that are on this instance together with the IO weight and the user values.

Again, these are things that you would not have had access to at this level of detail without the CloudWatch agent being installed.

Now I do encourage you to explore all of the different metrics that you now have access to as well as to how the log groups and log streams look with this agent installed.

But this is the end of what I had planned for this demo lesson.

So as always, we want to clear up all of the infrastructure that we've created within this demo lesson.

So to do that, I want you to move back to the EC2 console, right click on this instance, go down to security, select modify IAM role, and then remove the CloudWatch role from this instance.

You'll need to confirm that by following the instructions.

So to detach that role, then click on services and move back to IAM, click on roles.

And I want you to remove the CloudWatch role that you created earlier in this demo.

So select it and then click delete role.

You'll need to confirm that deletion.

What we're not going to do is delete the parameter value that we've created.

So if we go to services and then move back to systems manager, go to parameter store, because this is a standard parameter.

This won't incur any charges.

And we're going to be using this later on in future lessons of this course and other courses.

So this is a standard configuration for the CloudWatch agent, which we'll be using elsewhere in the course.

So we're going to leave this in place.

The last piece of cleanup that you'll need to do is to go back to the CloudFormation console.

You should have the single CW agent stack in place that you created at the start of the demo using the one click deployment.

Go ahead and select the stack, click on delete, and then confirm that deletion.

And once that's completed, all of the infrastructure you've used in this demo will be removed and the account will be back in the same state as it was at the start of this demo.

Now that's everything that I wanted you to do in this demo.

I just wanted to give you a brief overview of how to manually install the CloudWatch agent within an EC2 instance.

Now there are other ways to perform this installation.

You can use systems manager or bake it into AMIs or you can bootstrap it in using the process that you've seen earlier in the course.

We're going to be using the CloudWatch agent during future demos of the course to get access to this rich metric and logging information.

So most of the demos which follow in the course will include the CloudWatch agent configuration.

At this point though, that is everything I wanted you to do in this demo.

Go ahead, complete this video, and when you're ready, I'll look forward to you joining me in the next. in the next.

Welcome back and welcome to this demo where together we'll be installing the CloudWatch agent to capture and inject logging data for three different log files into CloudWatch logs as well as giving us access to some metrics inside the OS that we wouldn't have otherwise had visibility of.

So it's going to be a really good demonstration to show you the power of CloudWatch and CloudWatch logs when combined with the CloudWatch agent.

Now in order to do this demo you're going to need to deploy some infrastructure.

To do so just make sure that you're logged in to the general AWS account, so the management account of the organization and as always make sure you've got the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment URL which will deploy the infrastructure that you'll be using during this demo.

So go ahead and click on that link.

This will take you to a quick create stack screen.

The stack name should be pre-populated with CW agent.

You just need to scroll all the way down to the bottom, acknowledge the capabilities and click on create stack.

Also attached to this lesson is a lesson commands document which will contain all of the commands you'll be using during this demo lesson.

So go ahead and open that in a new tab.

Now you're going to need to let this cloud formation stack move into a create complete state before you continue the demo.

So go ahead and pause the video, wait for the status to change to create complete and then you're good to continue.

Okay so now this stack is in a create complete state then we good to continue the demo.

Now during this demo lesson you're going to be installing the cloud watch agent on an EC2 instance and this EC2 instance has been provisioned by this one-click deployment.

So the first thing that we need to do is to move across to the EC2 console and connect to this instance.

Once you're at the EC2 console click on instances running.

You should see one single EC2 instance called A4L WordPress.

Just go ahead and select this, right-click on it, select connect.

We're going to connect into this instance using EC2 instance connect so make sure that's selected.

Make sure also that the username is set to EC2-user and then connect into the instance.

Now if everything's working as it should be you should see the animals for life custom login banner when you log into the instance.

In my case I do see that and that means everything's working as expected.

So this demonstration is going to have a number of steps.

First we need to download the cloud watch agent then we need to install the agent then we need to generate the configuration file that this install of the agent as well as any future installs of the agent could use and then we need to get the cloud watch agent to read this config and start capturing and injecting those logs into cloud watch logs.

So step one is to download and install the agent and the command to do this is inside the lesson commands document which is attached to this lesson and that will install the agent but crucially it won't start it.

What we need to do before we can start the agent is to generate the config file that we'll use to configure this and any future agents but because we also want to store that config file inside the parameter store and because we also want to give this instance permissions to interact with cloud watch logs before we continue we need to attach an IAM role to this instance an EC2 instance role so that's the next step.

So we need to move back to the EC2 console click on services and then open the IAM console because we'll be creating an IAM role to attach to this instance you'll need to go to roles create role it'll be an AWS service role using EC2 so select EC2 and then click on next we'll need to attach two managed policies to this role so I've included the names of those managed policies in the lesson commands document the first is cloud watch agent server policy so make sure you type that in the filters policy and then check that box and the second is Amazon SSM full access so type that in the box select that policy and then scroll down and click on next and we'll call this role cloud watch role so enter cloud watch role and click on create role and once we've done that we can attach this role to our EC2 instance so we need to move back to the EC2 console go to instances right click on the instance go to security and then modify IAM role and then click on the drop down and select cloud watch role which is the role that you've just created then click update IAM role now that we've allocated that instance with the permissions that it needs to perform the next set of steps go ahead and connect to that instance again the tab may have timed out that you previously had open so if it doesn't respond you need to close it down and reopen it if it does respond that's fine keep the existing tab once we back in the terminal for the instance we need to start the cloud watch agent configuration wizard and the command to do that is also in the lesson commands document attached to this lesson so go ahead and paste that in and press enter and that will start off the configuration wizard for the cloud watch agent now for most of these values we can accept the defaults but we need to be careful because there are a number of them that we can't so press enter and accept the default for the operating system it should automatically detect Linux press enter it should automatically detect that it's running on an EC2 instance press enter to use the root user again that should be the default press enter for stats D press enter for the stats D port press enter for the interval press enter for the aggregation interval press enter to monitor metrics from collect D again that's the default press enter to monitor host metrics so CPU and memory press enter to monitor CPU metrics per core press enter for the additional dimensions press enter to aggregate EC2 dimensions press enter for the default resolution so 60 seconds for the default metric config that you want the default will be basic go ahead and enter 3 for advanced this captures additional operating system metrics that we might actually want so use 3 for this value press enter to indicate that we satisfied with the above config next we'll move to the log configuration part of this wizard so press enter for the default of no we don't have an existing cloud watch log agent config to import press enter which is the default for yes we do want to monitor log files you'll be asked for the log file path to monitor so the first one that we want to monitor and again these are in the lesson commands document so the first log path is forward slash var forward slash log forward slash secure press enter you'll be asked for the log group name the default is just the log name itself so secure but we're going to enter the full path I always prefer using the full path for the log group names for any system logs so going to enter var log secure again you'll be asked for the log stream name remembering the theory part of this lesson I talked about how a log stream will be named after the instance which is injecting those logs so the default choice is to do that to use the instance ID so press enter it's here where you can specify a log group retention in days we're just going to accept the default for the log group retention value the default will be yes we do want to specify additional log files so press enter the log file path for this one will be var log HTTP d access underscore log so enter that the log group name again will default to the name of the actual log we want the full path so enter the full path again press enter the log stream name the default for this is again the instance ID which is fine just press enter go ahead and accept the default for the log group retention in days press enter again we've got one more log file that we want to enter this time the log file path is var log HTTP d error underscore log again the log group name will default the name of the actual log we want to use the full path so enter the same thing again the default choice for log stream name will be again the instance ID that's fine press enter go ahead and accept the default for the log group retention in days and now we finished adding log files we won't want to log any additional files so press 2 and that will complete this logging section of this wizard it's asking us to confirm that we're happy with this configuration file and it's telling us that the configuration file is stored at forward slash opt forward slash aws forward slash amazon hyphen cloud watch hyphen agent forward slash bin and then config dot json in that folder now that's where it stores it on the local file system but we can also elect to store this json configuration inside the parameter store and I thought that since we've previously talked about the theory of the parameter store and done a little bit of interaction it would be useful for you to see exactly how it can be used in a more production like setting so the default is to store the configuration in the parameter store so we're going to allow that press enter it'll ask us for the parameter name to use and the default is Amazon cloud watch hyphen linux and that's fine so press enter it'll ask us for the region to use because parameter store is like many other services a regional service and the default region is the one where the instance is in so it automatically detects that we're in us east one which is northern Virginia so go ahead and accept that default choice it'll ask us for the credentials that it can use to send that configuration into the parameter store now these credentials will be obtained from the role that we've attached to this instance in the previous step so you can accept the default choice it'll use those credentials to store that configuration inside the parameter store and if we move back to the ec2 console and we switch back to the parameter store so just type SSM to move to systems manager which is the parent product of parameter store if we go down to the parameter store item on the menu on the left we'll be able to see this single parameter Amazon cloud watch hyphen linux and if we open that up and just scroll down we can see that the value is a JSON document with the full configuration of the cloud watch agent so we can now use this parameter to configure the agent on this ec2 instance as well as any other ec2 instances we want to deploy so if you create the cloud watch configuration once and then store it into parameter store then when you create ec2 instances at scale as you'll see how to do later in the course when we talk about auto scaling groups then you can use the parameter store to deploy this type of configuration at scale in a secure way okay so this is the end of part one of this lesson it was getting a little bit on the long side and so I wanted to add a break it's an opportunity just to take a rest or grab a coffee part 2 will be continuing immediately from the end of part one so go ahead complete the video and when you're ready join me in part 2.

Welcome back.

So far in the course you've had a brief exposure to CloudWatch and CloudWatch logs and you know that CloudWatch monitors certain performance and reliability aspects of EC2 but crucially only those metrics that are available on the external face of an EC2 instance.

There are situations when you need to enable monitoring inside an instance so have access to certain performance counters of the operating system itself.

So be able to look at the processes running on an instance, the memory consumption of those processes, so have access to certain operating system level performance metrics that you cannot see outside the instance.

You also might want to allow access to system and application logging from within the EC2 instance.

So application logs and system logs also from within the operating system of an EC2 instance.

So in this lesson I want to step through exactly how this works and what you need to use to achieve it.

So let's get started.

Now a quick summary of where we're at so far in the course relevant to this topic.

So I just mentioned you know now that CloudWatch is the product responsible for storing and managing metrics within AWS and you also know that CloudWatch logs is a subset of that product aimed at storing, managing and visualizing any logging data but neither of those products can natively capture any data or any logs that's happening inside of an EC2 instance.

The products aren't capable of getting visibility inside of an EC2 instance natively.

The inside of an instance is opaque to CloudWatch and CloudWatch logs by default.

To provide this visibility the CloudWatch agent is required and this is a piece of software which runs inside an EC2 instance.

So running on the operating system it captures OS visible data and sends it into CloudWatch or CloudWatch logs so that you can then use it and visualize it within the console of both of those products.

And logically for the CloudWatch agent to function it needs to have the configuration and permissions to be able to send that data into both of those products.

So in summary in order for CloudWatch and CloudWatch logs to have access inside of an EC2 instance then there's some configuration and security work required in addition to having to install the CloudWatch agent and that's what I want to cover over the remainder of this lesson and the upcoming demo lesson.

Architecturally the CloudWatch agent is pretty simple to understand.

We've got an EC2 instance on its own.

For example the animals for life WordPress instance from the previous demos.

It's incapable of injecting any logging into CloudWatch logs without the agent being installed.

So to fix that we need to install the CloudWatch agent within the EC2 instance and the agent will need some configuration.

So it will need to know exactly what information to inject into CloudWatch and CloudWatch logs.

So we need to configure the agent.

We need to supply the configuration information so that the agent knows what to do.

The agent also needs some way of interacting with AWS, some permissions.

We know now that it's bad practice to add long-term credentials to an instance so we don't want to do that but that aside it's also difficult to manage that at scale.

So best practice for using this type of architecture is to create an IAM role with permissions to interact with CloudWatch logs and then we can attach this IAM role to the EC2 instance providing the instance or more specifically anything running on the instance with access to the CloudWatch and CloudWatch logs service.

Now the agent configuration that will also need to set up that configures the metrics and the logs that we want to capture and these are all injected into CloudWatch using log groups.

We'll configure one log group for every log file that we want to inject into the product and then within each log group there'll be a log stream for each instance performing this logging.

So that's the architecture.

One log group for each individual log that we want to capture and then one log stream inside that log group for every EC2 instance that's injecting that logging data.

Now to get this up and running for a single instance you can do it manually.

You can log into the instance, install the agent, configure it, attach a role and start injecting the data.

At scale you'll need to automate the process and potentially you can use cloud formation to include that agent configuration for every single instance that you provision.

Now CloudWatch agent comes with a number of ways to obtain and store the configuration that it will use to send this data into CloudWatch logs and one of those ways is we can actually use the parameter store and store the agent configuration as a parameter and because we've just learned about parameter store I thought it would be beneficial as well as demonstrating how to install and configure the CloudWatch agent.

We should also utilize the parameter store to store that configuration and so that's what we're going to do together in the next demo lesson.

We're going to install and configure the CloudWatch agent and set it up to collect logging information for three different log files.

We're going to set it up to collect and inject logging for forward slash var forward slash log forward slash secure which shows any events relating to secure log ins to the EC2 instance and we're also going to collect logging information for the access log and the error log which are both log files generated by the Apache web server that's installed on the EC2 instance and by using these three different log files it should give you some great practical experience to how to configure the CloudWatch agent and how to use the parameter store to store configuration at scale.

So that's it for the theory for now you can go ahead and finish off this video and then when you're ready you can join me in the next demo lesson where we'll be installing and configuring the CloudWatch agent.

Welcome back and in this demo lesson, I'm just wanting to give you some practical experience with interacting with the parameter store inside AWS.

So to do that, make sure you're logged into the IAM admin user of the management account of the organization and you'll need to have the Northern Virginia region selected.

Now there's also a lesson commands document linked to this lesson, which contain all of the commands that you'll need for this lessons demonstration.

So before we start interacting with the parameter store from the command line, we need to create some parameters.

And the way that we do that is first move to systems manager.

So the parameter store is actually a sub product of systems manager.

So move over to the systems manager console.

And once you're there, you'll need to select the parameter store from the menu on the left.

So it should be about halfway down on the left and it's under application management.

So go ahead and select parameter store.

Now once you're in parameter store, the first thing that you'll need to do to remove this default welcome screen logically is to create a parameter.

So go ahead and click on create parameter.

Now when you create a parameter, you're able to pick between standard or advanced.

Standard is the default and that meets most of the needs that most people have for the product.

And you can create up to 10,000 parameters using the standard tier.

With the advanced tier, you can create more than 10,000 parameters.

The parameter value can be longer at eight kilobytes versus the four kilobytes of standard.

And you do gain access to some additional features.

But in most cases, most parameters are fine using the default, which is the standard tier.

With the standard tier, there's no additional charge to use this up to the limit of 10,000 parameters.

The only point at which parameter store costs any extra is if you use the faster throughput options or make use of this advanced tier.

And we won't be doing that at any point throughout the course.

We'll only be using standard.

And so there won't be any extra parameter store related charges on your bill.

Now I mentioned that a parameter is essentially a parameter name and a parameter value.

And it's here where you set both of those.

There's an optional description that you can use and you can set the type of the parameter.

The options being string, string list, which is a comma separated list of individual strings and then secure string, which utilizes encryption.

So we're gonna go ahead at this point and create some parameters that we're then going to interact with from the command line.

So the first one we'll create is one that's called forward slash my-cat-app forward slash DB string.

So this is the name of a parameter and it will also establish a hierarchy.

So anytime we use forward slashers, we're establishing a hierarchy inside the parameter store.

So imagine this being a directory structure.

Imagine this being the root of the structure.

Imagine my-cat-app being the top level folder and inside there, imagine that we've got a file called DB string.

So we're going to store this, we're going to store this hierarchy and we need to set its value.

So we'll keep this for now as a string and this is going to be the database connection string for my-cat-app.

So we'll just enter the value that's in the lesson commands document.

So DB dot all the cats dot com colon 3306.

And 3306 of course is the my SQL standard port number.

At this point, we could enter an optional description.

So let's go ahead and do that.

Connection string for cat application.

So just type in a description here.

It doesn't matter really what you type and then scroll down and hit create parameter.

So that's created our first parameter, my-cat-app forward slash DB string.

Now we're going to go ahead and do the same thing but for DB user.

So click on create parameter and then just click in this name box and notice how it presents you with this hierarchy.

So now we've got two levels of this hierarchical structure.

We've got the my-cat-app at the top and then we've got the actual parameter that we created at the bottom here.

So this has already established this structure.

So let's go ahead and create a new parameter.

This time it's going to be forward slash my-cat-app forward slash DB user.

We'll not bother with the description for this one.

We'll keep it at the default of standard and it will also be a string.

And then for the value, it'll be boss cat.

So enter all that and click on create parameter.

Next let's create a parameter again.

If we click in this name this time, we've got this hierarchy that's ever expanding.

So we've got the top level at the top and then below it two additional parameters, DB string and DB user.

And we're going to create a third one at this level.

So this time it's going to be called forward slash my-cat-app forward slash DB password.

This time though, instead of type string, it's going to be a secure string so that it encrypts this parameter.

And it's going to use KMS to encrypt the parameter.

And because it's using KMS, we'll need to select the key to use to perform the cryptographic operations.

We can either select a key from the current account, so the account that we're in, or we can select another AWS account.

And in either case, we'll need to pick the key ID to use and by default, it uses the product default key for SSM.

So that's using alias forward slash AWS forward slash SSM.

And you always have the option of clicking on this dropdown and changing it if you want to benefit from the extra functionality that you get by using a customer managed KMS key.

This is an AWS managed one.

So you won't be able to configure rotation and you won't be able to set these advanced key policies.

But in most cases, you can use this default key.

So at this point, we'll leave it as the default and we'll enter our super secret password, amazing secret password, 1337, and then click create parameter.

We're not finished yet though, click on create parameter again.

And I like to be inclusive, so not everything in my course is going to be about cats.

We're going to create another parameter, my-dog-app forward slash DB string.

We'll keep standard, we'll keep the type as string and then the value for connecting to the my-dog application.

So the DB string is going to be DB if we really must have dogs.com colon 3306.

So type that in and then click on create parameter.

And then lastly, we're going to create one more parameter.

This time the name is going to be forward slash rate my lizard.

So rate hyphen my hyphen lizard forward slash DB string.

The tier is going to be standard again.

The type is going to be string.

And for the value, it will be DB.

This is pretty random.com colon 3306.

So type that in and then click on create parameter.

So now we've created a total of five parameters.

We've created the DB string, the DB user, and the DB password for the cat application.

And then the DB string for the dog application as well as the rate my lizard application.

So a total of five parameters and one of them is using encryption.

So that's the DB password for the my cat application.

So now let's switch over to the command line and interact with these parameters.

And to keep things simple, we're going to use the cloud shell.

So this is a relatively new feature made available by AWS.

And this means that we don't have to interact with AWS using our local machine.

We can do it directly from the AWS console.

So click on the cloud shell icon on the menu on the top.

This will take a few moments to provision because this is creating a dedicated environment for you to interact with AWS using the command line interface.

So you'll need to wait for this process to complete.

So go ahead and pause the video and wait until this logs you into the cloud shell environment at which point you can resume the video and we're good to continue.

It'll say preparing your terminal and then you'll see a familiar looking shell much like you would if you were connected to a Linux instance.

And now you'll be able to interact with AWS using the command line interface, using the credentials that you're currently logged in with.

Now to interact with parameter store using the command line, we start by using AWS and then a space, SSM, and then a space, and then the command that we're going to use is get-parameters.

Now by default, what we need to provide the get-parameters command with is the path to a parameter.

So in this case, if we wanted to retrieve the database connection string for the rate my lizard application, then we could provide it with this name.

So forward slash rate-my-lizard/dvstring.

And this directly maps back through to the parameter that we've just created inside the parameter store.

So this parameter.

So if you go ahead and type that and press enter, it's going to return a JSON object.

Inside that JSON object is going to be a list of parameters and then for each parameter, so everything inside these inner curly braces, we're going to see the name of the parameter that we wanted to retrieve, the type of the parameter, the value of the parameter.

In this case, db.this_is_pretty_random.com/3306, the version number of the parameter because we can have different version numbers, the last modified date, the data type, and then the unique arn of this specific parameter.

And so this is an effective way that you can store and retrieve configuration information from AWS.

Now we can also use the same structure of command to retrieve all of those other parameters that we stored within the parameter store.

So if we wanted to get the db string for the my-dog-app, then we could use this command.

And again, it would return the same data structure.

So a JSON object containing a list of parameters and each of those parameters would contain all of this information.

I'll clear the screen to keep this easy to see.

We could do the same for the my-cat-app, retrieving its database connection string.

And again, it would return the same JSON object with the parameters list.

And then for each parameter, this familiar data structure.

Now what you can also do, and I'm going to clear the screen before I run this, is instead of providing a specific path to a parameter.

So if you remember, we had almost a hierarchy that we created with these different names.

So we have the my-cat-app hierarchy and then inside there db-pastword, db-string, db-user.

We have my-dog-app and inside there db-string and then rate-my-lizard and also db-string.

So rather than having to retrieve each of these individual parameters by specifying the exact name, we can actually use get parameters by path.

So let's demonstrate exactly how that works.

So with this command, we're doing a get-parameters-by-path and we're specifying a path to a group of a number of parameters.

So in this case, my-cat-app is actually going to be the first part of the path of db-pastword, db-string and db-user.

So by creating a hierarchical structure inside the parameter store, we can retrieve multiple parameters at once.

So this time we're returning a JSON structure.

Inside this JSON structure, we have a list of parameters and then we're retrieving three different parameters, db-pastword, db-string and db-user.

Now note how db-pastword is actually a type of secure string and by default, if we don't specify anything, we return the encrypted version of this parameter.

So the ciphertext version of this parameter.

This ensures that we can interact with parameters without actually decrypting them and this offers several security advantages.

Now I've cleared the screen to make this next part easy to see because it's very important.

Because we're using KMS to encrypt parameters, the permissions to access KMS keys to perform this decryption, this is separate than the permissions to access the parameter store.

So if this user, I am admin in this case, has the necessary permissions to interact with KMS to use the keys to decrypt these parameters, then we can also ask the parameter store to perform that decryption whilst we retrieve the parameters.

The important thing to understand is the permissions to interact with the parameter store are separate than the permissions to interact with KMS.

So to perform a decryption whilst we're retrieving the parameters, we would use this command.

So it's the same command as before, aws, SSM, get-parameters-by-path, and then we're specifying the my-cat-app part of the hierarchy.

So remember, this represents these three parameters.

Now, if we ran just this part on its own, which was the command we previously ran, this would retrieve the parameters without performing decryption.

But by adding this last part, this is the part that performs the decryption on any parameter types which are encrypted.

And if you recall, one of the parameters that we created was this DB password, which is encrypted.

So if we run this command, this time it's going to retrieve the /my-cat-app/db password parameter, but it's going to decrypt it as part of that retrieval operation and return the plain text version of this parameter.

And just to reiterate, that requires both the permissions to interact with the parameter store, as well as the permissions to interact with the KMS key that we chose when creating this parameter.

Now, we're logged in as the IAM admin user, which has admin permissions, and so we do have permissions on both of those, on SSM and on KMS, so we can perform this decryption operation.

Now, you're going to be using the parameter store extensively for the rest of the course and my other courses.

It's a great way of providing configuration information to applications, both AWS and bespoke applications within the AWS platform.

It's a much better way to inject configuration information when you're automatically building applications or you need applications to retrieve their own configuration information.

It's much better to retrieve it from the parameter store than to pass it in using other methods.

So we're going to use it in various different lessons as we move throughout the course.

In this demo lesson, I just wanted to give you a brief experience of working with the product and the different types of parameters.

But at this point, let's go ahead and clear up all of the things that we've created inside this demo lesson.

So close down this tab, open to Cloud Shell.

Back at the parameter store console, just go ahead and check the box at the top to select all of these existing parameters.

If you do have any other parameters, apart from the ones that you've created within this demo lesson, then do make sure that you uncheck them.

You should be using an account dedicated for this training, so you shouldn't have any others at this point.

But if you do, make sure you uncheck them.

You should only be deleting ones for 8-mile lizard, my dog app, and my cat app.

So make sure that all of those are selected and then click on delete to delete those parameters, and you'll need to confirm that deletion process.

And at this point, that's everything that I wanted you to do in this lesson.

You've cleared up the account back to the point it was at the start of this demo lesson.

So go ahead, complete this video, and when you're ready, I'll look forward to you joining me in the next.

Welcome back.

In this lesson, I want to cover the Systems Manager parameter store, a service from AWS which makes it easy to store various bits of system configuration.

So strings, documents, and secrets, and store those in a resilient, secure, and scalable way.

So let's step through the products architecture, including how to best make use of it.

If you remember earlier in this section of the course, I mentioned that passing secrets into an EC2 instance using user data was bad practice because anyone with access to the instance could access all that data.

Well, parameter store is a way that this can be improved.

Parameter store lets you create parameters, and these have a parameter name and a parameter value, and the value is the part that stores the actual configuration.

Many AWS services integrate with the parameter store natively.

CloudFormation offers integrations that you've already used, which I'll explain in a second and in the upcoming demo lessons, and you can also use the CLI tooling on an EC2 instance to get access to the service.

So when I'm talking about configuration and secrets, parameter store offers the ability to store three different types of parameters.

We've got strings, string lists, and secure strings.

And using these three different types of parameters, you can store things inside the product, such as license codes, database connects and strings, so host names, ports.

You can even store full configs and passwords.

Now parameter store also allows you to store parameters using a hierarchical structure.

Parameter store also stores different versions of parameters.

So just like we've got object versioning in S3 inside parameter store, we can also have different versions of parameters.

Parameter store can also store plain text parameters, and this is suitable for things like DB connection strings or DB users, but we can also use cipher text, and this integrates with KMS to allow you to encrypt parameters.

So this is useful if you're storing passwords or other sensitive information that you want to keep secret.

So when you encrypt using cipher text, you use KMS, and that means you need permissions on KMS as well.

So there's that extra layer of security.

The parameter stores also got the concept of public parameters, so these are parameters publicly available and created by AWS.

You've used these earlier in the course.

An example is when you've used cloud formation to create EC2 instances, you haven't had to specify a particular AMI to use, because you've consulted a public parameter made available by AWS, which is the latest AMI ID for a particular operating system in that particular region, and I'll be demonstrating exactly how that works now in the upcoming demo lessons.

Now, the architecture of the parameter store is simple enough to understand.

It's a public service, so anything using it needs to either be an AWS service or have access to the AWS public space endpoints.

Different types of things can use the parameter store, so this might be things like applications, EC2 instances, all the things running on those instances, and even Lambda functions.

And they can all request access to parameters inside the parameter store.

As parameter store is an AWS service, it's tightly integrated with IAM for permissions, so any accesses will need to be authenticated and authorized, and that might use long-term credentials, so access keys or those credentials might be passed in via an IAM role.

And if parameters are encrypted, then KMS will be involved and the appropriate permissions to the CMK inside KMS will also be required.

Now, parameter store allows you to create simple or complex sets of parameters.

For example, you might have something simple like myDB password, which stores your database password in an encrypted form, but you can also create hierarchical structures.

So something like /wordpress/ and inside there, we might have something called dbUser, which could be accessed either by using its full name or requesting the WordPress part of the tree.

We could also have dbPassword, which again, because it's under the WordPress branch of the tree, could be accessed along with the dbUser by pulling the whole WordPress tree or accessed individually by using its full name, so /wordpress/dbPassword.

Now, we might also have applications which have their own part of the tree, for example, my-cat-app, or you might have functional division in your organization, so giving your dev team a branch of the tree to store their passwords.

Now, permissions are flexible and they can be set either on individual parameters or whole trees.

Everything supports versioning and any changes that occur to any parameters can also spawn events.

And these events can start off processes in other AWS services.

And I'll introduce this later.

I just want to mention it now so you understand that parameter store parameter changes can initiate events that occur in other AWS products.

Now, parameter store isn't a hugely complex product to understand.

And so at this point, I've covered all of the theory that you'll need for the associate level exam.

What I want to do now is to finish off this theory lesson.

And immediately following this is a demo where I want to show you how you can interact with the parameter store via the console UI and the AWS command line tools.

Now, it will be a relatively brief demo and so you're welcome to just watch me perform the steps.

Or of course, as always, you can follow along with your own environment and I'll be providing all the resources that you need to do that inside that demo lessons folder in the course GitHub repository.

So at this point, go ahead, finish this video.

And when you're ready, you can join me in the next demo lesson.

Welcome to this demo lesson where you're going to get the experience of working with EC2 and EC2 instance roles.

Now as you learned in the previous theory lesson, an instance role is essentially a specific type of IAM role designed so that it can be assumed by an EC2 instance.

When an instance assumes a role which happens automatically when the two of them are linked, that instance and any applications running on that instance can gain access to the temporary security credentials that that role provides.

And in this demo lesson you're going to get the experience of working through that process.

Now to get started you're going to need some infrastructure.

Make sure that you're logged in to the general AWS account, so that's the management account of the organization and as always you'll need to be within the Northern Virginia region.

Assuming you are, there's a one click deployment link which is attached to this lesson so go ahead and click that link.

That will take you to a quick create stack page.

The stack name will be pre-populated with IAM role demo and all you need to do is to scroll down to the bottom, check this capabilities box and then click on create stack.

This one click deployment will create the animals for live VPC and EC2 instance and an S3 bucket.

Now in order to continue with this demo we're going to need this stack to be in a create complete state.

So go ahead and pause the video and then when the stack moves into a create complete status then we're good to continue.

Okay so this stacks now in a create complete state and we're good to continue.

So to do so go ahead and click on the services drop down and then type EC2, locate it, right click and then open that in a new tab.

Once you're at the EC2 console click on instances running and you should be able to see that we only have the one single EC2 instance.

Now we're going to connect to this to perform all the tasks as part of this demo.

So right click on this instance, select connect, we're going to use EC2 instance connect.

Just verify that the username does say EC2-user and then click on connect.

Now the AMI that we use to launch this instance is just the standard Amazon Linux 2 AMI.

And so if we type AWS and press enter it comes with the standard install of the AWS CLI version 2.

Now it's important to understand that right now this instance has no attached instance role and it's not been configured in any way.

It's the native Amazon Linux 2 AMI that's been used to launch this instance.

And so if we attempt to interact with AWS using the command line utilities, for example by running an AWS S3 LS, the CLI tools will tell us that there are no credentials configured on this instance and will be prompted to provide long term credentials using AWS Configure.

Now this is the method that you've used to provide credentials to your own installed copy of the CLI tools running on your local machine.

So you've used AWS Configure and set up two named configuration profiles.

And the way that you provide these with authentication information is using access keys.

Now this instance has no access keys configured on it and so it has no method of interacting with AWS.

We could use AWS Configure and provide these credentials but that's not best practice for an EC2 instance.

What we're going to do instead is use an instance role.

So to do that you're going to need to move back to the AWS console.

And once you're there click on services and in the search box type IAM.

We're going to move to the IAM console so right click and open that in a new tab.

As I mentioned earlier an instance role is just a specific type of IAM role.

So we're going to go ahead and create an IAM role which our instance can assume.

So click on roles and then we're going to go ahead and click on create role.

Now the create role process presents us with a few common scenarios.

We can create a role that's used by an AWS service, another AWS account, a web identity or a role designed for SAML 2.0 Federation.

In our case we want a role which can be assumed by an AWS service specifically EC2.

So we'll select the type of trusted entity to be an AWS service then we'll click on EC2 and then we'll click on next.

Now for the permissions in this search box just go ahead and type S3 and we're looking for the Amazon S3 read only access.

So there's a managed policy that we're going to associate with this role.

So check the box next to Amazon S3 read only access and then we'll click on next.

And then under role name we're going to call this role A4L instance role.

So it's easy to distinguish from any other roles we have in the account.

So go ahead and enter that and click on create role.

Now as I mentioned in the theory lesson about instance roles when we do this from the user interface.

It's actually created a role and an instance profile of the same name and it's the instance profile that we're going to be attaching to the EC2 instance.

Now from a UI perspective both of these are the same thing.

You're not exposed to the role and the instance profile as separate entities but they do exist.

So now we're going to move back to the EC2 console and remember currently this instance has no attached instance role and we're unable to interact with AWS using this EC2 instance.

To attach an instance role using the console UI right click, go down to security and then modify IAM role.

Select that and we'll need to choose a new IAM role.

You have the option of creating one directly from this screen but we've already created the one that we want to apply.

So click in the drop down and select the role that you've just created.

In this case A4L instance role.

So select that and then click on save.

Now if we select the instance and then click on security you'll be able to confirm that it does have an IAM role attached to this instance.

So this is the instance role that this EC2 instance can now utilize.

So now we're going to interact with this instance again from the operating system.

Now if it's been a few minutes since you've last used instance connect you might find when you go back it appears to have frozen up.

If that's the case that's no problem just close down these tabs that you've got connected to that instance.

Right click on the instance again, select connect, make sure the username is EC2-user and then click on connect.

And this will reconnect you to that instance.

Now if you recall last time we were connected we attempted to run AWS S3LS and the command line tools informed us that we had no credentials configured.

Let's attempt that process again.

AWS space S3 space LS and press enter.

And now because we have the instance role associated with this EC2 instance the command line tools can use the temporary credentials that that role generates.

Now the way that this works and I'm going to demonstrate this using the curl utility these credentials are actually provided to the command line tools via the metadata.

So this is actually the metadata path that the command line tools use in order to get the security credentials.

So the temporary credentials that a role provides when it's assumed.

So if I use this command and press enter you'll see that it's actually using this role name.

So you'll see a list of any roles which are associated with this instance.

If we use the curl command again but this time on the end of security credentials we specify the name of the role that's attached to this instance and press enter.

Now we can see the credentials that command line tools are using.

So we have the access key ID the secret access key and the token and all of these have been generated by this EC2 instance assuming this role because these are temporary credentials.

They also have an expiry date.

So in my case here we can see that these credentials expire on the 7th of May 2022 at 552 47 UTC.

And that really is all I wanted to show you in this demo lesson about instance roles.

Essentially you just need to create an instance role and then attach it to an instance.

And once you do that instance is capable of assuming that role gaining access to temporary credentials and then any applications installed on that instance, including the command line utilities are capable of interacting with AWS using those credentials.

Now the process of renewing these credentials is automatic.

So as long as the application that's running on the instance periodically checks the metadata service, it will always have access to up to date and valid credentials.

The EC2 service once this expiry date closes in and once the expiry date is in the past, these credentials will be renewed and a new valid set of credentials will automatically be presented via the metadata service to any applications running on this EC2 instance.

Now just one more thing that I do want to show you before we finish up with this demo lesson.

And I have made sure that I've attached this link to the lesson.

This link shows the configuration settings and precedence that the command line utilities use in order to interact with AWS.

So whenever you use the command line interface, each of these is checked in order.

First, it looks at command line options.

Then it looks at environment variables to check whether any credentials are stored within environment variables.

Then it checks the command line interface credentials file.

So this is stored within the dot AWS folder within your home folder and then a file called credentials.

Next, it checks the CLI configuration file.

Next, it checks container credentials.

And then finally, it checks instance profile credentials.

And these are what we've just demonstrated.

Now, this does mean that if you manually configure any long term credentials for the CLI tools as part of using AWS Configure, then they will be used as a priority over an instance profile.

But you can use an instance profile and attach this to many different instances as a best practice way of providing them with access into AWS products and services.

So that's really critical to understand.

But at this point, that is everything that I wanted to cover in this demo lesson.

And all that remains is for us to tidy up the infrastructure that we've used as part of this demo.

So to tidy up this infrastructure, I want you to go back to the IAM console.

I want you to click on roles and I want you to delete the A4L instance role that you've just created.

So select it and then click on delete role.

Once you've deleted that role, go back to the EC2 console, click on instances, right click on public EC2, go to security, modify IAM role.

Now, even though you've deleted the IAM role, note how it's still listed.

That's because this is an instance profile.

This is showing the instance profile that gets created with the role, not the role itself.

So what we're going to do, and I just wanted to do this to demonstrate how this works, we're just going to select no IAM role and then click on save.

We'll need to confirm that.

So to do that, we need to type detach into this box and then confirm it by clicking detach.

That removes the instance role entirely from the instance.

And then we can finish up the tidy process by moving back to the cloud formation console.

Selecting the IAM role demo stack and then clicking on delete and confirming that deletion.

And that will put the account back in the same state as it was at the start of this demo lesson.

So this has been a very brief demo.

I just wanted to give you a little bit of experience of working with instance roles.

So that's EC2 instances combined with IAM roles in order to give an instance and any applications running on that instance, the ability to interact with AWS products and services.

And this is something that you're going to be using fairly often throughout the course, specifically when you're configuring any AWS services to interact with any other services on your behalf.

That's a common use case for using IAM roles and we'll be using instance roles extensively to allow our EC2 instances to interact with other AWS products and services.

But at this point, that is everything that I wanted to cover in this demo lesson.

So go ahead, complete the video and when you're ready, I'll look forward to you joining me in the next.

Welcome back and I've mentioned a few times now within the course that I am roles are the best practice way that AWS services can be granted permissions to other AWS services on your behalf.

Allowing a service to assume a role grants the service the permissions that that role has.

EC2 instance roles are roles that an instance can assume and anything running in that instance has the permissions that that role grants and there is some detail involved which matters so let's take a look at how this feature of EC2 works architecturally.

Instance role architecture isn't really all that complicated it starts off with an I am role and that role has a permissions policy attached to it so whoever assumes the role gets temporary credentials generated and those temporary credentials give the permissions that that permissions policy would grant.

Now an EC2 instance role allows the EC2 service to assume that role which means there's an EC2 instance itself can assume it and gain access to those credentials but we need some way of delivering those credentials into the EC2 instance so that applications running inside that instance can use the permissions that the role provides so there's an intermediate piece of architecture the instance profile and this is a wrapper around an I am role and the instance profile is the thing that allows the permissions to get inside the instance when you create an instance role in the console an instance profile is created with the same name but if you use the command line or cloud formation you need to create these two things separately when using the UI and you think you're attaching an instance role direct to an instance you're not you're attaching an instance profile of the same name it's the instance profile that's attached to an EC2 instance.

We know by now that when I am roles are assumed you're provided with temporary security credentials which expire but these credentials grant permissions based on the roles permissions policy will inside an EC2 instance these credentials are delivered via the instance metadata.

An application running inside the instance can access these credentials and use them to access AWS resources such as S3.

One of the great things about this architecture is that the credentials available inside the metadata they're always valid EC2 and the secure token service liaise with each other to ensure that the credentials are always renewed before they expire as long as your application inside the EC2 instance keeps checking the metadata it will never be in a position where it has expired credentials.

So to summarize when you use EC2 instance roles the credentials are delivered via the instance metadata specifically inside the metadata there's an IAM tree in there there's a security credentials part and then in there is the role name and if you access this you'll get access to these temporary security credentials and they're always rotated they're always valid as long as that instance role remains attached to the instance anything running in the instance will always have access to these valid credentials applications running in the instance of course need to be careful about caching these credentials and just check the metadata before the credentials expire or do it periodically.

You should always use roles where possible I'm going to keep stressing that throughout the course it's important for the exam roles are always preferable than storing long-term credentials so access keys into an EC2 instance it's never a good idea to store long-term credentials such as access keys anywhere which aren't securely stored so for example on your local machine.

In fact the AWS tooling such as the CLI tools will use instance role credentials automatically so as long as the instance role is attached to the EC2 instance any command line tools running inside that instance can automatically make use of those credentials.

So at this point that's everything I wanted to cover thanks for watching go ahead and complete this video and when you're ready join me in the next lesson.

Welcome back and in this brief demonstration you'll have the opportunity to create an EC2 instance with WordPress bootstrapped in ready and waiting to be configured.

But this time you'll be using an enhanced CloudFormation template which uses CFN init and creation policies rather than the simple user data that you used in the previous demonstration.

To get started just make sure you are logged in to the general AWS account as the I am admin user and as always make sure you've got the northern Virginia region selected.

Now attached to this lesson are two one click deployment links.

Go ahead and use the first one which is the VPC link.

Everything should be pre-populated.

All you'll need to do is scroll down to the bottom, check the acknowledgement box and click on create stack.

Once it's moved into a create complete status you can resume and we'll carry on with the demo.

I'll assume that that's now in a create complete status and now we're going to apply another CloudFormation template.

This is the template that we'll be using.

It's just an enhancement of the one that you used in the previous lesson.

This time instead of using a set of procedural instructions, so a script that are passed into the user data, this uses the CFN init system and creation policies.

So let's have a look at exactly what that means.

If I scroll down and locate the EC2 instance logical resource, then here we've got this creation policy.

This means that CloudFormation is going to create a hold point.

It's not going to allow this resource to move into a create complete status until it receives a signal.

And it's going to wait 15 minutes for this signal.

So a timeout of 15 minutes.

Now scrolling down and looking at the user data, the only things we do in a procedural way, we use the CFN init command to begin the desired state configuration.

That will either succeed or not.

And based on that we use the CFN signal command to pass that success or failure state back to the CloudFormation stack.

And that's what uses this creation policy.

So the creation policy will wait for a signal and it's this command which provides that signal, either a success signal or a failure signal.

Now what we're interested in specifically for this demo lesson is this CFN init command.

So this is the thing that pulls the desired state configuration from the metadata of this logical resource.

I'll talk all about that in a second.

But it pulls that down by being given the stack ID and it uses this substitution command.

So instead of this being passed into the instance, what's actually passed instead of this variable name, so the stack ID variable name, is the actual stack ID.

And then likewise, instead of this variable name, aws colon region is passed to the actual region that this template is being applied into.

So that's what the substitution function does.

It replaces any variable or parameter names with the values of those variables or parameters.

So the CFN init process is then able to consult the CloudFormation stack and retrieve the configuration information.

That's all stored in the metadata section of this logical resource.

Now I just want to draw your attention to this double hyphen config sets wordpress underscore install.

This tells us what set of instructions we want CFN init to run.

So if I just expand the metadata section here, we've got one or more config sets defined.

In this case, we've only got the one which is wordpress underscore install.

And this config set runs five individual items, one after the other.

And these are called config keys.

So install CFN, software install, configure instance, install wordpress and configure wordpress.

Now these reference the config keys defined below.

So you'll see that the same name install CFN, software install, configure instance, install wordpress and configure wordpress.

You'll recognize a lot of the commands used because they're the same commands that install and configure wordpress.

So in the software install config key, we're using the DNF package manager to install various software packages that we need for this installation, such as WGet, MariaDB, the Apache web server and various other utilities.

Then another part is services and we're specifying that we want these services to be enabled and to be running.

So this means that the service will be set to start up on instance boot and it will make sure that it's running right now.

The next config key is configure instance.

The files component of this can create files with a certain content.

So we're creating a file called etc update-motd.d/40-cow.

This is the part that we had to do manually before and this is the thing that adds the cow say banner.

Then we're running some more procedural commands to set the database root password and to update this banner.

Then we've got install wordpress, which uses a sources option to expand whatever is specified here into this directory.

So this automatically handles the download and the unjzip and untarring of this archive into this folder and it can even do that with authentication if needed.

We're creating another file this time to perform the configuration of wordpress and another file this time to create the database for wordpress.

Then finally we've got the configure wordpress which fixes up the permissions and creates these databases.

So this is doing the same thing as the procedural example in the previous demo.

Instead of running all of these commands one by one, this is just using desired state.

Now there is one more thing that I wanted to point out right at the top.

This is the part that configures CFN init to keep watching the logical resource configuration inside the cloud formation stack.

And if it notices that the metadata for EC2 instance inside the stack changes, then it will run CFN init again.

Remember how in the theory lesson I mentioned that this process could cope with stack updates.

So it doesn't only run once like user data does.

Well, this is how it does that.

This configures this automatic update that keeps an eye on the cloud formation stack and reruns CFN init whenever any changes occur.

This is well beyond what you need for the associate exam.

I just want you to be aware of what this is and how it works.

Essentially we're setting up a process called CFN hop and making it watch the cloud formation stack for any configuration changes.

And then we're setting it up so that the CFN hop process is enabled and running so that it can watch the resource configuration constantly.

So that's it for this template.

What we'll do now is apply it.

So go ahead and click on the second one click deployment link attached to this lesson.

It should be called A4LEC2CFN init.

So click that link.

All you'll need to do is scroll down to the bottom and then click on create stack.

Now this time remember we're using a creation policy.

So cloud formation is not going to move this logical ID and to create complete when EC2 signals that the launch process is completed.

Instead it's going to wait until the instance itself signals the successful completion of the CFN init process.

So because we're using this creation policy it's going to hold until the instance operating system using CFN-signal provide a signal to cloud formation to say yep everything's okay and at that point the logical resource will move into create complete.

So that's going to take a couple of minutes.

The EC2 instance will need to actually launch itself and pass its status checks and then the CFN init process will run, perform all of the configuration required and then assuming the status code of that is okay then CFN-signal will take that status code and respond to the cloud formation stack with a successful completion and then the process will move on then cloud formation will mark the particular resources complete and the stack is complete.

Now that will take a few minutes so just keep hitting refresh and you should see the status update after two to three minutes but go ahead and pause the video and resume it once your stack moves into the create complete status.

And there we go at this point the stack has moved into the create complete status and I just want to draw your attention to this line.

You won't have seen this before.

This is the line where our EC2 instance has run the CFN init process successfully and then the CFN signal command has taken that success signal and delivered it to the cloud formation stack.

So this is the signal that cloud formation was waiting for before moving this resource into a create complete status and that's what's needed before the stack itself could move into a create complete status.

So now we explicitly know that the configuration of this instance has actually been completed.

So we're not relying on EC2 telling us that the instance status is now running with two out of two checks.

Instead the operating system itself the CFN init process that's completed successfully and the CFN signal process has explicitly indicated to cloud formation that that whole process has been completed.

So if we move across to the EC2 console we should be able to connect to the instance exactly as we've done before.

Look for the running instance and select it.

Copy the public IP version 4 IP address and open that in a new tab.

All being well you should see the familiar WordPress installation screen.

If you're right click on that instance and put connect.

Go to instance connect and hit connect that will connect you into the instance and you should be greeted by the cow themed login banner.

This time if we use curl to show us the contents of user data this time it's only a small number of lines because the only thing that runs is the CFN init process and the CFN signal process.

Notice though how all of these variable names have been replaced with their values so the stack IDs and the region.

So this is how it knows to communicate with the right stack in the right region inside cloud formation.

If we do a CD space forward slash var forward slash log and then do a listing we've still got these original two files so cloud hyphen init dot log and cloud hyphen init hyphen output dot log.

So these are primarily associated with the user data output.

But now we've also got these new log files so CFN hyphen init hyphen CMD dot log and that is an output of the CFN init process.

So if we cat that so shudu space cat space and then the name of that log file this will show us an output of the CFN init process itself.

So we can see each of the individual config keys running and what individual operations are being performed inside each of those keys.

So it's a more complex but a more powerful process.

And at this point that's everything I wanted to cover.

It was just to give you practical exposure to an alternative to raw user data and that was CFN hyphen init.

It's a much more powerful system especially when combined with cloud formation creation policies which allow us to pause the progress of a cloud formation stack waiting for the resource itself to explicitly say yes I finished off all of my bootstrapping process you're good to carry on and that's done using the CFN hyphen signal command.

Now at this point let's just clean up the account move back to cloud formation.

Once you there go ahead and delete the EC2 CFN init stack wait for that process to complete and once you've done that go ahead and delete the A4L VPC stack and that will return the AWS account into the state that you had it at the start of this demo.

At that point thanks for doing this demo I hope it was useful.

You can go ahead and complete this video now and when you're ready you can join me in the next.

Welcome back and in this demo lesson you're going to get the experience of bootstrapping an EC2 instance using user data.

So this is the ability to run a script during the provisioning process for an EC2 instance and automatically add a certain configuration to that instance during the build process.

So this is an alternative to creating a custom AMI.

Earlier in the course you created an Amazon machine image with the WordPress installation and configuration baked in.

Now that's really quick and simple but it does limit your ability to make changes to that configuration.

So the configuration is baked into the AMI and so you're limited as to what you can change during launch time.

With boot strapping you have the ability to perform all the steps in the form of a script during the provisioning process and so it can be a lot more flexible.

Now to get started we need to create the Animals for Life VPC within our general AWS account.

So this is the management account of the organization.

So make sure that you're logged into the IAM admin user of this account and as always make sure you have the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment link so go ahead and open that.

This is going to take you to the quick create stack page and everything should be pre-populated.

The stack name should be bootstrap everything else has appropriate default so just scroll down to the bottom, check the capabilities acknowledgement box and then go ahead and click on create stack.

Now this will create the Animals for Life VPC which contains the public subnets that we'll be launching our instance into and so we're going to need this to be in a create complete state before we move on.

So go ahead and pause the video and once your stack changes from create in progress to create complete then we good to continue.

Okay so now that that stack has moved into a create complete state we good to continue.

Now also attached to this lesson is another link which is the user data that we're going to use for this demo lesson so go ahead and open that link.

This is the user data that we're going to use to bootstrap the EC2 instance so what I want you to do is to download this file to your local machine and then open it in a code editor or alternatively just copy all the text on screen now and paste that into a code editor.

So I've gone ahead and opened that file in my text editor and if you look through all of the different commands contained within this user data .txt file then you should recognize some of them.

These are basically the commands that we ran earlier in the course when we manually installed word press and when we created the Amazon machine image.

So we're essentially installing the MariaDB database server, the Apache web server, Wget and Cowsay.

We're installing PHP and its associated libraries.

We're making sure that both the database and the web server are set to automatically start when the instance reboots and are explicitly started when this script is run.

We're setting the root password of the MariaDB database server.

We're downloading the latest copy of the WordPress installation archive.

We're extracting it and we're moving the files into the correct locations.

Then we're configuring WordPress by copying the sample configuration file into the final and proper file name so wp-config.php and then we're performing a search and replace on those placeholders and replacing them with our actual chosen values for the database name, the database user and the database password.

And then after that we're fixing up the permissions on the web root folder with the WordPress installation files inside so we're making sure that the ownership is correct and then we're fixing up the permissions with a slightly improved version of what we've used previously.

Then we're creating our DB.setup script in the same way that we did when we were manually installing WordPress.

We're logging into the database using the MySQL command line utility, authenticating as the root user with the root password and then running this script and this creates the WordPress database, the user sets the password and gives that user permissions on the database.

And then finally we're configuring the Cowsay utility so we're setting up the message of the day file we're outputting our animals for life custom greeting and then we're forcing a refresh of the login banner.

So these are all of the steps that you've previously done manually so I hope it's still fresh in your memory just how annoying that manual installation was.

Okay so at this point this user data is ready to go and I want to demonstrate to you how you can use this to bootstrap an EC2 instance.

So let's go ahead and move back to the AWS console.

Once we're at the AWS console this CloudFormation 1 click deployment has created the Animals for Life VPC.

So what we're going to do is to click on the services drop down and then move to the EC2 console and go ahead and click on launch instance followed by launch instance again.

So first things first the instance is going to be called a4l for animals for life - manual WordPress so go ahead and enter that in the box at the top then scroll down select Amazon Linux and then make sure Amazon Linux 2023 is selected in the drop down and then make sure that you've got 64-bit x86 selected.

I want you to pick whichever type is free tier eligible within your account and region in my case it's t2.micro but you should pick the one that's free tier eligible.

Under key pair go ahead and pick proceed without a key pair then scroll down to network settings and click on edit and there are a few items on this page that we need to explicitly configure.

The first is we need to select the Animals for Life VPC next to network so select a4l -vpc1 next to subnet I want you to go ahead and pick sn -web -a so that's the web or public subnet within availability zone a then make sure auto assign public IP is set to enable we'll be using an existing security group so check that box and then in the drop down so click the drop down and select the bootstrap -instance security group so bootstrap was the name of the cloud formation stack that we created using the one-click deployment we won't be making any changes to the storage configuration and next we need to scroll down to an option that we've not used before we're going to enter some user data so scroll all the way down and under advanced details expand this if it isn't already and you're looking for the user data box what we're going to do is paste in the user data that you just downloaded so in my case this is the user data.txt which I downloaded so I'm going to go ahead and select all of the information in this user data.txt making sure I get everything including the last line and I'm going to copy that into my clipboard now back at the AWS console we need to paste that in to the user data box now by default EC2 accepts user data as base64 encoded data so we need to provide it with base64 encoded data and we're not we're just giving it a normal text file so in this case the user interface can actually do this conversion for us so if what you're pasting in is not base64 encoded and what we're pasting in isn't then we don't need to do anything else if we're pasting in data which is already base64 encoded we need to check this box below the user data box we don't need to worry about that because we're not pasting in anything with base64 encoding so we can just paste in our user data directly into this box and this will be run during the instance launch process so this is where our automatic configuration comes from this is what will bootstrap the EC2 instance okay so that's everything we need to configure so go ahead and click on launch instance now at this point while this is launching I want you to keep in mind that in the previous demo examples in this course we manually launched an instance and then once the instance was in a running state we had to connect into it download WordPress install WordPress and then configure WordPress along with all of the other associated dependencies that WordPress requires so that was a fairly time-intensive process that was open to errors in the AMI example we followed that same process but at the end we created the Amazon machine image so keep that in mind and compare it to what your experience is in this demo lesson so now we've launched the instance and it's now in a running state and we've provided some user data to this instance so I want you to leave it a couple of minutes after it's showing in a running state just give it a brief while to perform that additional configuration after a few minutes go ahead and right click on that instance and select connect we're going to be using EC2 instance connect so make sure that's selected make sure the user is set to EC2 - user and then just click connect now what you should see if we've given this enough time is our custom animals for life login banner and that means that the bootstrapping process has completed think about this for a minute as part of the launch process EC2 has provisioned us an EC2 instance and it's also run a relatively complex installation and configuration script that we've supplied in the form of user data and that's downloaded and installed WordPress and configured our custom login banner if we go back to EC2 select instances and then if we copy the public IP address into our clipboard so copy the actual IP address do not click on this link because this will open it using HTTPS which we haven't configured if you take that IP address and open that in a new tab you'll see the installation dialogue for WordPress and that's because the bootstrapping process using the user data has done all the configuration process that previously we've had to do manually now if we go back to the instance I want to demonstrate architecturally and operationally exactly how this works what we can do is use the curl utility to review the instance metadata now because we're using Amazon Linux 2023 we need to do this slightly differently we need to use version 2 of the metadata service so first we need to run this command to get a token which we can use to authenticate to the metadata service so run this next we can run this command which gets us the metadata of the instance and this uses the 169254 169254 address or as I like to call it 169.254 repeating now if we use this with meta hyphen data on the end then we get the metadata service but as we know user data is a component of the metadata service so instead of using forward slash latest forward slash metadata we can replace metadata with user data and this will allow us to see the user data supplied to the instance and don't worry all of these commands will be attached to the lesson so you should recognize this this is the user data that we passed into the instance so this is performed a download a configuration and an installation of Apache the database server and WordPress as well as our custom login banner so that's how the user data gets into the EC2 instance and there's a service running on the EC2 instance which takes this data and automatically performs these configuration steps essentially this is run as a script on the operating system now something else we can do is to move into the forward slash VAR forward slash log folder and this is a folder which contains many of the system logs and if we do an LS space hyphen LA we'll see a collection of logs within this folder there are two logs in particular that are really useful for diagnosing bootstrapping related problems these logs are cloud hyphen init dot log and cloud hyphen init hyphen output dot log and both of these are used for slightly different reasons so what I want to do is to output one of these logs and show you the content so we're going to output using shudu first to get admin permissions and then cat and we're going to use the cloud hyphen init hyphen output dot log and I'm going to press enter and that's going to show you the contents of this file and you'll be able to see using this log file exactly what's been executed on this EC2 instance so you'll be able to see all of the actual commands and the output from those commands as they've been executed on this EC2 instance so you'll be able to see all of the WordPress related downloads and copies the replacements of the database usernames and passwords the permissions fix section the database creation user creation and then permissions on that database as well as the command that actually executes those and then right at the bottom is where we configure our custom login banner so this is how you can see exactly what's been run on this EC2 instance and if you ever encounter any issues with any of the demo lessons within this course or any of my courses then you can use this file to determine exactly what's happened on the EC2 instance as part of the bootstrapping process okay so this is the end of part one of this lesson it was getting a little bit on the long side and so I wanted to add a break it's an opportunity just to take a rest or grab a coffee part two will be continuing immediately from the end of part one so go ahead complete the video and when you're ready join me in part two.

Welcome back and in this demo lesson you're going to be creating an ECS cluster with the Fargate cluster mode and using the container of CATS container that we created together earlier in this section of the course, you're going to deploy this container into your Fargate cluster.

So you're going to get some practical experience of how to deploy a real container into a Fargate cluster.

Now you won't need any cloud formation templates applied to perform this demo because we're going to use the default VPC.

All that you'll need is to be logged in as the IAM admin user inside the management account of the organization and just make sure that you're in the Northern Virginia region.

Once you've confirmed that then just click in Find Services and type ECS and then click to move to the ECS console.

Once you're at the ECS console, step one is to create a Fargate cluster.

So that's the cluster that our container is going to run inside.

So click on clusters, then create cluster.

You'll need to give the cluster a name.

You can put anything you want here, but I recommend using the same as me and I'll be putting all the CATS.

Now Fargate mode requires a VPC.

I'm going to be suggesting that we use the default VPC because that's already configured, remember, to give public IP addresses to anything deployed into the public subnets.

So just to keep it simple and avoid any extra configuration, we'll use the default VPC.

Now it should automatically select all of the subnets within the default VPC, in my case all six.

If yours doesn't, just make sure you select all of the available subnets from this dropdown, but it should do this by default.

Then scroll down and just note how AWS Fargate is already selected and that's the default.

If you wanted to, you could check to use Amazon EC2 instances or external instances using ECS anywhere, but for this demo, we won't be doing that.

Instead, we'll leave everything else as default, scroll down to the bottom and click create.

If this is the first time you're doing this in an AWS account, it's possible that you'll get the error that's shown on screen now.

If you do get this error, then what I would suggest is to wait a few minutes, then go back to the main ECS console, go to cluster again and then create the all the cats cluster again.

So follow exactly the same steps, call the cluster all the cats, make sure that the animals for live default VPC is selected and all those subnets are present, and then click on create.

You should find that the second time that you run this creation process, it works okay.

Now this generally happens because there's an approval process that needs to happen behind the scenes.

So if this is the first time that you're using ECS within this AWS account, then you might get this error.

It's nothing to worry about, just rerun the process and it should create fine the second time.

Once you've followed that process through again, or if it works the first time, then just go ahead and click on the all the cats cluster.

So this is the Fargate based cluster.

It's in an active state, so we're good to deploy things into this cluster.

And we can see that we've got no active services.

If I click on tasks, we can see we've got no active tasks.

There's a tab here, metrics where you can see cloud watch metrics about this cluster.

And again, because this is newly created and it doesn't have any activity, all of this is going to be blank.

For now, that's fine.

What we need to do for this demonstration is create a task definition that will deploy our container, our container of cats container into this Fargate cluster.

To do that, click on task definitions and create a new task definition.

You'll need to pick a name for your task definition.

Go ahead and put container of cats.

And then inside this task definition, the first thing to do set the details of the container for this task.

So under container details under name, go ahead and put container of cats web.

So this is going to be the web container for the container of cats task.

Then next to the name under image URI, you need to point this at the docker image that's going to be used for this container.

So I'm going to go ahead and paste in the URI for my docker image.

So this is the docker image that I created earlier in the course within the EC2 docker demo.

You might have also created your own container image.

You can feel free to use my container image or you can use yours.

If you want to keep things simple, you should go ahead and use mine.

Yours should be the same anyway.

Now just to be careful, this isn't a URL.

This is a URI to point at my docker image.

So it consists of three parts.

First we have docker.io, which is the docker hub.

Then we have my username, so acantral.

And then we have the repository name, which is container of cats.

So if you want to use your own docker image, you need to change both the username and the repository name.

Again, to keep things simple, feel free to use my docker image.

Then scrolling down, we need to make sure that the port mappings are correct.

It should show what's on screen now, so container port 80, TCP.

And then the port name should be the same or similar to what's on screen now.

Don't worry if it's slightly different and the application protocol should be HTTP.

This is controlling the port mapping from the container through to the Fargate IP address.

And I'll talk more about this IP address later on in this demo.

Everything else looks good, so scroll down to the bottom and click on next.

We need to specify some environment details.

So under operating system/architecture, it needs to be linux/x86_64.

Under task size for memory, go ahead and select 1GB and then under CPU, 0.5 vCPU.

That should be enough resources for this simple docker application.

Scroll down and under monitoring and logging, uncheck use log collection.

We won't be needing it for this demo lesson.

That's everything we need to do.

Go ahead and click on next.

This is just an overview of everything that we've configured, so you can scroll down to the bottom and click on create.

And at this point, the task definition has been created successfully.

And this is where you can see all of the details of the task definition.

If you want to see the raw JSON for the task definition itself, you don't need this for the exam, but this is actually what a task definition looks like.

So it contains all of this different information.

What it has got is one or more container definitions.

So this is just JSON.

This is a list of container definitions.

We've only got the one.

And if you're looking at this, you can see where we set the port mapping.

So we're mapping port 80.

You can see where it's got the image URL, which is where it pulls the docker image from.

This is exactly what a normal task and container definition look like.

They can be significantly more complex, but this format is consistent across all task definitions.

Okay, so now it's time to launch a task.

It's time to take the container and task definitions that we've defined and actually run up a container inside ECS using those definitions.

So to do that, click on clusters and then select the all the cats cluster.

Click on tasks and then click on run a new task.

Now, first we need to pick the compute options and we're going to select launch type.

So check that box.

If appropriate for the certification that you're studying for, I'll be talking about the differences between these two in a different lesson.

Once you've clicked on launch type, make sure Fargate is selected in the launch type drop down and latest is selected under platform version.

Then scroll down and we're going to be creating a task.

So make sure that task is selected.

Scroll down again and under family, make sure container of cats is selected.

And then under revision, select latest.

We want to make sure the latest version is used and we'll leave desired tasks at one and task group blank.

Scroll down and expand networking.

Make sure the default VPC is selected and then make sure again that all of the subnets inside the default VPC are present under subnets.

The default is that all of them should be in my case six.

Now the way that this task is going to work is that when the task is run within Fargate, an elastic network interface is going to be created within the default VPC.

And that elastic network interface is going to have a security group.

So we need to make sure that the security group is appropriate and allows us to access our containerized application.

So check the box to say create a new security group and then for security group name and description, use container of cats -sg.

We need to make sure that the rule on this security group is appropriate.

So under type select HTTP and then under source change this to anywhere.

And this will mean that anyone can access this containerized application.

Finally make sure that public IP is turned on.

This is really important because this is how we'll access our containerized application.

Everything else looks good.

We can scroll down to the bottom and click on create.

Now give that a couple of seconds.

It should initially show last status.

So the last status should be set to provisioning and the desired state should be set to running.

So we need to wait for this task provisioning to complete.

So just keep hitting refresh.

You'll see it first change into pending.

Now at this point we need this task to be in a running state before we can continue.

So go ahead and pause the video and wait for both of these states.

So last status and desired status both of those need to be running before we continue.

So pause the video, wait for both of those to change and then once they have you can resume and will continue.

After another refresh the last status should now be running and in green and the desired state should also be running.

So at that point we're good to go.

We can click on the task link below.

We can scroll down and our task has been allocated a private IP version for address in the default VPC and also a public IP version for address also in the default VPC.

So if we copy this public IP into our clipboard and then open a new tab and browse to this IP we'll see our very corporate professional web application.

If it fits, I sits in a container in a container.

So we've taken a Docker image that we created earlier in this section of the course.

We've created a Fargate cluster, created a task definition with a container definition inside and deployed our container image as a container to this Fargate cluster.

So it's a very simple example, but again this scales.

So you could deploy Docker containers which are a lot more complex in what functionality they offer.

In this case it's just an Apache web server loading up a web page but we could deploy any type of web application using the same steps that you've performed in this demo lesson.

So congratulations, you've learned all of the theory that you'll need for the exam and you've taken the steps to implement this theory in practice by deploying a Docker image as a container on an ECS Fargate cluster.

So great job.

At this point all that remains is to tidy up.

So go back to the AWS console.

Just stop this container.

Click on stop.

Click on task definitions and then go into this task definition.

Select this.

Click on actions, deregister and then click on deregister.

Click back on task definitions and make sure there's no results there.

That's good.

Click on clusters.

Click on all the cats.

Delete the cluster.

You'll need to type delete space all the cats and then click on delete to confirm.

And at that point the Fargate cluster has been deleted.

The running container has been stopped.

The task definitions been deleted and our account is back in the same state as when we started.

So at this point you've completed the demo.

You've done great and you've implemented some pretty complex theory.

So you should already have a head start on any exam questions which involve ECS.

We're going to be using ECS a lot more as we move through the course and we're going to be using it in some of the Animals for Life demos as we implement progressively more complex architectures later on in the course.

For now I just wanted to give you the basics but you've done really well if you've implemented this successfully without any issues.

So at this point go ahead, complete this video and when you're ready join me in the next.

Welcome back and in this demo lesson you're going to learn how to install the Docker engine inside an EC2 instance and then use that to create a Docker image.

Now this Docker image is going to be running a simple application and we'll be using this Docker image later in this section of the course to demonstrate the Elastic Container service.

So this is going to be a really useful demo where you're going to gain the experience of how to create a Docker image.

Now there are a few things that you need to do before we get started.

First as always make sure that you're logged in to the I am admin user of the general AWS account and you'll also need the Northern Virginia region selected.

Now attached to this lesson is a one-click deployment link so go ahead and click that now.

This is going to deploy an EC2 instance with some files pre downloaded that you'll use during the demo lesson.

Now everything's pre-configured you just need to check this box at the bottom and click on create stack.

Now that's going to take a few minutes to create and we need this to be in a create complete state.

So go ahead and pause the video wait for your stack to move into create complete and then we're good to continue.

So now this stack is in a create complete state and we're good to continue.

Now if you're following along with this demo within your own environment there's another link attached to this lesson called the lesson commands document and that will include all of the commands that you'll need to type as you move through the demo.

Now I'm a fan of typing all commands in manually because I personally think that it helps you learn but if you are the type of person who has a habit of making mistakes when typing along commands out then you can copy and paste from this document to avoid any typos.

Now one final thing before we finish at the end of this demo lesson you'll have the opportunity to upload the Docker image that you create to Docker Hub.

If you're going to do that then you should pre sign up for a Docker Hub account if you don't already have one and the link for this is included attached to this lesson.

If you already have a Docker Hub account then you're good to continue.

Now at this point what we need to do is to click on the resources tab of this stack and locate the public EC2 resource.

Now this is a normal EC2 instance that's been provisioned on your behalf and it has some files which have been pre downloaded to it.

So just go ahead and click on the physical ID next to public EC2 and that will move you to the EC2 console.

Now this machine is set up and ready to connect to and I've configured it so that we can connect to it using Session Manager and this avoids the need to use SSH keys.

So to do that just right-click and then select connect.

You need to pick Session Manager from the tabs across the top here and then just click on connect.

Now that will take a few minutes but once connected you should see this prompt.

So it should say SH- and then a version number and then dollar.

Now the first thing that we need to do as part of this demo lesson is to install the Docker engine.

The Docker engine is the thing that allows Docker containers to run on this EC2 instance.

So we need to install the Docker engine package and we'll do that using this command.

So we're using shudu to get admin permissions then the package manager DNF then install then Docker.

So go ahead and run that and that will begin the installation of Docker.

It might take a few moments to complete it might have to download some prerequisites and you might have to answer that you're okay with the install.

So press Y for yes and then press enter.

Now we need to wait a few moments for this install process to complete and once it has completed then we need to start the Docker service and we do that using this command.

So shudu again to get admin permissions and then service and then the Docker service and then start.

So type that and press enter and that starts the Docker service.

Now I'm going to type clear and then press enter to make this easier to see and now we need to test that we can interact with the Docker engine.

So the most simple way to do that is to type Docker space and then PS and press enter.

Now you're going to get an error.

This error is because not every user of this EC2 instance has the permissions to interact with the Docker engine.

We need to grant permissions for this user or any other users of this EC2 instance to be able to interact with the Docker engine and we're going to do that by adding these users to a group and we do that using this command.

So shudu for admin permissions and then user mod -a -g for group and then the Docker group and then EC2 -user.

Now that will allow a local user of this system, specifically EC2 -user, to be able to interact with the Docker engine.

Okay so I've cleared the screen to make it slightly easier to see now that we've added EC2 -user the ability to interact with Docker.

So the next thing is we need to log out and log back in of this instance.

So I'm going to go ahead and type exit just to disconnect from session manager and then click on close and then I'm going to reconnect to this instance and you need to do the same.

So connect back in to this EC2 instance.

Now once you're connected back into this EC2 instance we need to run another command which moves us into EC2 user so it basically logs us in as EC2 -user.

So that's this command and the result of this would be the same as if you directly logged in to EC2 -user.

Now the reason we're doing it this way is because we're using session manager so that we don't need a local SSH client or to worry about SSH keys.

We can directly log in via the console UI we just then need to switch to EC2 -user.

So run this command and press enter and we're now logged into the instance using EC2 -user and to test everything's okay we need to use a command with the Docker engine and that command is Docker space ps and if everything's okay you shouldn't see any output beyond this list of headers.

What we've essentially done is told the Docker engine to give us a list of any running containers and even though we don't have any it's not erred it's simply displayed this empty list and that means everything's okay.

So good job.

Now what I've done to speed things up if you just run an LS and press enter the instance has been configured to download the sample application that we're going to be using and that's what the file container.zip is within this folder.

I've configured the instance to automatically extract that zip file which has created the folder container.

So at this point I want you to go ahead and type cd space container and press enter and that's going to move you inside this container folder.

Then I want you to clear the screen by typing clear and press enter and then type ls space -l and press enter.

Now this is the web application which I've configured to be automatically downloaded to the EC2 instance.

It's a simple web page we've got index.html which is the index we have a number of images which this index.html contains and then we have a docker file.

Now this docker file is the thing that the docker engine will use to create our docker image.

I want to spend a couple of moments just stepping you through exactly what's within this docker file.

So I'm going to move across to my text editor and this is the docker file that's been automatically downloaded to your EC2 instance.

Each of these lines is a directive to the docker engine to perform a specific task and remember we're using this to create a docker image.

This first line tells the docker engine that we want to use version 8 of the Red Hat Universal base image as the base component for our docker image.

This next line sets the maintainer label it's essentially a brief description of what the image is and who's maintaining it in this case it's just a placeholder of animals for life.

This next line runs a command specifically the yum command to install some software specifically the Apache web server.

This next command copy copies files from the local directory when you use the docker command to create an image so it's copying that index.html file from this local folder that I've just been talking about and it's going to put it inside the docker image in this path so it's going to copy index.html to /var/www/html and this is where an Apache web server expects this index.html to be located.

This next command is going to do the same process for all of the jpegs in this folder so we've got a total of six jpegs and they're going to be copied into this folder inside the docker image.

This line sets the entry point and this essentially determines what is first run when this docker image is used to create a docker container.

In this example it's going to run the Apache web server and finally this expose command can be used for a docker image to tell the docker engine which services should be exposed.

Now this doesn't actually perform any configuration it simply tells the docker engine what port is exposed in this case port 80 which is HTTP.

Now this docker file is going to be used when we run the next command which is to create a docker image.

So essentially this file is the same docker file that's been downloaded to your EC2 instance and that's what we're going to run next.

So this is the next command within the lesson commands document and this command builds a container image.

What we're essentially doing is giving it the location of the docker file.

This dot at the end contains the working directory so it's here where we're going to find the docker file and any associated files that that docker file uses.

So we're going to run this command and this is going to create our docker image.

So let's go ahead and run this command.

It's going to download version 8 of UBI which it will use as a starting point and then it's going to run through every line in the docker file performing each of the directives and each of those directives is going to create another layer within the docker image.

Remember from the theory lesson each line within the docker file generally creates a new file system layer so a new layer of a docker image and that's how docker images are efficient because you can reuse those layers.

Now in this case this has been successful.

We've successfully built a docker image with this ID so it's giving it a unique ID and it's tagged this docker image with this tag colon latest.

So this means that we have a docker image that's now stored on this EC2 instance.

Now I'll go ahead and clear the screen to make it easier to see and let's go ahead and run the next command which is within the lesson commands document and this is going to show us a list of images that are on this EC2 instance but we're going to filter based on the name container of cats and this will show us the docker image which we've just created.

So the next thing that we need to do is to use the docker run command which is going to take the image that we've just created and use it to create a running container and it's that container that we're going to be able to interact with.

So this is the command that we're going to use it's the next one within the lesson commands document.

It's docker run and then it's telling it to map port 80 on the container with port 80 on the EC2 instance and it's telling it to use the container of cats image and if we run that command docker is going to take the docker image that we've got on this EC2 instance run it to create a running container and we should be able to interact with that container.

So if you go back to the AWS console if we click on instances so look for a4l-public EC2 that's in the running state.

I'm just going to go ahead and select this instance so that we can see the information and we need the public IP address of this instance.

Go ahead and click on this icon to copy the public IP address into your clipboard and then open that in a new tab.

Now be sure not to use this link to the right because that's got a tendency to open the HTTPS version.

We just need to use the IP address directly.

So copy that into your clipboard open a new tab and then open that IP address and now we can see the amazing application if it fits i sits in a container in a container and this amazing looking enterprise application is what's contained in the docker image that you just created and it's now running inside a container based off that image.

So that's great everything's working as expected and that's running locally on the EC2 instance.

Now in the demo lesson for the elastic container service that's coming up later in this section of the course you have two options.

You can either use my docker image which is this image that I've just created or you can use your own docker image.

If you're going to use my docker image then you can skip this next step.

You don't need a docker hub account and you don't need to upload your image.

If you want to use your own image then you do need to follow these next few steps and I need to follow them anyway because I need to upload this image to docker hub so that you can potentially use it rather than your own image.

So I'm going to move back to the session manager tab and I'm going to control C to exit out of this running container and I'm going to type clear to clear the screen and make it easier to see.

Now to upload this to docker hub first you need to log in to docker hub using your credentials and you can do that using this command.

So it's docker space login space double hyphen username equals and then your username.

So if you're doing this in your own environment you need to delete this placeholder and type your username.

I'm going to type my username because I'll be uploading this image to my docker hub.

So this is my docker hub username and then press enter and it's going to ask for the corresponding password to this username.

So I'm going to paste in my password if you're logging into your docker hub you should use your password.

Once you've pasted in the password go ahead and press enter and that will log you in to docker hub.

Now you don't have to worry about the security message because whilst your docker hub password is going to be stored on the EC2 instance shortly we're going to terminate this instance which will remove all traces of this password from this machine.

Okay so again we're going to upload our docker image to docker hub so let's run this command again and you'll see because we're just using the docker images command we can see the base image as well as our image.

So we can see red hat UBI 8.

We want the container of cats latest though so what you need to do is copy down the image ID of the container of cats image.

So this is the top line in my case container of cats latest and then the image ID.

So then we need to run this command so docker space tag and then the image ID that you've just copied into your clipboard and then a space and then your docker hub username.

In my case it's actrl with 1L if you're following along you need to use your own username and then forward slash and then the name of the image that you want this to be stored as on docker hub so I'm going to use container of cats.

So that's the command you need to use so docker tag and then your image ID for container of cats and then your username forward slash container of cats and press enter and that's everything we need to do to prepare to upload this image to docker hub.

So the last command that we need to run is the command to actually upload the image to docker hub and that command is docker space push so we're going to push the image to docker hub then we need to specify the docker hub username so again this is my username but if you're doing this in your environment it needs to be your username and then forward slash and then the image name in my case container of cats and then colon latest and once you've got all that go ahead and press enter and that's going to push the docker image that you've just created up to your docker hub account and once it's up there it means that we can deploy from that docker image to other EC2 instances and even ECS and we're going to do that in a later demo in this section of the course.

Now that's everything that you need to do in this demo lesson you've essentially installed and configured the docker engine you've used a docker file to create a docker image from some local assets you've tested that docker image by running a container using that image and then you've uploaded that image to docker hub and as I mentioned before we're going to use that in a future demo lesson in this section of the course.

Now the only thing that remains to do is to clear up the infrastructure that we've used in this demo lesson so go ahead and close down all of these extra tabs and go back to the cloud formation console this is the stack that's been created by the one click deployment link so all you need to do is select this stack it should be called EC2 docker and then click on delete and confirm that deletion and that will return the account into the same state as it was at the start of this demo lesson.

Now that is everything you need to do in this demo lesson I hope it's been useful and I hope you've enjoyed it so go ahead and complete the video and when you're ready I look forward to you joining me in the next.