Last Week In AWS Podcast

LinksFollow Last Week In AWS on TwitterTranscriptCorey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.Pete: Hello, welcome to AWS Morning Brief. I am Pete Cheslock, and I am here yet again with Jesse DeRose.Jesse: Hello. Pete: We here to talk about the best service announced not during AWS Storage Day 2020.Jesse: So, close.Pete: So, close, though. It was announced a few days after, and that is the AWS S3 Storage Lens service, which I think I've got that naming right. I know sometimes it's ‘AWS thing,’ sometimes it's ‘Amazon thing,’ and to be honest, I never know which is which. Jesse: Yeah.Pete: AWS S3 Storage Lens is honestly one of the best new services that I've seen out, released thus far. I guess we're still pre-re:Invent announcements in a lot of this stuff. But what it is is a—from their site it says, “S3 Storage Lens delivers organization-wide visibility into object storage usage, activity trends,” blah, blah, blah, blah, blah, marketing speak. Basically, it allows you to get a view of your S3 usage across accounts. Which, that's mindblowing, right?Jesse: Yeah. This feature has so much potential; I'm really excited to see where they go with it.Pete: Yeah. And so when I first saw this blog post on Amazon’s site talking about it, my mind just started going crazy because again, we work in Duckbill Group as cloud economists with a lot of different clients, and because Amazon organizations may be the reason why, made it very easy to spin up new accounts, maybe also the adage, the design principle of creating many Amazon accounts to kind of segment workloads or to provide you to—segment your workloads in a way for cost reasoning or security reasons. But all of those things—somewhat related, somewhat not—have caused a lot of our clients to have lots of Amazon accounts. I mean, you could see hundreds, in some cases, of Amazon accounts. And the issue that I've always kind of had, and especially an issue we deal with in helping our clients analyze their costs and optimize their costs is how do you aggregate S3 usage? Because S3 is normally in the top five of services that we see in usage, how do you pull that together? And I guess we do that a lot of different ways. Jesse, maybe you can chat a little bit about what are some of the ways that we try to analyze this spend currently?Jesse: Yeah. Pete, I think I'm really excited about this feature because AWS already offers aggregate looks at metrics for other top services by spend. Like, for EC2, you've got Compute Optimizer. We don't have anything for RDS yet, but I feel like that might be not far off, given Compute Optimizer’s existence. And we already have other tools that allow you to look across multiple accounts to look at metrics, especially if you're looking at Cost Explorer, for example, you can see metrics across multiple accounts, you can see spend across multiple accounts. So, I feel like this makes sense. I'm really excited to see that you can look at all of your S3 storage metrics in one place because right now, the only way that we're able to get any kind of representation of S3 usage is through Cost Explorer. And there are ways that you can go about filtering and slicing that data to get usage information and certain metrics, slicing and dicing on different filters for accounts and cost allocation tags, but it's all at the bucket level, or at the usage level, and if you really want to dig in deeper, you don't have a lot of options.Pete: Yeah, it's a service that they're operating on your behalf. So, your only insight is what they give you insight into. Maybe some of that is CloudWatch metrics, there's obviously the S3 storage analytics that can give you some idea in your storage—based on access—that can help you kind of optimize, but nothing really again at the—ability to see it across multiple accounts is I think, really the big game-changer too.Jesse: And I think what's really amazing here is that the majority of metrics that they're offering are free. And we'll get into that in a minute, but I'm really impressed that so many of these metrics are shared free of charge. You just have to turn it on. And then you have access to all of this great information that you can work with. Pete: Yeah. I think that's a great point that we haven't mentioned yet, that this is—the basic form of this is free. And the metrics that you can get are pretty useful in the free tier. Also, this is actually something that is turned on in your account right now. If you have an Amazon account, go into S3, it's actually under S3, it'll be on the left-hand column—at least it should be unless they go move stuff around—but you'll see a drop-down for Storage Lens, and you'll see an option for dashboards. And when you go into the dashboards, there will be a default dashboard already pre-configured with the free metrics enabled for your account. Now, that could be super helpful if, let's say, you just have one account, you can get some real good high-level metrics around your storage based on bucket. You can go into that dashboard and really quickly see total storage across all your buckets. You can see trend analysis with, day-by-day, week-by-week change comparison, how are things growing. There was one thing that I saw that I was really blown away by because this is something we deal with a lot is they have broken the metrics out in kind of a high-level summary, focusing on data protection, like being able to see data percentage replicated or encrypted, but also based on cost efficiency, too, being able to see if you have versioning enabled, obviously, there's a cost for that. How many old versions of this thing do you have, but also incomplete multipart uploads? That is potentially a large and in many ways, super hidden cost for some users of Amazon S3. If you are uploading a multipart file, and it fails, it lives in this purgatory, storage purgatory, where you're charged for it, but you may not see it in an obvious way. Jesse: And we see that with a lot of our clients who have multipart uploads and end up with these incomplete multipart uploads that just take up space. There's no clear metrics right now, prior to Storage Lens, that say, here's all of this stale multi-part upload usage that you're paying for, that's effectively just taking up wasted space. But now we have metrics for that; now we have information that can clearly tell us where they are, how much space they're taking, and you can actually do something about it.Pete: Right. Yeah, it gives you this intelligence that you can act upon. To talk about those metrics, since we're kind of on that stage, when ...

Want to give your ears a break and read this as an article? You’re looking for this link.SponsorsNew RelicLinodeNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of November 30, 2020 with Corey Quinn.

LinksFollow Last Week In AWS on TwitterTranscriptCorey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock, and I am here yet again with Jesse DeRose. Jesse, welcome back. Jesse: Thanks for having me, Pete. Pete: But it's not just the two of us. We have a very special guest: we are also joined with one of the newest hires to The Duckbill Group, Amy Negrette. Amy, hello.Amy: Hello. And one might say the most special of guests; that person would be me.Pete: The most special of guests. Jesse: [laugh].Pete: Well, we are pleased to have you. So, in honor of Thanksgiving—American Thanksgiving, for anyone outside of the United States, or who doesn't celebrate. But this is the American Thanksgiving holiday week. We wanted to take a little different approach to this week's episode. And Amy, you were the one who kind of came up with this idea, and so that's why we forced you to join us because—Jesse: One of us. One of us.Pete: [laugh]. Because you had such a good idea, and we wanted to make sure that we just pulled this together and really did a Thanksgiving theme to this podcast. So, I don't know about either of you, but my family has some very clear requirements about what dishes do and do not constitute Thanksgiving. And you can always expect the turkey and the stuffing. It's just not Thanksgiving without those core components. Jesse: But then your cousin's boyfriend shows up with the candied vegetables that nobody asked to be candied. And, you know, you put a little bit on your plate because you want to be nice. You don't want to start World War III in the middle of Thanksgiving dinner. And you say, “Oh, yeah, this is good.” But then you're definitely giving those food scraps to the dog under the table and you don't go back for seconds.Pete: I mean, a metric ton of sugar is probably the only way to make turnips taste good.Jesse: Yeah.Pete: So, with that in mind, we wanted to talk about what AWS services are those core services that you expect the customers kind of using to leverage the cloud, what services would kind of represent a Thanksgiving meal? Which ones constitute the turkey, or the stuffing, or the green bean casserole which, while preparing this, there seem to be some conflicting thoughts about the quality of a green bean casserole.Jesse: There are some hot takes. Some hot, hot, hot takes in this discussion, putting this list together.Pete: So, I'll kick us off with an easy, softball one because why not? But it's EC2, right? This is the turkey. It's the main course. And it's also what you'll be eating three to five times a day for every day for the next week or two because you're going to have a lot extra. It's just going to be around for a long time. Jesse: Yeah, I feel like EC2 is one that you're going to get in some capacity, anywhere. Whether it is straight-up EC2 instances, whether it is Fargate, ECS, you're going to be using this compute resource in some capacity if you're using AWS. I don't think I know of any AWS customer that is not using some level of compute with EC2. Except for the few people who have managed to move entirely serverless to Lambda, which I am thoroughly impressed if you've been able to do that. Pete: So, that is actually a great one which is Amy you do a lot with the serverless community. What do you think Lambda would be as a Thanksgiving side dish?Amy: It is the canned cranberry sauce because everyone who I hear talk about it they seem to hate it, but I love it. I love not having to work for anything. It tastes the same and the sauce itself tastes like jelly and Lambda packages everything in a way where I don't have to deal with it, and to me that makes everything else super easy.Pete: I think it's the slow oozing out of the can it does that really kind of makes me not want to like it, and those just too perfect ridges from the form of it. But I don't know what it is about it; when you just slice through that and put it on your plate, so delicious. And don't at me with your fancy homemade cranberry sauce, whatever. None of that can hold a candle. So, I actually think Lambda is the special smoked turkey. Because it's a new trend. Lambda being in the new trend, serverless is a new trend. And of course, everyone who is doing a smoked turkey or has a smoker just can't stop talking about it, much like serverless. They just can't stop talking about it.Jesse: Yeah. I mean, I think that ever since you bought your smoker, you have not stopped telling us all about the meats that you're smoking on a recurring basis.Pete: I mean, I got a 16-pound turkey for $14, and I got turkey for days. Jesse: What I love is that not only do you have a smoker and you talk about it, but you have a monitoring system that you set up so that you can monitor the temperature of the smoker at any given time. Pete: I'm a bit of a Luddite at home. I don't like IoT powered anything because I think they're all generally terrible, but for some reason, yeah, my smoker has a little whatever, cellular—powered, connects to my wifi, but I can get to it from the app on my cell phone, can check the temperature of the turkey, out of the store running errands. “Oh, got to get home soon, my turkey’s almost done.”Jesse: Okay, I’ve got another easy one for us. S3 is your mashed potatoes. It's good, it's on everyone's plate, there appears to always be an infinite amount of it. Everybody's going to want some. And most importantly, if you leave a bucket of it open overnight, you're going to regret it. Pete: Yeah, that's going to turn to glue pretty fast, not Amazon Glue, which actually if we are going to talk about Amazon Glue and Lake Formation, and that weird amalgamation of Amazon services, we actually have one for that. This is something called the piecaken, which I had never heard about until I saw an Instagram ad because that's a thing. But a piecaken is a pecan pie—pecan or pecan? Let's not, do that.Jesse: Oh, God, don't start.Pete: Okay. Pumpkin pie, spice cake, and an apple pie filling. It's like three pies stacked into a cake. And that's what I think of when I think about the whole Lake Formation/Glue setup when you're trying to query or analyze your data lake. Jesse: Yeah, my arteries just clogged ...

Want to give your ears a break and read this as an article? You’re looking for this link.SponsorsGravitationalLinodeNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of November 23, 2020 with Corey Quinn.

LinksFollow Last Week In AWS on TwitterTranscriptCorey: Gravitational is now Teleport because when way more people have heard of your product than your company, maybe that’s a sign it’s a time to change your branding. Teleport enables engineers to quickly access any computing resource, anywhere on the planet. You know, like VPNs were supposed to do before we all started working from home, and the VPNs melted like glaciers. Teleport provides a unified access plane for developers and security professionals seeking to simplify secure access to servers, applications, and data across all of your environments without the bottleneck and management overhead of traditional VPNs. This feels to me like it’s a lot like the early days of HashiCorp’s Terraform. My gut tells me this is the sort of thing that’s going to transform how people access their cloud services and environments. To learn more, visit goteleport.com.Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock, and I'm also here, again, with Jesse DeRose. Hey, Jesse, how's it going?Jesse: Not too bad. Thanks for having me.Pete: It is part two of AWS Storage Day. If you haven't had the chance to listen to last week's episode, Jesse and I dove into some of the new features really focusing on what we would think is the biggest feature of AWS Storage Day, which was the S3 Intelligent Tiering. Go back and listen to it if you didn't hear about it. But essentially, Amazon keeps extending out features [00:01:34 unintelligible] this Intelligent Tiering platform. And we talked a little bit about it last week. But there were a lot of announcements as part of Storage Day, some pretty impressive, and some that were maybe a little underwhelming. We'll let you be the judge of that because some of these things could be incredibly important for you as—maybe—someone who operates on Amazon. So, now what we're going to do is we're going to dive into some of the other features, not only additional interesting S3 features, but there were a lot of new features announced around EBS, and EFS, and FSx, and all of the different ways that you can interact with AWS storage. I don't want to call it the biggest feature of this section because I think—let's be honest—they're all equally meh features, right, Jesse?Jesse: Yeah.Pete: I think that's going to be the common thread. Again, you might look at some of these features and go, “Finally, my life is so much better because they've announced this feature.” But I got to say, outside of Intelligent Tiering, Storage Day felt a little weak. But let's dive in anyway. S3 Replication; if you are replicating your data from one S3 bucket to another bucket, another region, which maybe you need to do for compliance reasons, disaster recovery reasons, some of the new features they added are around replication metrics and notifications. Now, previously, these metrics and notifications were only available if you used the Time Control Replication, and that is a additional charge to get a predictable SLA for your data to be backed up. They made these metrics now available for anyone, so that's actually awesome to hear that they’ve really just extended that out and are kind of giving you something for free. Additionally, they now replicate delete markers, which I swear I looked at a bunch of documents to understand better what delete markers mean, and the best I got to it, I don't actually really understand the problem from before, other than as you delete a version of something in the source, the delete marker moves over. But then maybe the previous versions are in the destination. That was my gist of it, Jessie, what was your gist of that one?Jesse: Yeah, I struggled a little bit with some of these previously because S3 replication always felt like this magical hand-wavy feature where you turned it on and then just waited, and eventually your objects would show up in your destination bucket or destination folder. But there wasn't really any clear path to what was going on behind the scenes. So, I'm really excited to see that now these metrics and notifications are available to everyone, not just to folks who were using the Replication Time Control feature, and allows everybody to more easily understand how their data is replicating between S3 buckets behind the scenes. So, I feel good about this one. I feel like this is definitely a step in the right direction. I'm really excited to see that this is now broadly available for everybody that's using S3. I think it will make using S3 Replication easier for a lot of folks who need it for business purposes or any other use case.Pete: Yeah, absolutely. Another really awesome feature—I was actually excited for this because, of course, it must affect me in my day-to-day—S3 object ownership is now available for all the Amazon regions and amazingly supported by CloudFormation, which I feel like is always an afterthought. But what this allows you to do is you can use this feature too, when you upload files, it'll make sure that the ownership is assumed by the bucket you've uploaded it into. And so this gets around a lot of hairy issues that come into S3 permissioning, IAM permissioning. I mean, S3 permissioning, in general, predates IAM. I don't know how many people actually know that. And I think because of it, there are some really gnarly edge cases people run into, and this is a big problem solver.Jesse: I am really, really excited about this feature release, I cannot say how many times we've run into this edge case with some of our internal tooling because we have effectively copied or synced data from a client's S3 bucket into our S3 bucket, and we don't gain ownership. And that becomes such a permissioning headache to be able to do anything with that data once we have it in our S3 bucket. So, I'm really excited to see that object ownership is now not only a first-class citizen but now is also built into and supported by AWS CloudFormation.Pete: Yeah, absolutely. Another new feature: it has to do with Outpost actually, and you can get S3 on Outposts now which, that's truly amazing if you think about it. Now, I don't know of anyone who actually is using Outposts, and I would love to chat with someone who can, if they're even allowed to, or if they're stuck under an NDA. But what an Outpost allows you to do is essentially purchase a rack of AWS; it's a rack of servers and storage with Amazon APIs. If you really just think about that for a second, that's pretty impressive. And if you are going to do hybrid cloud, and you have maybe some data locality requirements like you really need data in a specific location and that's not a region that Amazon supports, or you have data centers, or there's always some requirements, you can now get S3 on there. And they said that they can support 48 or 96 terabytes of S3 capacity per Outpost. What that actually means—like, is that a rack? Is that a whole rack? Is that just a single S3 configuration? Hard to really know. There's no API to go and provision an Outpost yet.Jesse: Yeah, I'm really curious about this one to see how folks end up using it because I'm super excited that this is a feature that's now available. I love the idea of Outposts, even though it may not be a business use case for us internally. But I'm really curious to see how thi...

Want to give your ears a break and read this as an article? You’re looking for this link.SponsorsGravitationalLinodeNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of November 16, 2020 with Brooke Mitchell.

LinksFollow Last Week In AWS on TwitterTranscriptCorey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock. Corey, while being back from his paternity leave, is still not here. We are having too much fun. And by we, I mean I'm joined again with Jesse DeRose. Hey, Jesse. Jesse: Thanks as always for having me, Pete. Pete: It's so much fun to again chat with people outside of my little family unit, that we've just decided not to give this back to Corey. And luckily, Corey has many other podcasts that he does, he was pretty happy to give it away.Jesse: I feel like you should never talk about your children that way, but he's got a plethora at this point. So, he's willing to kind of share the wealth.Pete: Exactly. And if you notice, we have a new theme song that came out, I think it was last week was the first week that we brought in the new theme song, which is I think much in line with a previous episode where we talked about ’80s breakdancing movies that the new theme song kind of has that vibe to it.Jesse: I hope you're wearing the Members Only jean jacket that I sent you, along with the shades to match the uniform.Pete: Yeah. I mean, I was born in ’80, so the ’80s for me, I was very young. I'm kind of waiting for the ’90s movies to come around again because I want to rock out my JNCO jeans and my wallet chain. Jesse: [laugh], yes.Pete: And all that good stuff.Jesse: I am ready.Pete: Exactly. Well, what are we talking about today? Well, earlier this week, AWS Storage Day 2020 happened on Tuesday. If you were a part of that, it was a free online event. As Amazon called it, a full day online event. Except it was only about four hours long, so kind of mailing it in on that one, huh?Jesse: Can we start discussing that with our boss and say that a full day of work is technically just four hours? Can we just start working with that going forward?Pete: Yeah, we'll just say it right now. So, hey, Corey, we're done for the day. Put in the old college four.Jesse: [laugh]. That's what you say, “I put in the old college try. I just did my full day of four hours, according to AWS. So, this has been great. I'll talk to you tomorrow.”Pete: Exactly. Well, Storage Day this year—it's the second year in a row if I'm remembering it correctly. 2019 was the last year they did that—and I feel like this kind of ties into the fact that there's just so many announcements that happened around re:Invent, that leading up into re:Invent, you have a lot of announcements to maybe soften the blow for a lot of folks. And Storage Day, really is just this whole day—well, four hours worth of a whole day—talking about everything related to storage. And we're talking about things like S3, EBS, EFS, FSx, for the five huge enterprises that probably use FSx. Although if you actually do use FSx, I'd be curious to hear about how you like it and what you think of it because we don't really hear a lot of people using it. But these are all the services, plus many more, that Amazon talked about as part of its Storage Day.Jesse: Yeah, it was a really interesting discussion. I greatly appreciate that AWS broke out this discussion prior to AWS re:Invent, but they dropped a lot of knowledge on us all at once, and in, like, rapid-fire succession, I was really, kind of… not necessarily surprised, but there's a lot of information that they shared all at once. And I have to admit that after sitting through this presentation, I now have a greater appreciation for Apple's slow presentation style. As much as I hate it; as much as I hate sitting for an hour and a half for one announcement while they toot their own horn, I have to say that the buildup and getting me involved in the story and bringing me along with them. It works, it absolutely works. And it was kind of hard for me to pick up on all the things that went on during AWS Storage Day this year because there was a lot of things going on.Pete: And honestly, the fact they give so much information is really amazing in, I guess, both their ability to tout, in many cases, minor feature changes that most SaaS businesses would just turn on and maybe blog about. But this is—obviously the engine of AWS is so good at discussing their wins. But you're right, it's just a huge amount. On Monday, Jeff Barr of course, wrote the blog post with a lot of these details, linking to countless other blog posts. And I think it really speaks to just how, probably every, or nearly every Amazon service ties into storage in some way. It's a huge, huge part of this ecosystem. Jesse: Absolutely. Pete: So, as you can imagine, there were so many new features that we're not even going to be able to cover them all throughout the course, but we did want to call out some of the big ones, or at least what we thought were the biggest ones, the most interesting new features, new product announcements that came out, and also just touch on some of the other things that we thought were pretty interesting as well. And yeah, there was a lot of fun stuff. I think the biggest one that was announced was the S3 Intelligent-Tiering, which is a class storage tier within S3, adds additional levels of archive access. So, if you imagine Intelligent-Tiering, you know, you have the automatic tiering of data from frequently accessed to infrequently accessed as things age out, they essentially automate that for you. So, as things are not accessed, you just start automatically paying less for them. And anything automatic in a cost savings world is going to help you save money. If you don't have to think about it and it just does it for you, it's fantastic. Well, Intelligent-Tiering added in these additional tiers—which they are Glacier—level tiers. They are additional places that your data can eventually move to as they start aging out based on a whole series of criteria. But there's caveats. There's more caveats now. Before, one of the interesting things that we actually learned as part of this—because it was buried in a pricing page footnote—is that when you store something into Intelligent-Tiering, there is a minimum storage time period that you will get charge...