The InfoQ Podcast

In this podcast, we are talking to Tyler Akidau, a senior engineer at Google, who leads the technical infrastructure and data processing teams in Seattle, and a founding member of the Apache Beam PMC and a passionate voice in the streaming space. This podcast will cover data streaming and the 2015 DataFlow Model streaming paper [http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf] and much of the concepts covered, such as why dealing with out-of-order data is important, event time versus processing time, windowing approaches, and finally preview the track he is hosting at QConf SF next week. Why listen to this podcast: - Batch processing and streaming aren’t two incompatible things; they are a function of different windowing options. - Event time and processing time are two different concepts, and may be out of step with each other. - Completeness is knowing that you have processed all the events for a particular window. - Windowing choice can be answered from the what, when, where, how questions. - Unbounded versus bounded data is a better dimension than stream or batch processing. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2AyBTAb You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2AyBTAb

In this podcast, Wes talks to Guy Podjarny (Founder/CEO Synk). The two discuss the space between open source software and third-party dependencies, including a discussion of the Equifax hack (and what we can learn from it), the role of serverless architectures today (and what it means to application surface area), and then finally they wrap with security hygiene best practices with OSS and serverless. Why listen to this podcast: - The majority of security vulnerabilities that exist in applications today comes from vulnerable third-party libraries, rather than the application’s own code. - An application shouldn’t permit total leak of all data because of a single vulnerability - defence in depth is important. - Equifax couldn’t have failed more spectacularly in the way they handled it. - The Equifax hack serves as a wake-up call to pay attention to vulnerabilities in dependencies. - If your build system breaks the build when a dependency vulnerability is found automatically, it will be applied sooner. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2ziAIat You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2ziAIat

In this podcast, QCon Chair Wesley Reisz talks to Julien Viet.  Viet is the project lead for Vert.x and a principal engineer at RedHat having taken over as project lead for Vert.x from Tim Fox in January 2016.  They talk about the newly released Vert.x 3.5.0, and the plans for Vert.x 4.0. Why listen to this podcast: * Vert.x adds RxJava2 support for streams and backpressure. * Vert.x is a polyglot set of APIs, custom aligned for the specific language. * It is unopinionated and can be used with any environments, since it doesn’t enforce a particular framework. * Verticles communicate in-VM or through peer-to-peer networking for distributed applications. * Vert.x 4.0 is on the roadmap for the future. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2z0BEQR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2z0BEQR

What can software learn from industries like aerospace, transportation, or even retail during national disasters? This week’s podcast is with Emil Stolarsky and was recorded live after his talk on the subject at Strangeloop 2017. Interesting points from the podcast include several stories from Emil’s research, including the origin of the checklist, how Walmart pushed decision making down to the store level in a national disaster, and where the formalized conversation structure onboard aircraft originated. The podcast mentions several resources you can turn to if you want to learn more and wraps with some of the ways this research is affecting incident response at Shopify. Why listen to this podcast: * Existing industries like aerospace have built a working history of how to resolve issues; it can be applicable to software issues as well. * Crew Resource Management helps teams work together and take ownership of problems that they can solve, instead of a command-and-control mandated structure. * Checklists are automation for the brain. * Delegating authority to resolve system outages removes bottlenecks in processes that would otherwise need managerial sign off. * When designing an alerting system, make sure it doesn’t flood with irrelevant alerts and that there’s clear observability to what is going wrong. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2zmCsfR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2zmCsfR

In this podcast, recorded live at Strange Loop 2017, Wes talks to Charity, cofounder and CEO of honeycomb.io. They discuss the social side of debugging and her Strange Loop talk “Observability for Emerging Infra: What got you Here Won't get you There”. Other topics include advice for testing in production, shadowing and splitting traffic, and sampling and aggregation. Why listen to this podcast: - Statistical sampling allows for collecting more detailed information while storing less data, and can be tuned for different event types. - Testing in production is possible with canaries, shadowing requests, and feature switches - Pulling data out of systems is just noise - it becomes valuable once someone has looked at it and indicates the meaning behind it. - Instrumenting isn’t just about problem detection - it can be used to ask business questions later - You can get 80% of the benefit from 20% of the work in instrumenting the systems. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2y6OP1b You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2y6OP1b

Nora Jones, a senior software engineer on Netflix’ Chaos Team, talks with Wesley Reisz about what Chaos Engineering means today. She covers what it takes to build a practice, how to establish a strategy, defines cost of impact, and covers key technical considerations when leveraging chaos engineering. Why listen to this podcast: - Chaos engineering is a discipline where you formulate hypotheses, perform experiments, and evaluate the results afterwards. - Injecting a bit of failure over time is going to make your system more resilient in the end. - Start with Tier 2 or non-critical services first, and build up success stories to grow chaos further. - As systems become more and more distributed, there becomes a higher need for chaos engineering. - If you’re running your first experiment, get your service owners in a war room and get them to monitor the results of the test as it is running. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2vJoimw You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extended shownotes? Check the landing page on InfoQ: http://bit.ly/2vJoimw

Shubha Nabar is a senior director of data science for Salesforce Einstein. Prior to working for Salesforce, she was a data scientist at LinkedIn and Microsoft. In the podcast she discusses Salesforce Einstein and the problem space that they are trying to solve, explores the differences between enterprise and consumer for machine learning, and then talks about the Optimus Prime Scala library that they use in Salesforce. Why listen to this podcast: * The volume of data, and hardware advances have made it possible to do machine learning to do them a lot faster. * AI is a science of building intelligent software, encompassing many aspects of intelligence that we tend to think of as human. * If you can’t measure something, you can’t fix it. * You have to think about what you can automate, rather than having a human to try and engineer out all those features. * Get feedback on design. Nora Jones, a senior software engineer on Netflix’ Chaos Team, talks with Wesley Reisz about what Chaos Engineering means today. She covers what it takes to build a practice, how to establish a strategy, defines cost of impact, and covers key technical considerations when leveraging chaos engineering. Why listen to this podcast: - Chaos engineering is a discipline where you formulate hypotheses, perform experiments, and evaluate the results afterwards. - Injecting a bit of failure over time is going to make your system more resilient in the end. - Start with Tier 2 or non-critical services first, and build up success stories to grow chaos further. - As systems become more and more distributed, there becomes a higher need for chaos engineering. - If you’re running your first experiment, get your service owners in a war room and get them to monitor the results of the test as it is running. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2vJoimw You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extended shownotes? Check the landing page on InfoQ: http://bit.ly/2xK7OxR

This week's podcast features Simon Brown well known for his work training software architects. Topics include the differences between a tech lead and an architect, how much documentation is enough and what that looks like in a continuous delivery environment. What you'll learn on this podcast: • As an industry we seem to have lost our knowledge of how to do architecture well in the context of modern agile software teams. • Architecture is about the expensive decisions; things that are costly to change later. • Ideally architects should code in the production code base. If you are not able to do this at least be involved in quality reviews and peer reviews in the production code so you can get feedback on your designs. • It is often said the the code is the only documentation you need but the code can’t tell you everything. You do need to document the things you can’t get from the code such as the architectural drivers, they key quality attributes and so on along with some high level diagrams and how you operate the system. • As you step into the role of architect go and find a mentor or a local meet-up. The major change is that you have to influence and lead people. This podcast is sponsored by AppDynamics. Software architects play a critical role in designi¬¬¬ng, executing, and migrating large infrastructures to the cloud. Download AppDynamic’s FREE eBook “10 Tips for Enterprise Cloud Migration” and launch your migration project with a proven plan. Download the eBook now at http://infoq.link/web_sndcld_appdynamics Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2xvq7qM

This week's podcasts features Yao Yue of Twitter. Yao spent the majority of her career working on caching systems at Twitter. She has since created a performance team that deals with edge performance outliers often exposed by the enormous scale of Twitter. In this podcast, she discusses standing up the performance team, thoughts on instrumenting applications, and interesting performance issues (and strategies for solving them) they’ve seen at Twitter. Why listen to this podcast: * Performance problems can be caused by a few machines running slowly causing cascading failure * Aggregating stats on a minute-by-minute basis can be an effective way of monitoring thousands of servers * Being able to record second-by-second is often too expensive to centrally aggregate, but can be stored locally * Distinguishing between request timeout and connection/network timeouts is important to prevent thundering herds * With larger scale organisations, having dedicated performance teams helps centralise skills to solve performance problems More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2wnBemB You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2wnBemB

On the InfoQ Podcast this week, Wes Reisz talks with the Queen of Patterns, Linda Rising. Linda discusses her thoughts on the importance of patterns, she answers questions about what really is a pattern, and how she became involved in working with them. Throughout the podcast she discusses a variety of organizational and personal patterns and finally wraps with patterns to apply when driving change and innovation. Why listen to this podcast: - You have to realise that there’s nothing you can do about other people. The only person you can affect is yourself. - A pattern is not a band-aid that you use once. You use it in a context where you use it in conjunctions with other patterns. - Take baby steps when driving change in an organisation, and seek out a pocket of receptive people to drive it. - Slack is an important part to have in life, so that if something comes along you can absorb it without having to stop doing something else. - Listen, Listen, Listen. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2vLIsMC You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2vLIsMC