The InfoQ Podcast

Anurag Goel, Founder and CEO of Render, shares insights into revolutionizing cloud platforms. He discusses the evolution of developer experiences and how Render simplifies Kubernetes deployments with user-friendly configurations. The conversation highlights the need for reduced complexity in cloud management, moving away from traditional DevOps demands. Anurag also touches on zero-downtime deployment practices and the significance of observability in cloud-native tech, all while navigating competition against established providers.

In this podcast, Daniel Bryant sat down with Greg Law, CTO at Undo. Topics discussed included: the challenges with debugging modern software systems, the need for “hyper-observability” and the benefit of being able to record and replay exact application execution; and the challenges with implementing the capture of nondeterministic system data in Undo’s LiveRecorder product for JVM-based languages that are Just-In-Time (JIT) compiled. Why listen to this podcast: - Understanding modern software systems can be very challenging, especially when the system is not doing what is expected. When debugging an issue, being able to observe a system and look at logging output is valuable, but it doesn’t always provide all of the information a developer needs. Instead we may need “hyper observability”; the ability to “zoom into” bugs and replay an exact execution. - Being able to record all nondeterministic stimuli to an application -- such as user input, network traffic, interprocess signals, and threading operations -- allows for the replay of an exact execution of an application for debugging purposes. Execution can be paused, rewound, and replayed, and additional logging data can be added ad hoc. - Undo’s LiveRecorder allows for the capture of this nondeterministic data, and this can be exported and shared among development teams. The UndoDB debugger, which is based on the GNU Project Debugger, supports the loading of this data and the execution and debugging in forwards and reverse execution of the application. There is also support for other debuggers, such as that included within IntelliJ IDEA. - Advanced techniques like multi-process correlation reveal the order in which processes and threads alter data structures in shared memory, and thread fuzzing randomizes thread execution to reveal race conditions and other multi-threading defects. - The challenges of using this type of technology when debugging (micro)service-based application lies within the user experience i.e. how should the multiple process debugging experience be presented to a developer? Live Recorder currently supports C/C++, Go, Rust, Ada applications on Linux x86 and x86_64, with Java support available in alpha. Supporting the capture and replay of data associated with JVM language execution, which contain extra abstractions and are often Just-In-Time (JIT) compiled, presented extra challenges. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37XLUa0 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37XLUa0

Today on The InfoQ Podcast, Wes Reisz speaks with CEO and founder of Solo Idit Levine. The two discuss the Three Pillars of Solo around Gloo, their API gateway, interoperability of service meshes (including the work on Service Mesh Interface), and on extending Envoy with Web Assembly (and the recently announced Web Assembly Hub). Why listen to this podcast: - Gloo is a Kubernetes-native ingress controller and API gateway. It’s built on top of Envoy and at its core is open source. - The Service Mesh Interface (SMI) is a specification for service meshes that runs on Kubernetes. It defines a common standard that can be implemented by a variety of providers. The idea of SMI is it’s an abstraction on top of service meshes, so that you can use one language to configure them all. - Autopilot is an open-source Kubernetes operator that allows developers to extend a service mesh control plane. - Lua has been commonly used to extend the service mesh data plane. Led by Google and the Envoy community, web assembly is becoming the preferred way of extending the data plane. Web assembly allows you to write Envoy extensions in any language while still being sandboxed and performant. - WebAssembly Hub is a service for building, deploying, sharing, and discovering Wasm extensions for Envoy. - Wasme is a docker like an open-source commandline tool from Solo to simplify the building, pushing, pulling, and deploying Envoy Web Assembly Filters. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37sYIoE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37sYIoE

Today, on The InfoQ Podcast, Wes Reisz talks with Gunnar Morling. Gunnar is a software engineer at RedHat and leads the Debezium project. Debezium is an open-source distributed platform for change data capture (CDC). On the show, the two discuss the project and many of its use cases. Additionally, topics covered on the podcast include bootstrapping, configuration, challenges, debugging, and operational modes. The show wraps with long term strategic goals for the project. Why listen to this podcast: - CDC is a set of software design patterns used to react to changing data in a data store. Used for things like internal changelogs, integrations, replication, and event streaming, CDC can be implemented leveraging queries or against the DB transaction log. Debezium leverages the transaction log to implement CDC and is extremely performant. - Debezium has mature source and sink connectors for MySQL, SQL Server, and MongoDB. In addition, there are Incumbating connectors for Cassandra, Oracle, and DB2. Community sink connectors have been created for ElasticSearch. - In a standard deployment, Debezium leverages a Kafka cluster by deploying connectors into Kafka Connect. The connectors establish a connection to the source database and then write changes to a Kafka topic. - Debezium can be run in embedded mode. Embedded mode imports Java library into your own project and leverages callbacks for change events. The library approach allows Debezium implementations against other tools like AWS Kinesis or Azure's Event Hub. Going forward, there are plans to make a ready-made Debezium runtime. - Out of the box, Debezium has a one-to-one mapping between tables and Kafka topic queues. The default approach exposes the internal table structure to the outside. One approach to address exposing DB internals is to leverage the Outbox Pattern. The Outbox Pattern uses a separate outbox table as a source. Inserts into your normal business logic tables also make writes to the outbox. Change events are then published to Kafka from the outbox source table. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3737GZB You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3737GZB

In this podcast, Daniel Bryant sat down with Kelsey Hightower, Staff Developer Advocate at Google. Topics covered included: the extensibility of Kubernetes, and why it has become the platform that other platforms are being built on top of; creating event-driven architectures and deploying these onto Function-as-a-Service (FaaS) platforms like the Kubernetes-based Knative and Google Cloud Run; and the benefits of learning, sharing knowledge, and building communities. Why listen to this podcast: - Kubernetes is a platform for building platforms. It may not be as opinionated as traditional Platform-as-a-Service (PaaS) offerings, but it has become popular due to its extensibility. There are PaaS-like solutions built on top of Kubernetes, such as OpenShift, Knative, and Cloud Run. - The creation of common interfaces within Kubernetes -- such as Custom Resource Definitions (CRDs), Container Networking Interface (CNI), and Container Runtime Interface (CRI) -- enabled the adoption of the platform by vendors and the open source community without everyone needing to agree on exactly how to implement extensions. - Although not every workload can be effectively implemented using an event-driven architecture, for those that can the Kubernetes-based Function-as-a-Service (FaaS) platforms like Knative and Cloud Run can handle a lot of the operational management tasks for developers. - Engineers may be able to get ~90% of the “service mesh” traffic management functionality they need from using a simple proxy. However, the separation of the control and data planes within modern service meshes, in combination with the rise in popularity of the sidecar deployment model, has provided many benefits within Kubernetes. - A lot of learning within software development and information technology is transferable. If you spend time going deep in a technology when you begin your career, much of what you learn will be useful when you come to learn the next technology. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/30alHC1 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/30alHC1

In this podcast, Daniel Bryant sat down with Katie Gamanji, Cloud Platform Engineer at Condé Nast International. Topics covered included: exploring the architecture of the Condé Nast Kubernetes-based platform; the importance of enabling self-service deployment for developers; and how the Kubernetes’ Federation API and Cluster API may enable more opportunities for platform automation. - Founded in the early 1900s, Condé Nast is a global media company that has recently migrated their application deployment platforms from individually-curated geographically-based platforms, to a standardised distributed platform based on Kubernetes and AWS. - The Condé Nast engineering team create and manage their own Kubernetes clusters, currently using CoreOS’s/Red Hat’s Tectonic tool. Self-service deployment of applications is managed via Helm Charts. - The platform team works closely with their “customer” developer teams in order to ensure their requirements are being met. - The Kubernetes Federation API makes it easy to orchestrate the deployment of applications to multiple clusters. This works well for cookie-cutter style deployments that only require small configuration differences, such as scaling the number of running applications based on geographic traffic patterns. - The Cluster API is a Kubernetes project to bring declarative APIs to cluster creation, configuration, and management. This enables more effective automation for cluster lifecycle management, and may provide more opportunities for multi-cloud Kubernetes use. - The Condé Nast platform Kubernetes Ingress is handled by Traefik, due to the good Helm support and cloud integration (for example, AWS Route 53 and IAM rule synchronization). The platform team is exploring the use of service mesh for 2020. - Abstractions, interfaces, and security will be interesting focal points for improvement in the Kubernetes ecosystem in 2020. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2FeYPrE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2FeYPrE

In this podcast, Daniel Bryant spoke to Joseph Jacks, Founder of OSS Capital and the Open Core Summit, and discussed topics including the open source and open core models, innovations within open source hardware and the RISC-V instruction set architecture, and current opportunities for disruption using commercial open source software. Why listen to this podcast: - Recently, open source software and the open core business model have driven a lot of innovation and created a lot of value, particularly within the cloud “as-a-service” space. - There has been some disagreement between the open source and commercially-focused communities, for example, in relation to the licencing models and how value is captured. - The Open Core Summit (OCS) is a new conference focusing on the intersection of commercialisation and open source software that aims to facilitate discussion in this space. - Organisations building around open source software can potentially look at large cloud vendors as partners. Public clouds can provide effective distribution, and typically focus on offering breadth of services rather than the depth of expertise that can be provided by a specialist company. - RISC-V is an open-source hardware instruction set architecture (ISA) based on the well-established reduced instruction set computer (RISC) principles. Leveraging RISC-V can reduce the time and cost of customising chip designs. - A lot of recent open source innovation has focused on the infrastructure layer within computing systems. This means that the application layer is now potentially ripe for disruption via commercial open source software. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2rDfYYU You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2rDfYYU

In this special year-end wrap-up podcast Wes Reisz, Shane Hastie, Daniel Bryant, and Charles Humble discuss what we’ve seen in 2019 and speculate a little on what we hope to see in 2020. Topics include business agility and Teal, what it means to be an ethical engineer, bringing your whole self to work, highlights from QCon and InfoQ during 2019, the rise of Python, and progress in quantum computing. Why listen to this podcast: * Business agility is one of the major themes that the InfoQ team has seen emerge this year, with stronger emphasis on outcomes over outputs. We’ve also seen a growing interest in ethics and the ethical implications of the work we all do. * On the programming languages front the rise of Python continues, driven largely by its popularity in data science. * As Kubernetes cements its dominant position we’re hoping to see a simplification of the workflows associated with it, as well as in areas like observability. * There have been several big announcements in quantum computing in the past year, and this is an area we continue to watch with interest. * Another key trend for next year is edge computing. The edge of the cloud infrastructure has an amazing amount of available compute resource, as does the device edge. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Z0Q9OI You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Z0Q9OI

Josh Wills, a software engineer working on data engineering problems at Slack, discusses the Slack data architecture and how they build and observe their pipelines. Josh, along with color commentary such as the move from IC to manager (and back), discusses recommendations, tips, tools, and lessons Slack engineering teams discovered while building products like Slack Search. The podcast covers machine learning, observability, data engineering, and general practices for building highly resilient software. Why listen to this podcast: - Slack has a philosophy of building only what they need. They have a don’t reinvent the wheel mindset. - Slack was originally a PHP monolith. Today, it is largely Hack-lang, HHVM, and several Java and Go binarys. On the data side, application logs are in Thrift (there is a plan to migrate to protobuf). Events are processed through a Kafka cluster that handles 100,000s of events per second. Everything is kept in S3 with a large Hive metastore. EMR is spun up on demand. Presto, Airflow, Slack, Snowflake (business analytics), Quiver (key value store) are all used. - ML worked best for Slack when it was used to help people answer questions. Things like Learn to Rank (LTR) become the most effective use of ML for Slack. - You can get pretty far with rules. Use machine learning when that’s all that’s left. - When you start applying observability to your data pipeline, a key lesson for Slack was to really focus on structured data, tracing, high cardinality events. This let them really use the tools they were already familiar with (ELK, Prometheus, Grafana) and go deep into understanding what’s happening in the systems. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PsVA4q You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2PsVA4q

In this podcast, Daniel Bryant sat down with Bryan Liles, senior staff engineer at VMware. Topics covered included: the challenges with deploying applications into Kubernetes, using the open source Octant tool to increase a user’s understanding of Kubernetes clusters, and how “serverless” technologies may influence the future approaches to building software. Why listen to this podcast: - Octant is a highly extensible platform for developers to better understand the complexity of Kubernetes clusters. Octant runs locally, using the local Kubernetes credentials. It currently displays information about a Kubernetes cluster and related applications as a web page. Soon this tool and resulting display will be provided as a standalone application. - The goal of Octant is to enable users to discover what they need to discover. The tool aims to provide context relevant to where a user is and what they are trying to achieve. The Octant plugin system allows integration with other tooling, such as logging and metrics frameworks. This aims to facilitate quick problem detection and resolution. - Cloud native platforms like Kubernetes are complicated, as there are lots of moving parts. The most important challenge to be tackled to increase the adoption of platforms like Kubernetes is “how do we move code from our IDEs to wherever it needs to run with the least amount of friction?”. Testing needs to be implicit, as does security verification, and the acts of deployment. Kubernetes needs its “Ruby on Rails” moment. - Creating “serverless” systems is an interesting approach, but we may currently be using this technology in a non-optimal way. For example, creating web applications using this technology enables scalability, but can lead to the creation of difficult to understand systems that also require a lot of boilerplate configuration. Arguably, a more interesting use case is implementing large-scale batch processing using simple event-driven models. - The Cloud Native Computing Foundation (CNCF) has created a series of communities of practice called Special Interest Groups (SIGs), such as SIG App Delivery. This allows folks with similar interests to work together as a community, focusing on solving a specific set of well-scoped problems. There are many ways to get involved, from discussions, to coding and creating documentation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37iUwIG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37iUwIG