Episoder
-
Implementing Data & Databases on K8s within the Dutch Government
Presented by Sebastiaan Mannem, Director at Mannem SolutionsA small walkthrough of projects within the Dutch government running databases on OpenShift. This talk shares success stories, provides a proven recipe to `get it done,` and debunks some of the FUD.Related LinksDoKC Website - https://dok.community/DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w
-
Unsticking Ourselves from Glue: Migrating PayItâs Data Pipelines to Argo Workflows and Hera
Presented by Matt Menzenski, Senior Software Engineering Manager, PayitgovAt PayIt, weâve been deploying applications to Kubernetes almost since the beginning of the company. Our data workloads, however, have run instead in AWS Glue. This has worked well enough for the reporting use cases that have been the main focus of this team historically. However, at the beginning of 2022, the PayIt data team began building out a new data platform, and in the process, ran into a number of challenges with Glue. In this talk, I will share the difficulties that we encountered with building, deploying, and orchestrating ETL pipelines in AWS Glue, our decision process for moving those workloads into Kubernetes, and the ELT architecture that weâve arrived at today.Related LinksDoKC Website - https://dok.community/DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w
-
Mangler du episoder?
-
Repel Boarders! How to find a Kubernetes operator that really protects your data
Presented by Robert Hodges, Altinity
Operators are a godsend for managing data in Kubernetes. But how about protecting it? We'll explore security threats to cloud native databases and show what protection you should look for in operators. Finally we'll introduce a new Data on Kubernetes Community project to develop security standards for database operators in Kubernetes.Related LinksDoKC Website - https://dok.community/DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w -
DoK + Apache Spark
Presented by Holden Karau, Spark Committer and Open Source Engineer at Netflix
In this brief talk, Holden will cover some of the best practices from trying to deploy both small and large scale Spark on Kube.Related LinksDoKC Website - https://dok.community/DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w -
DoK @ Comcast: Delivering Business Outcomes & Improved DevX with Data Services Running on KubernetesPresented by Greg Otto, Executor Director, DevX Platforms & Charles Ju, Principal EngineerTransforming how to deliver measurable value using data on Kubernetes, while providing psychological safety. If you just sighed, youâre one of the many people leaders and engineers who know the challenges of making these changes. In this talk, we will share our transformation journey, the âMonths to Minutesâ outcomes we achieved, the architecture approach, and the human journey from one of our engineers.Related LinksDoKC Website - https://dok.community/DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w
-
Abbey Russell, PM at Cockroach Labs, shared the backstory on how and why Kafka was created.
Along the way, you'll learn about
- Who Franz Kafka was
- Kafka's earliest use at Linkedin in 2010
- Why organizations like Uber/Coursera/Mailchimp use it today
- Future of Data StreamingTo find out more about how organizations are benefitting from running streaming workloads on Kubernetes, check out our 2022 research report- https://dok.community/data-on-kubernetes-2022-report/
https://dokcommunity.slack.com/
https://dok.community/About Data on Kubernetes Community (DoKC)
DoKC is an independent tech community where practitioners go to run data workloads on Kubernetes. It is managed by Constantia.io. We produce a range of resources for end users looking to run DoK including:- DoK Landscape An interactive landscape of products and services for running data workloads on Kubernetes. (https://dok.community/landscape/)
- Videos and Blogs Hundreds of videos and blogs including how tos, tutorials, use cases, case studies, and more. (https://dok.community/landscape/) (https://dok.community/blog/)
- DoK Reports DoKC works with an independent research firm to annually produce the industryâs only DoK-focused report. (https://dok.community/dok-reports/)
- Livestreams and Local Meetups Weekly livestreams on DoK-related topics including demos, tutorials, visionary talks, panels, and more. (https://www.meetup.com/data-on-kubern...)
- DoK Days @ KubeCon We host hybrid events twice a year for people to get together and learn in-person or online. Details are published on our main website. (https://dok.community/)
Connect With Us:
https://dok.community/
https://twitter.com/DoKcommunity
https://www.linkedin.com/company/6667...
https://github.com/dokc -
https://go.dok.community/slack
https://dok.community/
https://youtu.be/KjiK6eXYO34
ABSTRACT OF THE TALK
In this talk Sergio is going to present different ways to store data at the edge using different databases and Long Horn as a storage class. All this running on a Raspberry Pi and showing and small application using a database running at the edge.BIO
Sergio MĂ©ndez is a systems engineer and professor of operating systems at USAC Guatemala university. His work at the university is related to teaching and researching cloud native technologies with his students.
He has experience working on DevOps, and MLOps using open source technologies at work. About open source communities, he is involved in the CNCF Community, promoting students into the CNCF Ecosystem and hosting a Cloud Native meetup in Guatemala. He has been a speaker at several conferences such as KubeCon, WTFisCloudNative, and Kubernetes Community Days.
He is also a Linkerd Ambassador and author of Edge Computing systems with Kubernetes with Packt publisher.KEY TAKE-AWAYS FROM THE TALK
How to storage data at the edge using databases
-
https://go.dok.community/slack
https://dok.community/
Link: https://youtu.be/n_thXwyJNSU
ABSTRACT OF THE TALKDeploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments.
BIO
Passionate about Cloud Native, Kubernetes, Developer relations
KEY TAKE-AWAYS FROM THE TALK
This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet
-
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/4cPVRWOK-_E
ABSTRACT
Apache Kafka is the de facto data streaming platform used for ingesting vast amounts of data and processing them in real-time. Low latency analytics are vital if users are to react to events as fast as possible and to effectively shape future decision making. Together with Kubernetes, it allows to develop cloud oriented analytics solutions which are highly scalable.
During this session, we will explore an Apache Kafka based solution running on Kubernetes to help a Formula 1 team gain insights during a race. The ârealâ data will come from a well-known racing game and will be processed by our application in real-time, making us feel like real track-side F1 engineers! -
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/Y4tdy9lctEI
ABSTRACTLearn how customers are increasingly deploying stateful applications on Kubernetes to benefit from portability, economies of scale, and built-in orchestration capabilities. This talk will include how customers choose between using Kuberentes, or a data Software as a Service (SaaS) and stateful capabilities of Kubernetes across two dimensions - the application orchestration and the storage layer. Also learn about MariaDB SKYSQL, a database software as a service that runs thousands of StatefulSet Pods across multiple zones and regions on Kubernetes.
-
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/A1ch4AhKoeQABSTRACT
If thereâs one thing that everyone can agree on - itâs that the sheer scale and complexity of Kubernetes operations is growing constantly. Whatâs more, cloud native environments are becoming more and more expensive to operate and manage, as well as increasingly difficult to secure. On the bright side, there is a growing ecosystem of exceptional open source tools to help overcome this complexity, and provide greater situational awareness to whatâs happening in your many and multiple Kubernetes clusters.
In this talk, weâll take a deep dive on how to understand everything that is happening with stateful workloads in your Kubernetes clusters from health to security - including the management and monitoring of your clusters, troubleshooting and debugging, through full security scanning in multiple data perspectives- from configMaps, Secrets, hostPaths and many more security driven checks for proper role-based access control, vulnerabilities and misconfigurations. We will demonstrate how you can have a 360Âș-view of your Kubernetes clusters in a single open source dashboard leveraging tools like Lens and Kubescape, and ultimately enforce policies with Open Policy Agent. -
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/LymPjH6HA3E
ABSTRACTStateless apps are easy to manage. More often than not, a Kubernetes Deployment, with a Service, Ingress, and Horizontal Pod Autoscaler (HPA) is enough. Almost everyone can do it. But, when it comes to stateful applications, things become a bit more complicated. We might need a database and storage. We might need to manage database users and schema. We might need to consider quite a few other things. Stateful apps are harder for everyone, especially if we want to shift left and enable developers to do it themselves.
In this talk, we'll try to make the management of stateful applications easy for everyone. We'll accomplish that by creating easy-to-consume services that are made specifically for the needs of our organizations. We'll see how to create new Kubernetes Custom Resource Definitions (CRDs) and controllers using Crossplane. Those controllers will envelop all the tools, resources, and processes we might need. As a result, ops can focus on creating such services while everyone else can consume them (create and manage everything related to stateful apps) without opening JIRA tickets and waiting for others to complete their tasks. -
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
ABSTRACT
Healthcare organizations are transforming their applications and embracing digital platforms for efficient patient care. Today, compute at the edge, plays a critical role in deploying innovative healthcare applications that promise new approaches to patient care. Connected care platforms at scale are turning to Kubernetes infrastructure for deployment of telehealth and telemedicine apps. These kinds of applications often need real time image processing and inferencing at the edge, which in turn, require a distributed application platform on Kubernetes. They also need secure foundations for a regulatory compliant framework for wider adoption.
To elaborate further, a distributed platform or a multi-cluster infrastructure is necessary for healthcare applications. All medical application platforms are data intense. For eg. Medical applications that need to stream video data, i.e move large quantities of data need secure connections from edge to cloud. Smart healthcare applications are also seeing the value of multi-cloud deployments.
The ability to choose different technologies like AI, data storage systems or some specific relational database systems is driving multi-cloud adoption in healthcare solutions. A multi-cluster connectivity fabric is thus needed for data intensive healthcare applications to distribute across geographical boundaries. The fabric needs to have inbuilt automation and the smarts to make the best use of compute resources in a distributed infrastructure, for improved patient healthcare.
-
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
ABSTRACT
A practical session about running Highly Available PostgreSQL in Kubernetes. The primary objective will be to demonstrate how to set up a reliable architecture in a Kubernetes cluster to achieve low RTO and RPO. This will be covered by going over the various Kubernetes native components of our fully open source operator CloudNativePG[0]. It will be demonstrated how these components allow for a simple adoption of PostgreSQL into Kubernetes environments. If time and technical constraints permit there will be a live demo at the end of the presentation; if for some reason that is not possible there will be a link to a recorded demo.https://cloudnative-pg.io/
-
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
ABSTRACT
In this talk youâll explore how to run a PostgreSQL cluster across multiple Kubernetes clusters. Learn what challenges arise when using asynchronous streaming replication in a set of Kubernetes clusters spanning across several geographical regions. It will be discussed how such a topology can be established using Kubernetes and Kubernetes Add-Ons. You will learn which qualities the system has, where such a distributed system plays its strengths and where it may be problematic to use it. At the end of the talk you will have an increased understanding about how to create multi-regional data service instances using PostgreSQL as an example. -
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
ABSTRACT
So youâre looking to run your Open Source Database on Kubernetes. What best practices should you follow and what pitfalls should you avoid ? In this presentation we will look at how to run stateful applications on Kubernetes overall as well as what is particularly important for databases - we will cover high availability, security, backups and disaster recovery. Finally we will show how these practices can be implemented with Percona Operators for MySQL, MongoDB, PostgreSQL - one of the leading solutions to run Open Source Databases on Kubernetes
-
From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
ABSTRACTIn the software industry weâre fond of terms that define major trends, like âcloud nativeâ, âKubernetes nativeâ and âserverlessâ. As more and more organizations move stateful workloads to Kubernetes, weâve started to see these terms applied to data infrastructure, where they can get overtaken by marketing hype unless we work to define them.
In this talk, weâll examine two different databases, TiDB and Apache Cassandra, in order to identify what it means for a database to be Kubernetes native and why it matters. Weâll look at points including:
- The differences between cloud native, Kubernetes native, and serverless
- How databases become Kubernetes native
- Benefits of Kubernetes native databases
- How Kubernetes can better support databases - Se mer
